WEBVTT 00:01.570 --> 00:07.120 In this video we learn how to perform split a play combine operations on a data frame with the group 00:07.130 --> 00:08.490 by method. 00:08.490 --> 00:12.910 So in the last videos we already learned how to split a data frame into groups. 00:13.090 --> 00:19.840 And now we apply a function to our group by object and by doing so we are creating a new combined data 00:19.840 --> 00:20.890 frame. 00:20.890 --> 00:24.750 This sounds complicated but you will see that it's really easy and straightforward. 00:24.760 --> 00:33.820 So let's start coding and we impart pandas and we are working with the Titanic data set and also this 00:33.820 --> 00:36.040 time we are working with our toy example. 00:36.040 --> 00:45.820 So we are slicing our Titanic data frame and we only use in the first 10 rows and columns sex and age. 00:45.880 --> 00:51.960 So yes our toy example and we have already learned that by applying the group by a method. 00:52.060 --> 00:58.540 And for example using the key sex we are splitting our data frame here into in this example two data 00:58.540 --> 01:03.880 frame so we have one data frame with all the rows where the passengers female and we have another data 01:03.880 --> 01:11.270 frame with the rows and where the passengers male and we can transform our group by object into a list 01:11.300 --> 01:18.530 and then we can slice our list and here our first data frame and our group by object is the the female 01:18.530 --> 01:25.640 data frame and the second data frame is that the male data frame now let's assume that you want to calculate 01:25.910 --> 01:33.290 the average age of the females and the average age of the male passengers and therefore we are using 01:33.290 --> 01:38.720 near the group by method on our slice and be grouped by the sex column. 01:38.730 --> 01:40.460 So male and female. 01:40.640 --> 01:46.700 And by doing so we are creating a group by object and then we have to apply a function on our group 01:46.700 --> 01:53.810 by object and the function will be applied on both the data friends and our group by object and after 01:53.810 --> 01:59.690 we have applied both functions here both data frames will be combined to 1 you data frame. 01:59.690 --> 02:06.970 So this sounds complicated but let's just do it and we are using the function mean and let's surround 02:06.980 --> 02:14.480 the cell and let's see what we get here so we are getting near a brand new data frame where we have 02:14.530 --> 02:21.630 this index here our sex columns of female and male and we have a single column age and t we have the 02:21.630 --> 02:28.700 mean or the average age for all females and four males so actually here twenty eight US and the mean 02:28.710 --> 02:34.860 are the average of all of the females here and now a split and also here for our males or twenty eight 02:34.860 --> 02:42.490 point two five is that the mean or the average of one male passengers here and by default pandas is 02:42.570 --> 02:49.440 just ignoring here the missing value or any end value and this is actually also with the group by method 02:49.620 --> 02:56.220 we split our original data frame and a female and a male data frame and then for the female and the 02:56.220 --> 03:02.640 male did a frame we calculated the average age and then we combined the average age of females and the 03:02.640 --> 03:12.470 average age of males into one final data frame here and let's also have a look at our slides here so 03:12.470 --> 03:20.700 here we had our original data frame then we use to the group by a method and used as a key or the sex 03:20.700 --> 03:28.360 column and by doing so we created a group by object so we split our data frame into all rows that we 03:28.370 --> 03:35.370 have few minutes and all the rows where we have males and then we apply to the function mean and by 03:35.370 --> 03:43.770 doing so pandas actually calculated the mean age of all females and the mean age of automates and then 03:43.770 --> 03:50.380 both mean values were combined into a brand new data frame so by applying the group by method we split 03:50.380 --> 03:56.700 it our data frame and by using here the main function we applied a function on both data frames and 03:56.820 --> 04:01.630 at the same time we combined our groups into a new data frame. 04:01.740 --> 04:03.600 All right so now let's go back coding 04:06.840 --> 04:13.230 and in theory as a function Yeah we can apply and any function and the most frequently use functions 04:13.230 --> 04:20.700 are actually aggregation so taking the mean are the some the men are the max of the groups so as a next 04:20.700 --> 04:28.200 step we could also use here the sum function on our group by object so let's see what we get here and 04:28.200 --> 04:35.460 we are using Yeah the group by and some function on our whole data frame so we are having all 891 rows 04:35.910 --> 04:44.860 and columns and let's see what we get here so again we split it our data frame here by the sex column 04:44.860 --> 04:46.450 so we are female and male. 04:46.960 --> 04:50.000 And then we ask for each column here actually the sum. 04:50.140 --> 04:56.680 So we have here these survived column and in total two hundred thirty three female passenger survived 04:56.710 --> 05:03.520 and only one hundred nine male passengers survived and we have also the sum of the P class on this case 05:03.530 --> 05:07.450 that doesn't make really sense and we have also the sum of the H. 05:07.450 --> 05:15.430 So all females together seven thousand two hundred and eighty six years old and we have also the sum 05:15.430 --> 05:18.000 of all siblings and spouses. 05:18.130 --> 05:25.840 So all female passengers together had two hundred eighteen siblings and spouses onboard and we have 05:25.840 --> 05:33.400 the same year for parents and children and also we have the sum of the fare so all females together 05:33.430 --> 05:40.270 paid about forty thousand dollars a pound and all male passengers together paid slightly more than the 05:40.270 --> 05:41.280 females. 05:41.530 --> 05:47.620 And you can see here that taking the sum does not make sense for all columns so let's assume we are 05:47.620 --> 05:54.700 only interested in these survived column and we can do so here by here just only choosing these survival 05:54.700 --> 05:58.610 columns that survived. 05:58.730 --> 06:00.650 So this is actually method chaining. 06:00.650 --> 06:07.700 So first of all we are using the group by a method on our Titanic data frame and we group our Titanic 06:07.700 --> 06:09.590 data framed by the sex column. 06:09.590 --> 06:15.630 Then we are only interested in the Sea Life column and for each groups of female and male. 06:15.770 --> 06:20.050 We want to know the amount of female and male who survived. 06:20.180 --> 06:27.290 All right so let's do this and we can see here that we only get aggregations for these survival columns 06:27.320 --> 06:32.960 so we have two hundred thirty three females who survived and only one hundred nine men. 06:32.960 --> 06:36.330 All right and of course we can select more than one column. 06:36.350 --> 06:43.460 So you've only selected the survival column but we can also select the fair and the H column so we have 06:43.460 --> 06:50.060 to pass here within our stat brackets a list with the fair and the H column and we split our data framed 06:50.060 --> 06:54.010 by the sex column and then we want to know the maximum. 06:54.170 --> 06:58.490 So we want to know the maximum fare paid by a female and by a male. 06:58.520 --> 07:02.090 And we want to know the maximum age of females and males. 07:02.090 --> 07:03.410 So let's do this year 07:06.160 --> 07:10.030 so the highest fair paid by a female is five hundred and twelve. 07:10.030 --> 07:14.240 And also the highest fair paid by a male is also five hundred and twelve. 07:14.440 --> 07:23.150 And the oldest female on board was sixty three years old and the oldest male on board was 80 years old. 07:23.170 --> 07:29.810 All right so now let's again split our whole Titanic data framed by the sex column and we want to calculate 07:29.810 --> 07:31.570 that they mean on all columns. 07:31.570 --> 07:41.320 So let's do this and we assign the variable a new data frame so let's have a look at new data from so 07:41.350 --> 07:41.890 PBF. 07:41.960 --> 07:44.580 They mean for all columns by female and male. 07:44.750 --> 07:52.520 So whenever it's 75 percent of all females survived at the Titanic disaster and only about 19 percent 07:52.520 --> 08:00.010 of all males and the average peasant class is a little bit higher for the males and also the male peasant 08:00.010 --> 08:04.880 just a bit older on average than the female passengers. 08:04.880 --> 08:11.120 And on average and the female peasant just paid more for their tickets than the male passengers. 08:11.150 --> 08:17.480 So on average from the female is paid 44 dollars a pound and the male is paid twenty five dollars a 08:17.480 --> 08:18.650 pound. 08:18.650 --> 08:19.120 All right. 08:19.130 --> 08:25.780 Now we can also visualise here our new data frame and we import met plot lib and we use in the Seabourn 08:25.790 --> 08:33.050 style and we want to have a bar plot for each column where we can see the difference between female 08:33.050 --> 08:33.610 and male. 08:33.620 --> 08:35.840 So let's do this and let's have a look here 08:40.020 --> 08:41.720 so here we use the subplots. 08:41.720 --> 08:42.130 True. 08:42.130 --> 08:44.900 So for each column here we have a subplot. 08:44.910 --> 08:52.030 And here the first subplot this fourth and survive and we can see here on the left side the females 08:52.050 --> 09:00.470 and on the right side the males and we can see that the survival rate is tremendously higher for females. 09:00.490 --> 09:03.580 And here we have the average of the passenger class. 09:03.640 --> 09:05.530 And here the average of the age. 09:05.530 --> 09:14.750 So we can see that on average the male passengers were a bit older than the female passengers and here 09:14.750 --> 09:17.030 we have the information about the fare. 09:17.330 --> 09:24.420 So on average the female passengers paid significantly more than the male passengers. 09:24.420 --> 09:24.870 All right. 09:24.870 --> 09:30.960 So now we are finished with our video and in the next video we will apply our new split apply combined 09:31.020 --> 09:34.710 skids to some more cases so hope to see you there by.