WEBVTT 00:01.090 --> 00:02.020 Hi everybody. 00:02.050 --> 00:09.010 So for this video we still have imparted penned us an umpire from the last video and we also have started 00:09.010 --> 00:12.450 the H column of our Titanic dataset and the variable H. 00:13.000 --> 00:20.230 And we have our H panda serious within the marital values so we can also check again if our H serious 00:20.230 --> 00:23.500 has an American values so it has its float objects. 00:23.500 --> 00:29.520 That's true and now we want to further analyze our H column R. H. 00:29.520 --> 00:37.210 Pandas areas and in the last video we already applied them the very powerful method describe them that 00:37.250 --> 00:38.780 return to us here. 00:38.890 --> 00:45.610 The number of non low values that the mean standard deviation Max and so on and we can also call all 00:45.610 --> 00:48.200 of these summary statistics. 00:48.250 --> 00:49.430 A single method. 00:49.600 --> 00:52.300 So for example here we have the count method 00:55.250 --> 01:00.320 and the count method that gives us all nominal values of our pandas series. 01:00.320 --> 01:02.960 So that's the press shift tap here. 01:03.140 --> 01:08.170 Return number from animal values and observations in our series. 01:10.700 --> 01:20.010 All right so let's go on and try the attribute size and check with the shift tab. 01:20.170 --> 01:27.770 So the attribute size the returns are the number of elements in our serious so obviously also including 01:27.770 --> 01:28.480 missing value. 01:28.490 --> 01:34.950 So any anywhere you saw also included here in the attribute size so the differentiate between count 01:34.980 --> 01:44.330 and sizes then that count does not include all missing values and we can also pass them our age pandas 01:44.330 --> 01:48.250 serious to the Python burden and function length. 01:48.250 --> 01:55.860 And here we get eight hundred ninety one elements in our pandas areas including missing values and we 01:55.860 --> 02:00.600 can also take the sum of all elements in our pandas serious age. 02:01.740 --> 02:04.040 So it's twenty one thousand. 02:04.260 --> 02:07.800 Here in the case of our H column so the sun doesn't really make sense. 02:07.800 --> 02:14.190 But I want to show you here and highlight the difference to the past and build and function some so 02:14.190 --> 02:17.460 we can pass our pandas serious here to the sum function 02:20.170 --> 02:26.900 and you will get an A and so this stands for our number terminal or our non value actually. 02:27.100 --> 02:28.770 And why is this the case. 02:28.780 --> 02:34.470 Because then the pace and burden function some cop cannot handle missing values and in contrast then 02:34.560 --> 02:40.840 the method some can handle missing the missing values and you can see here by pressing shift tab that 02:40.840 --> 02:51.660 by default that is a parameter call it skip and a check here. 02:51.980 --> 02:52.550 Here it is. 02:52.620 --> 02:54.770 And uh by default that's true. 02:54.800 --> 02:58.700 So it excludes uh and a null values when computing the result. 03:00.710 --> 03:09.230 So we can also try it here skip and a and by default it's true so by default and some method to skips 03:09.380 --> 03:17.570 and a values missing values and still we get here twenty one thousand but if we do not skip and a values 03:20.000 --> 03:26.650 then also a year we get the non selection panels cannot calculate the sum year because we have uh low 03:26.670 --> 03:33.560 values of missing values and now we're serious and they actually they you can see the power of the pandas 03:33.560 --> 03:39.860 to the library of that it's specifically designed for handling real world data where we have they are 03:39.920 --> 03:47.960 typically many missing values and by default in most of the cases pandas just yet ignores actually the 03:47.960 --> 03:50.330 missing values. 03:50.450 --> 03:53.620 All right so now let's go over to the method mean. 03:54.170 --> 03:59.530 So the mean method obviously calculates the average or the mean value of our pandas serious. 03:59.530 --> 04:03.610 So the average peasant job was at twenty nine point six years old. 04:04.130 --> 04:08.800 And also here we have the parameter skip and a. 04:10.610 --> 04:21.840 So and by default it's uh set to true and B could also change this to forms and include the missing 04:21.840 --> 04:25.610 values and to our mean calculation and then we get normal. 04:25.620 --> 04:32.020 So you're by default and the and a values are ignored actually. 04:32.030 --> 04:37.170 And that's that's good actually for us because on most of the cases we have and a values and they are 04:37.190 --> 04:44.310 we should ignore it like when we calculate the mean values and that's also the method the standard deviation 04:44.310 --> 04:52.590 which calculates them the standard deviation and also that is a method that returns us to the minimum 04:52.590 --> 04:54.720 value in our pandas series. 04:54.720 --> 05:03.270 So we already know there was a passenger that was only 0.01 two years old and with the max method we 05:03.270 --> 05:03.960 can get them. 05:04.040 --> 05:09.480 Actually the oldest passenger on board of the Titanic and he was 80 years old. 05:09.480 --> 05:16.890 And there's also the P2P method and we can check here pressing shift tab. 05:16.890 --> 05:22.890 So this returns and the difference between the maximum value and the minimum value in our panda serious. 05:22.890 --> 05:26.910 So obviously the maximum value is 80 and the minimum 0.01 too. 05:27.090 --> 05:35.110 So we should get the difference here and it's seventy nine point five eight and with the median method 05:35.140 --> 05:39.930 we can get to the median of our pandas serious it's twenty eight years 05:42.810 --> 05:49.770 and now we are coming to the very powerful a unique method and the unique method is one of the methods 05:49.790 --> 05:57.180 then that are exclusively only available for pandas serious but not for panda data friends so we can 05:57.330 --> 06:02.320 run near the site and have a look and we can also check here with the shift tab. 06:03.790 --> 06:08.560 So the unique method returns unique values of our series object 06:12.310 --> 06:14.350 so this are actually the unique value. 06:14.380 --> 06:17.760 So in total we have 800 uh 91 percent just. 06:17.770 --> 06:21.200 But these are the unique ages actually. 06:21.220 --> 06:27.130 So we have for example passengers that are 22 years old and we have passengers that are 38 years old 06:29.000 --> 06:33.350 and you have the next question would be how many unique values do we have. 06:33.350 --> 06:42.100 And we can apply here the length function and we get eighty nine unique values and as you can see here 06:42.100 --> 06:50.920 by applying the unique method also missing values and values are included so eighty nine here should 06:50.920 --> 06:59.370 be all unique values including missing values and with the end unique method we can also count the number 06:59.370 --> 07:02.130 of unique values and it's eighty eight. 07:02.190 --> 07:07.010 So by default the end unique method excludes missing or any end values. 07:07.020 --> 07:13.990 But if you press shift tab here you can also see here that there apparently the drop and a. 07:15.210 --> 07:16.770 And by default that's true. 07:17.340 --> 07:20.850 So it ignores the missing values. 07:20.850 --> 07:30.510 And if we switch here to fours we also got eighty nine unique values because now also the end they end 07:30.500 --> 07:33.750 on missing values are included. 07:33.750 --> 07:39.810 All right so next we come to the also very helpful and yes super power powerful method. 07:39.810 --> 07:44.360 Value counts and with the value counts. 07:44.430 --> 07:49.840 Let's run here and check here shift tab. 07:50.240 --> 07:57.690 So value counts the returns here an object containing counts of unique value is and the resulting object 07:57.690 --> 08:03.040 will be in descending order so that the first element is the most frequently occurring element. 08:05.310 --> 08:10.350 So let's see here on the left side are the values of our panel series. 08:10.350 --> 08:19.360 So we have passengers who are 24 years old and 22 year old and 25 years is actually the most frequent. 08:19.510 --> 08:21.410 Value in our panel series. 08:21.410 --> 08:26.600 So there were actually 30 passengers on the Titanic that were 24 years old. 08:26.620 --> 08:31.540 And and there were 27 passengers and that were 22 years old. 08:31.540 --> 08:36.750 And we can scroll down here and in the end we can see that. 08:36.800 --> 08:38.710 There's only one passenger who is uh. 08:38.800 --> 08:44.090 Point 4 2 years old and uh for example uh there's only one passenger service. 08:44.180 --> 08:52.070 Seventy 74 years and also value counts by default ignores the missing or any end well. 08:52.080 --> 08:56.480 So actually you're the default parameter set to true. 08:56.970 --> 09:01.170 But we can actually change this to Ford's 09:05.930 --> 09:10.590 and then we can see we have from the value counts including missing values and at the top. 09:10.590 --> 09:18.330 We have here one hundred seventy seven missing values and you can also see that by default value counts. 09:18.340 --> 09:22.460 Returns um a new series with a slotted values. 09:22.490 --> 09:25.990 So the most frequent value is at the top end. 09:26.020 --> 09:30.940 The less frequent values are at the bottom here of our new panel areas. 09:30.970 --> 09:32.890 And we can also change for this year. 09:32.890 --> 09:39.280 And that is a parameter quite smart and we can set this to false. 09:39.370 --> 09:42.060 So by default these sort parameter is true. 09:43.030 --> 09:48.470 And let's see what we get here. 09:48.590 --> 09:54.950 We got actually here the unsorted uh value count serious starting from uh 22 years old. 09:54.950 --> 09:58.270 So this is actually the first value in our H. 09:58.280 --> 09:58.990 Serious. 09:59.000 --> 10:03.040 And 27 percent just 22 years old. 10:03.110 --> 10:06.640 So but it definitely makes sense to authority our aside. 10:06.740 --> 10:10.340 So let's again change it to true 10:13.110 --> 10:19.200 and you can see here that we have a descending order so starting with the most frequent number of value 10:20.220 --> 10:28.170 and we can also change this by setting the ascending parameter which is by default to set to false and 10:28.170 --> 10:31.780 we can set this to true. 10:31.950 --> 10:39.170 Now we have an ascending order starting with the less frequent values of our H serious so in the most 10:39.170 --> 10:50.250 cases for our analyzing purposes it makes more sense to have a look at the most frequent items here. 10:50.500 --> 10:50.960 All right. 10:50.950 --> 10:54.690 And that's a not the parameter quite normalized. 10:54.700 --> 11:01.630 So here we effort the absolute numbers are amounts our counts of our values and we can normalize this. 11:01.680 --> 11:08.190 So sample here and 30 percent just at 24 years old and we can normalize 30. 11:08.200 --> 11:11.220 So we can divide this by the total number of passengers. 11:11.230 --> 11:13.630 It's the net seven a look here 11:16.290 --> 11:20.650 we have actually 714 peasant us where we know the H 11:23.900 --> 11:32.790 and by setting here the normalized parameter to true we should get here 30 divided by 714 so let's check 11:32.790 --> 11:36.230 it here. 11:36.250 --> 11:40.610 So now you do not get the absolute quantities but the relative quantities. 11:40.620 --> 11:49.750 So still 24 year olds and the most frequent value and actually 5.2 percent of all passengers were 24 11:49.750 --> 11:50.700 years old. 11:51.010 --> 11:56.140 And because we have to drop an equal to the missing values were excluded for this calculation. 11:56.140 --> 11:58.380 But we can change this also. 11:58.450 --> 11:59.900 And last but not least. 12:00.000 --> 12:07.140 There's another parameter for our value counts uh method and this has been slow let's have a look here 12:07.150 --> 12:08.290 press shift tab 12:11.240 --> 12:18.500 and what the pins does them rather than count values that groups them into half open bins and let's 12:18.500 --> 12:19.700 see what we get here 12:22.990 --> 12:24.600 so we get to your five bins. 12:24.630 --> 12:32.620 Um yeah actually with equal lengths so starting from 0 point fifty nine and ten sixteen years and ending 12:32.620 --> 12:37.220 with them the oldest pin from 64 years to eighty years. 12:37.240 --> 12:46.330 And for each pin we get the amount of values that are in our H a serious so we have eleven passengers 12:46.540 --> 12:54.650 that are between 64 and 80 years old and the most frequent pin here is between 16 and 32 years old. 12:54.660 --> 12:59.670 So 346 passengers are between 16 and 32. 12:59.710 --> 13:05.740 All right so now we have finished the analysis of our numerical serious and in the next session we will 13:05.740 --> 13:09.490 have to look at the non numerical serious so hope to see there by.