WEBVTT 00:01.180 --> 00:04.020 In this video we learn how to filter our data frames. 00:04.030 --> 00:10.850 And so far we're selected draws of a data frame only by index position or index labels. 00:10.870 --> 00:18.490 So for example we selected the first 20 rows or we selected all rows with the index labeled Michael 00:18.490 --> 00:21.820 Phelps and our Summer Olympics data set. 00:22.240 --> 00:28.330 And actually in the real world the more important cases to filter a data frame based on information 00:28.330 --> 00:29.950 in one or more columns. 00:29.950 --> 00:35.710 So for example in our Titanic dataset we want to filter all male passengers on the Titanic. 00:35.740 --> 00:43.180 All we want to filter up hasn't just them that are younger than 14 years old and vessel for before we 00:43.240 --> 00:49.480 start let's import penned us and we are working with the Titanic data set and we will have a look at 00:49.480 --> 00:52.930 the first 10 rows here. 00:53.160 --> 00:59.120 And what do you want to do now is that we want to filter out all rows where the passengers are made. 00:59.130 --> 01:05.970 So in the end we want to create a data frame with only passengers that are male and therefore we select 01:06.000 --> 01:14.450 our sex column and we will have a look here at the first ten elements in our sex column so he can see 01:14.450 --> 01:24.340 our sex column and now we want to check element bias each element of um the element is male or female. 01:24.340 --> 01:28.820 And this works is essentially the same way as some pie arrays. 01:28.890 --> 01:34.360 So let's use here the double equal sign and let's see what we get here. 01:37.380 --> 01:40.960 We get actually here a brand new Panda's serious service. 01:41.010 --> 01:44.690 Boolean values and pandas does this. 01:44.700 --> 01:48.590 Now back to Reisman also we do not need any for loops like this lists. 01:48.810 --> 01:50.970 So pandas checks here for each element. 01:50.970 --> 01:54.340 So let's start Chivas Element Zero. 01:54.430 --> 02:00.280 Here we are female and male is equal to male so we get it through here then we have three times female 02:00.290 --> 02:02.240 and then we get here three times a. 02:02.260 --> 02:05.580 And then again for truces. 02:05.590 --> 02:10.310 So now here we have a panda serious so that we can values for each peasant. 02:10.330 --> 02:12.970 Or if it's in a male or female. 02:13.000 --> 02:21.850 So from next positions that are to 890 here and now we can fit our data frame and we can pass here our 02:21.850 --> 02:29.950 panda seriously with boolean values and to spark brackets of the Titanic dataset and this works essentially 02:29.950 --> 02:31.640 the same as as an umpire race. 02:31.670 --> 02:37.900 So by passing of a boolean serious here within the step brackets we filter out all the rows of our the 02:37.960 --> 02:38.890 peasant just made it. 02:38.950 --> 02:40.070 So let's try here. 02:44.230 --> 02:50.910 And we get you a brand new data frame with only a male passengers and what Pena does here partner's 02:51.010 --> 02:54.750 checks the row buys if we have a true or false in a row. 02:54.780 --> 03:00.960 So for example let's go back here to our pandas serious and we can see here. 03:00.990 --> 03:03.260 So in row as 0 we have a true. 03:03.810 --> 03:12.600 And in our new made data frame we have also the row with the next position zero and in the next position 03:12.990 --> 03:19.550 1 2 and 3 we have a false son our new data frame we do not have the rows of us that the next position 03:19.560 --> 03:27.160 1 2 and 3 and then again for the next positions 4 5 6 7 we have a true. 03:27.170 --> 03:36.640 We have a male and now a new data frame we have here the rows 4 5 6 7 All right 03:40.050 --> 03:41.060 so let's copy here. 03:41.070 --> 03:41.730 Our code 03:44.400 --> 03:50.300 and what we can also do instead of having here the square brackets notation we can also use the local 03:50.300 --> 03:50.970 notation 03:53.680 --> 03:56.980 and filter for all rows where the sex columnist made it. 03:56.980 --> 03:59.170 So this works as well. 03:59.170 --> 04:05.000 So we have here the same result actually the same new data frame. 04:05.090 --> 04:10.820 And if you ask me which notation to use I would clearly recommend the low notation and that for two 04:10.820 --> 04:12.170 reasons. 04:12.170 --> 04:16.790 So first of all of us of the log notation we can also filter here not only for rows. 04:16.790 --> 04:19.480 So here we want to have all the rows that we have a male. 04:19.490 --> 04:25.100 And in addition we can also filter for columns so we can make a comma and then we can say OK I want 04:25.100 --> 04:31.990 to have only the fifth column so for all male passengers I want to know the fare. 04:32.010 --> 04:32.610 They paid 04:35.440 --> 04:40.100 and now here we have a panda serious them with all the fastened that remain. 04:40.150 --> 04:44.750 Just paid and with the prep records notation. 04:44.780 --> 04:45.930 You cannot do this. 04:45.980 --> 04:49.740 But then here once back bracket so let's go back. 04:49.820 --> 04:57.040 So he in this case if you want to a filter only select the fat column you have to add here and others 04:57.080 --> 05:02.780 back bracket and filter for the fat column so discuss them. 05:02.780 --> 05:04.880 Actually yeah obviously the same result. 05:05.540 --> 05:11.210 So the first advantage of the low notation is actually that you only have fewer once back bracket and 05:11.630 --> 05:15.050 the code is actually easier to read and it's more comfortable. 05:15.050 --> 05:21.240 And here you have to spell brackets and this is a so-called chained indexing. 05:21.290 --> 05:26.780 So what they are doing here you actually change index operations so first of all your filter the data 05:26.780 --> 05:36.120 frame for only made it's and then the second step you select only the fat column and change index thing 05:36.120 --> 05:41.780 is actually a very bad way to filter your data frame because it creates huge problems and confusion 05:41.780 --> 05:46.320 and comes to manipulating data frames so changing elements in the data frame. 05:46.650 --> 05:50.090 And we will see and check this late on our costs or change. 05:50.110 --> 05:52.640 Next thing is not a good idea. 05:52.840 --> 05:58.020 And if you are already familiar with pandas you might have learned filtering here with this pap records 05:58.030 --> 06:02.380 notation but this is a clear recommendation from my site. 06:02.530 --> 06:03.900 Please use here. 06:04.000 --> 06:05.680 The local station. 06:05.680 --> 06:11.020 So again as I said when it comes to manipulating data frames you will have less less problems. 06:11.120 --> 06:15.380 So the local station then here with this bad record station. 06:15.850 --> 06:16.330 All right. 06:16.330 --> 06:17.800 So now let's move on here 06:21.080 --> 06:25.520 and as a matter of taste we could also call our pandas serious. 06:25.610 --> 06:27.130 It's the boolean values here. 06:27.120 --> 06:29.600 Titanic dot sex equals male. 06:29.630 --> 06:32.220 We can also assigned a variable mosque. 06:32.300 --> 06:42.400 So here mosque one and again here we get our boolean serious and to increase the readability of our 06:42.400 --> 06:47.520 code we can now also pass the mosque one to our local operator. 06:47.530 --> 06:50.710 Yeah and filter our Titanic data set in this manner. 06:50.740 --> 06:53.600 So this gives us of a them the same result. 06:53.620 --> 06:57.790 But that's the a bit clearer and it improves their readability. 06:57.790 --> 07:03.390 So this is a better style than passing year the whole line year into with a snap records. 07:03.880 --> 07:05.830 But the result is the same of course. 07:08.230 --> 07:14.350 And what we also did here actually we saved our brand new data frame with only the male rose to the 07:14.350 --> 07:19.410 variable Titanic mail and we can now print and check our new data frame. 07:19.420 --> 07:26.410 We check the first five rows and we FCA our brand new Titanic made a data frame with them only made 07:26.410 --> 07:33.460 it past themselves and with the logo notation we not only can fill it up for specific rows but we can 07:33.460 --> 07:39.700 also fill the specific columns so we learned that before but we can do more than just filtering for 07:39.710 --> 07:46.480 example the fact column while the H column what we can do actually we can check here the data types 07:46.660 --> 07:55.150 of our columns with the attribute Titanic that D types and TV got them the data types of our columns. 07:55.150 --> 08:03.350 So in this survived and peasant a class column we have us in the eights and Fair column we have float 08:03.350 --> 08:10.990 objects and also V and these the best P and patriots in the top tax and in the SEC's embarked. 08:11.000 --> 08:15.910 And that column we have actually strings and now what. 08:15.910 --> 08:22.260 If we want to filter our data frame so we want to have all rows where we have a male MTV only want have 08:22.420 --> 08:30.390 the columns with them and numeric objects so we want to filter for the columns survive passenger class 08:30.440 --> 08:38.310 the best p part and fair and we do not want to have them the string columns the sex and barked and tech. 08:38.440 --> 08:46.900 So now we could check for each column if the t type is object and call it mosque to. 08:47.020 --> 08:52.110 So now we have here for the sixth column the embarked column and the column a true because here we have 08:52.110 --> 08:58.920 a string objects and then we can also filter out authentic data frame both antelope notation. 08:58.940 --> 09:04.740 So we want to f all rows and we want to f the opposite of mosque two of us. 09:04.780 --> 09:06.830 The symbol here. 09:06.830 --> 09:13.200 So if you would fill the form asks who you would get our columns with the String objects and the to 09:13.260 --> 09:18.080 add to this a movie get all the columns service they are actually numeric values here. 09:18.080 --> 09:19.330 So let's try this one. 09:22.760 --> 09:30.080 So here we get all the rows of our Titanic data frame with only in the columns here and we can further 09:30.080 --> 09:35.500 customize our filtering and pass mosque one and the opposite of Mosque two. 09:36.620 --> 09:42.940 And then we get here all rows we have we have a male passenger and we only get the columns that have 09:43.070 --> 09:43.930 numeric data. 09:43.960 --> 09:49.340 So we have survived the passenger class age the best P parts and fare. 09:49.890 --> 09:56.610 So give it to the local station we can not only filled up four rows but also four columns and we set 09:56.620 --> 09:58.100 this back brackets notation. 09:58.100 --> 10:05.340 This wouldn't be as easy as we have it here with the local notation All right so now we are finished. 10:05.340 --> 10:12.090 So the key message is when you want to fit that data frames based on a condition that is in column you 10:12.090 --> 10:14.270 should always use the local station. 10:14.300 --> 10:17.010 So that's here a clear recommendation from my side. 10:18.080 --> 10:20.090 All right so I'll see you in the next session by.