WEBVTT 00:00.930 --> 00:06.240 This video is a short summary of what we have learned in the section and also we learned somewhat advanced 00:06.330 --> 00:10.210 indexing techniques with the re index method. 00:10.440 --> 00:17.180 So we import pandas and we are working with our temperature data set and we can actually create a daytime 00:17.220 --> 00:25.160 index by passing the column daytime time to the apparently does pass dates and index columns. 00:25.290 --> 00:31.470 So here we have the first five rows and the last five rows and on the left hand side we can see our 00:31.470 --> 00:35.940 daytime index with our Li timestamps. 00:36.300 --> 00:42.600 And also we have learned that we can change the frequency of our data so we can change from our lead 00:42.630 --> 00:43.860 to daily. 00:44.310 --> 00:49.230 And this is called down sampling and we can do this with the sample method. 00:49.590 --> 00:57.420 So we posture daily and we actually calculate daily means and then we store the new data frame in the 00:57.420 --> 01:01.820 available temperature daily. 01:01.850 --> 01:09.140 So now here we are on the left hand side the daily timestamps and we have also learned that we can index 01:09.140 --> 01:16.130 lost lives our data frame for specific time stamps and let us assume that we want to know the weather 01:16.130 --> 01:18.190 in New York and Los Angeles. 01:18.220 --> 01:27.230 Um during Christmas for the Yes to 14 to 15 and to 16 and we can actually create these uh three time 01:27.230 --> 01:31.650 stamps with the PD drop the date range method. 01:31.850 --> 01:36.590 So the end date should be the 24th of December to 16. 01:36.620 --> 01:43.280 We want to have three periods and we want to have a frequency of one year but we want to have exactly 01:43.310 --> 01:46.580 the 24th of December for each year. 01:46.640 --> 01:55.300 So here we have to pass the PD up the date offset method and the frequency is one year. 01:55.430 --> 02:00.160 And by doing so they are getting a daytime index with the sweet time stamps. 02:00.170 --> 02:08.810 So we have Christmas to 14 to 15 and to 16 and now we can actually filter our temperature daily data 02:08.810 --> 02:16.660 frame for the day time index here off father three time stamps and we can do this by passing the our 02:16.660 --> 02:18.810 date time index. 02:19.040 --> 02:26.360 So we started our day time index and the variable birthday and we can pass here but stay within the 02:26.360 --> 02:27.800 square brackets of the low. 02:27.800 --> 02:28.580 Operator 02:31.320 --> 02:40.080 And by doing so we are selecting the three days as three time stamps the 24th of December to 14 to 15 02:40.080 --> 02:47.820 and to 16 and obviously Christmas to 15 was in New York a quiet Christmas. 02:47.830 --> 02:56.230 So we had here on ever 17 degrees and actually in this case the special case was that all time stamp 02:56.240 --> 03:03.670 C on our day time index birthday well also represented here in our data frame temperature daily. 03:04.180 --> 03:08.300 So we FBI daily data from 232 to sixteen. 03:08.440 --> 03:12.510 So Christmas to 14 to 15 and to 60 not included. 03:12.730 --> 03:22.430 But let's assume now that you want to have the weather data for the last 10 years starting in 2 18 and 03:22.430 --> 03:31.600 you set periods to 10 so you have now of a new day time index birthday with the last 10 Christmases. 03:31.610 --> 03:39.790 So starting from the 24th of December 2009 onto the 24th of December 2009 18. 03:39.830 --> 03:47.570 And obviously not all time stamps here in the daytime Index are also included in our data frame temperature 03:47.570 --> 03:48.140 daily. 03:48.620 --> 03:56.850 But let's try to pass you all saw this the daytime and next year to the local operator and first of 03:56.850 --> 04:00.770 all we are getting here a warning message but let's have a look here. 04:00.780 --> 04:07.230 So far the time stamps that were also included in our data frame we get to hear the bell use. 04:07.230 --> 04:14.610 So for example here the hot Christmas in New York into 15 and for all of the other time stamps that 04:14.610 --> 04:20.430 are actually included here on our birthday time in the daytime and next but that are not included in 04:20.430 --> 04:24.170 the data frame we get here missing values. 04:24.210 --> 04:31.390 So a passing list likes to the lock operate with any missing label that race a key area on the future. 04:32.100 --> 04:38.910 And as an alternative we should use the re index method and let's try this out here. 04:38.910 --> 04:43.800 So we use the re index method on our data frame and let's have a look here 04:46.760 --> 04:53.120 and it says here the reason next method can form a data frame to a new index with the option of filling 04:53.120 --> 04:58.120 logic so we can also define what to do with the missing values. 04:58.150 --> 05:03.460 But first of all it's simply running near the center and the other is light. 05:03.470 --> 05:08.240 Actually it looks the same on a site tentacle as you have with the loke operator. 05:08.960 --> 05:11.100 So you for the far yes. 05:11.110 --> 05:11.350 Yeah. 05:11.360 --> 05:15.590 We have data and for the other yes we have missing values. 05:15.590 --> 05:17.600 But here it's a very indexed method. 05:17.600 --> 05:21.920 We can also take cap out to the missing value so let's have a look here. 05:22.310 --> 05:24.130 And here we have the pyramid method. 05:24.140 --> 05:28.040 And by default it's set to non and it's SCA. 05:28.060 --> 05:30.390 This is the method to use fulfilling holds. 05:30.430 --> 05:36.570 In re index data frame and we can use either numbers so we don't fill the gaps. 05:36.580 --> 05:42.970 We can use format data and by doing so actually using the last valid observation to fill the missing 05:42.970 --> 05:51.430 values and we can also use the backfill be and TV use of the next valid observation to fill the gap. 05:51.580 --> 05:52.930 We can use the nearest. 05:53.170 --> 05:57.640 And by doing so we are using actually the nearest developed observation to fill the gaps. 05:57.670 --> 06:02.140 So first of all let's call yet the method per meter. 06:02.920 --> 06:04.440 And that's possible. 06:04.450 --> 06:13.090 For example beef the fallback filler and TV can see four Christmases before to 13 so far to twelve eleven 06:13.090 --> 06:14.510 ten nine. 06:14.770 --> 06:17.360 We get to your values and the. 06:17.550 --> 06:22.930 Let's also look here at the first five rows of our total data frame. 06:23.290 --> 06:27.450 So the last valid observation is see at the very first observation that. 06:27.500 --> 06:29.340 January the 1st. 06:29.350 --> 06:32.530 And by setting the method parameter to backward. 06:32.870 --> 06:40.060 We are actually filling order and a valid use before the first of 10 area to 13 with the various in 06:40.060 --> 06:41.660 10 year to 13. 06:41.680 --> 06:46.160 So for L.A. We have eight point eight and for New York minus 0 point four. 06:46.750 --> 06:56.790 And let's try also here for about four years so f fill and we can see here for all missing values after 06:56.790 --> 07:00.320 two sixteen we get our values now. 07:00.550 --> 07:07.480 And let's also have a look at the last five rows of our data frame and we can see here actually that 07:07.540 --> 07:14.320 the missing values are food that's actually the very last the valid observation and this is here on 07:14.320 --> 07:22.960 the thirty first of December to 16 13 degrees for an A and one point two degrees for New York and last 07:22.960 --> 07:27.420 but not least we can also select your nearest as some method 07:31.470 --> 07:36.890 and TV Guide actually for all missing value is the nearest the valid observation. 07:36.890 --> 07:44.090 So for example here for missing value is before 213 we get here hear the values that January the 1st 07:44.090 --> 07:53.590 and 13 and for the timestamps after to 16 we get to the values of the very last timestamp into 16. 07:53.590 --> 08:00.770 So here on the thirty first of December and actually in this case serve as a yearly data also filling 08:00.770 --> 08:01.920 missing values. 08:02.000 --> 08:06.470 This data that is one year ago one year ahead does not really make sense. 08:06.460 --> 08:12.260 So here in this case you would actually a leaf here that mess up the prior perimeter as it is. 08:12.290 --> 08:20.370 So we could show to fill the missing values but later in this course we will see that if uh by using 08:20.370 --> 08:27.310 near the re index method if we are creating here some missing values in between and uh the increments 08:27.310 --> 08:33.710 that only hours or minutes then it definitely makes sense to fill the missing values either with that 08:33.790 --> 08:36.060 beef or forward. 08:37.070 --> 08:42.300 But in this case the best solution is to leave the missing values as they are. 08:42.680 --> 08:47.960 All right with this we are finished now and I hope to see you also in the next section by.