WEBVTT 00:00.790 --> 00:05.990 And this video we are going to assemble time series data with the read sample method. 00:06.130 --> 00:12.550 Slow up sampling is the opposite of down sampling it means that we increase the frequency in our data 00:12.880 --> 00:19.600 and decrease the time increments between timestamps and the one simple example would be converting data 00:19.600 --> 00:23.500 from daily to hourly data typically wind up sampling data. 00:23.530 --> 00:29.870 We create missing values and dealing with these missing values is actually the major challenge. 00:29.890 --> 00:36.980 So first of all it's important to us and we are still working with our stocks that are set and actually 00:36.980 --> 00:44.250 here on the left hand side we have daily data but we only have trading on business days so there are 00:44.260 --> 00:48.970 no Saturday Sundays and back holidays and the index. 00:48.970 --> 00:55.730 So essentially we have fewer business days and converting business days to calendar days. 00:55.750 --> 01:03.490 This is already up sampling because we increase the amount of time stamps so we add Saturdays and Sundays 01:03.520 --> 01:11.320 and also bank holidays and we can simply up sample our dataset with the free sample method and we can 01:11.320 --> 01:13.320 pass these for daily. 01:13.510 --> 01:17.470 And by doing so we create a re sample object and to be safe. 01:17.470 --> 01:24.570 The reason people object in the web to read sample object so let's have a look here so that's a day 01:24.570 --> 01:25.540 time index. 01:25.540 --> 01:26.730 The re sampler. 01:27.270 --> 01:31.070 And actually when we don't sample data we have to chain an aggregation method. 01:31.110 --> 01:39.060 But the question is here now with aggregation a functional method we can use for up sampling and actually 01:39.060 --> 01:42.150 we can you see the s the freak method. 01:42.270 --> 01:43.320 So let's have a look here. 01:44.460 --> 01:48.930 And the s freq method actually returns that the values of the new frequency. 01:49.380 --> 01:54.030 And essentially it works like um the re index method. 01:54.270 --> 01:55.370 So let's have a look here. 01:58.090 --> 02:01.620 And now we have your daily data in terms of calendar days. 02:01.630 --> 02:07.040 So we also have here the first of January which is a bank holiday. 02:07.090 --> 02:10.210 Then we have the second of January a Saturday. 02:10.210 --> 02:16.510 And also the third of January a Sunday and that's no trading took place on these days. 02:16.510 --> 02:22.040 We have missing values since that off stock prices and the Christmas. 02:22.040 --> 02:25.420 Now how to deal with these missing values and. 02:25.520 --> 02:29.310 For stocks typically we use forward failing. 02:29.360 --> 02:36.110 So actually the best price for the bank holiday and for the weekends is the the immediately our most 02:36.290 --> 02:44.000 recently price here on the thirty first of December and therefore we simply sought to replace missing 02:44.000 --> 02:47.790 values by the immediately preceding price. 02:47.860 --> 02:57.410 And this is actually a forward filling and we can do this with uh the math that f Phil and AFL actually 02:57.410 --> 03:06.570 forward fits the values so let's run the sell here so we're actually on the first second and third the 03:06.570 --> 03:13.210 immediately preceding prices and actually that's also a second method. 03:13.240 --> 03:15.070 So be Phil or backfill. 03:15.640 --> 03:21.070 And typically in finance the more commonly used as the forward method. 03:21.400 --> 03:28.330 But there might be other cases where Beckford filling is more appropriate and we can do this with P 03:28.420 --> 03:37.180 Phil and here we have now on the first the second and third thirty point five seven and this is actually 03:37.180 --> 03:39.960 the trading price on the fourth of January. 03:39.970 --> 03:41.680 So let's have a look again. 03:41.710 --> 03:42.370 So here 03:45.950 --> 03:51.350 and actually for the forward film method and Deepak fought the film method there is one parameter and 03:51.380 --> 03:53.450 this is the limit parameter. 03:54.140 --> 03:55.240 So let's have a look here. 03:59.890 --> 04:02.430 And here we can actually limit of how many values to fill. 04:02.860 --> 04:06.390 And the default setting is that it is unlimited. 04:06.850 --> 04:14.430 But for example we can also pass to here to the limit parameter and let's have a look here so here we 04:14.440 --> 04:18.020 only backward for the two most recent timestamps here. 04:18.040 --> 04:20.380 The third and the second. 04:20.380 --> 04:24.640 And we still have missing values on the 1st of January. 04:24.640 --> 04:24.990 All right. 04:25.000 --> 04:32.260 Now let's go on and let's again have a look at the up sample the data without filling the missing values 04:32.260 --> 04:32.610 here. 04:35.860 --> 04:38.770 And that's actually also a third alternative. 04:38.770 --> 04:42.430 So we can also interpolate here the missing values. 04:42.430 --> 04:51.040 So we can interpolate between the immediately preceding trading price and the next trading price and 04:51.040 --> 04:54.460 we can actually do this with the interpolate method. 04:54.460 --> 04:56.440 And there are quite some options. 04:56.440 --> 05:00.850 And the the most easiest one would be to have a linear interpolation. 05:01.180 --> 05:04.050 So that's the default setting here. 05:04.060 --> 05:06.880 So here we have the method and we use linear 05:10.940 --> 05:17.930 so here we have the linear interpolation between the thirty first of December and the fourth with the 05:17.990 --> 05:23.330 incremental love 0 point 1 2 0 point eleven. 05:23.450 --> 05:26.330 And finally we can also further reduce the frequency. 05:26.360 --> 05:34.550 Let's assume that we want to f uh increments of 12 hours then we can pass a twelve hours to the read 05:34.550 --> 05:36.020 sample method. 05:36.020 --> 05:38.930 And again we can you see the s freq method 05:42.880 --> 05:49.150 so here we have additional information on the time and we have timestamps on midnight and actually noon 05:49.150 --> 05:50.570 twelve o'clock. 05:50.680 --> 05:53.800 And apparently here we have created many more missing values. 05:54.160 --> 05:57.190 So typically we have values at midnight. 05:57.220 --> 06:05.250 But we do have a missing value some 12:00 and it actually might make sense to use the interpolation. 06:05.330 --> 06:09.290 So thanks for watching and I'll see you also in the next video by.