How to sort and rearrange time series data into random groups?

Hi, I have 100 years time series data.

Here is one month data as example:

= = = = = = = = = = = = = = = = = = = = = = = = = = = = == = = = = = = = == = = = = = = = = = = = = = = = = = = = = = = =

Date temperature differrnce

01-Mar-2011 00:00:00 1.5764

02-Mar-2011 00:00:00 2.0028

03-Mar-2011 00:00:00 1.9728

04-Mar-2011 00:00:00 1.7604

05-Mar-2011 00:00:00 1.6378

06-Mar-2011 00:00:00 1.3025

07-Mar-2011 00:00:00 1.9193

08-Mar-2011 00:00:00 2.0134

09-Mar-2011 00:00:00 2.1462

10-Mar-2011 00:00:00 2.1556

11-Mar-2011 00:00:00 2.14

12-Mar-2011 00:00:00 1.9263

13-Mar-2011 00:00:00 2.9042

14-Mar-2011 00:00:00 2.8062

15-Mar-2011 00:00:00 1.6993

16-Mar-2011 00:00:00 2.3935

17-Mar-2011 00:00:00 1.9432

18-Mar-2011 00:00:00 1.9997

19-Mar-2011 00:00:00 2.0371

20-Mar-2011 00:00:00 1.632

21-Mar-2011 00:00:00 2.2297

22-Mar-2011 00:00:00 1.8848

23-Mar-2011 00:00:00 1.129

24-Mar-2011 00:00:00 1.5175

25-Mar-2011 00:00:00 1.7745

26-Mar-2011 00:00:00 1.5799

27-Mar-2011 00:00:00 1.8735

28-Mar-2011 00:00:00 2.3939

29-Mar-2011 00:00:00 1.9723

30-Mar-2011 00:00:00 1.3674

31-Mar-2011 00:00:00 1.1814

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

How to rearrange this data into random groups based on date and time?

For example:

Group1: 01-Mar-2011 00:00:00 to 20-Mar-2011 00:00:00

Group2: 06-Mar-2011 00:00:00 to 14-Mar-2011 00:00:00

Thanks in advance.

[Walter Roberson]

Is the implication that there can be overlaps?

The groups are not fixed length? Is there a maximum and mininum length? But they are also continuous, no gaps right?

[Parthu P]

Thanks Walter,

Yes, the group will overlap and there is no fixed length and no gaps.

I have to sort and rearrange these data randomly (only based on date/time) from continuous 100 years daily time-series.

Answer 1

[Walter Roberson]

readtable(). If necessary, add the duration variable corresponding to the time, to the datetime variable for the date -- though in what you show us, the times are all 0:0:0 so you probably do not need to do that.

Now sort() the table based upon the time.

After that it is just a matter of using

sort(randi(height(TheTable),2,number_of_sequences))

to have the first row correspond to the row index in the table for the start, and the second row correspond to the row index in the table to stop at.

Note: this does not distribute sizes equally. Smaller sequences are favoured; the descrease looks to be pretty much linear as size of the sequence increases.

[Parthu P]

Thanks very much Walter,

I tried this:

1. A = readtable('mydata') - worked well
2. B = sortrows(A, 'Date') - worked well

3. C = sort(randi(height(TheTable),2,01/03/2011-20/03/2011))

It gave:

C 2×0 empty double matrix

What shoul I change in step 3 to have correct values?

Thanks in advance.

[Walter Roberson]

Where I wrote number_of_sequences, that was the number of different random sequences you want to create, not an indication of which part of the table you want to select from.

timetable() objects can be indexed by date, with there being a helper function for time ranges: https://www.mathworks.com/help/matlab/ref/timerange.html

For you, instead,

number_of_sequences = 83;  %set as appropriate
mask = isbetween(A.Date, datetime(2011,1,3), datetime(2011,3,20) );
subtable = B(mask,:);
rand_subsets = sort(randi(height(subtable), 2, number_of_sequences));
rand_subsets = cell(number_of_sequences,1);
for K = 1 : number_of_sequences
    rand_subsets{K} = subtable(rand_subsets(1,K):rand_subsets(2,K), :);
end

[Parthu P]

Thank you very much.

It works well.