How to extract multiple matrices of different dimensions from a large matrix?

Hello everyone,

I'm having troubles trying to extract multiple smaller sized matrices from a huge matrix.

Situation:

1. I have a matrix of dimension matrix(110848,15) - everything numeric matrix(:,1) - the first row of the matrix contains a time-stamp (in miliseconds for 15 minutes). matrix(:,2:15) contain numeric values of something I measured.

2. I have another column vector which contains exact points in time (time-stamps) at which I want to break the above mentioned matrix: time(125,1). 125 is given as example.

time(1,1)=5;
time(2,1)=7.5;
time(3,1)=12.5;
time(4,1)=15;
time(5,1)=20;
time(6,1)=22.5;
time(7,1)=27.5;
time(8,1)=30;
time(9,1)=35;
time(10,1)=37.5;
time(11,1)=42.5;
time(12,1)=45;
etc.

This sequencing from 5 to 2.5 seconds goes on for the equivalent of 15 minutes.

Let's say I want to extract all data from matrix for the first 5 seconds which go from row1 (second=0) to row641 (second=5) in a different matrix, call it matrix_period1. So matrix_period1 will initially have 641 observations for 15 columns. As such, time(1,1) = 5 and matrix(641,1) = 5.

In matrix_period2, I want to have all observations from matrix(642:j,1) for which time(j=2,1) - hence for the second observation of time (7.5 seconds).

In matrix_period3, I want to have all observations from matrix(j:next,1) for which time(j=next,1) - hence for the third observation of time (12.5 seconds).

And so on... for all 8 matrices.

However, after going through all matrices for 8 periods, matrix_period1 will have to take values from matrix(j=last_period from matrix_period8,1) - hence the 9th observation of time.

The process repeats itself.

Does anyone have any idea on how to help me?

I can provide more information of course if I was not clear.

Thank you!

[Andrew Shum]

So let me see if I understand correctly. Your data is in one giant matrix. In a vector, you have times which indicate the end of each data set. You want to separate the data sets into groups such that a group contains every 8th data set.

[Mihai Toma]

Hello,

Sorry I've only now seen your reply.

Yes, my data is one giant matrix and I have a time column vector with points that indicate the end of each data set.

I'm not sure about your last sentence.

End goal: 8 matrices. Matrix1 = data measured from 0 to 5 seconds; (diff = 5) Matrix2 = data measured from 5 seconds to 7.5 seconds; (diff = 2.5) Matrix3 = data measured from 7.5 seconds to 12.5 seconds; (diff = 5) Matrix4 = data measured from 12.5 seconds to 15 seconds; (diff = 2.5) Matrix5 = data measured from 15 seconds to 20 seconds; (diff = 5) Matrix6 = data measured from 20 seconds to 22.5 seconds; (diff = 2.5) Matrix7 = data measured from 22.5 seconds to 27.5 seconds; (diff = 5) Matrix8 = data measured from 27.5 seconds to 30 seconds; (diff = 2.5)

However, after the 30th seconds, the data in matrix goes on towards second ~900 (equivalent of 15mins). Then, instead of matrix9, the data from 30-35 seconds will go to Matrix1, as:

Matrix1=data from 0-5 seconds and underneath data from 5 seconds the data from seconds 30-35; Matrix2=data from 5-7.5 seconds and underneath data from 5 seconds the data from seconds 35-37.5; Matrix3=data from 7.5-12.5 and then 37.5-42.5; Matrix4=data from 12.5-15 and then 42.5-45; Matrix5=data from 15-20 and then 45-50; Matrix6=data from 20-22.5 and then 50-52.5; Matrix7=data from 22.5-27.5 and then 52.5-57.5; Matrix8=data from 27.5-30 and then 57.5-60;

And so on... So these 8 matrices will have to be filled with data from the huge matrix (for time-series analysis purposes differentiated by groups/chunks of data).

I'm attaching a small sample code (which does not work, as I've been struggling with it in vain apparently).

 %time_seconds has size of (900,1) ~ 15mins
%matrix has size of (110848,1) ~ data measured during those 15 minutes (millisecond intervals)

i=1; for j=1:2 %length(time_seconds(:,1)) I'm just trying for the first 2 j from time_seconds

    for i=i:1:length(matrix(:,1))
        if matrix(i,1) == time_seconds(j,1)
           matrix_test = matrix(1:i,1);
           %matrice_per1 = matrix(1:i,1);
           %i2=size(matrice_per1,1);
           i=size(matrice_test,1);
           matrix_per1 = matrix_test(1:i,1);
           break;
        end
    end

    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    i2 = size(matrix_test,1)+1;
    i2_start = size(matrix_test,1)+1;

    for i2=i2:1:length(matrix(:,1))
        if matrix(i2,1) == time_seconds(j,1)
           i2_end = i2;
           matrix_test =matrix(i2_start:i2_end,1);
           %matrice_per2 = matrix(1:i2,1); 
           %i3=size(matrice_per2,1);
           matrix_per2 = matrix_test(i2_start:i2_end,1);
           break;
        end
    end
end

I think the issue is that after going through the 1st J of the first for, then it overwrites... I'm still working on it.

Thank you.

[Stephen23]

Identify the groups using division and rounding (or whatever other method that suits your needs), then use mat2cell or accumarray to split the matrix into matrices in a cell array. A search of this forum will find hundreds of threads discussing how to split a numeric matrix into smaller matrices, e.g.:

https://www.mathworks.com/matlabcentral/answers/?term=split+matrix&sort=updated+desc

Do NOT waste your time writing bad code that creates separate variables with numbered variable names: "A frequent use of the eval function is to create sets of variables such as A1, A2, ..., An, but this approach does not use the array processing power of MATLAB and is not recommended."

source: http://www.mathworks.com/help/matlab/matlab_prog/string-evaluation.html

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

Answer 1

[Andrei Bobrov]

[Small fix]

% Let A - your matrix [110848 x 15]

t = [0;time(1:8)];
t2 = seconds(milliseconds(A(:,1)));
ii = discretize(t2,t);
lo = ~isnan(ii);
Awork = A(lo,:);
g = ii(lo);
out = accumarray(g,(1:numel(g))',[],@(x){Awork(x,:)});

[Mihai Toma]

Thank you, it seems really interesting and elegant, however, I get the following error:

Error using discretize Too many output arguments.

I then change and write it: Y = discretize(t2,t);

But then I get this error: Error using discretize Expected input number 1, x, to be one of these types:

numeric, logical

Instead its type was duration.

Error in discretize (line 57) validateattributes(x, {'numeric','logical'}, {'real'}, funcname, 'x', 1)

I then change t and t2 to numerical and convert t to same unit (milliseconds). But I receive the following error:

Error using discretize Expected input number 2, edges, to be non-decreasing valued.

Error in discretize (line 58) validateattributes(edges, {'numeric','logical'},{'vector', 'real', ...

I will continue to investigate, thank you!

[Andrei Bobrov]

What version of MATLAB do you use and attach a piece of your data.

[Mihai Toma]

Hello Andrei,

Again I'm sorry for this rather late e-mail.

I'm using R2016a.

I'm attaching the time file and an example of the data (file is too big).

Thank you again.

[Andrei Bobrov]

for your datas

time1 = xlsread('time.xls');
T = readtable('matrix.xls');
t = [0;time1(1:8,1)];

ii = discretize(T.Time,t);
lo = ~isnan(ii);
Twork = T(lo,:);
g = ii(lo);
out = accumarray(g,(1:numel(g))',[],@(x){Twork(x,:)});

[Mihai Toma]

Thank you very much!

This worked!!!

I had to go around it a little bit to get all the data in (with readtable it doesnt get everything), but this is it.

Thank you, Andrei!

[Mihai Toma]

Hello Andrei,

Sorry to come back again.

The solution indeed worked and maybe I'm not this used to working with such large chunks of data in Matlab, but how can I convert the resulting T (for which each row is a cell of a numeric matrix) to those 8 matrices?

I'm trying something like:

matrix_1 = out{1,:}; matrix_2 = out{2,:}; matrix_3 = out{3,:}; matrix_4 = out{4,:}; matrix_5 = out{5,:}; matrix_6 = out{6,:}; matrix_7 = out{7,:}; matrix_8 = out{8,:};

for i=9:8:n matrix_1_final = cell2mat(out(i,:)); end

But then matrix_1_final is overwritten. I'm trying to figure out how to place the data in those 8 separate matrices. As in: matrix_1 will have data from points 9, 17, 25 etc.

matrix_2 will have data from 10, 18, 26 etc.

And so on.

Thank you!