Average sections of a timetable

I have a data timetable. One of the variables is a flag that determines sample time and background time. For the purposes of my data processing, I want to average the sample times to 1 minute intervals.

Currently, I extract the rows with filterstate == 0 and use retime, then I concatenate the timetables and sort the rows in time. This creates an artifact where the first sample time will be rounded to the previous minute and creates a data processing issue.

Is there a better way to do this, or could I specify in retime to round to the next minute?

load example_tt.mat

% EDIT to show raw data
example_tt(6612:6635,:)

ans = 24×4 timetable
           Time                           Signal                                  power                    filter state              Tau           
    ___________________    ____________________________________    ____________________________________    ____________    ________________________

    2022-08-13 22:35:53    11.236    4.9098    3.1298    3.4968    591.27    613.06    454.61       699         1          6.9647e-05    2.7618e-05
    2022-08-13 22:35:54    10.923    4.1274    3.2449    2.7163    591.57    612.87    451.74    698.23         1          6.9801e-05    2.7604e-05
    2022-08-13 22:35:55    12.134    4.7342    2.9405    4.6952    591.98    609.79    451.75    696.94         1           6.988e-05    2.7639e-05
    2022-08-13 22:35:56    11.795    5.1227    2.5436    3.9818    591.49    602.53    452.69    695.75         1          6.9703e-05     2.764e-05
    2022-08-13 22:35:57    12.738    4.7315    2.4259    3.2908    591.95    595.63    453.57    695.02         1          6.9796e-05    2.7626e-05
    2022-08-13 22:35:58    12.309    5.6532    4.7778    3.8169    591.76    592.55    452.97    694.57         1          6.9668e-05    2.7635e-05
    2022-08-13 22:35:59    13.019    5.1435    3.7599    4.7426    591.61    600.95    452.89    693.76         1          6.9812e-05    2.7603e-05
    2022-08-13 22:36:00    12.539    4.9198    3.3075    3.9967    591.59     609.3    454.07     692.3         1          6.9839e-05      2.76e-05
    2022-08-13 22:36:01    11.001    5.2377    3.1239     3.392     592.7    609.82    451.41    691.07         1           6.982e-05    2.7604e-05
    2022-08-13 22:36:02      12.1    4.1433    3.8361     4.071    592.07    608.77    452.16    690.28         1          6.9765e-05    2.7612e-05
    2022-08-13 22:36:03    11.941    6.0223    3.9551    3.6893    592.15    602.95    451.41    689.82         1          6.9816e-05    2.7644e-05
    2022-08-13 22:36:04    12.381    4.9697    2.2283    4.6778    591.67    603.88    452.09    688.76         1          6.9754e-05    2.7607e-05
    2022-08-13 22:36:05    11.899    5.0105    3.7079    4.2653    591.85    609.52    452.29    688.46         1          6.9838e-05    2.7611e-05
    2022-08-13 22:36:06    12.787    5.2006    3.1608    3.8318    592.04    617.04    452.26    686.84         1          6.9734e-05     2.761e-05
    2022-08-13 22:36:07    11.274    6.1099    4.2194    3.2188    592.07    616.28    451.56     686.4         1          6.9806e-05    2.7615e-05
    2022-08-13 22:36:08    11.485    4.7791     3.541    4.2523    592.23    611.85    451.32    685.88         1          6.9813e-05    2.7648e-05

example_avg = retime_sample(example_tt);
example_avg(885:910,:)

ans = 26×4 timetable
           Time                           Signal                                  power                    filter state              Tau           
    ___________________    ____________________________________    ____________________________________    ____________    ________________________

    2022-08-13 22:35:53    11.236    4.9098    3.1298    3.4968    591.27    613.06    454.61       699         1          6.9647e-05    2.7618e-05
    2022-08-13 22:35:54    10.923    4.1274    3.2449    2.7163    591.57    612.87    451.74    698.23         1          6.9801e-05    2.7604e-05
    2022-08-13 22:35:55    12.134    4.7342    2.9405    4.6952    591.98    609.79    451.75    696.94         1           6.988e-05    2.7639e-05
    2022-08-13 22:35:56    11.795    5.1227    2.5436    3.9818    591.49    602.53    452.69    695.75         1          6.9703e-05     2.764e-05
    2022-08-13 22:35:57    12.738    4.7315    2.4259    3.2908    591.95    595.63    453.57    695.02         1          6.9796e-05    2.7626e-05
    2022-08-13 22:35:58    12.309    5.6532    4.7778    3.8169    591.76    592.55    452.97    694.57         1          6.9668e-05    2.7635e-05
    2022-08-13 22:35:59    13.019    5.1435    3.7599    4.7426    591.61    600.95    452.89    693.76         1          6.9812e-05    2.7603e-05
    2022-08-13 22:36:00    12.245    5.8375    3.4445    4.2451    590.33    603.19    455.57    706.19         0          6.9969e-05    2.6849e-05
    2022-08-13 22:36:00    12.539    4.9198    3.3075    3.9967    591.59     609.3    454.07     692.3         1          6.9839e-05      2.76e-05
    2022-08-13 22:36:01    11.001    5.2377    3.1239     3.392     592.7    609.82    451.41    691.07         1           6.982e-05    2.7604e-05
    2022-08-13 22:36:02      12.1    4.1433    3.8361     4.071    592.07    608.77    452.16    690.28         1          6.9765e-05    2.7612e-05
    2022-08-13 22:36:03    11.941    6.0223    3.9551    3.6893    592.15    602.95    451.41    689.82         1          6.9816e-05    2.7644e-05
    2022-08-13 22:36:04    12.381    4.9697    2.2283    4.6778    591.67    603.88    452.09    688.76         1          6.9754e-05    2.7607e-05
    2022-08-13 22:36:05    11.899    5.0105    3.7079    4.2653    591.85    609.52    452.29    688.46         1          6.9838e-05    2.7611e-05
    2022-08-13 22:36:06    12.787    5.2006    3.1608    3.8318    592.04    617.04    452.26    686.84         1          6.9734e-05     2.761e-05
    2022-08-13 22:36:07    11.274    6.1099    4.2194    3.2188    592.07    616.28    451.56     686.4         1          6.9806e-05    2.7615e-05

function [raw_TT_retimed] = retime_sample(raw_TT)
% Take a raw data timetable and convert the sample times to 1 minute
% averages

sample_times = raw_TT(raw_TT.('filter state') == 0,:);
sample_times = retime(sample_times,"regular",'mean','TimeStep',minutes(1));
raw_TT_retimed = [sample_times;raw_TT(raw_TT.('filter state')==1,:)];
raw_TT_retimed = sortrows(raw_TT_retimed,'Time');

end

[Cris LaPierre]

How can you be sure that won't introduce the same data processing issue into your data?

[Poison Idea fan]

The artifact is still present, but the erroneous point ends up being added to the beginning of each filter time. When I select the background time, that point does not appear in the time range I need for background.

Answer 1

[Poison Idea fan]

Use 'IncludedEdge','right'