how to calculate the standard deviation for each month in monthly time series data of 46 years?

Hi,

i have a time series of monthly means for 46 years. its a univariate time series. i have shifted it to a timetable of size 557x1. since dat is from Jan 1971 - May 2017. it is attached here with.

i need to calculate the standard deviation for each month, using all the yeas in the data. my idea is to reshape the timetable so as to show years in rows and months in columns. so i tried to reshape this timetable to 46X12 sized timetable by using the following code, after wards, the resultant timetable 'A' woulld have 12 columns for each year. thus for each column of month , standard devviation may then be calculated.

but i am getting an error. please correct me, if i am wrong? or any better solution is also appreciated.

thanks in advance Sir.

monthlymeans_ObsData=retime(ObservedData,'monthly',@nanmean); 
x = monthlymeans_ObsData.timmendorf_time;
A = reshape(monthlymeans_ObsData,46,12);

 %% i get the following error 
%  Error using tabular/reshape (line 155)
% Undefined function 'reshape' for input arguments of type 'timetable'.
mean_y = nanmean(A);
A_std = nanstd(A);
plot(x, mean_y, 'b', x, mean_y + A_std, 'r',x, mean_y - A_std, 'g');

Answer 1

[Stephan]

Hi,

the following code reshapes your table the way you want it:

load('monthlymeans_ObsData.mat')
monthlymeans_ObsData.Properties.RowTimes.Format = 'MMM';

% reshape the data as needed
waterlevel = nan(564,1);
waterlevel(1:557) = monthlymeans_ObsData.timmendorf_waterlevel;
waterlevel = reshape(waterlevel,12,47);

% New table
T = splitvars(table(waterlevel));

% Variable Names for new table - columns
years = cell(1,47);
for y = 1:47
    years{y} = sprintf('Year_%d',1970+y);
end
T.Properties.VariableNames = years;

% Variable Names for new table - rows
months = cell(1,12);
month_names = string(monthlymeans_ObsData.Properties.RowTimes(1:12));
for m = 1:12
    months{m} = sprintf('%s',month_names(m));
end
T.Properties.RowNames = months;

% clean up
clear waterlevel y years months month_names m ans

With this result you can access all data for the months May and June over all years that way:

T_may_june = T({'May', 'Jun'},:);

which results in a new table. Calculate mean and standard deviation for all years for may and june and append those values as columns:

T_may_june.mean = mean(T_may_june{:,:},2,'omitnan');
T_may_june.std = std(T_may_june{:,:},1,2,'omitnan');

If you like an array more you can use:

may_june = T_may_june{:,:};

Best regards

Stephan

[bushra raza]

Hi,

Thanx a lot Mr. Stephan for your reply. logically it sounds great as i wanted to solve.

i am using Matlab R2017b.

i tried it, i got this message "Undefined function or variable 'splitvars'."

i searched splitvar, and save this splitvar function in my workspace. and then again execute your advised code, its giving me this message

"Error using splitvar

Too many output arguments. "

any idea Sir?

thanx once again for your time and guidance.

Best Regards,

BR

[Stephan]

Always a problem, if there is no Matlab Release provided... Im sure there is a smart solution. splitvars was introduced in R2018a. My solution is, that i will attach a .mat file containing the results as soon as i have access to my computer in a few hours...

Think about installing the newest release...

EDIT:

.mat-file is attached at my answer now. Also note the useful answer from Akira Agata.

Answer 2

[Akira Agata]

If your goal is to calculate the standard deviation for each month, you don't need to reshape the data to 46X12.

The following is one possible solution for R2017b.

load('monthlymeans_ObsData.mat');
[group,month] = findgroups(monthlymeans_ObsData.timmendorf_time.Month);
stdVal = splitapply(@std,monthlymeans_ObsData.timmendorf_waterlevel,group);

tblResult = table(month,stdVal);

FYI: If you can update to R2018a or later, you can utilize groupsummary function to do this task.

monthlymeans_ObsData.month = monthlymeans_ObsData.timmendorf_time.Month;
tblResult = groupsummary(monthlymeans_ObsData,'month','std');

[bushra raza]

Sorry for late reply,

Thank you so much both of you Sir Stephan, and Sir Agata.

your solutions helped me.

Best Regards,

BR