how to take out specific month from data

sir i have 95 year daily model output rainfall data form 2006-1-1 to 2100-12-31.i want to take out nov to feb which include leap years. i have program for this.this is working perfectly for who does not include leap year but i want to include leap years then please suggest me what will change in this program for this. in between 2006 to 2100 have 24 leap years but my data include only 23 leap years,this is not not include end(last) leap year. this is daily data.sir my precipitaion data size is ( 20x22x34698) (lat,lon.time) for india region.sir please help me. this is include leap years.please help me.thank you

dten = datenum([2006 1 1 ;2100 12 31])

[~,M] = datevec(dten(1):dten(2));

xx = find(ismember(M,[11 12 1 2]));

[m,n,k] = size(prec_all_years1);

ii = cumsum(eomday(2014,1:12));

x0 = rem(0:k-1,365)+1;

xx_logic = x0 <= ii(2) | x0 > ii(10);

p = prec_all_years1(:,:,xx_logic);

max_rainfall = max(reshape(p,m*n,[]));

[Azzi Abdelmalek]

Can you clarify what exactly is your problem? What problems do you have with leap years?

[ravi]

Azzi Abdelmalek sir i want to take out nov to feb month from the data. my data size is 12x16x34675 (lat,lon.time) for india region who include leap years but this program only for those data whos does not include leap years please suggest me what will change in this program for take out nov to feb month from the data

[dpb]

What do you mean by "take out", precisely? You mean you want to remove (or selectively keep) those months' data and ignore the rest? Your M variable and ismember will find those. If you're looking to find the leap years and either keep or discard only them, then do the same kind of thing keeping Y as well and build the logical vector of which year is a leap year for the addressing. That's easily done by

isLpYr=(eomday(Y,2)==29);  % logical vector T for Y==leap year

[ravi]

it's mean i want to make another data set for nov to feb from 12 month data (12x16x34675 (lat,lon.time) for india region who include leap years).i want to cut only nov to feb my data set but this program is right only for those who does not include leap year then suggest me what will change in this program for my requirement.please help me,thank you

Answer 1

[dpb]

As noted above, you were already there--

dten = datenum([2006 1 1 ;2100 12 31]);   
[~,M] = datevec(dten(1):dten(2));
p = prec_all_years1(:,:,ismember(M,[11 12 1 2]));

IOW, just throw away all the leap year stuff if don't want it. But if want to go back, the preceding logical vector is much simpler way to select/not select those years than the initial logic.

[Y,M] = datevec(dten(1):dten(2));
p = prec_all_years1(:,:,ismember(M,[11 12 1 2]) & ~(eomday(Y,2)==29));

will select Nov-Feb for the non-leap years in the range of years Y

Look up Using Logicals in Array Indexing in the documentation for more details.

[ravi]

i can not understand.what are you saying? can you give me complete program for this?please help me.thank you.

[ravi]

sir this is not working please help me.please give me all lines of program after changing above program.

[dpb]

A) It is, after all, your program/work/job/research so it shouldn't be too much to expect you put in some effort to debug/fix any donated suggestions, and

B) What does "is not working" mean, specifically? I can't see your terminal from here.

One observation -- 95*365 --> 34675

whereas

length(datevec(dten(1):dten(2))) --> 34698

It thus appears that your model doesn't know anything about leap years? Is this the problem that your resulting vectors aren't the commensurate length? If so, either fix the calendar in the model to generate full time histories or remove the leap-year day from the time vector you've generated.

ADDENDUM:

Again on the assumption it's the above length mismatch that's the problem and that you'd rather use the data from the model as is rather than fix its broken time stamp, one way to proceed would be to eliminate the leap days from the M vector. The simplest way to do this means returning the day as well as month but don't really need the year altho I returned it anyway on general principles...

>> [Y,M,D] = datevec(dten(1):dten(2));  % create the date vector over period
>> ix29= (M==2) & (D==29);              % find Feb 29 locations
>> sum(ix29)    % just observe how many there are
ans =
  23
>> Y(ix29)=[];M(ix29)=[];D(ix29)=[];    % remove those from Y,M,D 
>> length(M)          % and observe resulting length is same as your data
ans =
     34675

Now just select for the months desired as before; the length of the logical vector will now match the number of planes in the data array. Since we've previously eliminated the leap days, now no longer any need for it in the selection logic

p = prec_all_years1(:,:,ismember(M,[11 12 1 2]));

[ravi]

sorry sir my observation is 95*365+24,(24 is the leap years) .95*365 --> 34675 observation does not include leap years.now please help me i don't want to remove these leap years.this is my question i want to include leap years.and want to cut only nov to feb data from my data(12x16x(95*365+24)).this is the daily data.

[ravi]

i don't want to remove these leap years.this is my question i want to include leap years.and want to cut only nov to feb data from my data(12x16x(95*365+24)).this is the daily data.

[dpb]

Your precise words have been "sir my precipitaion data size is 12x16x34675". That's not enough to include the leap day for those 24 years (of which as noted before, one is NOT a leap year). That would require as noted above an array size of 34698 planes.

The selection I gave earlier that you claimed "does not work" does either depending on what you do want but the vector sizes are not going to be commensurate unless the data are consistent.

You need to show explicitly what "doesn't work" including the error message in context of the code you tried.

It would be good if you also included the result of

size(prec_all_years1)
size(M)

for absolute confirmation of what we're dealing with. We need hard data/code to debug with; not descriptions as we can't see your terminal or know what you know inherently and the language barrier doesn't help so it's got to be something in a common unambiguous language and that's Matlab code and data.

ADDENDUM:

And, what do you mean by "cut"? Is that what you want to retrieve or not retrieve?

[ravi]

sir i am very confuse.i want to make nov to feb month data from my data( 20x22x34698) for analysis the extreme event.please help me.thank you

ADDENDUM:

dten = datenum([2006 1 1 ;2100 12 31]);

[Y,M] = datevec(dten(1):dten(2));

for i=1:95

p = prec_all_years(:,:,ismember(M,[11 12 1 2]) & (~eomday(Y,2)==29));

end

when run this program then result is

after whos commond

M 1x34698 277584 double

X1(lon)                  20x1                         160

Y                    1x34698                  277584

Y1(lat)                  22x1                         176  

ans                  1x14                         28  

dten                 2x1                          16  

i                    1x1                           8  double 

p                   20x22x0                        0  

prec_all_years      20x22x34698            122136960  double

[dpb]

OK, now we're getting somewhere...so, indeed, your raw data is 34698 in length...in that case the first response above is all you need. What you don't seem to recognize is the vectorization of operations in Matlab. The result of

ismember(M,[11 12 1 2])

is a logical array of the length( M) (commensurate with the number of planes in prec_all_years) which is T for the months in the selection vector. Logical addressing in Matlab will return the entries in the array for the dimension in which that vector is used based on whether the value is T or F; hence there's no need to loop over the year--that's what the vector does automagically.

By putting it in a loop, you've done the same thing 95 times but have also included the code to exclude the leap year which isn't needed since you do want those years as well. All you need is the one line

p = prec_all_years(:,:,ismember(M,[11 12 1 2]));

Note if I just use the time vector M at the command line--

sum(ismember(M,[11 12 1 2]))
ans =
     11423
>> 95*120+23
ans =
     11423
>>

the sum() of the result of ismember is precisely the number of days in 95 years of the months of [N D J F]=[30+31+31+28 -->120]*95+23. Again, remember that 2100 is not a leap year. The sum of the logical vector is the number of T elements in it so you've got exactly the right number selected.

Oh, and while you're not wanting to exclude the leap years after all, the reason you got no results from your selection is that I had a misplaced parens in the exclusion condition on the leap year flag--

(~eomday(Y,2)==29)  should have been ~(eomday(Y,2)==29)

to negate the overall test vector, not just the function value. As it was written, it returns no T values at all.

So, the exclusion condition if it were to be used would be

p = prec_all_years(:,:,ismember(M,[11 12 1 2]) & ~(eomday(Y,2)==29))

We can also test this easily enough at the command line --

>> sum(ismember(M,[11 12 1 2]) & ~(eomday(Y,2)==29))
ans =
      8640
>> (95-23)*120
ans =
      8640
>>

[ravi]

Thank you so much sir

[Mustapha Adamu]

Hello Sir, thanks for such a wonderful explanation. In my case my data is monthly data and I want to select specific months from it. for example I want to select data of october, november, december, Jan, Feb and then analyse. Please can you help me.

[Mustapha Adamu]

thanks

[dpb]

That's just changing the months to include October as well rather than beginning in November--

p = prec_all_years(:,:,ismember(M,[10:12 1 2]));

[Poulomi Ganguli]

Hello:

I came across the same issue. My data are arranged in a matrix containing year, month, day and value. If I run this code, I am encountering foll. error message:

The logical indices in position 3 contain a true value outside of the array bounds

[dpb]

Well, the above code is for retrieving from a 3D array, not 2D...you'll need to write the proper expression to match your different storage arrangement. But, can't see your terminal from here so have no definite knowledge to go on.

Answer 2

[dpb]

2100 is NOT a leap year; it's the "divided by 100" exception of the "divisible by four" rule. Then there's the 400 yr correction as well but we'll likely not be seeing it.

I am wondering, however, about how you've managed to accumulate daily precipitation data for years from 2015-2100 as yet...