Loading weird data and without extension

Hello

I'm trying to load that data I got, I'm attaching a few files as an example, it generally goes from 1882 to 2005. Files to up to and including 2002 have no extension, 2002-2005 are dat files. I've tried loading them but with very mixed results, and the older the data goes the more data is missing. I've been using data=importdata but it reads it as three columns (I believe that might be because the headers go Day Month Year).

I would like to import the data and then store it in some structure (I generally would like to extract them column by column, for example a column 13 for each year and each day and store it in a way that makes sense, but at this point I can't even import it).

That's info I got about it:

And I'm interested in data from rows 7 to 13 and columns 13 to 19.

Can anyone think of some way of doing that? I'd greatly appreciate any help.

Thank you

[OcDrive]

Hi Cris

Thank you for your answer, I have edited my original post, does it make more sense now?

[Cris LaPierre]

It looks like you have a fixed-width file. Have you tried using fixedWidthImportOptions?

[OcDrive]

I haven't, never heard of it. I will be able to try it in around 2 hours time. Thanks for the tip.

Answer 1

[Cris LaPierre]

It does help. So each variable has a width of 6

There are 36 columns of data, 16 rows for each day.

Something like this would work.

unzip('data.zip')

filename = "NH_MSLP_daily_1882";
c = linspace(-180,170,36)

c = 1×36

-180 -170 -160 -150 -140 -130 -120 -110 -100 -90 -80 -70 -60 -50 -40 -30 -20 -10 0 10 20 30 40 50 60 70 80 90 100 110

c = [c(c<0)+"W" c(c>0)+"E"];
r = linspace(90,15,16) + "N";

% Extract the date (assumes daily data)
yr = str2double(extract(filename,digitsPattern));

d = datetime(yr,1,1):days(1):datetime(yr,12,31);
D = repmat(d,length(r),1);
degree = repmat(r(:),length(d),1);

% Set import options
opts = fixedWidthImportOptions('NumVariables',length(c));
opts.VariableWidths = ones(1,length(c))*6;
opts.Whitespace = "-";
opts.MissingRule = "omitrow";
opts.VariableTypes = repmat({'double'},1,length(c));
opts.VariableNames = string(c);
opts.VariableNamingRule = "preserve";

% Read the data to a timetable. Times are added to the first column
T = readtimetable(filename,opts,'RowTimes',D(:));
T = addvars(T,degree,'Before',1)

T = 5840×37 timetable
       Time        degree    -180W    -170W    -160W    -150W    -140W    -130W    -120W    -110W      -100W          -90W          -80W          -70W          -60W          -50W          -40W          -30W          -20W          -10W          10E           20E           30E           40E           50E           60E           70E           80E           90E        100E     110E     120E     130E     140E     150E     160E     170E     ExtraVar1 
    ___________    ______    _____    _____    _____    _____    _____    _____    _____    _____    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    __________    _____    _____    _____    _____    _____    _____    _____    _____    __________

    01-Jan-1882    "90N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "85N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "80N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "75N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "70N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768    1.0248e+05    1.0166e+05    1.0083e+05    1.0053e+05    1.0024e+05         99950         99665         99628         99591    1.0049e+05    1.0138e+05    1.0125e+05    1.0111e+05    1.0064e+05    1.0017e+05         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "65N"     32768    32768    32768    32768    32768    32768    32768    32768         32768    1.0303e+05     1.022e+05    1.0148e+05    1.0064e+05         99669         99072         98808         99039         98545         99111     1.008e+05    1.0176e+05    1.0106e+05    1.0094e+05    1.0025e+05         99838         99823         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "60N"     32768    32768    32768    32768    32768    32768    32768    32768         32768    1.0276e+05    1.0183e+05    1.0096e+05    1.0012e+05         99269         98697         98869         99859         99515         99575    1.0111e+05    1.0116e+05    1.0084e+05    1.0084e+05    1.0037e+05         99985         99810         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "55N"     32768    32768    32768    32768    32768    32768    32768    32768    1.0306e+05    1.0275e+05    1.0186e+05    1.0086e+05         99850         99735         99913    1.0063e+05    1.0112e+05     1.006e+05    1.0074e+05    1.0158e+05    1.0175e+05     1.017e+05     1.009e+05    1.0063e+05    1.0096e+05    1.0017e+05    1.0034e+05    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "50N"     32768    32768    32768    32768    32768    32768    32768    32768     1.026e+05    1.0242e+05    1.0188e+05    1.0082e+05    1.0006e+05     1.004e+05    1.0124e+05    1.0194e+05    1.0205e+05     1.012e+05    1.0152e+05    1.0201e+05    1.0231e+05    1.0212e+05    1.0205e+05    1.0174e+05    1.0185e+05    1.0122e+05         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "45N"     32768    32768    32768    32768    32768    32768    32768    32768    1.0272e+05    1.0233e+05    1.0156e+05    1.0088e+05    1.0096e+05    1.0114e+05    1.0233e+05    1.0272e+05    1.0232e+05    1.0118e+05    1.0151e+05    1.0216e+05    1.0277e+05    1.0262e+05    1.0277e+05    1.0292e+05    1.0255e+05         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "40N"     32768    32768    32768    32768    32768    32768    32768    32768    1.0254e+05    1.0232e+05    1.0134e+05    1.0134e+05    1.0146e+05    1.0174e+05    1.0274e+05    1.0273e+05    1.0213e+05    1.0097e+05    1.0194e+05    1.0216e+05    1.0277e+05     1.028e+05    1.0321e+05    1.0346e+05         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "35N"     32768    32768    32768    32768    32768    32768    32768    32768    1.0265e+05    1.0219e+05     1.008e+05     1.015e+05    1.0164e+05    1.0244e+05    1.0284e+05    1.0282e+05    1.0194e+05    1.0147e+05      1.02e+05    1.0261e+05     1.024e+05     1.027e+05    1.0338e+05         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "30N"     32768    32768    32768    32768    32768    32768    32768    32768         32768    1.0166e+05     1.009e+05    1.0171e+05    1.0194e+05    1.0254e+05    1.0294e+05    1.0263e+05    1.0154e+05    1.0156e+05    1.0279e+05     1.028e+05    1.0261e+05    1.0285e+05         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "25N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768    1.0202e+05    1.0213e+05    1.0212e+05    1.0262e+05    1.0282e+05    1.0252e+05    1.0172e+05     1.017e+05         32768         32768         32768         32768         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "20N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768         32768    1.0168e+05    1.0189e+05     1.023e+05    1.0241e+05    1.0221e+05     1.017e+05     1.015e+05         32768         32768         32768         32768         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}
    01-Jan-1882    "15N"     32768    32768    32768    32768    32768    32768    32768    32768         32768         32768         32768    1.0146e+05    1.0158e+05      1.02e+05     1.019e+05    1.0179e+05    1.0148e+05         32768         32768         32768         32768         32768         32768         32768         32768         32768         32768    32768    32768    32768    32768    32768    32768    32768    32768    {'-32768'}