How to import specific data from .f06 file in MATLAB?

I want to write a code which imports specific data from the .f06 file to plot graphs.

[Walter Roberson]

Which data do you want to import?

[Omkar Samel]

Hi Walter. The first 4 columns of the displacement vector.

Answer 1

[Walter Roberson]

S = fileread('data.txt');
SS = regexprep(S, '^.*POINT ID.*?\n', '');
datacell = textscan(SS, '%f %s %f %f %*f %*f %*f %*f');
%the line after the last row happens to start with a number, and
%the %s is happy to eat the field after that, but then the %f for the
%third column fails to match. So the first two cells might be one
%entry longer than the third and fourth. Trim away excess
minlen = min(cellfun(@length, datacell));
datacell = cellfun(@(V) V(1:minlen, :), datacell, 'uniform', 0);
%everything is same length now
col1 = datacell{1};  %numeric
col2 = datacell{2};  %cell array of character vectors, probably single characters
col3 = datacell{3};  %numeric
col4 = datacell{4};  %numeric

[Walter Roberson]

Change the colon after the assignment to col1 into a semi-colon.

[Walter Roberson]

The fileread() reads the entire file in as one big character vector. The character vector will have embedded line terminators. Those terminators might be carriage return followed by newline if the file was created for compatibility with old PC programs, and will be just newline for Mac/Linux or PC use that is not stuck 30 years in the past.

The regexprep takes the character vector, searches starting from the beginning (^ means beginning) and looks for any number (*) of any character (.) followed by POINT ID . In other words it searched from the beginning of the file looking for the string POINT ID. Then the .*? means to look for any number (*) of any character (.) but use as few characters as possible (?) and then find the newline (\n) . In other words, after the POINT ID phrase, extend the search to the end of that line . So we know have a search range that started from the beginning of file, and extends to the end of the line that has POINT ID on it. And then the '' as the third parameter of regexprep means that that entire search range of characters is to be replaced in the string by nonthingness -- in other words, all of those characters are to be deleted. Probably we should add the option 'once' at the end of the regexprep() call.

So now what we have in SS is all of the input, but with everything from the beginning to the end of the header line that included POINT ID deleted. The beginning of SS is now where the columns of numbers start.

We then run that character vector through textscan(), telling it to look for a number (%f) followed by a string (%s) followed by two numbers (%f %f), and then to expect four more numbers on the same line but to just pass through them and throw them away (%*f %*f %*f %*f) .

textscan() will keep trying to apply that format %f %s %f etc., until it encounters end of file or encounters a field that cannot match the format code it is supposed to be looking for. The line after the table of displacement values happens to start with 1 (a historical artifact that in that context codes advancing to the next printer page), followed by a trailer string . The %f will accidentally read in the 1 and the %s will accidentally read in the trailer string because the inputs happen to be valid to match those format codes, but the bit after that in the input is another string and that other string does not match %f numeric format, so the textscan stops. textscan() returns one cell array entry for each % format that is not surpressed with *, so the %f %s %f %f tells it to return a cell array with four entries.

Because of the accidental matches after the table, the first two cell array entries have one more row than the other two cell array entries. The cellfun code there checks what the shortest row of input was, and truncates all of the rows down to that size. In the case of your input data, that takes the pattern 28 28 27 27 down to 27 27 27 27 in length.

The unwanted spare data now having been rejected, you can extract the individual parts. If all of the columns had been numeric I probably would have used cell2mat() to create a simple numeric array, but the second column happens to be character so you cannot just merge everything into numeric form.

[Omkar Samel]

Thank you Walter for all the help!

[Omkar Samel]

Hi. So I tried this to read a different file. But the problem is that the new file has multiple lines which start with "POINT ID". This script returns the last part that starts with POINT ID. Is there a way to select which lines I want in SS. Also, here I need data from two point id lines. I have attached the file to make it easier to understand what I am talking about.

[Walter Roberson]

Do you just want the displacement vectors, or the load vectors as well?

Do you want all of the displacement vectors merged together in one array, or do you want them separated out in blocks (like they are in the file) ?

[Omkar Samel]

Just the displacements merged together. I deleted the load vectors and merged the displacement vectors together in the data file to make the code work. But I was wondering if there was a better way to do it. Also, is there a way to structure the script to read the file by line instead of point ID.

[Walter Roberson]

S = regexp(fileread('data.txt'), '\r?\n', 'split');

Now S will be a cell array of character vectors, one entry per line of input.

Sometimes it can be worth processing in that form, even if just to figure out which entries you want to deal with. But textscan() cannot handle a cell array of character vectors so you would need to strjoin() then back into a block of text if you wanted to use textscan() to do the numeric work.

str2double() also cannot handle multiple numbers per cell array... and besides you have the non-numeric type data in the middle of the line.

str2num() can handle multiple numbers per character vector but str2num() does not permit a cell array of character vectors. str2num() is also pretty dangerous as it is happy to take any character string that appears and execute it as a function call.

sscanf() can handle multiple numbers per character vector, but it too does not permit a cell array of character vectors as input.

So... when you have blocks of inputs on separate lines, your choices reduce to

1. Use one single character vector as input to textscan(), even if this requires using strjoin() to put cell array of character vectors into one character vector
2. break up the cell array of character vectors into fields, ending up with a rectangular cell array of character vectors; str2double() can process that. But str2double() cannot process a cell array that contains cell arrays that contain character vectors, so the step about arranging all of the fields into a rectangular array is important
3. cellfun() textscan() or sscanf() invocations so each call is working on a separate character vector. This has a comparitively high overhead.
4. careful use of regexp() or regexprep() such as using named tokens, probably followed by some str2double() calls
5. or fopen() the file, process it line by line, making whatever decisions are appropriate. Sometimes fscanf() in cases where you either know how many lines you will be matching or else expect the pattern to go to end of file. Otherwise fgetl() to fetch one line, examine it to figure out what situation you are in, and potentially sscanf() to extract the values.
6. Or use other tools. Especially if you are on Mac or Linux, sometimes invoking external tools such as sed can be really worthwhile, even if just to preprocess the file.

[Omkar Samel]

Thanks for the explanation. I'll work on this tomorrow and get back to you with my results. Only getting the 3rd and 4th column works for me as well if it makes the coding easy. I have a windowns pc in regards to your last suggestion. Thank you again for the all the help. This makes my work very easy and I also get to learn more about MATLAB.