How do I save an array of strings preferably as a .mat file?

I am trying to pull all of the values from multiple text files and manipulate them. I would like to know what file a certain data point was pulled from. For the numbers I can read them to a variable and save as a .mat file, but the file name gives me this error when I use save to save the variable:

Error using save

Unable to write to MAT-file Y:\M1\sf.mat.

The file may be corrupt.

Error in Datapull_A (line 51)

save ('sf.mat','sf', '-v7.3')

function [directory] = Datapull_A(f1,f2)
%UNTITLED3 %works.  pulls file path for all files in raw data and puts them in a
%structure called files uses subdir (downloaded from matlab) stored on
%matlab computer.  
%A rev also compiles a list of file names and saves it so that the file
%name and time stamp can be pulled later.
clear files directory HPo USEo timeo Currento Ro file fid

files = struct2cell(subdir(f1));
%these files are huge.  files is an array of path and other info
files=files';
directory=size(files,1); %determines how many files

HP=[];R=[];USE=[];time=[];Current =[];sf=[];% initialize variables, loop size is unclear

tic %starts timer

for i=1:directory
   file=string(files(i,1));%identify i file path in list
   fid = fopen(file); %open i file in list
   out=textscan(fid,'%s  %s  %s  %s   %s  %s  %s  %s %s '); 
   % convert file to structure each cell is all of the data in one column in a tab deliniated data file
   fclose(fid);
   timeo = str2double(strrep(out{1,1}, ',', '.')); %moneyline replace comma (metric decimal point) with decimal convert to double
   timeo([1,2],:) = [];% remove first two lines they are data labels
   sfo=cell(size(timeo,1),1);
   sfo(:)={file};
   HPo = str2double(strrep(out{1,2}, ',', '.'));
   HPo([1,2],:) = [];
   USEo = str2double(strrep(out{1,4}, ',', '.'));
   USEo([1,2],:) = [];
   Ro = str2double(strrep(out{1,6}, ',', '.'));
   Ro([1,2],:) = [];
   Currento = str2double(strrep(out{1,8}, ',', '.'));
   Currento([1,2],:) = [];
   time=cat(1,time, timeo);%add this files data to the end of variable
   HP=cat(1,HP, HPo);
   USE=cat(1,USE, USEo);
   R=cat(1,R, Ro);
   Current=cat(1,Current, Currento);
   sf=cat(1,sf, sfo);
end
toc %display time elapsed

cd(f2)
save time.mat time -v7.3 %save data as .mat file
save HP.mat HP -v7.3
save USE.mat USE -v7.3
save R.mat R -v7.3
save Current.mat Current -v7.3
save ('sf.mat','sf', '-v7.3')

% data = [time HP USE R Current];% make array of all relivant data
% save data.mat data %save data as .mat file
toc %display time elapsed
end

[Stephen23]

Note that the clear at the start of your function serves absolutely no purpose: those variables only get defined later in the function, so clearing non-existent variables does nothing useful.

Well written code rarely needs to call clear anyway.

Also note that your code would be faster and more efficient if you avoid cd and use absolute/relative filenames when accessing the data files.

[Andy Thistle]

the for loop takes 12-15 hours to run, everything else totals to a few minutes. Do you think the absolute/relitive filenames would help in the for loop? Does this somehow impact fopen?

[Cris LaPierre]

Why are you using subdir? The most recent comment from the author says

"The dir function gained the ability to search subdirectories as of R2016b, at which point it also started returning the parent folder of each file it listed. Prior to that, it didn't return the folder, and didn't search recursively. So for newer releases, you're probably better off just using dir; this function fills the gap for earlier releases."

You list your version as 2018a, so why not just use dir?

[Andy Thistle]

I read that, tried dir, got an error and switched back to subdir. I didn't spend much time on it.

Answer 1

[Cris LaPierre]

As a first go, you have formatted the save command for sf differently than that of the other save commands. If the others are working and this one is not, what happens if you change it to match the formatting of the others?

...
save Current.mat Current -v7.3
save sf.mat sf -v7.3

[Andy Thistle]

save does not seem to be able to handle large arrays of strings. The doubles that make up the other variables work great. for reference I have data sets sizes of 10^9. sf is a string filename.

[Guillaume]

save has no problem handling large arrays of strings. What makes you think that it is the problem?

[Cris LaPierre]

I created a mockup to test. I was able to save reasonably large sets (10^5 rows) without issues, but my MATLAB locked up when I tried to save a variable with ~10^10 rows. I let it go for a couple hours without it ever completing.

At this point, I'd suggest reaching out to tech support via the Contact Us link and file a report. They will better be able to determine the source of the issue and identify potential solutions.

Answer 2

[madhan ravi]

https://www.mathworks.com/help/matlab/ref/save.html#btox10b-1-version

[Andy Thistle]

I had reviewed this, but I just tried using v7 instead of v7.3. It worked for a small fraction of the data set, but will likely end up double the size limit of a v7 .mat file. it also seemed computationally expensive. I think I can shorten the number of characters, but is there a way to just brute force this?