Rearrange Data in text file

I have data in a text file and would like to rearange the data in a different sequencial order.

For instance I have:

Name,proj2;

Edition,59;

Start;

Variable,1;

DataReg,00.0,00.0;

Area,520.00;

300,221.7467,424.1668,801.0146;

390,117.4175,507.8583,29.2203;

Area,530.00;

300,488.6090,963.0885,488.8977;

390,578.5251,546.8057,624.0601;

Datareg,00.0,30.0;

Area,520.00;

300,367.4366,913.2868,335.3568;

390,987.9820,796.1839,679.7280;

Area,530.00;

300,106.7619,715.0371,698.7458;

390,653.7573,903.7206,197.8098;

DataReg,00.0,60.0;

Area,520.00;

300,291.9841,167.1684,489.6876;

390,431.6512,106.2163,339.4934;

Area,530.00;

300,522.6770,5147.8709,3201.4549;

390,7137.8581,9142.7370,7101.0988;

DataReg,30.0,00.0;

Area,520.00;

300,2121.7467,4214.1668,8201.0146;

390,1217.4175,5107.8583,219.2203;

Area,530.00;

300,4188.6090,9163.0885,4288.8977;

390,5178.5251,5146.8057,6224.0601;

Datareg,30.0,30.0;

Area,520.00;

300,3167.4366,9113.2868,3135.3568;

390,9287.9820,7296.1839,6279.7280;

Area,530.00;

300,1106.7619,1715.0371,1698.7458;

390,1653.7573,1903.7206,1197.8098;

DataReg,30.0,60.0;

Area,520.00;

300,2921.9841,1672.1684,2489.6876;

390,4321.6512,1206.2163,2339.4934;

Area,530.00;

300,522.6770,57.8709,3021.4549;

390,7327.8581,9422.7370,7021.0988;

DataReg,60.0,00.0;

Area,520.00;

300,316.4366,911.2868,313.3568;

390,92.9820,729.1839,627.7280;

Area,530.00;

300,110.7619,171.0371,169.7458;

390,165.7573,190.7206,119.8098;

DataReg,60.0,30.0;

Area,520.00;

300,292.9841,167.1684,248.6876;

390,432.6512,120.2163,233.4934;

Area,530.00;

300,52.6770,5.8709,302.4549;

390,732.8581,942.7370,702.0988;

DataReg,60.0,60.0;

Area,520.00;

300,7292.9841,7167.1684,7248.6876;

390,7432.6512,7120.2163,7233.4934;

Area,530.00;

300,752.6770,75.8709,7302.4549;

390,7732.8581,7942.7370,7702.0988;

(Sorry it's so long, just wanted to paint a vivid picture)

Notice how the lines that include "DataReg" change in the following sequencial order:

00.0,00.0;

00.0,30.0;

00.0,60.0;

30.0,00.0;

30.0,30.0;

30,0,60.0;

60.0,00.0;

60.0,30.0;

60.0,60.0

What I would like instead is for the data to be in the following sequencial order:

00.0,00.0;

30.0,00.0;

60.0,00.0;

00.0,30.0;

30.0,30.0;

60.0,30.0;

00.0,60.0;

30.0,60.0;

60.0,60.0;

Of course, keep the 6 lines of data below each "DataReg" line to stay with it's corresponding DataReg when rearanged. Also keep in mind the actual data set is much larger than the one provided. Include many descriptive comments in the script please. Thank you kindly!

[Ruger28]

Easiest way (in my opinion) is to read it in, sort how you want it, and re-write the entire file. I have messed around with EOF and such, and it can become a headache.

[Rik]

I agree. As long as the file will fit in your computer memory that is the easiest way. What have you tried so far yourself?

[Timbo]

So far I've tried to use regexp() to change the data around, but since each line of data is unique and there are so many lines in my .txt file, using regexp() is not really plausable.

[Rik]

I don't think you should be using regexp to split the file into the parts. It can of course be done, but I think it will be easier to parse the file line by line to group your data.

You can get my readfile function from the FEX. If you are using R2017a or later, you can also get it through the AddOn-manager. That will read your file to a cell array with each line in one cell (preserving empty lines). That should give you a start. Don't be afraid to use loops, Matlab is fairly good at optimizing code if it is easy to read and/or in a loop.

Answer 1

[Ruger28]

So you want to sort on "DataReg", then you can read the file in, search the lines for "DataReg", and then run your regexp on those lines (cells) only. Something like

fname = 'C:\Users\<username>\Documents\MATLAB\Sandbox\testfile.txt';
% read in data
MyData = importdata(fname);
% reshape it
ReshapedData = reshape(MyData(5:end),7,[]);
% find your values of DataReg
DataRegValues = regexp(ReshapedData(1,:),'\d\d.\d,\d\d.\d','match','once');
e = cellfun(@(x) strsplit(x,','),DataRegValues,'uni',0);
e = vertcat(e{:});
% Sort them based on second column
[~,SortedIDX] = sort(str2double(e(:,2)));
ReshapedData = ReshapedData(:,SortedIDX);
% Create new data that has same format as file, but sorted to your liking
NewMyData = [MyData(1:4);reshape(ReshapedData,[],1)];
% Create and write to file
fname = 'C:\Users\<username>\Documents\MATLAB\Sandbox\testfile_modified.txt';
FID = fopen(fname,'wt+');
for ii = 1:length(NewMyData)
   fprintf(FID,'%s\n',NewMyData{ii});
end
fclose(FID);

Hope this helps.

[Timbo]

Works like a charm! I'm going to try to apply it to larger data files and will let you know how it works with those soon.

[Timbo]

I tried it on a very large data set (100,000+ lines) and it worked! Thanks for the help!

[Ruger28]

@Timbo Im glad I was able to help out and it worked! Thanks for the update.