Matlab to write binary files?

I am using an old dos program that states in its documentation:

input files (*._IN) are in binary Format. They cannot be edited in a text editor (e.g. Notepad) without being corrupted.

I know that with matlab I can write to binary files using fwrite, but how do I follow the format that the dos program documentation lists:

char identifier[7] = "dos 00";   //  input file identifier (Text)
char version[5]="1.02 00";   // Program version (Text)
short int [2 bytes] end_line;   // Total number of input file lines (short int, 0x0008, shows
       // up in the file as 08 00)
char description[56] = (description line in program input file editor);   // description (Text)
struct input_data{   // This block is repeated for "end_line"
   char process[12];   // Process (Text) (see  program manual for text strings)
   char sin_string[9];    // Starting index (Text)
   char units_sin[5];     // Starting index units (Text)
   char ein_string[9];    // Ending index (Text)
   char units_ein[5];     // Ending index units (Text)
   char time_string[7]; // Time of process (Text)
   char units_time[4];  // Units for time of process (Text)
};

nulls at end of each
each line is input data structure

What do the numbers in [] mean and how can i replicate this structure in matlab to generate *IN files?

Answer 1

[Walter Roberson]

The numbers in [] indicate array sizes. So for example,

char identifier[7] = "dos 00"

means that identifier is an array of 7 char and it is to be assigned the initial value ['d', 'o', 's', ' ', '0', '0', char(0)] . Here char(0) is a binary 0 not the character '0'. Every C string is terminated with a binary 0.

The line

char version[5]="1.02 00";   // Program version (Text)

is a problem. It says that 5 characters are to be reserved for version but then it tries to initialize with 8 characters, ['1', '.', '0', '2', ' ', '0', '0', char(0)] . Depending on the exact order that the code executes in, that could end up corrupting your executable. It would make sense if it had been "1 00" or "1 02" or "1.02", but not what it is now.

"short int" corresponds to int16. The comment that 0x0008 shows up in the file as 08 00 tells you that binary values must be written in "little endian" format.

Store your data in a structure named input_data with fields named process, sin_string, units_sin, ein_string, units_ein, time_string, units_time . Do not worry about getting the length of each string exactly right, in that the code below will automatically chop or pad the values as necessary. The code takes care of the null terminator that is part of every C string.

RowV = @(C) reshape(C, 1, []);
FirstN = @(C, N) C(1:min(N,end));
ZPadTo = @(C, N) [C, zeros(1, N - length(C), 'uint8')];
CString = @(C, N) ZPadTo( FirstN(RowV(uint8(C)), N-1), N );

identifier = 'dos 00';
version = '1.02 00';    %this is going to get chopped to '1.02'
end_line = int16(length(input_data));
description = 'Some Description';

fid = fopen('YourNewFile.IN', 'w', 'ieee-le');
fwrite(fid, CString(identifier,7));
fwrite(fid, CString(version,5));
fwrite(fid, end_line, '*int16');
fwrite(fid, CString(description,56));

for IDX = 1 : length(input_data)
  this = input_data(IDX);
  fwrite(fid, CString(this.process, 12));
  fwrite(fid, CString(this.sin_string, 9));
  fwrite(fid, CString(this.units_sin, 5));
  fwrite(fid, CString(this.ein_string, 9));
  fwrite(fid, CString(this.units_ein, 5));
  fwrite(fid, CString(this.time_string, 7));
  fwrite(fid, CString(this.units_time, 4));
end

fclose(fid);

[matlabuser12]

Thanks Walter, quick question, how do I write multiple lines of processes to the same binary file? My struct looks like:

input_data = struct('process',{'heat'}, 'sin_string',{'cobalt'},'units_sin',{'grams'},...etc. )

but if I want to to do multiple lines of processes, I cannot duplicate the elements of the struct since they already exist. I also need a carriage return or new line to start at each new struct, how would I code that into the binary file?

[Walter Roberson]

Binary structures do not use carriage return or new lines to separate them. They use a precise amount of memory.

You can proceed in multiple ways:

A)

input_data(1) = struct('process',{'heat'}, 'sin_string',{'cobalt'}, 'units_sin', {'grams'}, ...etc. );
input_data(2) = struct('process',{'cool'}, 'sin_string',{'argon'}, 'units_sin', {'mol'}, ...etc. );
...

This makes it clear what each different element is.

B)

input_data = struct('process', {'heat', 'cool'}, 'sin_string',{'cobalt', 'argon'}, 'units_sin', 'grams',...etc. );

Here you can use a mix of {} and non-{} elements. The non-{} elements like I show for units_sin are copied to all of the members of the resulting structure. The {} elements like {'heat', 'cool'} will store the first value inside {} for the first member of the structure, the second for the second member of the structure and so on. Every {} that you use must be either exactly one long (in which case it is copied to all) or the same size as the others. The above would be equivalent to

input_data(1) = struct('process', 'heat', 'sin_string', 'cobalt', 'units_sin', 'grams', .....);
input_data(2) = struct('process', 'cool', 'sin_string', 'argon', 'units_sin', 'grams', ......);

[matlabuser12]

I used style B and it worked fine! One last question, if I want to generate multiple files over a range of values, for example:

input_data = struct('process', {'heat', 'cool'}, 'sin_string',{'cobalt', 'argon'}, 'units_sin', {'grams' ,'' },'value',{'500', '400',..want a series here with a step size} . );

How would I go about adding a series of values or numbers into the structure? and for each step size generate a separate input file. One thing I did realize I had to do to get multiple lines read in the dos program was to make sure that all of the fields are full, either with values in order or with '' spaces as I have shown above. Otherwise it just populates the next entry and the ordering gets messed up.

Would it be best to autogenerate an array or structures, and then cycle through those to generate the files? As opposed to making one very very large structure since there may be cases where I need to change 2 or more of the entries to be a range. Such as the gram weight, the time, etc. in order to get all the permutations over a range.

[Walter Roberson]

To use the code I posted, you should use a different input_data structure for each output file. The code I posted extracts the size of the input_data structure as the number of "lines" to store in the file, and it loops over all of the members of the structure outputting each. Although it would be possible to create an index saying how many to write out at each point, calling the routine once for each output file is easier. Unless, that is, that you want to only ever write out one structure member for each .IN file to be created?

For the sequence '500' '400' '300' '200' '100' you can use

seq = cellstr( num2str( (500:-100:100).' ) ).' ;

and then put in

'value', seq

It is important for this purpose that the vector 500:-100:100 be transposed to become a column vector, or else num2str will write it out all on one line.

Alternate code would be

seq = arrayfun( @(V) sprintf('%d', V), 500:-100:100, 'Uniform', 0);

which is easier to read.

[matlabuser12]

Would the seq variable go in the structure where I normally put the value?

Unless, that is, that you want to only ever write out one structure member for each .IN file to be created?

I am not sure what you mean by this, here is an example of the structure I wrote:

input_data = struct('process',{'Heat','Cool','Hold'},'sin_string',...
{'cobalt','aluminum','titanium'},'units_sin',...
{'grams','lbs','grams','grams'},...
'ein_string',{'5','','4.3',''},'units_ein',...
{'C','','F',''},'time_string',...
{'','3.5','10','6'},'units_time',...
{'','HRS','MIN','HRS'});

That whole structure becomes an input file, then a new structure is used where ein_string for example is ticked up by 1 for the 1st entry (5 in this case), and that new structure with a minor change is turned into an input file, etc.

[Walter Roberson]

master_data = struct('process',{'Heat','Cool','Hold'}, ...
  'sin_string', {'cobalt','aluminum','titanium'}, ...
  'units_sin', {'grams','lbs','grams','grams'}, ...
  'ein_string', {'5','','4.3',''}, ...
  'units_ein', {'C','','F',''}, ...
  'time_string', {'','3.5','10','6'}, ...
  'units_time', {'','HRS','MIN','HRS'});

for ein = 5 : .1 : 7.3    %for example
  input_data = master_data;
  input_data(1).ein_string = sprintf('%g', ein);
  .... now write input_data to an appropriate file
end

[matlabuser12]

Thanks Walter. Having one issue with that though. It does not generate separate files, I seem to be getting only the one file at the end. I am trying to use the sprintf to count up the file names but that isn't workign for me correctly either:

for k = 1:length(ein)

fid = fopen(sprintf('YourNewFile' '%k''.IN'), 'w', 'ieee-le');

end

[Image Analyst]

You need to use %d and put in k after the single quotes, like this:

for k = 1:length(ein)
    filename = sprintf('YourNewFile%d.IN', k);
    fid = fopen(filename, 'w', 'ieee-le');
    % fwrite.....
    fclose(fid);
end

[matlabuser12]

Should I also use the k in the input_data(1).ein_string line? so that I get a new input data file or does it automatically overwrite the structure at each ein step?

[Image Analyst]

I don't know. What is an example of what it looks like? If the string needs to have the value of k embedded in it, then yes, you'll need to use sprintf. If it doesn't need k, then don't.

Answer 2

[Image Analyst]

It can be a lot of work to write a file reader utility. I'm not going to do that for you but I can give you a function where I read some stuff out of a binary file. In this code I read a bunch of bytes from a binary file and stick the interpreted results (different things in the header that have different meanings) onto a structure as different fields of the structure, stHeader. See the comments to get an idea of what they are. Feel free to adapt this code as needed to your particular format:

% Read_RAW_Header
% Returns the image header information and some extra information
% Usage: stHeader = Read_RAW_Header(filename)
% Input
%   filename - RAW format file name
% Output
%   stHeader - header structure containing information on the data
% Header is 44 bytes.
% The header of an RAW file will look something like this:
%  stHeader.TotalVoxels:  Array size - total size of the array (not the whole file) - number of voxels.
%  stHeader.NumberOfDimensions:  Number of dimensions - 3 for a 3D image.
%  stHeader.x_size:  Size (number of voxels) in the X direction.
%  stHeader.y_size:  Size (number of voxels) in the Y direction.
%  stHeader.z_size:  Size (number of voxels) in the Z direction.
%  stHeader.XStart:  Position of the center of the first pixel in the X direction - it's 0.
%  stHeader.YStart:  Position of the center of the first pixel in the Y direction - it's 0.
%  stHeader.ZStart:  Position of the center of the first pixel in the Z direction - it's 0.
%  stHeader.XEnd:  Position of the center of the last pixel in the X direction - it's voxelWidth * (sizeX - 1).
%  stHeader.Yend:  Position of the center of the last pixel in the Y direction - it's voxelWidth * (sizeY - 1).
%  stHeader.ZEnd:  Position of the center of the last pixel in the Z direction - it's voxelWidth * (sizeZ - 1).
% Following the header in the data file, are the voxel values, but of course we do not return these in the header.
%
% I also add these fields to the returned header argument:
%  stHeader.HeaderSize = 44;
%  stHeader.BytesPerVoxel
%  stHeader.FileSizeInBytes
%  stHeader.DataSizeInBytes
%  stHeader.Endian
%  stHeader.EndianArg
%
% Example usage:
% fullFileName_RAW = fullfile(cd, 'SimulatedData_Distance.RAW')
% stHeader = Read_RAW_Header(fullFileName_RAW)
%
%------------------------------------------------------------------------------

function stHeader = Read_RAW_Header(fullFilename)
  % Check syntax.  Must have at least one input argument, the full filename. 
  if (nargin ~= 1)
    error('Usage: stHeader = Read_RAW_Header(fullFilename)');
  end 

  if (ischar(fullFilename)~=1)
    error('Requires a string filename as an argument.');
  end 

  % These aren't in the header specification for this type of file,
  % but other formats use it and it's a useful thing to add to the header.
  stHeader.Endian = 'Big';
  stHeader.EndianArg = 'ieee-be';

  % Open the file for reading.
  fileHandleID = fopen(fullFilename, 'rb', stHeader.EndianArg);
  if (fileHandleID == -1)
    error(['Error opening ', fullFilename, ' for input.']);
  end

  % Go to the beginning of the file.
  % Shouldn't be necessary, but can't hurt.
  fseek(fileHandleID, 0, 'bof');

  % Read the total number of voxels in the image.
  % Read bytes 1-4.
  stHeader.TotalVoxels = fread(fileHandleID, 1, '*int32');
  % Note: this may be unreliable, and can be zero!
  % Better to take the x, y, and z sizes and multiply them together.

  % The next 4 bytes are the number of dimensions - 2 or 3.
  % Read bytes 5-8.
  stHeader.NumberOfDimensions = fread(fileHandleID, 1, 'int32');

  % Read in the dimensions for the different directions.
  % They'll be in bytes 9-20.
  stHeader.x_size = fread(fileHandleID, 1, '*int32');
  stHeader.y_size = fread(fileHandleID, 1, '*int32');
  stHeader.z_size = fread(fileHandleID, 1, '*int32');
  stHeader.TotalVoxels = stHeader.x_size * stHeader.y_size * stHeader.z_size;

  % Read in the position of the center of the first pixel.
  % They'll be in bytes 21-32.
  stHeader.XStart = fread(fileHandleID, 1, '*float');
  stHeader.YStart = fread(fileHandleID, 1, '*float');
  stHeader.ZStart = fread(fileHandleID, 1, '*float');
  % Read in the position of the center of the last pixel.
  % They'll be in bytes 33-44.
  stHeader.XFieldOfView = fread(fileHandleID, 1, '*float');
  stHeader.YFieldOfView = fread(fileHandleID, 1, '*float');
  stHeader.ZFieldOfView = fread(fileHandleID, 1, '*float');
  % Note: the fields of view are based on (pixel center) - to - (pixel center).

  % Calculate the voxel width.
  stHeader.VoxelWidth = stHeader.XFieldOfView / single(stHeader.x_size - 1);

  % Assign some other useful information.
  % It's not in the header but may come in useful anyway.

  % Assign the bytes per voxel.
  %fileInfo = imfinfo(fullFilename);  % Note: doesn't work!
  % fileSizeInBytes = fileInfo.FileSize;
  % This works:
  fileInfo = dir(fullFilename); 
  % MATLAB returns the information in a structure with fields:
  %    name
  %    date
  %    bytes
  %    isdir
  %    datenum
  stHeader.HeaderSize = 44;
  fileSizeInBytes = fileInfo.bytes;
  dataSizeInBytes = double(fileSizeInBytes) - double(stHeader.HeaderSize);
  stHeader.BytesPerVoxel = int32(round(dataSizeInBytes / double(stHeader.TotalVoxels)));
  stHeader.FileSizeInBytes = fileSizeInBytes;
  stHeader.DataSizeInBytes = dataSizeInBytes;

  % Close the file.
  fclose(fileHandleID);  % Close the file.

[matlabuser12]

THanks I will take a look, I am actually trying to write binary files. I have a dos program from the early 90s that can read them and perform some calculations.

[matlabuser12]

Here is the code I am working off of:

input_data = struct('process',{'Heat','Cool','Hold','Heat'}, ...
  'sin_string', {'cobalt','aluminum','titanium','helium'}, ...
  'units_sin', {'grams','grams','grams','grams'}, ...
  'ein_string', {'5','','4.3','5.5'}, ...
  'units_ein', {'C','','C','C'}, ...
  'time_string', {'','3.5','10','9'}, ...
  'units_time', {'HRS','HRS','HRS','HRS'});

RowV = @(C) reshape(C, 1, []);
FirstN = @(C, N) C(1:min(N,end));
ZPadTo = @(C, N) [C, zeros(1, N - length(C), 'uint8')];
CString = @(C, N) ZPadTo( FirstN(RowV(uint8(C)), N-1), N );

   ein = 5 : .1 : 7.3;  %for example

for k=1:length(ein) %to generate multiple input files for the range of ein_string values

      input_data(1,k).ein_string = sprintf('%g', ein(1,k)); %to edit only the first entry in the ein_string struct segment

identifier = 'dos 00';
version = '1.02 00';    %this is going to get chopped to '1.02'
end_line = int16(length(input_data));
description = 'My Test';

fid = fopen('YourNewFile._IN', 'w', 'ieee-le');
fwrite(fid, CString(identifier,7));
fwrite(fid, CString(version,5));
fwrite(fid, end_line, '*int16');
fwrite(fid, CString(description,56));

for IDX = 1 : length(input_data)
  this = input_data(IDX);
  fwrite(fid, CString(this.process, 12));
  fwrite(fid, CString(this.sin_string, 9));
  fwrite(fid, CString(this.units_sin, 5));
  fwrite(fid, CString(this.ein_string, 9));
  fwrite(fid, CString(this.units_ein, 5));
  fwrite(fid, CString(this.time_string, 7));
  fwrite(fid, CString(this.units_time, 4));
end

fclose(fid);

end

[Image Analyst]

OK - did it work? Does it produce a file that can be used by your other, old DOS program?