I have a huge image file (.dat file 17Gb) of columns x rows x frames. I need to import this file as fast as possible, but only every 10th frame or so.

I have a huge image file (binary .dat) which takes minutes to import into memory and process. My calculations are under control after I have it in memory but I really only need every Nth frame in order to speed things up (for my calculations, and if possible loading into memory). The issue (I believe) is my for loop while loading the .dat file. I'm sure there is a better way to import this data. Thanks!

function [rawImage] = read_image_raw(loadFile,colPix, numRowsPerFrame, totalNumFrames)

% Calculate how many pixels to read in each frame
pixToRead = colPix * numRowsPerFrame;

% Open the file for reading
fid = fopen(loadFile,'r');

% Allocate space to the image variables
fileMarker = 0;
imageBlock = zeros(numRowsPerFrame,colPix,totalNumFrames);

% Loop through and read the image          
for iFrame = 1:totalNumFrames
    fseek(fid,fileMarker,'bof');

    imageData = fread(fid,pixToRead,'uint16=>double','ieee-le');
    imageBlock(:,:,iFrame) = reshape(imageData,[colPix,numRowsPerFrame])';

    fileMarker = fileMarker + 2.*colPix*numRowsPerFrame;
end

% Close the file
fclose(fid);

rawImage = imageBlock;

Answer 1

[Asvin Kumar]

The fseek command takes as second argument the offset which is the number of bytes to move from origin.

To obtain every Nth frame, you can modify the appropriate statement from the for loop as follows:

fileMarker = fileMarker + N*2*colPix*numRowsPerFrame;

Here’s a link to the documentation for fseek: https://www.mathworks.com/help/matlab/ref/fseek.html