pre-define a struct/array of structs in attempt to get rid of out of memory error.

Hello, how can i pre-define an array/struct of structs? Motivation. I was dynamically creating an array that grew with every iteration using the sample code below:

ultimate_result = [];
% then in my loop
for i=1:length(items)
    items(i).name = item_name; % string
    items(i).size = item_size; % double 
    items(i).results = item_result; % another struct 
    ultimate_result = [ultimate_result items(i)];
end

now some of you have already predicted, I run out of memory (which is actually the main problem and reason i need this) as the item size grows even with my 32gb or ram. I know before hand the number of items i will need to process.

I have tried something like

ultimate_result(2000) = struct; % 

but MATLAB gives the following error.

Unable to perform assignment because the indices on the left side are not compatible with the size of the right side.

Any suggestions to help get rid of the problem will be greatly appreciated. Of course i have tried increasing matlab priority in the task manager in vain,

Am using windows 10, Matlab 2020a, with about 32gb ram.

[Stephen23]

"pre-define a struct/array of structs in attempt to get rid of out of memory error"

Preallocating a structure will not reduce its memory consumption.

[MatlabEnthusiast]

@Stephen Cobeldick indeed the out of memory error persists. What are my options I suspsect that i do indeed run out of memory.

Answer 1

[Matt J]

It is not at all clear from what you posted why you might run out of memory, nor why you get the error message you have posted, which has nothing to do with memory limits.

However, assuming "ultimate_ressult" is a typo and should really be "ultimate_result", then you are needlessly updating ultimate_result inside your loop and slowing things down. The whole thing, with proper pre-allocation, can be done as follows.

N=___ % number of "items"

items(N).name=[];
items(N).item_size=[];
items(N).item_result=[];

% then in my loop
for i=1:N
    items(i).name = item_name; % string
    items(i).size = item_size; % double 
    items(i).results = item_result; % another struct 
end

ultimate_result=items;

[MatlabEnthusiast]

Indeed "ultimate_ressult" is a typo and should really be "ultimate_result". I am not exactly sure why I run out of memory. MATLAB simply says some error. Forexample:

In one instance it said,

Error using horzcat  
Out of memory. Type "help memory" for your options ....

But i get your point, and i am going to try it out asap.

[MatlabEnthusiast]

The pre allocations worked ffine, thanks, but the out of memory error persists. What are my other optiions store this information in files or what?

[Matt J]

The memory error isn't coming from code that I suggested to you. There are no horzcat operations in the version I posted.

Answer 2

[Joseph Cheng]

the pre-allocation can be done (though can't really find the right way

ultimateR = [];
temp = struct('name',[],'size',[],'results',[]);
temp(2000).name = 'dummy'; %to be overridden when you actually get to index 2000

for ind = 1:10
    temp(ind).name = 'dummy';
    temp(ind).size = 10;
    temp(ind).results = temp(1);
    ultimateR = [ultimateR temp(ind)];
end

I don't think you need your line ultimate_ressult as you're already saving the data in items(i).'parameter'. you're doubling the stored data by re-saving it into ultiate_results which the data is contained in items. i don't get the same error but perhapse something is predefined such that the left and right hand some of one of the above lines isn't right.