Pre-calculating memory usage of repmat.m to avoid out of memory errors

5 views (last 30 days)
A and B are equal length vectors of my data. The max value in A is 100 and the max value in B is 20,000 and n = 2,500. I wish to run the following code
n = length(A); % length(A) == length(B);
a = 1; b =1;
gridx1 = [1 : a : max(ceil(A))];
gridx2 = [1 : b : max(ceil(B))];
[gridx2,gridx1] = meshgrid(gridx2,gridx1);
x1 = repmat(gridx1, [1,1,n]);
x2 = repmat(gridx2, [1,1,n]);
mu1 = repmat(A,[length(gridx1),length(gridx2),1]);
mu2 = repmat(B,[length(gridx1),length(gridx2),1]);
However I get an out of memory error. Hence I need to reduce the sampling frequency of the grids.
Lets say I wish to allocate up to 3GB of RAM to this process, how can I find the sampling frequency, "a" and "b".
Obviously I wish to sample at the maximum frequency possible (subject to the 3GB constraint) to get the most accurate answer.
thank you
  2 Comments
Jason Ross
Jason Ross on 7 Feb 2013
I'll state what I hope is pretty obvious: If you are regularly hitting "Out of memory" issues, you really should move to a 64-bit operating system. The hardware and operating systems have supported it for years now. Barring a few specialized use cases (generally related to hardware drivers), there is no downside to moving to a 64-bit operating system. RAM is also quite inexpensive, 16 GB can be had for less than $100, 24 GB less than $150, 32 GB less than $200 and even 64 GB is running ~$350. This, of course, assumes a desktop host with a motherboard that can take these types of chips.

Sign in to comment.

Accepted Answer

Matlab2010
Matlab2010 on 7 Feb 2013
Thank you for your suggestions. very helpful.
I think the below script does want I need.
function testLimit()
%%3GB in bytes
maxLimit = 3 * 1024 * 1024 * 1024;
%%make some fake data
A = [1: 1: 100];
B = [1 : 200: 20E3];
n = length(A);
maxA = max(A);
maxB = max(B);
a=1; %lets keep this equal to one.
% Now calculate b
nom = (n*(maxB)^2 + 2*n*(maxA/a)^2*(maxB)^2);
denom = ((maxLimit / 64) - n*(maxA/a)^2);
b = sqrt(nom/ denom); %this is the sampling frequency of the grid
%%Now lets test the answer to make sure its ok
x1Size = n * (maxA / a)^2;
x2Size = n * (maxB / b)^2;
mu1Size = n * (maxA / a)^2 * (maxB / b)^2;
mu2Size = mu1Size;
tot=64* sum([x1Size x2Size mu1Size mu2Size]);
if(tot>=(maxLimit+1))
disp('too large')
else
disp('fine')
end
end

More Answers (1)

cr
cr on 7 Feb 2013
Edited: cr on 7 Feb 2013
say, Lg1 is length(gridx1) and Lg2 is length(gridx2). Then your variables need this much memory x1: Lg1*Lg2*n*8 bytes (assuming you are using doubles), x2: Lg1*Lg2*n*8 bytes This is where the problem lies. Other not so big. Use singles or integer types if doubles are not necessary. Or, do your operations sequentially clearing intermediate variables.

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by