MATLAB array size limit error - why?

I am receiving an unexpected error because of an array allegedly exceeding the maximum array size limit with a non-double type variable.

Here is a short example to reproduce the error:

First, to keep the example variables small, go to preferences -> MATLAB -> workspace and set the limit of the maximum array size temporarily to 1% of the RAM.

You may provoke the correct error with a double array demanding too much memory:

[~,sys] = memory;
testLen = round(sys.PhysicalMemory.Total/100/2);
N = zeros(testLen,1);

Error using zeros

Requested 62859407x1 (0.5GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.

(for 12GB of RAM)

A uint8 array of the same length is ok:

n = zeros(testLen,1,'uint8');

returns no error. You can manipulate it by

n(end) = uint8(1);

as expected.

N(end-5:end).'

gives

    0    0    0    0    0    1

But why do

 n(end-1:end) = uint8(1);
% or
 n(end-1:end) = uint8([1;1]);

and similar manipulations error:

Requested 62859405x1 (0.5GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.

?

Is MATLAB (R2015b) simply miscalculating the array memory size for non-doubles here or am I doing something wrong?

[dpb]

Looks like the former to me...like end is coded only for double and not the class of the array at hand would be the likely candidate for the problem.

I hadn't realized that was a user preference; don't see it in R2012b so gather mayhaps it is a newer introduction. If so, would seem quite possible it's not been too thoroughly test as yet; I'd suggest bug report is in order (altho you might want to see if anybody else has alternate explanations first).

[Andres]

Thanks for your comment, dpb. Meanwhile I'm convinced this is a bug or at least an unintended behaviour, too. I have sent a bug report.

Answer 1

[Andres]

Thanks for your answer ( Link), Walter. The error first occured inside a function, but my example works in plain matlab workspace, too.

Based on your idea, I did some more analysis with different data types, trying out different test lengths. If the manipulation should temporarily duplicate the memory demand of a variable, the error occurence would depend on the data type size (8/16/32/64 bit).

clear k l m n p q

k = zeros(testLen,1,'uint8');       
l = false(testLen,1);
m = zeros(testLen,1,'uint16');       
n = zeros(testLen,1,'uint32');       
p = zeros(testLen,1,'single');
q = zeros(testLen,1);

k(end-1:end) = uint8(1);
l(end-1:end) = true;
m(end-1:end) = uint16(1);
n(end-1:end) = uint32(1);
p(end-1:end) = single(1);
q(end-1:end) = 1;

Near the maximum length that will allow double (64bit) type creation, the allowableness of the manipulation of all smaller data types changes, too:

[~,sys] = memory;

testLen = floor(sys.PhysicalMemory.Total/100/8);
% maximum length that will work with all types (k l m n p q)
% x(end-1:end) = .. will work on all variables

testLen = floor(sys.PhysicalMemory.Total/100/8)+1;
% creates variables only up to 32bit types (k l m n p)
% (i.e. smallest length that will error on double creation)
% x(end-1:end) = .. will work on all variables created

testLen = floor(sys.PhysicalMemory.Total/100/8)+2;
% creates variables only up to 32bit (k l m n p)
% does NOT allow x(end-1:end) = .. on ANY of the variables created

(maximum array size was limited to 1% of RAM, as above)

I think this is not a reasonable or intended behaviour, so I tend to send a bug report.

Btw, is there a way to get or set the maximum array size including the percentage value found in the preferences tab programmatically? The preferences tab only shows a slider without displaying a value.

[Andres]

The bug has been confirmed.

I have received some code to set the array size limit programmatically, but it is undocumented, so use at your own risk:

% get and set current array size limit percentage
s = Settings();
s.matlab.desktop.workspace.ArraySizeLimit
s.matlab.desktop.workspace.ArraySizeLimitEnabled

ramPercent = 1;   % 1..100
set(s.matlab.desktop.workspace, 'ArraySizeLimit',uint8(ramPercent))

Thanks to anyone who helped, including Mathworks service.

[Andres]

By the way it makes no difference if you pass "end" or a corresponding number to the colon operator. But, at least, it seems you can work around this bug by passing small numbers to the colon operator first and then add an offset:

k(end+(-1:0)) = uint8(1);
l(end+[-1:0]) = true;
m(end+[-1:0]) = uint16(1);
n(end+[-1:0]) = uint32(1);
p(end+[-1:0]) = single(1);
q(end+[-1:0]) = 1;

That did the trick with R2015b, but I do not know if it also works for 2015a (that has a very similar bug). The Mathworks service say that the only viable workaround is to disable the array size limit check for now.

[Andres]

The issue has been fixed with R2016a!

[Faiza Bukenya]

Iam using R2016a but still facing the same error

[Zoe]

I unchecked that limit check and then I could run my code. Thank you very much!

[Mishary Alfarah]

I uchecked that limit and on my activity monitor it showed that it took 63 GB of RAM! then it immediately quit. I think keeping it on maximum maybe is better than uncheck it.

Answer 2

[Walter Roberson]

n(end-1:end) = uint8(1) may require that n be duplicated if n is an array that is currently shared. For example if n is a parameter passed in, and is not being used in the restricted context

n = SomeFunction(n)

whose header is

function n = SomeFunction(n)

then MATLAB may well need to duplicate the array.