On define a large scale arrays

Hi, all, I want to define and store a large scale arrays, like ones(3^N,1000,N) and N can be 15. However, matlab says it is out of memory, because of the large scale.

I wonder is there any way to resolve it? I mean, to fulfill the purpose but satisfy the requirement of matlab? Actually ones(3^N, 1000,1) is still very large scale.

Thanks.

Answer 1

[Image Analyst]

Does it need to be double? Can it be single, uint8, or logical instead? Will it have any values other than 1 or small integers?

[C Zeng]

Good idea! No, for ones(3^N,1000,N) array, only 0 1 or 2, that is uint8. However, for (3^N,1000,1) array, can be any positive fraction number.

I defines ones to give initial values.

[Image Analyst]

OK, then does

yourArray = ones(rows, columns, N, 'uint8');

solve your memory problem?

[C Zeng]

Hi, sorry, still not working here. The problem is that 3^N can be very large, for example N can be 14, so 3^N*1000*N is very large, I think it exceeds the limit of Matlab. I created the array but it takes a lot of time, stay busy...

[Image Analyst]

Are you sure that you need to whole matrix in memory at one time? Can't you work in "chunks"? Like work on a smaller cube of it at a time? Will you eventually access each and every one of those elements? Or can you use "sparse" matrices? Have you read the Mathworks guidance on this issue: Memory Usage

[C Zeng]

I may not need all matrix at a time and yes I can work in chunks. Let me double check the problem. Thank you for your suggestion!

Answer 2

[Matt J]

Are the entries mostly zeros? Could you use the SPARSE command?

If you need to work with n-dimensional sparse arrays, there is this:

http://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays

[C Zeng]

Thanks, will take a look. Unfortunately, entries may not be mostly zeros.

[Matt J]

If they are mostly ones, you can express the array as

 1 + sparse_array

and rewrite your computations in terms of that.

[C Zeng]

Yes, could be helpful, let me find it out. Thanks.