How to simulate a large number of probabilities efficiently?

I'm running a program where I need to generate a large matrix of boolean values based on a probability, gamma. I have to regenerate the probabilities at least a million times for every iteration of the program, and this single line takes about 25% of the runtime in the code analyzer. This is my current method of generating the values:

for i = 1 : 2.25 * 10 ^ 6

* code

bernoulliRandomVariables = rand(neuronCount, numInputLines) < gamma;

* code

end

Is there a more efficient way to do this?

[Mike Croucher]

How big are neuronCount and numInputLines

[Cooper Scher]

Highly variable for both depending on the run. For a recent series of runs, neuronCount = 900 and numInputLines = 390. Another thing to note is that gamma is generally very small, generally less than 0.001.

Answer 1

[Harsh Mahalwar]

Hi Cooper,

From what I can gather, you are trying to create a rand array inside a for loop which is going to iterate 2.25 million times.

To optimize the generation of a large matrix of Boolean values based on a probability, you can try the methods mentioned below:

1. Reducing precision:

If your simulation can tolerate it, consider using a lower precision for the random numbers. By default, rand generates double-precision floating-point numbers. You might not need this level of precision. Using single precision can reduce memory usage and potentially speed up computations, here is an example how you can achieve this:

bernoulliRandomVariables = single(rand(neuronCount, numInputLines)) < gamma;

2. Use parallel computing:

Given that you're generating many of these matrices in a loop, parallelizing this operation could offer significant speedups, here is an example on how you can do so:

parfor i = 1 : 2.25 * 10^6 
    % Your code 
    bernoulliRandomVariables = rand(neuronCount, numInputLines) < gamma; 
    % Rest of your code 
end 

Note: You will need Parallel Computing toolbox and a suitable hardware setup (like a multicore processor or a computer cluster) to get the most out of it.

You can also try running the simulation with C/C++ code which can be easily generated with MATLAB Coder, as there are no overheads like garbage collection, etc in the case of C/C++, they are much faster at executing programs.

To learn more about C/C++ code generation with MATLAB refer the following document:

https://www.mathworks.com/help/coder/optimize-speed-of-generated-code.html

I hope this helps, thanks!