workers idling after a while when using parfor loop

I have a script that fits 2D gaussians in captured images from my measurement setup. Since I generate ~100.000 images every measurement that's quite a task. I have a parfor loop to analyze these images and it works fine, but after several hours only about a quarter of the workers are still busy, the rest is idling. This leaves the system underused and I can't figure out why, especially because it starts out working fine but breaks somewhere along the way. Can anybody help me to debug this?

[Edric Ellis]

parfor uses a static means of distributing work, but it attempts to load-balance so that this situation does not occur. Do you know if the analysis can take a different amount of time for different images? If so, that can cause load imbalances.