How to analyze a csv (txt or xls) file without opening it

Hello,

I have an optimization problem in which I need to simultaneously run it several times using par-for. The reason is that my optimization problem is very difficult and the algorithm might get traped in a local minima. So, I need to run it in parallel in a hope that one of them does not get traped.

To this end, I need to save the results of each of them in a csv (txt or xls). However, I would like to update this file whenever each solvers finds a new solution. So, once one of the solvers finds a new solution this csv file should be read and if the new result is better the file should be updated by this better result.

The problem is computing time. It takes around 0.03 seconds to update the file (as I explained in above). Considering the fact that I have 12 cores on my PC this is a very expensive task (my optimization problem is already time-consumming).

So, the best for me is to manipulate an existing csv file without reading it (and without saving the new results). Is this possible in matlab?

Thanks in advance,

Babak

[dpb]

"... manipulate an existing csv file without reading it (and without saving the new results). Is this possible in matlab?"

Taken literally, this is impossible without the yet-to-be-released Clairvoyant Toolbox™.

Practically, what you're describing is to simply keep the data in a shared memory area. I've never had the ML parallel programming TB so don't know how it deals with it; with what I have done one uses semaphores to handle the access to the shared data from the different processes -- gridlock is a potential issue when the number of processes gets large, I would suspect, but I've never worked in anger with more than a couple processors at a time so new techniques are bound to have been developed since...