How to send files from worker to client in MDSC?

In the client I opened several workers by MDSC with MJS cluster "MyMJScluster" using multiple computers as node PC.

parpool('MyMJScluster')

Then I run a function which has a spmd block.

spmd
...
end

During the running of job, several files are generated on the local hard drive of each node PC. These files are under the same path of each node, such as "D:\data1\output\*.*", but filenames are different and unpredicatble, but could be dir from worker PC.

I want these files to transfered to the client under the same path to pool them together and do further combined small-scale analysis of these files at client.

Is there a way to do such a transfer? The addAttachedFiles only transfer the specified files from client to workers, but not the reverse.

Thanks.

Answer 1

[Edric Ellis]

There's no facility to transfer files from the workers back to the client. If you can, I would suggest reading the files into memory on the workers, and then transferring the variables back from your spmd block as Composite objects. Something like

spmd
  ... do stuff ...
  myData = loadAllOutputs('D:\data1\output\*.*');
end
% Pull all the data back from the workers
localData = myData(:);
% localData is now a cell array of length 'numlabs'
% containing all the data loaded on each worker

[raym]

I also come up with the same idea last night. Thanks.