How does Matlab interact with custom Diffusion models?

Matlab has been developing the DL Toolbox for many years, and I want to know what it can do now. I know there are already many official examples, but their overall cognitive effect is limited or even outdated.

For example, I want to start LDF-VFI now, which is a Diffusion model, but there are not many related examples. What extent can Matlab achieve this? By calling PyTorch through Matlab to reduce meaningless dependencies; by calling the converted ONNX through Matlab to break away from Python; even optimizing this scheme to reduce actual VRAM and other resource overheads?

If you could provide the specific code flow, I would greatly appreciate it.

[Torsten]

The best way to learn what a Toolbox can do is to get a test licence and read its user guide:

https://uk.mathworks.com/help/pdf_doc/deeplearning/index.html

I think your question is too special that anybody here could answer (or at least understand) it.

Maybe there are experts in the Mathworks employee's team who know what you are talking about:

https://uk.mathworks.com/support/contact_us.html

Answer 1

[Matt J]

From what I've read, simple diffusion U-Nets are well within Matlab's reach. For very advanced models like LDF-VFI, Matlab can do it, but PyTorch is better optimized computationally, particularly for training. The main reason to use Matlab would be to simplify integration with other things you might already have in Matlab. If that's your priority, then you could use PyTorch for training and Matlab for deployment, but it is also unclear how well very intricate PyTorch models will import into Matlab. It would be worthwhile importing an LDF-VFI architecture from PyTorch as a test, just to see if all the components cross over successfully.