About VR removed in future versions of MAtlab/Simulink and connection with Unreal engine

Hi!

So I saw that in the next versions of Matlab/Simulink, the blocks related to virtual reality (vr sink etc.) will be removed. Now I have built the GUI in the virtual world which then goes with Simulink desktop real time, connected via UDP to a planar robot exchanging data etc.

I have seen that it is recommended for future versions to use Simulink 3d animation with unreal engine, now the problem I am experiencing for example if I use the "Simulation 3 scene configuration" block is that it does not connect to unreal (because of plugins I cannot find, for example "Mathworks interface"). I am using Matlab version 2023b and Unreal engine 5.4.1. Please let me know if anyone has already tried it or can help me thank you very much.

Here in the picture I have attached what my interface looks like in the VR sink block, I wanted to understand how I could translate this with the simulink 3D and unreal engine blocks, thank you

Answer 1

[Nishan Nekoo]

Hi Giada, thank you for reaching out about this! Specific versions of MATLAB can only be used with specific versions of Unreal Engine. For example, MATLAB R2024a only works with Unreal Engine 5.1. See more documentation here: https://www.mathworks.com/help/sl3d/unreal-engine-simulation-environment-requirements-and-limitations.html

However, I am wondering if your workflow even requires the Unreal Editor at all. You have access to Unreal Engine within the MATLAB environment (without needing to download Unreal Engine separately) and I am wondering if you can just leverage that for your workflow (see the examples below).

For example, you can use the Simulation 3D Actor block to create graphical primitives as in the picture you are showing. Here's a simple example using Simulink: https://www.mathworks.com/help/sl3d/build-actor-3d-graphic-primitives-using-simulink.html but you can also use MATLAB to do this programmatically: https://www.mathworks.com/help/sl3d/build-actor-from-3d-graphic-primitives-using-matlab.html

I hope that helps and please let me know if you have any other questions, I am happy to help with your transition from the virtual reality functionality to Unreal Engine functionality. If you have any features to request or run into any issues, definitely reach out as well because we want to make sure you are supported through this transition!

Nishan

[Giada Parodi]

Thank you very much for your reply!

Yes I have seen about the compatibility between the different versions. Yes then I had already added the two actors (the cursor (sphere) and target (sphere)), my problem would be the background which I would like to make simply a black background (where I don't think there is the function to set the colour in sim3d.World() ) and also add some other actors which are, however, texts that appear and disappear during the experiment (like 'go' or 'finish'). So for these things I thought I would have to use Unreal engine.

Thanks again for the support.

[Nishan Nekoo]

Hi Giada,

To make a black background, change the Weather settings in the Simulation 3D Scene Configuration block by setting the Sun Altitude to -90 degrees.

You can also add annotations to the world using the sim3d.graphics APIs: https://www.mathworks.com/help/sl3d/sim3d.graphics.text.html

Please let me know if you have any questions or need any further assistance!

Nishan

[Giada Parodi]

Thanks for the reply.

Ok though through this class is there a way to connect or pass this data into simulink, because I generally work in simulink where I already have a model that also communicates with a robot through udp etc. so I was trying to create everything in simulink, and I don't really know how to interface this sim3d class with simulink. And above all, then integrate this text into the simulation block 3d scene configuration.

Thanks again, Giada.

[Nishan Nekoo]

Hi Giada, absolutely!

In the Simulation 3D Actor block, you can use the initialization script to create the text or annotations that you need. See the screenshot below for an example.

Unfortunately, the text cannot be changed at runtime when used in this context for now, but as a workaround you can create multiple actors for each annotation, then enable the Scale port and set it to 1 or 0 to make the annotation appear or disappear.

I hope that helps and feel free to reach out if you have any more questions!

Nishan

[Giada Parodi]

Thanks again, yes if I'm not mistaken it was like that when working on vr blocks. So I tried it the way you told me, it seems to compile, but the text cannot be displayed on the 3d scene. I don't know why, I can see the other actors created but not the text.

[Nishan Nekoo]

Hi Giada, that is interesting. During the simulation, if you right click and view the actor tree, do you see the actor for the text in the tree? If you click on it, does it come into focus?

If you could send your model here, I'm happy to take a look as well!

[Giada Parodi]

Yes in the end it was a problem of putting the sun at -90° because it was obscuring the text, indeed the other actors were also a bit strange, for now I should have solved it by putting it at -20°.

[Nishan Nekoo]

Ah I see, I am glad you figured it out!

[Giada Parodi]

Hi again, I was trying a bit to integrate two actors and the 3d scene into my simulink model, so I gave the input translation of my "cursor" and "target" blocks values of x,y,z, where in the case of the cursor is the position of my robot (via UDP connection). But it gives me this error, where could the problem be?

[Nishan Nekoo]

Hi Giada, this error is because your model is attempting to generate code. In general Simulation 3D blocks do not support code generation as described in the product limitations: https://www.mathworks.com/help/sl3d/unreal-engine-simulation-environment-requirements-and-limitations.html#mw_fe62d6bb-5157-4613-909f-542c0a7bbfd8

Are you using Accelerator/Rapid Accelerator mode or Fast Restart in your model? These could be triggering code generation. If not, please provide a complete screenshot of your model or attach it here so that I can take a look.

Nishan

[Giada Parodi]

No it is in normal mode, is there any way around this problem? if 3D graphics with UNREAL are incompatible with SDRT, that's a big problem for me... in this case I have to think of alternatives

[Nishan Nekoo]

Hi Giada, using 3D graphics with Unreal in a Real Time use case is definitely possible, but the configuration and interfaces will need to be set up correctly. Please refer to this video from around 6:00 onwards: https://www.mathworks.com/videos/hardware-in-the-loop-simulation-1698333473102.html

What blocks are you using in your model from SDRT? Can you tell me more about your workflow and what hardware/communication interfaces you are using?

Please reach out to me directly or contact [e-mail] for any additional questions. Without being able to see your model via screenshare or understand the rest of your model/workflow, it is a little tricky for me to find the right answers for you.

Nishan

[Giada Parodi]

Hi Nishan, look I tried to follow the webinar you sent me, but there the model configuration is for simulink real time, not desktop real time. So it is not specific for my problem. I will attach my model, please let me know.

With SDRT is giving me this error "The selected system target file 'sldrt.tlc' is not currently supported for concurrent execution."