Simulink-Unity-ROS for robot simulation

Hey community, I wanted to explore an option of Using Matlab/Simulink-Unity-ROS2 together. My idea is build a Simulator for robots based on Unity (which has simulation environment) that includes the robot model (modelled in simulink) and is connected with ROS2 simultaneously which controls the behaviour of the robot based on received sensor inputs. I have been working with Gazebo & ROS2 for simulation but considering the limitations associated with the robot model, thought of exploring this option to create a high-fidelity robot model using simulink.

Answer 1

[Pratyush]

Hi Rajkumar,

Integrating MATLAB/Simulink, Unity, and ROS2 for a high-fidelity robot simulator involves several key steps:

• Use Simulink for modeling and simulating the robot's control systems and dynamics.
• Create a new Unity project for the simulation, importing 3D models of your robot and the environment.
• Use the Simulink 3D Animation toolbox or custom plugins to facilitate communication between Simulink and Unity, allowing for control commands and sensor data exchange.
• Install ROS2 and set up a bridge (e.g., "ros2-web-bridge") for Unity-ROS2 communication, enabling Unity to subscribe to sensor topics and publish control commands.
• Begin with simple tests to ensure communication works, then gradually increase complexity, using ROS2 tools for debugging and Unity for performance optimization.
• Keep detailed records of your setup and share your findings with the community for feedback and improvements.

This approach leverages the strengths of each platform for a comprehensive simulation environment, requiring careful integration and communication management between MATLAB/Simulink, Unity, and ROS2.

[Rajkumar]

Hi Pratyush,

Thank you very much for the detailed infomation. Your breakdown provides a clear roadmap for achieving seamless communication and integration between these platforms. I will certainly look into the suggested approach and share my findings.

In the meanwhile I have a Doubt regarding Simulink 3D Animation, does it work with Unity as well? Because while reading about it, I came across a line saying "Simulink 3D Animation connects Simulink models and MATLAB algorithms to Unreal Engine®"