Simulink model output heavily dependent on relative tolerance

I have a relatively large Simulink model that I am validating using a data set. This model is using the ECEF 6DOF block with only a gravitational force as the input.

I have observed that the Simulink model output varies considerably depending on the relative tolerance selected (ranging from 1e-5 to 1e-12), with the output approaching the data set values as the relative tolerance passes below 1e-10. Decreasing the relative tolerance has a significant negative impact on model runtime, so I would like to keep the relative tolerance at a reasonable value while maintaining accuracy in the model results.

Is a significant variation with changing relative tolerance (below the obvious candidates such as >= 1e-5) expected for a typical Simulink model?

Answer 1

[Fangjun Jiang]

Not typical but is for a stiff system. Choose a stiff solver.

https://www.mathworks.com/help/matlab/math/solve-stiff-odes.html