Is this code suitable for solving a system of ODEs ?

Can i use this code for a system of ODE and in what way ?

x = linspace(0,1,10000)';

inputSize = 1;

layers = [

featureInputLayer(inputSize,Normalization="none")

fullyConnectedLayer(10)

sigmoidLayer

fullyConnectedLayer(1)

sigmoidLayer];

dlnet = dlnetwork(layers);

numEpochs = 15;

miniBatchSize =100;

initialLearnRate = 0.1;

learnRateDropFactor = 0.3;

learnRateDropPeriod =5 ;

momentum = 0.9;

icCoeff = 7;

ads = arrayDatastore(x,IterationDimension=1);

mbq = minibatchqueue(ads,MiniBatchSize=miniBatchSize,MiniBatchFormat="BC");

figure

set(gca,YScale="log")

lineLossTrain = animatedline(Color=[0.85 0.325 0.098]);

ylim([0 inf])

xlabel("Iteration")

ylabel("Loss (log scale)")

grid on

velocity = [];

iteration = 0;

learnRate = initialLearnRate;

start = tic;

% Loop over epochs.

for epoch = 1:numEpochs

% Shuffle data.

mbq.shuffle

% Loop over mini-batches.

while hasdata(mbq)

iteration = iteration + 1;

% Read mini-batch of data.

dlX = next(mbq);

% Evaluate the model gradients and loss using dlfeval and the modelGradients function.

[gradients,loss] = dlfeval(@modelGradients3, dlnet, dlX, icCoeff);

% Update network parameters using the SGDM optimizer.

[dlnet,velocity] = sgdmupdate(dlnet,gradients,velocity,learnRate,momentum);

% To plot, convert the loss to double.

loss = double(gather(extractdata(loss)));

% Display the training progress.

D = duration(0,0,toc(start),Format="mm:ss.SS");

addpoints(lineLossTrain,iteration,loss)

title("Epoch: " + epoch + " of " + numEpochs + ", Elapsed: " + string(D))

drawnow

end

% Reduce the learning rate.

if mod(epoch,learnRateDropPeriod)==0

learnRate = learnRate*learnRateDropFactor;

end

end

ModelGradients

function [gradients,loss] = modelGradients2(dlnet, dlX, icCoeff)

y = forward(dlnet,dlX);

% Evaluate the gradient of y with respect to x.

% Since another derivative will be taken, set EnableHigherDerivatives to true.

dy = dlgradient(sum(y,"all"),dlX,EnableHigherDerivatives=true);

% Define ODE loss.

eq = dy + y/5 - exp(-(dlX / 5)) .* cos(dlX);

% Define initial condition loss.

ic = forward(dlnet,dlarray(0,"CB")) - 0 ;

% Specify the loss as a weighted sum of the ODE loss and the initial condition loss.

loss = mean(eq.^2,"all") + icCoeff * ic.^2;

% Evaluate model gradients.

gradients = dlgradient(loss, dlnet.Learnables);

end

[Paola]

I tried to use the similar code available at

https://it.mathworks.com/help/deeplearning/ug/solve-odes-using-a-neural-network.html

on the same example of differential equation (y = exp(-x^2))

but unfortunately I get this error message:

Error using dlfeval (line 43)

Layer 'input': Invalid input data. Invalid number of spatial dimensions. Layer expects 0 but received 2.

what is it? how can this problem be solved?

Many thanks in advance!

Paola

Answer 1

[Vishnu]

Hi Dimitris,

Yes, you can use this code for a system of ordinary differential equations (ODEs). The code implements a neural network model using the deep learning toolbox in MATLAB.

The ODE system is defined in the `modelGradients2` function, where the ODE is represented by the equation

`dy + y/5 - exp(-(dlX / 5)) .* cos(dlX)`. The initial condition is specified as `forward(dlnet,dlarray(0,"CB")) - 0`.

The neural network architecture consists of a feature input layer, a fully connected layer, a sigmoid layer, another fully connected layer, and another sigmoid layer. The `dlnet` variable represents the network.

The code then trains the network using the SGDM optimizer and updates the network parameters based on the computed gradients. The loss function is defined as a weighted sum of the ODE loss and the initial condition loss.

You can modify this code to suit your specific ODE system by adjusting the architecture of the neural network, the loss function, and the optimization parameters.

[Paola]

I tried to use the similar code available at

https://it.mathworks.com/help/deeplearning/ug/solve-odes-using-a-neural-network.html

on the same example of differential equation (y = exp(-x^2))

but unfortunately I get this error message:

Error using dlfeval (line 43)

Layer 'input': Invalid input data. Invalid number of spatial dimensions. Layer expects 0 but received 2.

what is it? how can this problem be solved?

Many thanks in advance!

Paola

[zahoor m]

Layer 'input': Invalid input data. Invalid number of spatial dimensions. Layer expects 2 but received

0.