# Why ODE for optimal control theory?

Robotics Asked by bicepjai on January 6, 2022

I am trying to understand optimal control theory which forms the base for reinforcement learning techniques in AI. Whenever I open a lecture or a book or any online notes, everything starts with an ODE and then derivation goes the payoff function which is straight forward.

I am trying hard to comprehend why an ODE models any system? Many say it easy to begin with but why this model?

$$dx/dt = f(x(t))$$

I could not find the reason and decided to ask for help.

Linear and nonlinear systems which form the basis for subjects like optimal control theory have rigorous math fundamentals which allow you to analyze ODEs without actually solving them and mathematically prove whether is system is stable, how fast your convergence will be or define a safe operating region. This makes representing systems as ODEs very useful in control systems. ODE representation also makes it clear how your states (eg: position, velocity, acceleration for cars) interact with each other (is there any coupling between the states?).

Coming back to optimal control, the cost functional is expressed as functions of your states and/or control inputs. Say you want to minimize distance, your cost functional will be the integral over the velocity for all time. As you expressed your system as an ODE, you already have access to this expression.

Answered by Arun Lakshmanan on January 6, 2022

The reason why ODE's are used is simply: physics. It would be great if any system could be modelled by a simple linear function like $x(t)=at$, but nature is not so simple, or linear. Even when you neglect nature, dynamical systems, like $dot{x}(t)=f(x(t))$ still pop up everywhere, like CroCo said, it is the basis of the mathematical modelling of many systems.

I would suggest looking into differential equations first before starting with reinforcement learning.

Answered by WalterJ on January 6, 2022

## Related Questions

### Jacobian transpose: How to calculate orientation error?

1  Asked on December 17, 2021 by karol-wajs

### How to calculate vehicle detection distance?

1  Asked on December 17, 2021 by assam-alzookery

### How does a clustered particle filter work?

1  Asked on December 17, 2021 by s-e-k

### How does 3D Lidar work?

1  Asked on December 17, 2021 by mjd-kassem

### How to send commands to iRobot Create 2?

1  Asked on December 17, 2021 by mauricio-moreno

### Vehicle Odometry Correction Using Lidar Contour Points (Localization)

1  Asked on December 7, 2021 by dong-jae-lee

### KUKA Robotics API IDE

3  Asked on December 4, 2021 by nikita-semenov

### Motor encoders randomly start counting backward, then go back to normal

2  Asked on November 30, 2021 by abdullah-ansari

### How to calculate power supply requirements for Closed Loop Stepper Motors?

1  Asked on November 28, 2021 by oscarm

### ROS sensor implementation

1  Asked on November 28, 2021

### Design optimization method using indices

1  Asked on November 28, 2021 by hamedmh

### Why can’t the Controller Spawner find the expected controller_manager ROS interface?

1  Asked on November 28, 2021 by shubham-wani

### Bias correction for multiple sensor fusion through Kalman Filtering

1  Asked on November 28, 2021

### Why aren’t my 4x 612 RPM motors driving with only 12 lb load?

2  Asked on October 3, 2021

### State propagation from uncertain control input

1  Asked on October 3, 2021 by astghik-hakobyan

### Inverse kinematics for 4-DOF Articualted manipluator

2  Asked on October 3, 2021 by oualid

### How to find inertia parameter of payload automatically

1  Asked on October 3, 2021 by j_f

### output velocity equation of compliance control

0  Asked on October 3, 2021 by rrrruo

### Bearing landmark localization

1  Asked on October 3, 2021 by r-go