Differential Equations

Calculus > Differential Equations

What is a Differential Equation?

A differential equation is an equation that involves an unknown function and its derivative. For example, the equation f'(x) + f(x) = x + 3 has f(x) as the unknown function. They differ from “regular” equations like x4 + 9 = x because (in addition to having a derivative as part of the expression) they have a function or set of functions as the solution, rather than a number or set of numbers. There are two main forms of differential equations: ordinary and partial.

Ordinary differential equations have one variable with respect to another. They are further classified according to the order of the highest derivative of the dependent variable. For example, Bessel’s differential equation is a second order differential equation:

First and second order differential equations are the most important of the set of differential equations.

A partial differential equation involves an unknown function as a function of multiple independent variables. The equation involves partial derivatives, which are functions of several variables and their derivatives with respect to one another. Like ordinary differential equations, partial differential equations are also classified by their order, but classification as elliptic, hyperbolic, or parabolic equations is particularly important for this type of differential equation.

Differential equations are further classified as homogeneous or nonhomogeneous differential equations. A homogeneous differential equation involves only the variable y: y” + 2y’ – 3y = 0. Nonhomogeneous differential equations are the same as homogeneous differential equations, with one difference; they can have the variable x on the right side: y” + 2y’ – 3y = x2+10.

Example Differential Equation

Sample question: Prove that y'(x)=e-3x is one solution to the differential equation y” + 2y’ – 3y = 0.
Step 1: Write a list of the function, the first derivative of the function, and the second derivative of the function:
y1(x)=e-3x
y'(x)=-3e-3x
y”(x)=9e-3x
Step 2: Substitute f(x), f'(x) and f”(x) from Step 1 into the differential equation:
y” + 2y’ – 3y = 0, so:
9e-3x + 2(-3e-3x)-3(e-3x)= 0
Multiplying through:
9e-3x + (-6e-3x)-(3e-3x)= 0
That’s it! (9-6-3 equals 0).

Note: There are often several solutions to differential equations. y2(x)=ex also works as a solution to y” + 2y’ – 3y = 0.

Fun fact: A differential equation is just an equation that contains derivatives. They are more common than you think: Newton’s second law, F=ma, is actually a differential equation; you’d have to rewrite it a little (find a full explanation of why here).

How to Solve an Ordinary Differential Equation.

A second order differential equation.

Ordinary differential equations are equations have a function as the solution rather than a number. An ordinary differential equation contains information about that function’s derivatives. If you want to solve an ordinary differential equation, you may have to solve an equation with an initial condition or it may be without an initial condition. For example, the differential equation dsdt=cos(x) is an ordinary differential equation, but dsdt=cos(x);y(π) = 0 is an ordinary differential equation with an initial condition, y(π)=0. You’ll find two procedures below: one for “with” and one for “without” initial conditions. Scroll down the page for more solved examples of both types.

Solve an ordinary differential equation without initial conditions

Sample problem: Solve dsdt = cos t + sin t

Step 1: Use algebra to make the equation integrable. For this question, you need to get ds on it’s own, so divide by dt to get:
ds = cos t + sin t dt

Step 2: Integrate both sides of the equation:
∫ds = ∫cos t + sin t dt →
∫1 ds = ∫cos t + sin t dt →
s = sin t – cos t + C

That’s it!

Solve an ordinary differential equation with initial conditions

dydx3x2-5x-9; y(1)=10.5

Step 1: Use algebra to make the equation integrable:
dy = 3x2 – 5x + 9 dx

Step 2: Integrate both sides of the equation:
∫dy = ∫3x2 – 5x + 9 dx →
∫1 dy = ∫3x2 – 5x + 9 dx →
y = 3x3 – 2.5x2 + 9x + C

Step 3: Insert the values from the initial condition into the equation:
10.5 = 3(13) – 2.5(1) + 9(1) + C
10.5 = 3 – 2.5 + 9 + C
C = 1

The solution to the differential equation is y = 3x3 – 2.5x2 + 9x + 1

What is a Differential Equation Particular Solution?

A problem that requires you to find a series of functions has a general solution as the answer — a solution that contains a constant, which could represent one of a possibly infinite number of functions. A particular solution requires you to find a single solution that meets the constraints of the question. For example, a problem with the differential equation dydv x3+8 requires a general solution with a constant for the answer, while the differential equation dydv x3+8; f(0)=2 requires a particular solution, one that fits the constraint f(0)=2.

Differential Equation Particular Solution Example

Sample problem #1: Find the particular solution for the differential equation dydx=5, where y(0)=2.

Step 1: Rewrite the equation using algebra to move dx to the right (this step makes integration possible):
dy = 5 dx

Step 2: Integrate both sides of the equation to get the general solution differential equation:
∫ dy = ∫ 5 dx →
∫ 1 dy = ∫ 5 dx →
y = 5x + C

Step 3: Rewrite the general equation to satisfy the initial condition, which stated that when x = 0, y = 2:
2 = 5(0) + C
C = 2
The differential equation particular solution is y = 5x + 2

Sample problem #2: Find the particular solution for the differential equation dydx= 18x, where y(5)=230.

Step 1: Rewrite the equation using algebra to move dx to the right:
dy = 18x dx

Step 2: Integrate both sides of the equation:
∫ dy = ∫ 18x dx →
∫ 1 dy = ∫ 18x dx →
y = 9x2 + C

Step 3: Rewrite the general equation to satisfy the initial condition, which stated that when x = 5, y = 230:
230 = 9(5)2 + C
C = 5
The differential equation particular solution is y = 5x + 5

That’s it!

How to Find the General Solution for a Differential Equation.

Problems with differential equations are asking you to find an unknown function or functions, rather than a number or set of numbers as you would normally find with an equation like f(x)=x2+9. For example, the differential equation dydx=10x is asking you to find the derivative of some unknown function y that is equal to 10x.

General Solution of Differential Equations: Example

Sample problem #1: Find the general solution for the differential equation dydx=2x.

Step 1: Use algebra to get the equation into a more familiar form for integration:
dydx=2x →
dy = 2x dx

Step 2: Integrate both sides of the equation:
∫dy = ∫2x dx →
∫1 dy = ∫2x dx →
y = x2 + C

Sample problem #2: Find the general solution for the differential equation dydx=x2-3

Step 1: Use algebra to get the equation into a more familiar form for integration:
dydx=x2-3→
dy = x2-3 dx

Step 2: Integrate both sides of the equation:
∫ dy = ∫x2-3 dx →
∫ 1 dy = ∫x2-3 dx →
y = x33 -3x + C

Sample problem #3: Find the general solution for the differential equation θ2 dθ = sin(t+0.2) dt.

Step 1: Integrate both sides of the equation:
∫θ2 dθ = ∫sin(t+0.2) dt→
θ3=-cos(t+0.2)+C
That’s how to find the general solution of differential equations!

Tip: If your differential equation has a constraint, then what you need to find is a particular solution. For example, dydx=2x ; y(0)=3 is an initial value problem that requires you to find a solution that satisfies the constraint y(0)=3.

How to Find a Solution to a Second Order Differential Equation.

A second order differential equation.

Differential equations are equations that have a derivative as part of the equation. For example, dy/dx=2x. Unlike in algebra, where there is usually a single number as a solution for an equation, the solutions to differential equations are functions. The equations represent the relationship between some varying quantity and it’s rate of change.

The order of a differential equation refers to the highest derivative you can find in the function. First order differential equations (sometimes called ordinary differential equations) contain first derivatives and therefore only require one step to solve to obtain the function. Second order differential equations contain second derivatives. Although they look a little intimidating at first, second order differential equations are solved in the exact same way as first order. They just require two steps to solve: one for the first derivative and one for the function itself.

Second Order Differential Equation: Steps:

Sample Problem: Solve the following second order differential equation with the initial conditions y'(0)=4; y(1)=5:
d2dx2 = 2-6x

Step 1: Solve for y’ by integrating the differential equation:
y’ = ∫ (2-6x)dx →
y’ = 2x – 6x22 + C

Step 2: Plug the first initial value y'(0) into the equation from Step 1. y'(0) = 4, so when x=0, y=4:
4 = 2(0) – 3(0)22 + C →
c = 4
Therefore, y’ = 2x-3x2+4

Step 3: Integrate the solution from Step 2 to get the function:
y = ∫2x – 3x2 + 4 →
y = x2 – x3 + 4x

Step 4: Plug the second initial value y(1) = 5 into the equation from Step 3.
5 = 12 – 13 + 4(1) + C
C = 1
Therefore, the solution to the second order differential equation is y=x2-x3+4x+1

That’s it!

How to Solve a Differential Equation with an Initial Condition.

When a differential equation specifies an initial condition, the equation is called an initial value problem. Initial conditions require you to search for a particular solution for a differential equation. For example, the differential equation dydx=19x2+10 needs a general solution of a function or series of functions (a general solution has a constant “c” at the end of the equation). But if an initial condition is specified, like dydx19x2+10;y(10)=5, then you must find a particular solution (a single function). Finding a particular solution for a differential equation requires one more step — simple substitution — after you’ve found the general solution.

Sample Problem: Solve the following differential equation, with the initial condition y(0)=2.

dydx = 10-x

Step 1: Use algebra to move the “dx” to the right side of the equation (this makes the equation more familiar to integrate):
dydx = 10-x →
dy = 10-x dx

Step 2: Integrate both sides of the equation.
∫dy = ∫10-x dx →
∫1 dy = ∫10-x dx →
y = 10x – x22+C

Step 3: Substitute in the values specified in the initial condition. In this sample problem, the initial condition is that when x is 0, y=2, so:
2 = 10(0) – 022+C
2 = 0 + C
C = 2
Therefore, the function that satisfies this particular differential equation with the initial condition y(0)=2 is y = 10x – x2⁄2 + 2

That’s it!

Initial Value Sample problem #2: Solve the following initial value problem: dydx = 9x2-4x+5; y(-1)=0

Step 1: Rewrite the equation, using algebra, to make integration possible (essentially you’re just moving the “dx”.
dydx = 9x2-4x+5 →
dy = (9x2-4x+5) dx

Step 2: Integrate both sides of the differential equation to find the general solution:
∫ dy = ∫(9x2-4x+5) dx →
∫ 1 dy = ∫(9x2-4x+5) dx →
y = (9x334x22 + 5x + C →
y=3x3-2x2+5x+C

Step 3: Evaluate the equation you found in Step 3 for when x=-1 and y=0.
0 = 3(-1)3-2(-1)2+5(-1)+C→
0 = -3-2-5+C→
0 = -10+C
c = 0
Therefore, the particular solution to the initial value problem is y = 3x3-2x2+5x+10

That’s it!

What is the Ornstein-Uhlenbeck Process?

The Ornstein-Uhlenbeck Process (OU Process) is a differential equation used in physics to model the motion of a particle under friction. In financial probability, it models the spread of stocks. It’s also used to calculate interest rates and currency exchange rates.

Pairs trading was developed at the end of the 1980s, after several crises in the economy hit the market in the previous decades. It’s a way to keep market exposure low while at the same time making a profit from trading. Pairs trading identifies two similar companies. The two companies should have a high correlation, cointegration or both. Equity securities for both companies should be trading outside their normal historical range. You buy the undervalued security and short sell the overvalued security, betting that the investments will return to their historical norm. Once you have identified the two companies, you’ll want to generate a way to generate trading signals. One way to do this is with the Ornstein-Uhlenbeck Process.

Ornstein-Uhlenbeck Process / OU Process.

Basically, a force exerts on a particle to bring the particle back to the mean; a greater the distance from the mean results in more force. The same principle works for modeling spread between a pair of stocks, enabling you to identify when the stock is below the mean (buy) and when it is above the mean (sell).

OU Process = dxt = θ(μ – xt)dt + σ dWt

Where:

• xt = the particle’s current position.
• θ = a mean reversion constant.
• μ = the mean particle position
• σ = a constant volatility
• dWt = a Wiener process (Brownian motion).

Calculation of the OU Process

Calculating the OU process is quite complex. Ideally, you should be familiar with stochastic calculus, Brownian motion and differential equations. This NYU article covers the basics of what you should know.

Considering there could be millions of dollars at stake, it’s highly unlikely you’ll want to calculate the OU process by hand. Instead, there are a multitude of software packages that will perform the calculations for you, including:

• Matlab: Daniel Charlebois uploaded code to the Mathworks file exchange (found here) that can calculate the “Exact numerical solution and plots of the Ornstein-Uhlenbeck (OU) process and its time integral – calculation and plotting of the probability density function (pdf) of the OU process is also performed.”
• R: Package ‘sde’ is for the simulation and inference of stochastic differential equations. You can find the package here.

All of the above tools have multiple regression options built in.

Least squares regression is probably your best bet for modeling the best fit of the data. For an example of how you would apply this to a set of data, check out Calibrating the Ornstein (originally from SITMO.com). It includes two methods (least squares and maximum likelihood).

A Caution on Using an Unmodified OU processes

It sounds easy to use, and (assuming you’re using software to do the calculations), it’s pretty simple to put into action. The problem is, if you’re using an unmodified OU process without a stop-loss, you could end up losing everything. The further the stock is from the mean, the more you risk and the bigger you trade. You could end up betting all of your capital and losing everything when the stock falls. It’s wise to include a stop-loss to prevent this from happening.

References:
D.S. Ehrmann. The Handbook of Pairs Trading. John Wiley & Sons,
Hoboken, New Jersey, 2006.