Statistics Definitions > Nonlinearity
A nonlinearity is a relation between data points that can’t be condensed into a neat linear graph. Models of systems such as biochemical reactions in living organisms, combustion and fluid turbulence all show examples of nonlinearities.
Nonlinearity vs. Linearity
To understand nonlinearity we first need to define what it means for relationship between two variables to be linear.
Linearity is easiest to understand visually. If your data is linear, you can graph it on a line plot, like the one above. With allowance for an error term, you’ll be able to draw a neat line through your points.
Key properties
A small change in your independent variable (input) will lead to a small change in your dependent variable (output). A large change in your independent variable will lead to a large change in your dependent variable. We call this proportionality. The other key property of linear functions is additivity: : f(x + y) = f(x) + f(y). This basically means that you can:
- Superimpose the different contributions to your independent variable (input) with simple adding.
- Split up your output to reflect what parts of the output depends on which different parts of the input.
- Break up your problem to analyze it, using the basic principles of addition and subtraction.
Nonlinearity happens when the rules of additivity and proportionality are violated. Small changes in the input may lead to significant changes in the output, or the input and output cannot be easily separated for analysis. Nonlinear systems sometimes show chaotic behavior or necessitate more complex models than linear systems. For example, if a test statistic of a random sample X is a nonlinear function of X, the sampling distribution is often unknown or tedious to calculate for finite sample sizes [1]. Nonlinear systems might be chaotic, or they might simply need a more complicated model than a basic line plot.
How do I identify nonlinearity?
Identifying linear and nonlinear regression models is not quite as easy as looking for straight lines, because linear regression models may include curves. Basically, a linear regression model is defined as a curve that can be written as a sum of a constant and first order parameters multiplied by variables. Y = a0 + b1 X1 + b2 X2 + b3 X3… To be a linear regression model, the parameters (b1, b2, b3must each be first order. But the variables (X1, X2, X3) may be squared, cubed, or raised to the nth power, so this could actually be the equation of a curve. You could even take the log or inverse of the independent variable, and this would not affect the linearity of the regression model if the parameters are linear (first order). If your regression model can’t be written Y = a0 + b1 X1 + b2 X2 + b3X3… , it is a nonlinear model.
Four types of nonlinear curves
A nonlinear curve has a curve whose slope changes as the value of one of the variables changes. In comparison, the slope of linear curve does not change when one of the variables changes. The four types of nonlinear relationships are [2]:
- Increasing gradually, then rising more steeply: This can happen in real life when a company has two customer segments with different retention rates.
- Decreasing gradually, then dropping quickly: This can be observed in mortgages. During the early years of a fixed-rate mortgage, the principal decreases slowly. For instance, in the first five years of a 30-year mortgage, the balance only decreases by about $15,000. However, by year 25, the balance drops below $45,000.
- Climbing quickly, then tapering off: A real life example is when a company focuses on volume as a driver of profit, rather than more impactful drivers such as price, which can lead to per unit profit tapering off over time.
- Falling sharply, then gradually. This can happen when a company enters a payback period: as the payback period increases, annual rate of return (ARR) drops steeply in the beginning and then more slowly.
References
[1] Hong, Y. (2017). Probability And Statistics For Economists. World Scientific.
[2] Harvard Business Review: Linear Thinking in a Nonlinear World. Retrieved August 23, 2023 from: https://hbr.org/2017/05/linear-thinking-in-a-nonlinear-world