Basis functions (called derived features in machine learning) are building blocks for creating more complex functions. In other words, they are a set of k standard functions, combined to estimate another function—one which is difficult or impossible to model exactly.
For example, individuals powers of x— the basis functions 1, x, x2, x3…— can be strung together to form a polynomial function. The set of basis functions used to create the more complex function is called a basis set.
It’s possible to create many complex functions by hand; IDeally, you’ll want to work with a set of as few functions as possible. However, many real-life scenarios involve thousand of basis functions, necessitating the need for a computer.
Common Basis Functions
The most common types of basis functions in calculus are:
- Polynomial basis: 1, x, x2, x3…—
- B-Spline basis: a set of k polynomial functions, each of a specified order d. An order is the number of constants required to define the function (Ramsay and Silverman, 2005; Ramsay et al., 2009). Popular for non-periodic data.
- Fourier basis: a set of sine functions and cosine functions: 1, sin(ωx), cos(ωx), sin(2ωx), cos(2ωx), sin(3ωx), cos(3ωx)&hekkip;. These are often used to form periodic functions. Derivatives for these functions are easy to calculate but aren’t suitable for modeling discontinuous functions (Svishcheva et al., 2015).
In machine learning, popular methods include:
- Artificial Neural Networks,
- Basis function regression (including Radial Basis Functions),
- k-Nearest Neighbors.
Hertzmann, A. et al. (2015). Nonlinear Regression.
Ramsay JO, Hooker G, Graves S (2009) Functional Data Analysis with R and Matlab.
New York: Springer-Verlag. 214 p.
Ramsay JO, Silverman BW (2005) Functional Data Analysis. New York: Springer
Series in Statistics. 430 p.
Svishcheva et al. (2015). Region-based association test for familial data under functional linear models. Retrieved August 9, 2020 from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4481467/