In hypothesis testing, the true error is the error rate of a hypothesis over a whole unknown distribution of examples; It is the probability a single randomly drawn example will be misclassified (Mitchell, 1997).
True error is also sometimes defined as the difference between the true value found by a calculation, and the approximate value found by using a numerical method. As an example, the derivative, which gives a precise value for the slope at a point, can be approximated by the equation f′(x) ≈ (f(x + h) – f(x)) / h; The difference between these two values is the true error.
What Causes True Error?
True errors can happen because of many reasons, including non-sampling error— a wide range of causes for errors that include:
- Poor data collection methods (like faulty instruments or inaccurate data recording),
- Selection bias, where your methods for choosing participants are faulty, like the healthy worker effect or Non response bias (where individuals don’t want to or can’t respond to a survey).
Increasing the sample size won’t reduce these types of errors and can make them worse (larger samples using the same faulty methods = more errors).
They key to minimizing true error is to make sure your experiment or survey is well designed. You should also make sure your measuring instruments are precise.
References
Bhattacharyay, S. Errors in Computational Analysis. Retrieved January 6, 2021 from: https://people.uwec.edu/BHATTAS/Errors_in_numer_anal.pdf
Mitchell, T. (1997). Machine Learning. 1st edition. McGraw-Hill.
Muth, J. (2006). Basic Statistics and Pharmaceutical Statistical Applications, Second Edition (Pharmacy Education Series). Chapman and Hall/CRC.