Statistics How To

Assumptions and Conditions for Regression

Probability and Statistics > Regression Analysis > Assumptions and Conditions for Regression

Assumptions and Conditions for Regression.

Regression can be a very useful tool for finding patterns in data sets. However, your data can’t always be fit to a regression line. Most software, like SPSS and Excel, will always give you a the best regression line it can find even if the regression line doesn’t make sense. It’s up to you to figure out beforehand if your data makes sense for regression analysis. How do you do that? By considering the following assumptions and conditions for regression before you run the test:

  1. The Quantitative Data Condition.
  2. The Straight Enough Condition (or “linearity”).
  3. The Outlier Condition.
  4. Independence of Errors
  5. Homoscedasticity
  6. Normality of Error Distribution

The Quantitative Data Condition/ Quantitative Variables Condition.

You can only perform regression on quantitative variables. In other words, if your data isn’t a set of numbers, regression isn’t a good method for finding a trend. Check that your variables have actual units and that they are measuring something that makes sense.

In order to find out if your data meets to quantitative data condition, you have to make sure you have quantitative data (numerical data) and not qualitative data. Qualitative data is data that fits into categories (that’s why it’s also called categorical data). See: Quantitative or Qualitative: How to Classify Variables.

Categorical Variables that Masquerade as Quantitative.

quantitative variables condition Sometimes in statistics you can assign numbers to categorical variables in order to force them to become quantitative (so you can perform computations). For example, a deck of cards is made up of quantitative variables (the numbers on the cards) and categorical variables (the suits: hearts, diamonds, spades, clubs). You can give the suits numbers in order to make them numeric:

  • Hearts = 1
  • Diamonds = 2
  • Spades = 3
  • Clubs = 4

However, giving numbers to categorical data does not turn them into quantitative variables; They are still categorical variables: just ones that have been assigned numbers. Therefore, you can’t perform regression on these types of variables because they do not meet to quantitative variables condition.

The Straight Enough Condition (Linearity).

(Linear Regression only). Regression lines will be very misleading if your data isn’t approximately linear. The best way to check this condition is to make a scatter plot of your data. If the data looks like it can roughly fit a line, you can perform regression. For other types of regression (like exponential regression), eyeball the scatter plot to make sure it roughly follows the shape of whatever regression you are performing.

The Outlier Condition.

Assumptions and Conditions for Regression

One outlier can dramatically affect your regression line.




Outliers can have a dramatic effect on regression lines and the correlation coefficient you get when you run regression analysis. If you do have an outlier in your data, it’s a good idea to run regression analysis twice: Once with the outlier and once without.

Independence of Errors

If your points are following a clear pattern, it might indicate that the errors are influencing each other. The errors are the deviations of an observed value from the true function value. The following image shows two linear regression lines; on the left, the points are scattered randomly. On the right, the points are clearly influencing each other.
Independence of Errors


If you don’t have random errors, you can’t run linear regression as your predictions won’t be accurate.

Homoscedasticity

With homoscedasticity, you basically want your points to look like a tube instead of a cone. Heteroscedasticity is where, like independence of errors, you see a trend in the errors but this time the trend is larger or smaller (as opposed to the errors clearly influencing each other). In the picture below, the left graph shows a linear regression line where the errors are getting larger. The shape is cone-like.
Heteroscedasticity


Running linear regression on data that shows heteroscedasticity will give you poor results.

Normality of Error Distribution

At any point in your x-values, the data points should be normally distributed around the regression line. Your values should be fairly close to the line, evenly distributed with only a few outliers. The following image shows data that is fairly normally distributed on the left. The data on the right has data that is either clustered to the line or far from the line. Linear regression should not be run on values that are not normally distributed.
Normality of Error Distribution

Questions about assumptions and conditions for regression? Post a comment and I’ll do my best to help!

Assumptions and Conditions for Regression was last modified: March 10th, 2016 by Andale

2 thoughts on “Assumptions and Conditions for Regression

  1. Zeyad

    Hello,
    Thanks for this post. I am trying to write a computer programme to make different regression types; linear, power, exponential, logarithmic, polynomial (power2, 3 , 4 , 5). However, I need to set assumptions and conditions for every regression type in order to crash of the programme(endless loop). How the above conditions can be set up in terms of equations/ mathematical statements.

    Thanks!

  2. Andale Post author

    Hello, Zeyad,
    Thanks for your comment. I’m not much of a programmer though, so you might want to try a programming forum. Some of the assumptions can probably be set up with “if” statements or something similar (like the data condition: something is either numerical or it’s not).
    Best of luck,
    Stephanie

Leave a Reply

Your email address will not be published. Required fields are marked *