Probability and Statistics > Regression Analysis > Logistic Regression / Logit Model
In order to understand logistic regression (also called the logit model), you may find it helpful to review these topics:
The Nominal Scale.
What is Linear Regression?
For a brief look, see: Logistic Regression in one picture.
Simple Logistic Regression
Simple logistic regression is almost identical to linear regression. However, linear regression uses two measurements and logistic regression uses one measurement and one nominal variable. The measurement variable is always the independent variable. It’s used when you want to find the probability of getting a certain nominal variable when you have a particular measurement variable.
Logistic Regression, ANOVA and Student’s T-Tests
ANOVA and Student’s T-Tests can also be used to analyze data that has one nominal variable and one measurement variable. Logistic regression is used when you want to predict the probability for the nominal variable. Here’s an example to clarify that statement:
You measure the BMI for a group of 50-year-old women, then ten years later you survey the women to see who had a myocardial infarction (a heart attack). You could evaluate your data in different ways, depending on your goal:
- Student’s T-Test: You can test the null hypothesis that BMI is not linked to myocardial infarction.
- Logistic Regression: You can predict the probability that a 50-year-old woman with a certain BMI would have a heart attack in the next decade.
Logistic Regression vs. Linear Regression
In linear regression, you must have two measurements (x and y). In logistic regression, your dependent variable (your y variable) is nominal. In the above example, your y variable could be “had a myocardial infarction” vs. “did not have a myocardial infarction.” However, you can’t plot those nominal variables on a graph, so what you do is plot the probability of each variable (from 0 to 1). For example, your study might show that a woman with a BMI of 30 has a 4% chance of having a heart attack within the next ten years; you could plot that as 30 for the X variable and 0.04 for the Y variable.
Comparison with Discriminant Analysis
Discriminant analysis is a classification method that gets its name from discriminating, the act of recognizing a difference between certain characteristics. The two goals are:
- Construction of a classification method to separate members of a population.
- Using the classification method to allocate new members to groups within the population.
Discriminant Analysis is used when you have a set of naturally formed groups and you want to find out which continuous variables discriminate between them. The simplest example of DA is to use a single variable to predict where a member will fall in a population. For example, using high school GPA to predict whether a student will drop out of college, graduate from college, or graduate with honors.
A more complex example: you might want to find out which variables discriminate between credit applicants who are a high, medium, or low risk for default. You could collect data on credit card holder characteristics and use that information to determine what variables are the best predictors for whether a particular person will be a high, medium, or low risk. New observations (in this case, new applicants) could then be allocated to a particular group.
As well as the credit and banking industries, other uses for Discriminant Analysis include:
- Developing facial recognition technology.
- Classifying biological species.
- Classifying tumors.
- Determining the best candidates for college admissions.
Logistic Regression is often preferred over Discriminant Analysis as it can handle categorical variables and continuous variables. Logistic Regression also does not have as many assumptions associated with it. For example, Discriminant Analysis requires the assumptions of equal variance-covariance within each group, multivariate normality, and the data must be linearly related. Logistic Regression does not have these requirements.
Related Articles
Hosmer-Lemeshow Goodness of Fit test.
What are Log Odds?
Check out our YouTube channel for hundreds of videos on elementary statistics and probability.