Statistics How To

Regression Analysis > Stepwise Regression Stepwise regression is a way to build a model by adding or removing predictor variables, usually via a series of F-tests or T-tests. The variables to be added or removed are chosen based on the test statistics of the estimated coefficients. While the technique does have its benefits, it requires … Read more

Regression Analysis > Lasso Regression You may want to read about regularization and shrinkage before reading this article. What is Lasso Regression? Lasso regression is a type of linear regression that uses shrinkage. Shrinkage is where data values are shrunk towards a central point, like the mean. The lasso procedure encourages simple, sparse models (i.e. … Read more

Non probability sampling > What is Modal Instance Sampling? The purpose of modal instance sampling is to sample the most typical members of a population. The term modal comes from the mode, which is the most common item in a set. Issues One problem with modal instance sampling is identifying what is the most “typical” … Read more

Statistics Definitions > Variance Inflation Factor You may want to read this article first: What is Multicollinearity? What is a Variance Inflation Factor? A variance inflation factor(VIF) detects multicollinearity in regression analysis. Multicollinearity is when there’s correlation between predictors (i.e. independent variables) in a model; its presence can adversely affect your regression results. The VIF … Read more

Probability > Theoretical Probability What is Theoretical Probability? The study of probability can be divided into two areas: Theoretical Probability is the theory behind probability. Experimental (empirical) probability is probability calculated during experiments, direct observation, experience, or practice. With theoretical probability, you don’t actually conduct an experiment (i.e. roll a die or conduct a survey). … Read more

Probability > Experimental Probability / Empirical Probability Empirical research, from where we get empirical probabilities, is where you conduct “hands on” experimentation. In other words, you get your results from actual experience rather than from a theory or belief. What is Empirical Probability? Empirical probability, also called experimental probability, is the probability your experiment will … Read more

Types of Variables > What is a Responding Variable? A responding variable is something that “responds” to changes you make in an experiment. It’s the effect or outcome in an experiment. The changes in an experiment are made to the independent variable (also called the manipulated variable); the responses that happen as a result of … Read more

Design of Experiments > Convergent Validity and Discriminant Validity You may find it helpful to read this article first: What is Construct Validity? What are Convergent Validity and Discriminant Validity? Convergent Validity is a sub-type of construct validity. Construct validity means that a test designed to measure a particular construct (i.e. intelligence) is actually measuring … Read more

Parsimonious Model > Akaike’s Information Criterion What is Akaike’s Information Criterion? Akaike’s information criterion (AIC) compares the quality of a set of statistical models to each other. For example, you might be interested in what variables contribute to low socioeconomic status and how the variables contribute to that status. Let’s say you create several regression … Read more

Maximum Likelihood Estimation > EM Algorithm (Expectation-maximization) You might want to read this article first: What is Maximum Likelihood Estimation? What is the EM Algorithm? The Expectation-Maximization (EM) algorithm is a way to find maximum-likelihood estimates for model parameters when your data is incomplete, has missing data points, or has unobserved (hidden) latent variables. It … Read more