Regression Analysis > Relative Weights
What are Relative Weights?
Johnson’s Relative Weights is a way quantify the relative importance of correlated predictor variables in regression analysis. “Relative importance” in this context means the proportion of the variance in y accounted for by xj. Put another way, it helps you figure out what variables contribute the most to r-squared.
The actual calculations are complex and are usually performed with software. Chao et. al outline the general steps as:
- Transform the predictor variables into a set of orthogonal (uncorrelated) variables. These variables are “maximally related” to the predictor variables from an unweighted least squares perpective.
- Regress the dependent variables on the new set of transformed variables.
For most statistical software programs (like SPSS or JMP), run principal components regression to produce the orthogonal variables. Next, run least squares regression, using the results from the PCR to predict y-variables. The combined relative weights should add up to the initial r-squared.
Comparison to Other Indices
According to Chao et. al, Many relative importance of indices have been proposed over the years, including the Product Measure, General Dominance Index and Squared semipartial correlation. Johnson’s Relative Weights is superior in that it has better theoretical underpinnings and it always produces clear results even if the predictors have very high collinearity. Although Relative Weights and General Dominance Index (Shapley regression) produce similar results, Shapley’s method is computationally complex for more than a dozen or so variables and so is less often used. For example, a 30 variable relative weights model will run almost instantaneously on a home computer, while a 30 variable Shapley’s regression could take days.
Chao, Yi-Chun. Quantifying the Relative Importance of Predictors in Multiple Linear Regression Analyses for Public Health Studies. Journal of Occupational & Environmental Hygiene Volume: 5 Issue 8 (2007) ISSN: 1545-9624
If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.Comments? Need to post a correction? Please post on our Facebook page.