In this article, we will be discussing r-squared and adjusted r-squared, its differences, its indication, formula, and more.
R-squared
The R-squared is a statistical measure that represents the proportion of the variance in a regression model for a dependent variable that is defined by an independent variable or variables. It’s a metric for determining how far or close the data is from the fitted regression line. In other words, a linear model explains a proportion of the variation in the response variable, which we call the r-squared.
Some other words for R-squared are:
- Coefficient of Determination and,
- Coefficient of Multiple Determination for Multiple Regression
Formula
The following is the formula for calculating r-squared:
where:
- RSS: The sum of squares attributable to regression
- TSS/Total Variance: The entire sum of squares
We can calculate the R-squared in the following way:
- The first step in its calculation is gathering data points from dependent and independent variables and determining the line of best fit from a model.
- Right after that, we compute the expected values, remove actual values, and square the results. The reason for performing the prior step is to find a list of mistakes squared. These mistakes squared are all added together and represents the unexplained variance.
- Now we must determine the TSS/total variance. This becomes possible by subtracting the average real value from each of the actual values.
- After that, we’ll square the results and add them together.
- Now that we have the explained and unexplained variance values, the last step involves us to divide the explained variance by the unexplained variance and subtract the result from one. The result we get from that is the r-squared value.
What Does It Indicate?
Firstly, you need be aware that r squared is always between 0 and 100%.
The 0 percent shows that the model explains none or zero portion of the variability in the response data around its mean. Whereas 100% means that the model explains all of the variability in the response data around its mean.
If the r-squared is greater, it signifies that the model perfectly reflects your data. A smaller r-squared, on the other hand, suggests that the model does not completely reflect your data.
Now, I’m sure you’re thinking that a high r squared is what you need to aim for because it’s a positive indicator. However, a high r-squared does not automatically imply that the regression model is excellent. The kind of variables in the model, the units of measurement of the variables, and the data transformation done all have an effect on the statistical measure’s quality.
Advantages of R-squared
The following are some advantages of r-squared:
- The relationship between the movements of a dependent variable and the movements of an independent variable can be illustrated through r-squared.
- It assists you in determining whether the model matches the original data.
- It can help you avoid overfitting a model.
Limitations of R-squared
The following are some limitations of r-squared:
- R-squared can not measure the goodness-of-fit test for a model.
- R-squared indicates nothing about prediction inaccuracy. It can be anything between 0 and 1 by varying the range of X. It does not keep track of prediction accuracy.
- We can not use r-squared to compare models with altered responses.
- Even though r-squared is a measure of the connection between the movements of a dependent variable and the movements of an independent variable, it does not indicate whether or not your chosen model is good. It doesn’t even say if the facts and predictions are faulty.
- We also previously discussed how a high r-squared isn’t always considered good. It is possible to get a low r-squared for a good model and high r-squared for a poor one.
Adjusted R-squared
An adjusted R-squared is a refined version of R-squared that takes into consideration factors in a regression model that are not really significant. In simple words, the adjusted R-squared indicates whether or not adding more factors will improve a regression model. Basically, it determines whether or not those additional factors contributes to the regression model. It tests the predictive power of regression models with varying levels of predictors. An adjusted r squared helps in comparing the goodness-of-fit of regression models with different numbers of independent variables. The goodness-of-fit testing is an important hypothesis test that determines how well sample data fits a normal distribution from a population. One of the most prevalent forms of goodness-of-fit tests is the chi-square test.
The following is the formula to calculate the adjusted r-squared:
where:
- dft: The measure of the population variance of the dependent variable with degrees of freedom n– 1
- dfe: The measure of the underlying population error variance with degrees of freedom n– p – 1
An adjusted r-squared can be determined based on the r-squared value, the number of independent variables, and the total sample size.
Can an Adjusted R-squared Increase or Decrease?
You’re probably wondering if an adjusted r-squared increases or decreases on its own, and if so, what may be the cause. Basically, an adjusted R-squared only increases if the new predictor improves the model beyond what might be predicted by accident. When a predictor improves the model less than what is actually predicted by accident, it will fall or diminish. Let me explain this in simple terms, if you add way too many variables which are useless, to a model, the adjusted r-squared will decrease. However, if you add a bunch of variables that are actually useful, to a model, then the adjusted r-squared will increase. An adjusted r squared is always going to be either less than or equal to r-squared.
Differences and Relationship in Between R-squared and Adjusted R-squared
The adjusted r-squared takes into account and evaluates different independent factors, whereas the r-squared does not. We just spoke about how an adjusted r squared rises when useful variables are added to the model, and vice versa. Always keep in mind that the r-squared, on the other hand, rises with each predictor added to a model. Unlike an adjusted r-squared, the r-squared never decreases. The more variables you add to the model, the better it will fit.