Problem on simple linear regression model evaluation

Section 2.3 in the book suggests three ways to numerically evaluate a simple linear regression model: the regression standard error, s, the coefficient of determination, R², and assessing the slope parameter, b₁, using a hypothesis test or confidence interval. For simple linear regression all three methods are mathematically equivalent, but their generalizations for multiple linear regression are not equivalent.

The purpose of this exercise is to demonstrate that a simple linear regression model with superior measures of fit (i.e., a lower s, a higher R², and a higher absolute t-statistic for the slope) does not necessarily imply the model is more appropriate than a model with a higher s, a lower R², and a lower absolute t-statistic for the slope. These three measures of model fit should always be used in conjunction with a graphical check of the model to make sure that it is appropriate (e.g., see section 2.4 in the book).

Download the simulated data from one of the following files (in SPSS, text, and Excel format, respectively): COMPARE.SAV, COMPARE.TXT, COMPARE.XLS. There is a single response variable, Y, and four possible predictor variables, X₁, X₂, X₃, and X₄.

1. Fit a simple linear regression using X₁ as the predictor. Make a note of the values of the regression standard error, s, the coefficient of determination, R², and the t-statistic for the slope, t. Also construct a scatterplot of Y (vertical) versus X₁ (horizontal) and add the least squares regression line to the plot.
2. Repeat part (a), but this time use X₂ as the predictor. Do the values of s, R², and t suggest a worse or better fit than the model from part (a)? Does the visual appearance of the scatterplot with the X₂ model confirm or contradict the numerical findings?
3. Repeat part (a), but this time use X₃ as the predictor. Do the values of s, R², and t suggest a worse or better fit than the model from part (a)? Does the visual appearance of the scatterplot with the X₃ model confirm or contradict the numerical findings?
4. Repeat part (a), but this time use X₄ as the predictor. Do the values of s, R², and t suggest a worse or better fit than the model from part (a)? Does the visual appearance of the scatterplot with the X₄ model confirm or contradict the numerical findings?

You should find that the model with X₁ has a higher s, a lower R², and a lower absolute t-statistic for the slope than the other three models (which all have the same values of s, R², and t). However, the model with X₁ is more appropriate than each of the other three models:

• The model with X₂ has a clear problem with increasing variance of the residuals as X₂ increases, so that the model fits very well for low values of X₂, but it fits less well for high values of X₂;
• The model with X₃ fails to pick up the clear curved trend in the data points;
• The model with X₄ seems overly influenced by the relatively small number of points in the top right of the plot, and the data seem to be in a few tight clusters rather than in a broad straight-line pattern.

Concluding message: measures of regression model fit like the regression standard error, s, the coefficient of determination, R², and the absolute t-statistic for the slope are only really meaningful when the assumptions of the model are broadly satisfied.

Last updated: September, 2006

The views and opinions expressed in this page are strictly those of the page author. The contents of this page have not been reviewed or approved by the University of Oregon.

© 2006, Iain Pardoe, Lundquist College of Business, University of Oregon