MS Module 20: Residuals and Standardized Residuals (overview)
(The attached PDF file has better formatting.)
Reading: §12.6 Assessing Model Adequacy (titled Aptness of the Model and Model Checking in first edition)
The residual is the observed value minus the fitted value. The standard deviation is the same for all points in a distribution, so the procedure to standardize the distribution is the same at all points.
For classical regression analysis, the variance of the error term is the same at all points. But the fitted value is more strongly drawn to the observed value at outlying (influencial) points, so the variance of the residual is greatest at the mean and lower at outlying points.
You will grasp this relation most quickly by forming regression equations with Excel’s regression add-in.
● Chose X-value of {1, 2, 3, 4, 5, 6, 7, 8, 9}.
● Chose any nine Y-values, and run the regression add-in.
● The add-in computes β1 and the residuals at each point.
Now change the Y value by one unit at one of the points.
● If you change the Y value at X=5, β1 doesn’t change but the residual at X=5 changes by 1/N.
● If you change the Y value at X=9, β1 change at lot but the residual at X=5 changes by less than 1/N.
The standardized residual is the residual divided by its standard deviation, which differs for each residual. The arithmetic is similar to that for confidence intervals and prediction intervals, but the signs differ: a larger width of the prediction interval is associated with a smaller standard deviation of the residual.
Final exam problems may give the set of x values, the residuals at two points, and the standardized residual at one of these points, and it may ask to derive the standardized residual at the other point.
Know the five diagnostic plots and their uses, and difficulties with the regression analysis that they indicate.
Take heed: The standardized residuals computed by the Regression add-in in Excel’s Analysis Pack differ from the standardized residuals in the textbook. See the discussion at the web page:
https://stats.stackexchange.com/questions/166533/how-exactly-are-standardized-residuals-calculated(The discussion is attached as a PDF file to this posting.)