Regression analysis Module 12: F test practice problems

(The attached PDF file has better formatting.)

** Exercise 12.1: F-Test

An F-statistic tests the hypothesis that all the slopes (ß’s) are zero.

The F-statistic using sums of squares is

-statistic = (RegSS / k) ÷ (RSS / (n – k – 1) )

ResSS + RSS = TSS, so some textbooks write this as

-statistic = (RegSS / k) ÷ ( (TSS – RegSS) / (n – k – 1) )

The F-statistic using R² is

-statistic = (R² / k) ÷ ( (1 – R²) / (n – k – 1) )

R² = ResSS / TSS, so the expression with R² is the expression with TSS and RegSS after dividing numerator and denominator and TSS.

The total sum of squares (TSS) is divided between the regression sum of squares (RegSS) that is explained by the regression equation and the residual sum of squares (RSS) that remains unexplained. If a greater percentage is explained by the regression line, R² is greater (ResSS is a greater percentage of TSS), the F-statistic is larger, and the regression is more likely to be significant.

(See Fox, Chapter 6, statistical inference, page 108)

** Exercise 12.2: Degrees of freedom of F-statistic

A regression model has N data points, k explanatory variables (ß’s), and an intercept.

The F-test says: "How much additional predictive power does the model under review have compared to what we would otherwise use, as a ratio to the total predictive power of the model under review?" Each part of this ratio is adjusted for the degrees of freedom.

The degrees of freedom in the numerator adjusts for the extra predictive power of the model under review stemming from additional explanatory variables. If the model under review has one extra explanatory variable, it predicts better even if this extra explanatory variable has no actual correlation with the response variable. The degrees of freedom is the number of extra explanatory variables, or q.

If the F-test has a p-value of P% with q degrees of freedom in the numerator, its p-value is more than P% with q+1 degrees of freedom in the numerator. A higher p-value means that it is more likely that the observed increase in predictive power reflects the spurious effects of additional explanatory variables.

The degrees of freedom for the model under review is N – k – 1; this is the degrees of freedom in the denominator of the F-ratio. As N increases but no other parameters change, the additional predictive power of the model under review is less likely to be spurious (more likely to be real), so the p-value decreases

** Exercise 12.3: F test

A linear regression Y_j =

The regression equation has one intercept, one explanatory variable, and five observations, so it has N – k – 1 = 5 – 1 – 1 = 3 degrees of freedom. The

RSS =

The F value = the regression sum of squares (RegSS / k) / (RSS / N – k – 1) = (RegSS / 1) / (4.3 / 3)

RegSS = (4.3 / 3) × 20.1628 = 28.900.

The R² of the regression is the regression sum of squares divided by the total sum of squares. The total sum of squares TSS = ResSS + RSS = 28.9 + 4.3 = 33.2, so the

R² = 28.9 / 33.2 = 0.87048.

The absolute value of the correlation is the square root of the R²:

0.87048 = 0.9330.

The t value for B, the ordinary least squares estimator for

value = 20.1628 = 4.4903

This t value is the ordinary least squares estimator for

standard error = 1.7 / 4.4903 = 0.3786.