TS Module 14 Model diagnostics residual analysis practice problems

(The attached PDF file has better formatting.)

Cryer and Chan chapter 8 model diagnostics residual analysis

** Exercise 14.1: MA(2) process

An AR(2) process is Y_t = ö₁ Y_t-1 + ö₂ Y_t-2 + è₀ + e_t.

ö₁ = 0.5, ö₂ = 0.3, and è₀ = 1

A.    What is the expected value for Y_T in terms of Y_t-1 and Y_t-2?

B.    What is the residual in Period T in terms of Y_T, Y_t-1, and Y_t-2?

C.    If Y₁ = 4, Y₂ = 6, and Y₃ = 5, what is e₃? 

Part A: The expected value of Y_t = ö₁ Y_t-1 + ö₂ Y_t-2 + è₀ = 0.5 Y_t-1 + 0.3 Y_t-2 + 1.

Part B: The residual in Period T = å_t = Y_t – (0.5 Y_t-1 + 0.3 Y_t-2 + 1).

Part C: 5 = 0.5 × 4 + 0.3 × 6 + 1 + e₃ = 4.8 ➾ e₃ = –0.2

This problem is simple. More complex problems use an ARMA(1,1) process. You figure out the residual in Period T from the actual and expected values and then use the residual to forecast the Period T+1 value.

** Exercise 14.2: Residuals

If a stationary ARMA process with n elements is correctly specified, which of the following are true?

A.    All the residuals = 0

B.    The sum of the residuals = 0

C.      (residual – mean residual)² / (n – 1) = ó²_e

D.    The residuals have (approximately) expected values of zero and constant variance

E.    The residuals approach zero asymptotically as n    

Part A: The residual å_t = y_t – ŷ_t. 

y_t is a random variable with a variance, so å_t is also a variance with a variance. It does not equal zero.

Part B: Each residual has a variance of ó²_ε. The residuals are independent, so the sum of the residuals has a variance of n × ó²_ε.

Jacob: In regression analysis, the sum of the residuals is zero. Why is that not true here?

Rachel: If we know the true regression coefficients, the sum of the residuals is not zero. If we don’t know the true regression coefficients, ordinary least squares estimation chooses values that force the sum of residuals to be zero. This exercise assumes the ARMA process is correctly specified: that is, the autoregressive and moving average parameters are correct, not the estimates from a sample.

Part C: As n ➝ ∞, ∑ (residual – mean residual)² / (n – 1) = ó²_e. In a finite sample with n < ∞, the equation is not exact.

Part D: The statement is correct. Each estimated value is the true expected value, so the residuals have expected values of zero. The variance of the residuals is constant: even if the residual is large in Period T, the residual in Period T+1 has an expected value of zero and the same variance as other residuals.

Part E: The residuals have a constant variance; they don’t approach zero asymptotically.

See Cryer & Chan, page 176: “If the model is correctly specified and the parameter estimates are reasonably close to the true values, then the residuals should have nearly the properties of white noise. They should behave roughly like independent, identically distributed normal variables with zero means and common standard deviations.”

** Exercise 14.3: An AR(1) process of 100 values (n = 100) has ö = 0.9.

A.                  What is the standard deviation of ?

B.      What is the standard deviation of ?

C.      What is the standard deviation of ?

Part A: The standard deviation of is ö / √n = 0.9 / 10 = 0.090.

See Cryer and Chan, page 180, equation 8.1.5.

Part B: The variance of is [1 – (1 – ö²) × ö² ] / n = [1 – (1 – 0.9²) × 0.9²] / 100 = 0.00846.

The standard deviation of is 0.00846^0.5 = 0.09198.

See Cryer and Chan, page 180, equation 8.1.6.

Part C: The standard deviation of is approximately 1 / √n = 0.10. This approximation is true for large n.

See Cryer and Chan, page 181.

** Exercise 14.4: Box-Pierce Q statistic

A.    What is the Box-Pierce Q statistic for a time series with n elements and k lags?

B.    What is the Ljung-Box Q statistic for a time series with n elements and k lags?

C.    Which is larger: the Box-Pierce Q statistic or the Ljung-Box Q statistic?

D.    What distribution does the Box-Pierce Q statistic have?

E.    How many degrees of freedom does the Box-Pierce Q statistic have?