TS module 12 method of moments for 3 observations

Estimating parameters for time series with few observations

(The attached PDF file has better formatting.)

If a time series has only two or three observations, we estimate the parameters by first principles. For moving average models, we do not know past residuals. For simple estimates, we assume residuals are zero and values equal the mean for periods before the first observation. Maximum likelihood estimation gives more exact results, but you are not responsible for the maximum likelihood procedures in this course.

This posting shows example for an MA(1) process and an IMA(1,1) process.

Exercise 12.1: MA(1) process

An MA(1) process with ì = 0 has Y₁ = 0, Y₂ = –1, and Y₃ = ½.

A.      What are the residuals in periods 1, 2, and 3? Explain why we don’t know the true residuals.

B.      If the residuals before Period 1 are zero, what are the expected values in periods 1, 2, and 3?

C.      If the residuals before Period 1 are zero, what are the residuals in periods 1, 2, and 3?

D.     What is the estimate of è?

E.      What is the estimated ó²?

Part A: The residual in period t depends on the residual in period t-1. For example,

           If the residual in period 0 is 1, the expected value in period 1 is 0 – 1è = –1 è.

           The residual in period 1 is 0 – (–1 è) = è.

           If the residual in period 0 is å₀, the expected value in period 1 is 0 – è å₀.

           The residual in period 1 is 0 – (–è å₀) = è å₀.

Even if we know è, we don’t know any of the residuals, since each residual depends on the previous one.

Part B: One solution (which is best for pencil and paper calculations) is to assume the residuals in periods before the first observed value are zero. This assumption is not the maximum likelihood estimate of è (that is, we can do better), but it gives reasonably close answers.

Parts B and C: Compute expected values and residuals period by period.

           The expected value in Period 1 is ì – è å₀ = 0.

           The residual in Period 1 is actual – expected = 0 – 0 = 0

           The expected value in Period 2 is ì – è å₁ = 0.

           The residual in Period 2 is actual – expected = –1 – 0 = –1

           The expected value in Period 3 is ì – è å₂ = 1è

           The residual in Period 3 is actual – expected = ½ – è

Part D: In periods 1 and 2, the expected values and residuals are not functions of è. If the mean of the MA(1) process is zero, all residuals before Period 1 are zero, and Y₁ = 0, the residual in period 1 is zero. The expected value in period 2 is ì + 0 × è = 0 regardless of the value of è. The residual in period 2 is –1.

The expected value of the residuals is zero.

           The residual in period 2 is –1, so the expected value in period 3 is 0 – (–1) × è.

           The actual value is Y₃ = ½, so we equate 0 + 1 è = ½   è = ½.

Part E: We estimate the noise variance ó² from the observed variance of the three Y values. We know that the mean of the Y values is zero, so the variance is

⅓ × [ (0 – 0)² + (–1 – 0)² + (½ – 0)² ] = 1.25 / 3 = 0.41667

ó² = 0.41667 / (1 + è²) = 0.41667 / 1.25 = 0.333

Intuition: The value in Period t has the random fluctuation from Period t minus half the random fluctuation from Period t-1. The random fluctuations are independent, so the variance of Y_t is (1 + (–½)² ) × ó² = 1.25 ó². 

Exercise 12.2: An IMA(1,1) process with no drift (that is, the underlying MA(1) process has a mean of zero) has Y₁ = 10, Y₂ = 9, and Y₃ = 9.5.

A.      What are the value of the underlying MA(1) process?

B.      What is the estimated è of the underlying MA(1) process?

Part A: The underlying MA(1) process is the first difference of the IMA(1,1) process.

           We don’t know the value in period 1.

           The value in Period 2 is 9 – 10 = –1.

           The value in Period 3 is 9.5 – 9 = 0.5.