TS module 12 method of moments for 3 observations
Estimating parameters for time series with few observations
(The attached PDF file has better formatting.)
If a time series has only two or three observations, we estimate the parameters by first principles. For moving average models, we do not know past residuals. For simple estimates, we assume residuals are zero and values equal the mean for periods before the first observation. Maximum likelihood estimation gives more exact results, but you are not responsible for the maximum likelihood procedures in this course.
This posting shows example for an MA(1) process and an IMA(1,1) process.
Exercise 12.1: MA(1) process
An MA(1) process with ì = 0 has Y1 = 0, Y2 = –1, and Y3 = ½.
A. What are the residuals in periods 1, 2, and 3? Explain why we don’t know the true residuals.
B. If the residuals before Period 1 are zero, what are the expected values in periods 1, 2, and 3?
C. If the residuals before Period 1 are zero, what are the residuals in periods 1, 2, and 3?
D. What is the estimate of è?
E. What is the estimated ó2?
Part A: The residual in period t depends on the residual in period t-1. For example,
If the residual in period 0 is 1, the expected value in period 1 is 0 – 1è = –1 è.
The residual in period 1 is 0 – (–1 è) = è.
If the residual in period 0 is å0, the expected value in period 1 is 0 – è å0.
The residual in period 1 is 0 – (–è å0) = è å0.
Even if we know è, we don’t know any of the residuals, since each residual depends on the previous one.
Part B: One solution (which is best for pencil and paper calculations) is to assume the residuals in periods before the first observed value are zero. This assumption is not the maximum likelihood estimate of è (that is, we can do better), but it gives reasonably close answers.
Parts B and C: Compute expected values and residuals period by period.
The expected value in Period 1 is ì – è å0 = 0.
The residual in Period 1 is actual – expected = 0 – 0 = 0
The expected value in Period 2 is ì – è å1 = 0.
The residual in Period 2 is actual – expected = –1 – 0 = –1
The expected value in Period 3 is ì – è å2 = 1è
The residual in Period 3 is actual – expected = ½ – è
Part D: In periods 1 and 2, the expected values and residuals are not functions of è. If the mean of the MA(1) process is zero, all residuals before Period 1 are zero, and Y1 = 0, the residual in period 1 is zero. The expected value in period 2 is ì + 0 × è = 0 regardless of the value of è. The residual in period 2 is –1.
The expected value of the residuals is zero.
The residual in period 2 is –1, so the expected value in period 3 is 0 – (–1) × è.
The actual value is Y3 = ½, so we equate 0 + 1 è = ½ è = ½.
Part E: We estimate the noise variance ó2 from the observed variance of the three Y values. We know that the mean of the Y values is zero, so the variance is
⅓ × [ (0 – 0)2 + (–1 – 0)2 + (½ – 0)2 ] = 1.25 / 3 = 0.41667
ó2 = 0.41667 / (1 + è2) = 0.41667 / 1.25 = 0.333
Intuition: The value in Period t has the random fluctuation from Period t minus half the random fluctuation from Period t-1. The random fluctuations are independent, so the variance of Yt is (1 + (–½)2 ) × ó2 = 1.25 ó2.
Exercise 12.2: An IMA(1,1) process with no drift (that is, the underlying MA(1) process has a mean of zero) has Y1 = 10, Y2 = 9, and Y3 = 9.5.
A. What are the value of the underlying MA(1) process?
B. What is the estimated è of the underlying MA(1) process?
Part A: The underlying MA(1) process is the first difference of the IMA(1,1) process.
We don’t know the value in period 1.
The value in Period 2 is 9 – 10 = –1.
The value in Period 3 is 9.5 – 9 = 0.5.
Part B: The solution is similar to that in the previous exercise. We assume that all residuals in prior periods are zero and all values are the mean, which is zero in this exercise. If so, the observed values are 0, –1, and 0.5, which are the same as in the previous exercise.