Module 7: Mixed autoregressive moving average (ARMA) practice problems
(The attached PDF file has better formatting.)
** Exercise 7.1: Mixed autoregressive moving average (ARMA) process
An ARMA(1,1) process has
ó2 = 1, è = –0.4, and ö = 0.8.
What is the value of
ã0?
What is the value of
ã1?
What is the value of
ñ1?
What is the value of
ñ2?
ARMA(1,1) means the process has 1
ö parameter and 1 è parameter. Know equations 4.4.3, 4.4.4, and 4.4.5 on page 78. These seem complex at first. Review the derivation in the textbook, so you recall the formulas.
After period 1,
ãk and ñk have exponential decay. ñ1 is ã1 / ã0. ã1 is a simple function of ã0, ö1, and è. The final exam problems test ñ1 most frequently.
Part A:
For an ARMA(1,1) process,
= ( 1 - 2 × 0.8 × -0.4 + (-0.4)
2 ) / (1 – (0.8)2 ) = 5
(See Cryer and Chan, chapter 4, page 78, equation 4.4.4)
Part B:
ã1 = ö ã0 – è ó2å = 0.8 × 5 – (-0.4) × 1 = 4.4
See Cryer and Chan, chapter 4, page 78, equation 4.4.3
Jacob:
How do we derive these formulas?
Rachel:
ã0 is the variance of Yt = E(Yt2).
To avoid superfluous parameters, subtract the mean from all values of the time series.
The variance of Yt, the covariances, and the autocorrelations do not change, and we don’t have to include a
ì parameter in the derivations.
For an ARMA(1,1) process with a mean of zero, Yt =
ö × Yt-1 + åt – è × åt-1
ã
0 = E(Yt2) = E(Yt × (ö × Yt-1 + åt – è × åt-1) = ö × E(Yt × Yt-1) + E(åt × Yt) – è × E(åt-1 × Yt)
ã
0 = (ö × ã1) + E(åt × Yt) – è × E(åt-1 × Yt)
Also:
ã1 = E(Yt-1 × Yt) = E(Yt-1 × (ö × Yt-1 + åt – è × åt-1) = ö × E(Yt-1 × Yt-1) + E(åt × Yt-1) – è × E(åt-1 × Yt-1).
For a stationary time series, E(Yt-1 × Yt-1) = E(Yt × Yt) =
ã0 and E(åt-1 × Yt-1) = E(åt × Yt), so
ã
1 = (ö × ã0) + E(åt × Yt-1) – è × E(åt × Yt)
If we can solve for E(
åt × Yt) and E(åt-1 × Yt) in terms of the ARMA(1,1) parameters ö, è, and ó2t, we have two equations in two unknowns (ã0 and ã1).
å
t is uncorrelated with Yt-1 and with åt-1, so
E(
åt × Yt) = E(åt × (ö × Yt-1 + åt – è × åt-1) ) = 0 + ó2t + 0 = ó2t
Similarly,
åt-1 is uncorrelated with åt and the correlation of åt-1 with Yt-1 is ó2t, so
E(
åt-1 × Yt) = E(åt-1 × (ö × Yt-1 + åt – è × åt-1) ) = ö × ó2t + 0 – è × ó2t = (ö – è) × ó2t
We now have two equations in two unknowns:
ã
0 = (ö × ã1) + ó2t – (è × (ö – è) × ó2t)
ã
0 = (ö × ã1) + ( [1 – è × (ö – è) ] × ó2t)
ã
1 = (ö × ã0) – (è × ó2t)
We use the expression for
ã1 in the formula for ã0 to get
ã
0 = (ö × [(ö × ã0) – (è × ó2t)] ) + ( [1 – è × (ö – è) ] × ó2t)
(1 –
ö2) × ã0 = (ö × (è × ó2t)] ) + ( [1 – è × (ö – è) ] × ó2t)
ã
0 = (1 – 2èö – è2) × ó2t) / (1 – ö2)
We express
ã1 as a function of ö, è, and ó2t:
ã
1 = (ö × ã0) – (è × ó2t) = { [ö × (1 – 2èö – è2) ] – [è × (1 – ö2) ] } × ó2t) / (1 – ö2)
ã
1 = [ (ö – è – 2 è ö2 + è ö2) × ] ó2t / (1 – ö2)
ã
1 = [ (ö – è) × (1 – 2èö2 + èö2) ] × ó2t/ (1 – ö2)
We express
ñ1 as a function of ö and è:
ñ
1 = ã1 / ã0. Both ã0 and ã1 have the multiplicative term ó2t / (1 – ö2), which cancels out of the ratio for ñ1:
ñ
1 = [ (ö – è) × (1 – 2èö2 + èö2) ] / (1 – 2èö – è2)
The moving average effect dies out after one period, and the autoregressive effect decays exponentially, so
ñ
k = ñk-1 × ö
ñ
k = ñ1 × ök-1
See Cryer and Chan, chapter 4, pages 77-78, equations 4.4.2-4.4.4. Cryer and Chan write the formulas for
ã0, ã1, and ñk. Rachel shows the derivation in more detail. Once you have read Rachel’s dialogue, you can see the intuition for the expressions. The equations appears on final exam problems, so follow Rachel’s derivation.
Part C:
For an ARMA(1,1) process:
Since we have solved for
ã0 and ã1, we use the relation ñ1 = ã1 / ã0 = 4.4 / 5 = 0.88
In practice, you use the autocorrelations
ñk more frequently than the autocovariances ãk.
Part D:
The autocorrelations
ñk decline geometrically at a rate ö ñ2 = ñ1 × ö = 0.88 × 0.8 = 0.704
In the module on method of moments, we start with sample autocorrelations of 0.88 and 0.704 and derive estimates of the
ö and è parameters.
** Exercise 7.2: Moving average and autoregressive processes
An autoregressive process with mean zero is Yt =
ö Yt-1 + åt.
Re-write this time series as a moving average process of infinite lag.
Solution 7.2:
Yt = et +
ö Yt-1
= et +
ö (et-1 + ö Yt-2)
= et +
ö et-1 + ö2 (et-2 + ö Yt-3)
Continue expanding to eliminate all the Yt and remain with an infinite series of
å’s.
Yt = et +
ö et-1 + ö2 et-2 + ö3å3 + …
See Cryer and Chan, chapter 4, page 70, equation 4.3.8
Cryer and Chan refer to the coefficients of the
å’s as ø weights or filter representation.
Jacob:
Why would we want to express an AR(1) process as a moving average process of infinite lag?
Rachel:
The values of an AR(1) process are autocorrelated. The
å terms are independent, so we can easily compute the variance of the autoregressive process as
variance (Yt) = (1 +
ö + ö2 + ö3 + … ) × ó2t = ó2t / (1 – ö2).
* Question 7.3: ARMA(1,1)
An ARMA(1,1) process has
ñ1 = 0. Which of the following is true?
ö
1 = è1 or ö1 × è1 = 1
ö
1 = è1 and ö1 × è1 = 1
ö
1 = –è1 or ö1 × è1 = –1
ö
1 = –è1 and ö1 × è1 = –1
ö
1 = è1 and ö1 × è1 = –1
Answer 7.3: A
For an ARMA(1,1) process:
for k 1
Notation:
For AR(1), MA(1), and ARMA(1,1) processes, Cryer and Chan drop the subscripts on the parameters, using
ö and è instead of ö1 and è1. The final exam problems may use either notation.
Intuition:
If the residual in period t increases one unit:
Moving average:
è1 causes the forecast to decrease è1 units
Autoregressive:
ö1 causes the forecast to increase ö1 units
If
ö1 = è1, these two effects offset each other. A change in the period t value does not affect the period t+1 value, so ñ1 = 0.
è
1 and 1/è1 produce the same autocorrelation function.
ö
1 = è1 has the same effect as ö1 = 1/è1 or ö1 × è1 = 1.