TS module 16 ARIMA forecasting with binary residuals
(The attached PDF file has better formatting.)
Standard ARMA processes assume a normal distribution for the residuals. This exercise uses a binary residual to clarify the variance of ARMA vs ARIMA processes.
Exercise 16.1: Binary Residuals: ARIMA vs ARMA
An MA(1) process has è1 = 1. The residual in each period is 1 or –1 with 50% chance of each. An ARIMA(0,1,1) process is the cumulative sum of the MA(1) process.
A. What are the variances of the 1, 2, & 3 period ahead forecasts for the ARMA process?
B. What are the variances of the 1, 2, & 3 period ahead forecasts for the ARIMA process?
Part A: The variance of the error term is [12 + (–1)2 ] / 2 = 1. This is a population variance, so we divide by N, not by N-1.
In a sample variance, the error terms are not necessarily +1 and –1.
The population variance uses the distribution of error terms.
The one period ahead forecast for the ARMA process is ì + åt – è1 × åt-1.
The parameters ì and è1 are scalars with no variance.
åt-1 has already occurred, so it is also a scalar.
åt has not yet occurred, so it is a random variable with a variance of 1.
➾ The variance of ì + åt – è1 × åt-1 is 1.
Part B: The one period ahead forecast for the ARIMA process is the one period ahead forecast for the ARMA process plus the most recent value of the ARIMA process.
The most recent value of the ARIMA process has already occurred, so it is a scalar.
The variance of the one period ahead forecast of the ARIMA process is the same as the variance of the one period ahead forecast of the ARMA process.
Part C: The two periods ahead forecast for the ARMA process is ì + åt+1 – è1 × åt.
The parameters ì and è1 are scalars with no variance.
åt and åt+1 have not yet occurred.
They are independent random variables with variances of 1.
The variance of è1 × åt-1 is è12 × var(åt-1).
The variance of the difference of two independent random variables is the sum of the variances of each random variable.
➾ The variance of ì + åt – è1 × åt-1 is 1 + è12 × 1 = 1 × (1 + è12) = 2, since è1 = 1.
Part D: The two periods ahead forecast for the ARMA process is the sum of
The most recent value of the ARIMA process: Yt-1
The one period ahead forecast of the ARMA process.
The two periods ahead forecast of the ARMA process.
= Yt-1 + ì + åt – è1 × åt-1 + ì + åt+1 – è1 × åt
= [Yt-1 + 2 × ì – è1 × åt-1] + [åt+1 + (1 – è1) × åt]
We have grouped the two periods ahead forecast into
a sum of scalars (with no variance) and
a sum of random variables.
è1 = 1 so (1 – è1) = 0, and the variance of the sum of random variables is var(åt+1).
Intuition: Let ì = 0 and Yt-1 = 0. The table below shows the possible values of the two periods ahead value of the ARMA process.
| Residuals | Values |
Scenario | åt | åt+1 | ARMA(t) | ARMA(t+1) | ARIMA(t+1) |
1 | -1 | -1 | -1 | 0 | -1 |
2 | -1 | 1 | -1 | 2 | 1 |
3 | 1 | -1 | 1 | -2 | -1 |
4 | 1 | 1 | 1 | 0 | 1 |
For the ARMA process in Period t+1:
The mean value is (2 + 0 + 0 + –2) / 4 = 0.
The variance is (22 + 02 + 02 + (–2)2 ) / 4 = 8 / 4 = 2.
For the ARIMA process in Period t+1:
The mean value is (–1 + 1 + –1 + 1) / 4 = 0.
The variance is ((–1)2 + 12 + (–1)2 + 12 ) / 4 = 4 / 4 = 1.