TS Module 9 Non-stationary ARIMA time series
(The attached PDF file has better formatting.)
Time series ARIMA processes practice problems
** Exercise 9.1: ARIMA(0,1,1) process
What is an ARIMA(0,1,1) process?
Write an ARIMA(0,1,1) process as a series of error terms.
What is the variance of an ARIMA(0,1,1) process?
What is the mean of an ARIMA(0,1,1) process?
Part A:
An ARIMA(0,1,1) process is a once-integrated MA(1), called an IMA(1,1) process in the text.
The first difference of an ARIMA(0,1,1) process is an MA(1) process.
If Yt is an ARIMA(0,1,1) process = an IMA(1,1) process, then
![](http://www.neas-seminars.com/Discussions/Image17.gif)
Part B:
Express Yt in terms of Yt-1 and Yt-1 in terms of Yt–2:
Yt = Yt-1 +
åt – è åt-1.
Yt-1 = Yt-2 +
åt-1 – è åt-2.
Yt = Yt-2 +
åt + åt-1 – è å1-1 – è å1-2 = Yt-2 + åt + (1 – è) å1-1 – è å1-2.
We continue in this fashion to get
Yt =
åt + (1 – è) å1-1 + (1 – è) å1-2 + (1 – è) å1-3 + …
Part C:
The error terms
åj are independent with a variance of ó2t for each one. The variance of Yt is
ó
2t + (1 – è)2 × ó2t + (1 – è)2 × ó2t + (1 – è)2 × ó2t + … = infinity.
Part D:
The mean of each error term
åj is zero, so one is tempted to say that the mean of Yt is zero. But the sum of an infinite number of random variables, each of which has a mean of zero, is not necessarily zero.
If the observed value of Yt is 1, the expected value of Yt+1 is 1; if the observed value of Yt is 2, the expected value of Yt+1 is 2. Given any observed value in the current period, the expected values in future periods differ. The values do not regress toward any point; that is, they have no mean.
Jacob:
What is the drift of an ARIMA process?
Rachel:
The drift of an ARIMA process with d = 1 is the mean of the underlying ARMA process. If the MA(1) process has a mean of
ì, the expected value of the ARIMA(0,1,1) process increases by ì each period.
ARIMA processes with fixed starting points.
Cryer and Chan begin with non-stationary processes that have a fixed starting point. If the time series has no starting point, its mean and variance are not defined. For an infinite ARIMA process, Yt has no mean or variance. Each entry in the non-stationary time series is the sum of an infinite number of random variables that do not die out. To explain the pattern of these processes, we assume they begin at some time t.
This is not really a restrictive condition. Most commonly, we forecast future values of a time series based on historical observations. Before the first observation, we assume all values of the time series are zero. We model the evolution of the mean and variance of the process.
**Exercise 9.2: ARIMA(0,1,1) process
Suppose
![](http://www.neas-seminars.com/Discussions/Image17.gif)
with Y
t = 0 for t < 1.
What is the variance of Y1?
What is the variance of Y2?
What is the variance of YT?
Part A:
Y1 = Y0 +
å1 – è å0 = 0 + å1 – è å0
The variance of
å is ó2å, so the variance of Y1 is (1 + è2) × ó2t.
Part B:
Y2 = Y1 +
å2 – è å1
Y1 = Y0 +
å1 – è å0 = 0 + å1 – è å0, so Y2 = å2 + (1 – è) å1 – è å0
The variance of
å is ó2å, so the variance of Y2 is (1 + (1 – è)2 + è2) × ó2t.
Part C:
We continue this process T times, giving the variance of Y2 = (1 + (T–1) × (1 –
è)2 + è2) × ó2t.
Jacob:
For an ARMA process, we evaluate the variance of Yt, where t may be any element of the time series. Each element has the same mean, so it is a random variable, which has a variance.
For this ARIMA(0,1,1) process, Y1, Y2, and YT are specific observations; they are values, not random variables. How can they have variances?
Rachel:
We have formulated the exercise as Cryer and Chan do in their textbook. We are standing at time t=0, and we are projecting the values at times t=1, t=2, and t=T. Now these are random variables; each has a variance.
Jacob:
If we don’t specify that Yt = 0 for t < 1, what is the variance of Yt?
Rachel:
An ARIMA(0,1,1) process is not stationary and has no variance.
Cryer and Chan, P94: equation 5.2.7, for IMA(1,1) process, is
= ![](http://www.neas-seminars.com/Discussions/Image19.gif)
Cryer and Chan assume that
Yt = 0 for periods t < –m. They assume we are now at Period 0, looking forward to estimate the variances at Periods 1, 2, 3, …. The first observed value is Period 1, but the process has been going on since Period –(m-1). We don’t observe the value at Period 0, but we know the time series process.
The Cryer and Chan scenario is complex, making it difficult to solve problems by first principles. Final exam problems use the scenario here: Yt = 0 for t < 1.
** Exercise 9.3: ARIMA(0,1,1) process
Suppose
![](http://www.neas-seminars.com/Discussions/Image17.gif)
with
è = 0.4 and ó2å = 4 for t > 0 and Yt = 0 for t < 1.
What is the variance of Y1?
What is the variance of Y2?
What is the variance of Y3?
Part A:
Y1 = Y0 +
å1 – è å0 = 0 + å1 – è å0
The variance of
å is 4, so the variance of Y1 is (1 + 0.42) × 4 = 4.64
Part B:
Y2 = Y1 +
å2 – è å1
Y1 = Y0 +
å1 – è å0 = 0 + å1 – è å0, so Y2 = å2 + (1 – è) å1 – è å0
The variance of
å is 4, so the variance of Y2 is (1 + (1 – 0.4)2 + 0.42) × 4 = 6.08
Part C:
The variance of YT is (1 + (T – 1) × (1 –
è)2 + è2) × ó2t.
The variance of Y3 is (1 + 2 × (1 – 0.4)22 + 0.42) × 4 = 7.5200
** Exercise 9.4: ARIMA Process
A time series is Yt =
á1 Yt-1 + á2 Yt-2 + (1 – á1 – á2) Yt-3 + et + â1 et-1 + â2 et-2
Write the time series in terms of Wt ( Yt).
What ARIMA process is this time series?
What are the
ö and è coefficients of this ARIMA process?
Part A:
Rewrite the ARIMA process as
Yt – Yt-1 = (
á1 – 1) Yt-1 + á2 Yt-2 + (1 – á1 – á2) Yt-3 + et + â1 et-1 + â2 et-2
Yt – Yt-1 = (
á1 – 1) Yt-1 – (á1 – 1) Yt-2 + (á1 – 1) Yt-2 + á2 Yt-2 + (1 – á1 – á2) Yt-3 + et + â1 et-1 + â2 et-2
Yt – Yt-1 = (
á1 – 1) Yt-1 – (á1 – 1) Yt-2 + (á1 + á2 – 1) Yt-2 – (á1 + á2 – 1) Yt-3 + et + â1 et-1 + â2 et-2
Wt = (
á1 – 1) Wt-1 + (á1 + á2 – 1) Wt-2 + et + â1 et-1 + â2 et-2
Jacob:
What is the procedure for this transformation?
Rachel:
The sum of the coefficients for the Y terms are equal on both sides of the equation.
The left side has a coefficient of 1. The right side has coefficients of
á1 + á2 + (1 – á1 – á2) = 1.
Part B:
The time series is an ARIMA(2,1,2) process.
d = 1: we took one difference ( Yt).
p = 2: we use Wt-1 and Wt-2.
q = 2: we use et-1 and et-2.
Part C:
The ARMA coefficients are
ö
1 = (á1 – 1)
ö
2 = (á1 + á2 – 1)
è
1 = –â1
è
2 = –â2
Illustration:
A time series is Yt = 1.4Yt-1 + 0.1Yt-2 – 0.5Yt-3 + et + 0.3et-1 + 0.2et-2
Write the time series in terms of Wt ( Yt).
What are the coefficients of this ARIMA process?
Part A:
Rewrite the ARIMA process as
Yt – Yt-1 = 0.4Yt-1 + 0.1Yt-2 – 0.5Yt-3 + et + 0.3et-1 + 0.2et-2
= 0.4Yt-1 – 0.4Yt-2 + 0.4Yt-2 + 0.1Yt-2 – 0.5Yt-3 + et + 0.3et-1 + 0.2et-2
= 0.4Yt-1 – 0.4Yt-2 + 0.5Yt-2 – 0.5Yt-3 + et + 0.3et-1 + 0.2et-2
Wt = 0.4 Wt-1 + 0.5Wt-2 + et + 0.3et-1 + 0.2et-2
Part B:
The ARIMA coefficients are
ö
1 = 0.4
ö
2 = 0.5
è
1 = –0.3
è
2 = –0.2
** Exercise 9.5: ARIMA Process
A time series is Yt =
è0 + á1 × Yt-1 + á2 × Yt-2 + á3 × Yt-3 + â1 × et + â2 × et-1 + â3 × et-2
Write the time series in terms of Wt ( Yt).
What is the ARIMA process followed by this time series?
Part A:
Rewrite the ARIMA process as
Yt – Yt-1 =
è0 + (á1 – 1) × Yt-1 + á2 × Yt-2 + á3 × Yt-3 + â1 × et + â2 × et-1 + â3 × et-2
Yt–Yt-1 =
è0 + [(á1 – 1) × Yt-1 – (á1 – 1) × Yt-2] + [(á1 – 1) × Yt-2 + á2 × Yt-2 + á3 × Yt-3] + â1 × et + â2 × et-1 + â3 ×et-2
The coefficient of Yt-1 – Yt-2 is (
á1 – 1).
For this to be an ARIMA(2,1,2) process, we must have
(
á1 – 1) + á2 = –á3
á
1 + á2 + á3 = 1
Jacob:
What about the
â coefficients?
Rachel:
Any
â coefficients are fine.
If
â1 = 1, then ö1 = –â2 and ö2 = –â3.
If
â1 1, then ö1 = –â2 / â1 and ö2 = –â3 / â1.
** Exercise 9.6: Non-Stationary Series
A time series Yt = 2 Yt-1 +
åt has ó2å = 3.
Yt = 0 for t < 1.
What is the variance of Yt for t = 1?
What is the variance of Yt for t = 2?
What is the variance of Yt for t = 3?
Is this time series stationary?
Part A:
Y1 = 2 Y0 +
å1 = å1, so the variance of Y1 = the variance of åt = ó2å = 3.
Part B:
Y2 = 2 Y1 +
å2 = 4 Y0 + 2 å1 + å3 = å1, so the variance of Y2 = 22 × ó2å + ó2å = 15.
Part C:
The variance of Yt is
ó2å + 22 ó2å + (22)2 ó2å + … + (22)t-1
=
ó2å × (22 × t – 1) / (22 – 1)
= × (4t – 1) ×
ó2å, which is equivalent to equation 5.1.4 on page 89.
The variance of Y3 is × (64 – 1) × 3 = 63.
Part D:
A stationary time series has the same mean and variance for all values of t. The variance of this time series depends on t.
Jacob:
What if the exercise did not say that Yt = 0 for t < 1. What the time series be stationary?
Rachel:
A stationary time series need havd no beginning. It is in a stochastic equilibrium: the mean and variance are the same in all periods. If this time series has no beginning, its variance is infinite and its has no mean.
Jacob:
Why does it have no mean? The mean of
å is zero, so isn’t the mean of Yt also zero?
Rachel:
If Yj = k, the expected value of Yj+1 is 2 × k, and the expected value of Yj+1 is 2 × 2 × k. if k is zero, these are all zero. But Yj has infinite variance, so k could be anything.
** Exercise 9.7: Combining error terms
Suppose Yt = Mt + et and Mt = Mt-1 +
åt
Write Yt as a function of Mt-1 and error terms.
What type of time series is Mt?
What type of time series is Yt?
What is Yt (the first difference of Yt)?
What is the variance of Yt?
What is the variance of Yt?
What is the covariance of Yt and Yt-1?
What is
ñ1, the autocorrelation of Yt and Yt-1?
Part A:
Yt = Mt + et = Mt-1 + et +
åt
Part B:
Mt is a random walk.
Part C:
Yt = Mt-1 + et +
åt = Yt-1 + åt + et – et-1. This is a random walk with a more complex error term.
Part D:
Yt = Yt – Yt-1 = Mt-1 + et +
åt – ( Mt-1 + et-1) = åt + et – et-1
Part E:
If the random walk has no beginning, the variance is infinite, so it does not exist. If the random walk has a beginning, the variance depends on the period.
Part F:
The variance of Yt = var(
åt + et – et-1 ). The three random variables are independent, so the variance = 2ó2e + ó2å.
Part G:
The covariance of Yt and Yt-1 is covariance (
åt + et – et-1 , åt-1 + et-1 – et-2 ) = –ó2e.
Part H:
The autocorrelation of Yt and Yt-1 (
ñ1) is –ó2e / 2ó2e + ó2å = –1 / (2 + ó2å / ó2e ). This is equation 5.1.10 on page 90.
**
Exercise 9.8: IMA(1,1) process
Each of the following time series is an IMA(1,1) process. What is the value of
è for each time series?
Yt = Yt-1 + et – 0.4et-1
Yt = Yt-1 – et – 0.4et-1
Yt = Yt-1 + 0.4et – 0.4et-1
Yt = Yt-1 – 0.4et – 0.4et-1
Part A:
The first difference of an IMA(1,1) is an MA(1) process.
The first difference of this time series is et – 0.4et-1, which is an MA(1) process with
è = 0.4.
Part B:
The first difference of this time series is – et – 0.4et-1. Use a change of the error term
åt = –åt, which gives a first difference of + et + 0.4et -1, which is an MA(1) process with è = –0.4.
Part C:
The first difference of this time series is 0.4 et – 0.4et-1. Use a change of the error term
åt = 2.5åt, which gives a first difference of + et – et -1, which is an MA(1) process with è = 1.
Part D:
The first difference of this time series is –0.4 et – 0.4et-1. Use a change of the error term
åt = –2.5åt, which gives a first difference of + et + et -1, which is an MA(1) process with è = –1.
(Cryer and Chan Page 93)
** Exercise 9.9: ARI(1,1) process
The time series Yt =
è0 + á Yt-1 + â Yt-2 + et is an ARI(1,1) process.
Write the time series in terms of Wt ( Yt).
What is the relation of
á and â?
What is the value of
ö for this ARI(1,1) process?
Part A:
Wt = Yt = Yt – Yt-1 =
è0 + (á – 1) × Yt-1 + â Yt-2 + et
Part B:
If
á – 1 = –â, we can write the time series as Yt – Yt-1 = è0 + (–â) × (Yt-1 – Yt-2) + et
Part C:
ö = –â = á – 1
** Exercise 9.10: Time series process
A time series is Yt =
è0 + 1.75 Yt-1 – 0.75Yt-2 + et is an ARI(1,1) process.
Write the time series in terms of Wt ( Yt).
What is the value of
ö1 for this ARI(1,1) process?
What is the value of
ö2 for this ARI(1,1) process?
Part A:
Wt = Yt = Yt – Yt-1 =
è0 + (á – 1) × Yt-1 + â Yt-2 + et
Part B:
If
á – 1 = –â, we can write the time series as Yt – Yt-1 = è0 + (–â) × (Yt-1 – Yt-2) + et
Part C:
ö = – â
** Exercise 9.11: IMA(1,1) process
An IMA(1,1) process is Yt = Yt-1 +
åt – 0.4 åt-1, with Yt = 0 for t < 1.
Write Yt as
ât × åt + ât-1 × åt-1 + … + â1 × å1 + â0 × å0
What is
ât?
What is
ât-1?
What is
â1?
What is
â0?
Part A:
Expand the time series period by period:
Yt = Yt-1 +
åt – 0.4 åt-1
Yt-1 = Yt-2 +
åt-1 – 0.4 åt-2
Yt-2 = Yt-3 +
åt-2 – 0.4 åt-3
The expanded time series is
Yt =
åt – 0.4 åt-1 + åt-1 – 0.4 åt-2 + åt-2 – 0.4 åt-3 + … + Y0 + å1 – 0.4 å0
Y0 = 0, so we have finished expanding. We group error terms with the same subscript to get the
ât values.
Part B:
ât is the coefficient of the åt term = 1.
Part C:
ât-1 is the coefficient of the åt-1 term = (1 – 0.4).
Part D:
â1 is the coefficient of the å1 term = (1 – 0.4).
Part E:
â0 is the coefficient of the å0 term = –0.4.