TS Module 9 Non-stationary ARIMA time series

Neas-Seminars

TS Module 9 Non-stationary ARIMA time series

http://33771.hs2.instantasp.net/Topic12733.aspx

By NEAS - 8/31/2013 2:50:45 PM

TS Module 9 Non-stationary ARIMA time series

(The attached PDF file has better formatting.)

Time series ARIMA processes practice problems

** Exercise 9.1: ARIMA(0,1,1) process

What is an ARIMA(0,1,1) process?

Write an ARIMA(0,1,1) process as a series of error terms.

What is the variance of an ARIMA(0,1,1) process?

What is the mean of an ARIMA(0,1,1) process?

Part A:

An ARIMA(0,1,1) process is a once-integrated MA(1), called an IMA(1,1) process in the text.

The first difference of an ARIMA(0,1,1) process is an MA(1) process.

If Y_t is an ARIMA(0,1,1) process = an IMA(1,1) process, then

Part B:

Express Y_t in terms of Y_t-1 and Y_t-1 in terms of Y_t–2:

Y_t = Y_t-1 +

å_t – è å_t-1.

Y_t-1 = Y_t-2 +

å_t-1 – è å_t-2.

Y_t = Y_t-2 +

å_t + å_t-1 – è å_1-1 – è å_1-2 = Y_t-2 + å_t + (1 – è) å_1-1 – è å_1-2.

We continue in this fashion to get

Y_t =

å_t + (1 – è) å_1-1 + (1 – è) å_1-2 + (1 – è) å_1-3 + …

Part C:

The error terms

å_j are independent with a variance of ó²_t for each one. The variance of Y_t is

²_t + (1 – è)² × ó²_t + (1 – è)² × ó²_t + (1 – è)² × ó²_t + … = infinity.

Part D:

The mean of each error term

å_j is zero, so one is tempted to say that the mean of Y_t is zero. But the sum of an infinite number of random variables, each of which has a mean of zero, is not necessarily zero.

If the observed value of Y_t is 1, the expected value of Y_t+1 is 1; if the observed value of Y_t is 2, the expected value of Y_t+1 is 2. Given any observed value in the current period, the expected values in future periods differ. The values do not regress toward any point; that is, they have no mean.

Jacob:

What is the drift of an ARIMA process?

Rachel:

The drift of an ARIMA process with d = 1 is the mean of the underlying ARMA process. If the MA(1) process has a mean of

ì, the expected value of the ARIMA(0,1,1) process increases by ì each period.

ARIMA processes with fixed starting points.

Cryer and Chan begin with non-stationary processes that have a fixed starting point. If the time series has no starting point, its mean and variance are not defined. For an infinite ARIMA process, Y_t has no mean or variance. Each entry in the non-stationary time series is the sum of an infinite number of random variables that do not die out. To explain the pattern of these processes, we assume they begin at some time t.

This is not really a restrictive condition. Most commonly, we forecast future values of a time series based on historical observations. Before the first observation, we assume all values of the time series are zero. We model the evolution of the mean and variance of the process.

**Exercise 9.2: ARIMA(0,1,1) process

Suppose

with Y

_t = 0 for t < 1.

What is the variance of Y₁?

What is the variance of Y₂?

What is the variance of Y_T?

Part A:

Y₁ = Y₀ +

å₁ – è å₀ = 0 + å₁ – è å₀

The variance of

å is ó²_å, so the variance of Y₁ is (1 + è²) × ó²_t.

Part B:

Y₂ = Y₁ +

å₂ – è å₁

Y₁ = Y₀ +

å₁ – è å₀ = 0 + å₁ – è å₀, so Y₂ = å₂ + (1 – è) å₁ – è å₀

The variance of

å is ó²_å, so the variance of Y₂ is (1 + (1 – è)² + è²) × ó²_t.

Part C:

We continue this process T times, giving the variance of Y₂ = (1 + (T–1) × (1 –

è)² + è²) × ó²_t.

Jacob:

For an ARMA process, we evaluate the variance of Y_t, where t may be any element of the time series. Each element has the same mean, so it is a random variable, which has a variance.

For this ARIMA(0,1,1) process, Y₁, Y₂, and Y_T are specific observations; they are values, not random variables. How can they have variances?

Rachel:

We have formulated the exercise as Cryer and Chan do in their textbook. We are standing at time t=0, and we are projecting the values at times t=1, t=2, and t=T. Now these are random variables; each has a variance.

Jacob:

If we don’t specify that Y_t = 0 for t < 1, what is the variance of Y_t?

Rachel:

An ARIMA(0,1,1) process is not stationary and has no variance.

Cryer and Chan, P94: equation 5.2.7, for IMA(1,1) process, is

Cryer and Chan assume that

Y_t = 0 for periods t < –m. They assume we are now at Period 0, looking forward to estimate the variances at Periods 1, 2, 3, …. The first observed value is Period 1, but the process has been going on since Period –(m-1). We don’t observe the value at Period 0, but we know the time series process.

The Cryer and Chan scenario is complex, making it difficult to solve problems by first principles. Final exam problems use the scenario here: Y_t = 0 for t < 1.

** Exercise 9.3: ARIMA(0,1,1) process

Suppose

with

è = 0.4 and ó²_å = 4 for t > 0 and Y_t = 0 for t < 1.

What is the variance of Y₁?

What is the variance of Y₂?

What is the variance of Y₃?

Part A:

Y₁ = Y₀ +

å₁ – è å₀ = 0 + å₁ – è å₀

The variance of

å is 4, so the variance of Y₁ is (1 + 0.4²) × 4 = 4.64

Part B:

Y₂ = Y₁ +

å₂ – è å₁

Y₁ = Y₀ +

å₁ – è å₀ = 0 + å₁ – è å₀, so Y₂ = å₂ + (1 – è) å₁ – è å₀

The variance of

å is 4, so the variance of Y₂ is (1 + (1 – 0.4)² + 0.4²) × 4 = 6.08

Part C:

The variance of Y_T is (1 + (T – 1) × (1 –

è)² + è²) × ó²_t.

The variance of Y₃ is (1 + 2 × (1 – 0.4)²² + 0.4²) × 4 = 7.5200

** Exercise 9.4: ARIMA Process

A time series is Y_t =

á₁ Y_t-1 + á₂ Y_t-2 + (1 – á₁ – á₂) Y_t-3 + e_t + â₁ e_t-1 + â₂ e_t-2

Write the time series in terms of W_t ( Y_t).

What ARIMA process is this time series?

What are the

ö and è coefficients of this ARIMA process?

Part A:

Rewrite the ARIMA process as

Y_t – Y_t-1 = (

á₁ – 1) Y_t-1 + á₂ Y_t-2 + (1 – á₁ – á₂) Y_t-3 + e_t + â₁ e_t-1 + â₂ e_t-2

Y_t – Y_t-1 = (

á₁ – 1) Y_t-1 – (á₁ – 1) Y_t-2 + (á₁ – 1) Y_t-2 + á₂ Y_t-2 + (1 – á₁ – á₂) Y_t-3 + e_t + â₁ e_t-1 + â₂ e_t-2

Y_t – Y_t-1 = (

á₁ – 1) Y_t-1 – (á₁ – 1) Y_t-2 + (á₁ + á₂ – 1) Y_t-2 – (á₁ + á₂ – 1) Y_t-3 + e_t + â₁ e_t-1 + â₂ e_t-2

W_t = (

á₁ – 1) W_t-1 + (á₁ + á₂ – 1) W_t-2 + e_t + â₁ e_t-1 + â₂ e_t-2

Jacob:

What is the procedure for this transformation?

Rachel:

The sum of the coefficients for the Y terms are equal on both sides of the equation.

The left side has a coefficient of 1. The right side has coefficients of

á₁ + á₂ + (1 – á₁ – á₂) = 1.

Part B:

The time series is an ARIMA(2,1,2) process.

d = 1: we took one difference ( Y_t).

p = 2: we use W_t-1 and W_t-2.

q = 2: we use e_t-1 and e_t-2.

Part C:

The ARMA coefficients are

₁ = (á₁ – 1)

₂ = (á₁ + á₂ – 1)

₁ = –â₁

₂ = –â₂

Illustration:

A time series is Y_t = 1.4Y_t-1 + 0.1Y_t-2 – 0.5Y_t-3 + e_t + 0.3e_t-1 + 0.2e_t-2

Write the time series in terms of W_t ( Y_t).

What are the coefficients of this ARIMA process?

Part A:

Rewrite the ARIMA process as

Y_t – Y_t-1 = 0.4Y_t-1 + 0.1Y_t-2 – 0.5Y_t-3 + e_t + 0.3e_t-1 + 0.2e_t-2

= 0.4Y_t-1 – 0.4Y_t-2 + 0.4Y_t-2 + 0.1Y_t-2 – 0.5Y_t-3 + e_t + 0.3e_t-1 + 0.2e_t-2

= 0.4Y_t-1 – 0.4Y_t-2 + 0.5Y_t-2 – 0.5Y_t-3 + e_t + 0.3e_t-1 + 0.2e_t-2

W_t = 0.4 W_t-1 + 0.5W_t-2 + e_t + 0.3e_t-1 + 0.2e_t-2

Part B:

The ARIMA coefficients are

₁ = 0.4

₂ = 0.5

₁ = –0.3

₂ = –0.2

** Exercise 9.5: ARIMA Process

A time series is Y_t =

è₀ + á₁ × Y_t-1 + á₂ × Y_t-2 + á₃ × Y_t-3 + â₁ × e_t + â₂ × e_t-1 + â₃ × e_t-2

Write the time series in terms of W_t ( Y_t).

What is the ARIMA process followed by this time series?

Part A:

Rewrite the ARIMA process as

Y_t – Y_t-1 =

è₀ + (á₁ – 1) × Y_t-1 + á₂ × Y_t-2 + á₃ × Y_t-3 + â₁ × e_t + â₂ × e_t-1 + â₃ × e_t-2

Y_t–Y_t-1 =

è₀ + [(á₁ – 1) × Y_t-1 – (á₁ – 1) × Y_t-2] + [(á₁ – 1) × Y_t-2 + á₂ × Y_t-2 + á₃ × Y_t-3] + â₁ × e_t + â₂ × e_t-1 + â₃ ×e_t-2

The coefficient of Y_t-1 – Y_t-2 is (

á₁ – 1).

For this to be an ARIMA(2,1,2) process, we must have

(

á₁ – 1) + á₂ = –á₃

₁ + á₂ + á₃ = 1

Jacob:

What about the

â coefficients?

Rachel:

Any

â coefficients are fine.

â₁ = 1, then ö₁ = –â₂ and ö₂ = –â₃.

â₁ 1, then ö₁ = –â₂ / â₁ and ö₂ = –â₃ / â₁.

** Exercise 9.6: Non-Stationary Series

A time series Y_t = 2 Y_t-1 +

å_t has ó²_å = 3.

Y_t = 0 for t < 1.

What is the variance of Y_t for t = 1?

What is the variance of Y_t for t = 2?

What is the variance of Y_t for t = 3?

Is this time series stationary?

Part A:

Y₁ = 2 Y₀ +

å₁ = å₁, so the variance of Y₁ = the variance of å_t = ó²_å = 3.

Part B:

Y₂ = 2 Y₁ +

å₂ = 4 Y₀ + 2 å₁ + å₃ = å₁, so the variance of Y₂ = 2² × ó²_å + ó²_å = 15.

Part C:

The variance of Y_t is

ó²_å + 2² ó²_å + (2²)² ó²_å + … + (2²)^t-1

ó²_å × (2^{2 × t} – 1) / (2² – 1)

= × (4^t – 1) ×

ó²_å, which is equivalent to equation 5.1.4 on page 89.

The variance of Y₃ is × (64 – 1) × 3 = 63.

Part D:

A stationary time series has the same mean and variance for all values of t. The variance of this time series depends on t.

Jacob:

What if the exercise did not say that Y_t = 0 for t < 1. What the time series be stationary?

Rachel:

A stationary time series need havd no beginning. It is in a stochastic equilibrium: the mean and variance are the same in all periods. If this time series has no beginning, its variance is infinite and its has no mean.

Jacob:

Why does it have no mean? The mean of

å is zero, so isn’t the mean of Y_t also zero?

Rachel:

If Y_j = k, the expected value of Y_j+1 is 2 × k, and the expected value of Y_j+1 is 2 × 2 × k. if k is zero, these are all zero. But Y_j has infinite variance, so k could be anything.

** Exercise 9.7: Combining error terms

Suppose Y_t = M_t + e_t and M_t = M_t-1 +

å_t

Write Y_t as a function of M_t-1 and error terms.

What type of time series is M_t?

What type of time series is Y_t?

What is Y_t (the first difference of Y_t)?

What is the variance of Y_t?

What is the covariance of Y_t and Y_t-1?

What is

ñ₁, the autocorrelation of Y_t and Y_t-1?

Part A:

Y_t = M_t + e_t = M_t-1 + e_t +

å_t

Part B:

M_t is a random walk.

Part C:

Y_t = M_t-1 + e_t +

å_t = Y_t-1 + å_t + e_t – e_t-1. This is a random walk with a more complex error term.

Part D:

Y_t = Y_t – Y_t-1 = M_t-1 + e_t +

å_t – ( M_t-1 + e_t-1) = å_t + e_t – e_t-1

Part E:

If the random walk has no beginning, the variance is infinite, so it does not exist. If the random walk has a beginning, the variance depends on the period.

Part F:

The variance of Y_t = var(

å_t + e_t – e_t-1 ). The three random variables are independent, so the variance = 2ó²_e + ó²_å.

Part G:

The covariance of Y_t and Y_t-1 is covariance (

å_t + e_t – e_t-1 , å_t-1 + e_t-1 – e_t-2 ) = –ó²_e.

Part H:

The autocorrelation of Y_t and Y_t-1 (

ñ₁) is –ó²_e / 2ó²_e + ó²_å = –1 / (2 + ó²_å / ó²_e ). This is equation 5.1.10 on page 90.

Exercise 9.8: IMA(1,1) process

Each of the following time series is an IMA(1,1) process. What is the value of

è for each time series?

Y_t = Y_t-1 + e_t – 0.4e_t-1

Y_t = Y_t-1 – e_t – 0.4e_t-1

Y_t = Y_t-1 + 0.4e_t – 0.4e_t-1

Y_t = Y_t-1 – 0.4e_t – 0.4e_t-1

Part A:

The first difference of an IMA(1,1) is an MA(1) process.

The first difference of this time series is e_t – 0.4e_t-1, which is an MA(1) process with

è = 0.4.

Part B:

The first difference of this time series is – e_t – 0.4e_t-1. Use a change of the error term

å_t = –å_t, which gives a first difference of + e_t + 0.4e_{t -1}, which is an MA(1) process with è = –0.4.

Part C:

The first difference of this time series is 0.4 e_t – 0.4e_t-1. Use a change of the error term

å_t = 2.5å_t, which gives a first difference of + e_t – e_{t -1}, which is an MA(1) process with è = 1.

Part D:

The first difference of this time series is –0.4 e_t – 0.4e_t-1. Use a change of the error term

å_t = –2.5å_t, which gives a first difference of + e_t + e_{t -1}, which is an MA(1) process with è = –1.

(Cryer and Chan Page 93)

** Exercise 9.9: ARI(1,1) process

The time series Y_t =

è₀ + á Y_t-1 + â Y_t-2 + e_t is an ARI(1,1) process.

Write the time series in terms of W_t ( Y_t).

What is the relation of

á and â?

What is the value of

ö for this ARI(1,1) process?

Part A:

W_t = Y_t = Y_t – Y_t-1 =

è₀ + (á – 1) × Y_t-1 + â Y_t-2 + e_t

Part B:

á – 1 = –â, we can write the time series as Y_t – Y_t-1 = è₀ + (–â) × (Y_t-1 – Y_t-2) + e_t

Part C:

ö = –â = á – 1

** Exercise 9.10: Time series process

A time series is Y_t =

è₀ + 1.75 Y_t-1 – 0.75Y_t-2 + e_t is an ARI(1,1) process.

Write the time series in terms of W_t ( Y_t).

What is the value of

ö₁ for this ARI(1,1) process?

What is the value of

ö₂ for this ARI(1,1) process?

Part A:

W_t = Y_t = Y_t – Y_t-1 =

è₀ + (á – 1) × Y_t-1 + â Y_t-2 + e_t

Part B:

á – 1 = –â, we can write the time series as Y_t – Y_t-1 = è₀ + (–â) × (Y_t-1 – Y_t-2) + e_t

Part C:

ö = – â

** Exercise 9.11: IMA(1,1) process

An IMA(1,1) process is Y_t = Y_t-1 +

å_t – 0.4 å_t-1, with Y_t = 0 for t < 1.

Write Y_t as

â_t × å_t + â_t-1 × å_t-1 + … + â₁ × å₁ + â₀ × å₀

What is

â_t?

What is

â_t-1?

What is

â₁?

What is

â₀?

Part A:

Expand the time series period by period:

Y_t = Y_t-1 +

å_t – 0.4 å_t-1

Y_t-1 = Y_t-2 +

å_t-1 – 0.4 å_t-2

Y_t-2 = Y_t-3 +

å_t-2 – 0.4 å_t-3

The expanded time series is

Y_t =

å_t – 0.4 å_t-1 + å_t-1 – 0.4 å_t-2 + å_t-2 – 0.4 å_t-3 + … + Y₀ + å₁ – 0.4 å₀

Y₀ = 0, so we have finished expanding. We group error terms with the same subscript to get the

â_t values.

Part B:

â_t is the coefficient of the å_t term = 1.

Part C:

â_t-1 is the coefficient of the å_t-1 term = (1 – 0.4).

Part D:

â₁ is the coefficient of the å₁ term = (1 – 0.4).

Part E:

â₀ is the coefficient of the å₀ term = –0.4.