5.11 Appendix

5.11.1 Derivations for AR(1) Model

The model is stated

\[Y_t = \delta + \phi_1Y_{t-1} + W_t\] where \(W_t \stackrel{iid}{\sim} N(0,\sigma^2_w)\) is independent Gaussian white noise with mean 0 and constant variance, \(\sigma^2_w\).

5.11.1.1 Approach 1: Assume Stationarity

Expected Value

\[E(Y_t) = E(\delta + \phi_1 Y_{t-1} + W_t) = E(\delta) + E(\phi_1 Y_{t-1})+ E(W_t) \] \[= \delta + \phi_1 E( Y_{t-1}) + 0 \] \[= \delta + \phi_1 E( Y_{t})\]

if the process is stationary (constant expected value).

If we solve for \(E(Y_t)\), we get that \(E(Y_t) = \mu = \frac{\delta}{1-\phi_1}\).

Variance

By the independence of current error and \(Y_{t-1}\) (as \(Y_{t-1}\) is only a function of past errors),

\[Var(Y_t) = Var(\delta + \phi_1 Y_{t-1} + W_t) = Var(\phi_1 Y_{t-1} + W_t)\] \[= Var(\phi_1 Y_{t-1}) + Var(W_t)\] \[= \phi_1^2 Var( Y_{t-1}) + \sigma^2_w\]

\[= \phi_1^2 Var( Y_{t}) + \sigma^2_w\]

if the process is stationary.

If we solve for \(Var(Y_t)\), then we find that

\[Var(Y_t) = \frac{\sigma^2_w}{1-\phi_1^2}\]

Since \(Var(Y_t)>0\), then \(1-\phi_1^2 > 0\) and therefore \(|\phi_1| < 1\).

Covariance

Let’s assume that the data have a mean of 0 (we’ve removed the trend and seasonality) such that \(\delta = 0\) and \(Y_t = \phi_1Y_{t-1} + W_t\).

Then the \(Cov(Y_t,Y_{t-k}) = E(Y_tY_{t-k})\). For \(k=1\) (observations 1 time unit apart),

\[\Sigma_Y(1) = Cov(Y_t,Y_{t-1}) = E(Y_tY_{t-1}) = E((\phi_1Y_{t-1} + W_t)Y_{t-1}) \] \[= E(\phi_1Y_{t-1}^2 + W_tY_{t-1}) \] \[= \phi_1E(Y_{t-1}^2) + E(W_t)E(Y_{t-1}) \text{ since }Cov(W_t,Y_{t-1}) = 0\] \[= \phi_1Var(Y_{t-1}) + 0E(Y_{t-1})\] \[= \phi_1Var(Y_{t}) \]

Thus,

\[\rho_1 = Cor(Y_t,Y_{t-1}) = \frac{Cov(Y_t,Y_{t-1}) }{Var(Y_t)} = \frac{\phi_1Var(Y_t) }{Var(Y_t)} = \phi_1\]

For \(k>1\), multiple each side of the model by \(Y_{t-k}\) and take the expectation.

\[Y_t = \phi_1Y_{t-1} + W_t\]

\[Y_{t-k}Y_t = \phi_1Y_{t-k}Y_{t-1} + Y_{t-k}W_t\] \[E(Y_{t-k}Y_t) = E(\phi_1Y_{t-k}Y_{t-1}) + E(Y_{t-k}W_t)\]

\[Cov(Y_{t-k},Y_t) = \phi_1Cov(Y_{t-k},Y_{t-1}) \] \[\Sigma_Y(k) = \phi_1\Sigma_Y(k-1) \]

If we work recursively, starting with \(\Sigma_Y(1)\), we get

\[\Sigma_Y(1) = \phi_1\Sigma_Y(0) \] \[\Sigma_Y(2) = \phi_1\Sigma_Y(1) = \phi_1^2\Sigma_Y(0) \] \[\Sigma_Y(3) = \phi_1\Sigma_Y(2) = \phi_1^3\Sigma_Y(0) \]

\[\Sigma_Y(k) = \phi_1^k\Sigma_Y(0) \]

So the correlation for observations \(k\) lags apart is

\[\rho_k = \frac{\Sigma_Y(k) }{Var(Y_t)} = \frac{\phi_1^kVar(Y_t) }{Var(Y_t)} = \phi_1^k\]

5.11.1.2 Approach 2: Write AR(1) as MA(\(\infty\))

\[Y_t = \delta + \phi_1Y_{t-1} + W_t\]

where \(t= ...,-3,-2,-1,0,1,2,3,...\)

By successive substitution, we can rewrite this model,

\[Y_t = \delta + \phi_1(\delta + \phi_1Y_{t-2} + W_{t-1}) + W_t =\delta(1 + \phi_1) + \phi_1^2Y_{t-2} + \phi_1W_{t-1} + W_t \] \[ =\delta(1 + \phi_1) + \phi_1^2(\delta + \phi_1Y_{t-3} + W_{t-2}) + \phi_1W_{t-1} + W_t \] \[=\delta(1 + \phi_1 + \phi_1^2) + \phi_1^3Y_{t-3} + \phi_1^2W_{t-2} + \phi_1W_{t-1} + W_t \] \[ =\delta(1 + \phi_1 + \phi_1^2 + \cdots) + W_t + \phi_1W_{t-1} +\phi_1^2W_{t-2} +\phi_1^3W_{t-3}+\cdots \]

Using infinite geometric series (\(\sum^\infty_{i=0}r^i = (1-r)^{-1}\) if \(|r|<1\)), the first part converges to \(\frac{\delta}{1-\phi_1}\) if \(|\phi_1|<1\). Similarly, the sum of random variables converges to a finite value when \(|\phi_1|<1\).

With this format, it is clear that for an AR(1) process,

\[E(Y_t) = E(\frac{\delta}{1-\phi_1} + W_t +\phi_1W_{t-1} + \phi_1^2 W_{t-2} + \cdots ) = \frac{\delta}{1-\phi_1}\]

\[Var(Y_t) = Var(\frac{\delta}{1-\phi_1} + W_t +\phi_1W_{t-1} + \phi_1^2 W_{t-2} + \cdots ) \] \[= \sigma^2_w(1+\phi_1^2 + \phi_1^4 + \cdots) \] \[ =\frac{\sigma^2_w}{1-\phi_1^2}\text{ if } |\phi_1|<1\]

Let \(h>0\), then

\[Cov(Y_t,Y_{t-h}) = Cov(\frac{\delta}{1-\phi_1} + W_t +\phi_1W_{t-1} + \phi_1^2 W_{t-2} + \cdots ,\frac{\delta}{1-\phi_1} + W_{t-h} +\phi_1W_{t-h-1} + \phi_1^2 W_{t-h-2} + \cdots ) \]

\[= Cov( W_t +\phi_1W_{t-1} + \phi_1^2 W_{t-2} + \cdots ,W_{t-h} +\phi_1W_{t-h-1} + \phi_1^2 W_{t-h-2} + \cdots ) \]

\[= \sum_{j = 0}^\infty Cov(\phi_1^{h+j}W_{t-h-j},\phi_1^{j} W_{t-h-j})\] \[= \sum_{j = 0}^\infty \phi_1^{h+2j}Var(W_{t-h-j}) \]

\[= \phi_1^h\sigma_w^2\sum_{j = 0}^\infty \phi_1^{2j} \] \[= \frac{\phi_1^h\sigma_w^2}{1-\phi_1^2}\text{ if } |\phi_1|<1\]

Thus,

\[Cor(Y_t,Y_{t-h}) = \frac{Cov(Y_t,Y_{t-h}) }{SD(Y_t)SD(Y_{t-h})} = \frac{\phi_1^h\sigma_w^2}{1-\phi_1^2}/\frac{\sigma^2_w}{1-\phi_1^2}\] \[= \phi_1^h\]