3.4 Estimation

3.4.1 Sample Covariance Matrix

To estimate \(\boldsymbol\Sigma\) with observed data sampled from the larger population (\(n\) realizations of these \(m\) random variables), we can calculate the sample covariance matrix,

\[\mathbf{S} = \frac{1}{n-1}\sum^n_{i=1} (\mathbf{y}_i - \bar{\mathbf{y}})(\mathbf{y}_i - \bar{\mathbf{y}})^T \] where \(\mathbf{y}_i = (y_{i1},y_{i2},...,y_{im})\) and \(\bar{\mathbf{y}} = n^{-1} \sum^n_{i=1} \mathbf{y}_i\).

Therefore, the sample variances (on the diagonal of \(\mathbf{S}\)) are the familiar estimates we saw in our introductory course,

\[ s^2_j = \frac{1}{n-1}\sum^n_{i=1}(y_{ij} - \bar{y}_j)^2\]

Note: This estimation is only possible for longitudinal data if we observed \(m\) repeated measurements for each case at the exactly same time. For time series and spatial data, we don’t have more than 1 realization of the process. For longitudinal data, if we have unbalanced data that was collected as irregular times, we’ll need to use one of the common model structures.

3.4.2 Sample Autocovariance Function

If we assume that observed data come from a stationary process, we can estimate the autocovariance function with the sample autocovariance function (ACVF),

\[c_k = \frac{1}{n}\sum^{n-|k|}_{t=1}(y_t - \bar{y})(y_{t+|k|} - \bar{y})\] where \(\bar{y}\) is the sample mean, averaged across time/space because we are assuming it is constant.

For any sequence of observations, \(y_1,...,y_n\),

  1. \(c_k = c_{-k}\)
  2. \(c_0\geq 0\) and \(|c_{k}| \leq c_{0}\)

3.4.3 Sample Semivariogram

If we assume that observed data come from a stationary process (or at least a intrinsic stationary process), we can estimate the sample semivariogram,

\[\gamma(h) = \frac{1}{2}Var(Y_{s+h} - Y_s)\]

To estimate the empirical semivariogram, let \(H_1,...,H_k\) be a partition of the space of possible distances or lags \(h\), with \(h_u\) being a representative spatial lag/distance in \(H_u\). Then use your stationary process \(y_t\) to estimate the empirical semivariagram.

\[\hat{\gamma}(h_u) = \frac{1}{2\cdot |\{i-j \in H_u\}|}\sum_{\{i-j\in H_u\}}(y_i - y_j)^2\]