Trends and Smoothing II

36-467/36-667, Fall 2020

10 September 2020

\[ \newcommand{\Expect}[1]{\mathbb{E}\left[ #1 \right]} \newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]} \newcommand{\Cov}[1]{\mathrm{Cov}\left[ #1 \right]} \newcommand{\TrueRegFunc}{\mu} \newcommand{\EstRegFunc}{\widehat{\TrueRegFunc}} \DeclareMathOperator{\tr}{tr} \DeclareMathOperator*{\argmin}{argmin} \DeclareMathOperator{\dof}{DoF} \DeclareMathOperator{\det}{det} \newcommand{\TrueNoise}{\epsilon} \newcommand{\EstNoise}{\widehat{\TrueNoise}} \]

In our last episode…

An example to make things concrete: GDP Per Capita

An example to make things concrete: GDPPC growth rate

Trend=mean?

The math for taking the trend to be the global mean

Trend=short moving average?

The math for moving averages

Trend=wider moving average?

Trend=one-sided moving average?

Data = trend + fluctuation

Some residuals

Residuals from using a constant trend:

Some residuals

Residuals from using an MA(5):

Some residuals

Residuals from using an MA(16) on the past:

Some math of residuals

\[\begin{eqnarray} \mathbf{\EstNoise} & = & \mathbf{x} - \mathbf{\EstRegFunc}\\ & = & \mathbf{x} - \mathbf{w}\mathbf{x}\\ & = & (\mathbf{I} - \mathbf{w})\mathbf{x} \end{eqnarray}\]

Expected residuals

\[\begin{eqnarray} \Expect{\mathbf{\EstNoise}} & = & \Expect{(\mathbf{I}-\mathbf{w})\mathbf{X}}\\ & = & (\mathbf{I}-\mathbf{w})\mathbf{\TrueRegFunc} \end{eqnarray}\]

Biased trend estimate \(\Leftrightarrow\) biased fluctuation estimate

Variance and covariance of the residuals

\[ \Var{\mathbf{\EstNoise}} = (\mathbf{I}-\mathbf{w}) \Var{\mathbf{\epsilon}} (\mathbf{I}-\mathbf{w})^T \]

IF \(\Var{\mathbf{\epsilon}} = \sigma^2 \mathbf{I}\), THEN \(\Var{\mathbf{\EstNoise}}= \sigma^2 (\mathbf{I}-\mathbf{w})(\mathbf{I}-\mathbf{w})^T\)

NB: Correlations from off-diagonal entries in \(\mathbf{w}\), even though there are no correlations for the true fluctuations

Splines

\[ \EstRegFunc = \argmin_{m}{\frac{1}{n}\sum_{i=1}^{n}{(x_i - m(t_i))^2} + \lambda\int{(m^{\prime\prime}(t))^2 dt}} \]

How do we pick \(\lambda\)?

Leave-one-out cross-validation (LOOCV)

Leave-one-out cross-validation (LOOCV)

Don’t have to re-fit linear smoothers \(n\) times

\[\begin{eqnarray} \EstRegFunc^{(-i)}(t_i) &= & \frac{({\mathbf{w} \mathbf{x})}_i - w_{ii} x_i}{1-w_{ii}}\\ x_i - \EstRegFunc^{(-i)}(t_i) & = & \frac{x_i - \EstRegFunc(t_i)}{1-w_{ii}}\\ LOOCV & = & \frac{1}{n}\sum_{i=1}^{n}{\left(\frac{x_i-\EstRegFunc(t_i)}{1-w_{ii}}\right)^2} \end{eqnarray}\]

Many variants

The moral on CV

Spline smoothing of economic growth

growth.ss <- with(na.omit(gdppc), smooth.spline(x = year, y = growth))
growth.ss
## Call:
## smooth.spline(x = year, y = growth)
## 
## Smoothing Parameter  spar= 0.2029895  lambda= 4.923235e-08 (16 iterations)
## Equivalent Degrees of Freedom (Df): 83.07802
## Penalized Criterion (RSS): 0.1871268
## GCV: 0.001244193

Summing up