Posterior consistency can be thought of as a theoretical
justification of the Bayesian method. One of the most popular approaches
to nonparametric Bayesian regression is to put a nonparametric prior
distribution on the unknown regression function using Gaussian processes.
In this paper, we study posterior consistency in nonparametric regression
problems using Gaussian process priors. We use an extension of the
theorem of Schwartz (1965) for nonidentically distributed
observations, verifying its conditions when using Gaussian process
priors for the regression function with normal or double
exponential (Laplace) error distributions. We define a metric
topology on the space of regression functions and then establish
almost sure consistency of the posterior distribution. Our metric
topology is weaker than the popular
L1
topology. With additional
assumptions, we prove almost sure consistency when the regression
functions have
L1
topologies. When the covariate (predictor) is
assumed to be a random variable, we prove almost sure consistency for
the joint density function of the response and predictor using the
Hellinger metrics.
Keywords:Almost sure consistency, Convergence in probability,
Hellinger metric, L1 metric, Laplace distribution