In this paper we show that the posterior distribution for feedforward
neural networks is asymptotically consistent. This paper extends
earlier results on universal approximation properties of neural
networks to the Bayesian setting. The proof of consistency embeds the
problem in a density estimation problem, then uses bounds on the
bracketing entropy to show that the posterior is consistent over
Hellinger neighborhoods. It then relates this result back to the
regression setting. We show consistency in both the setting of the
number of hidden nodes growing with the sample size, and in the case
where the number of hidden nodes is treated as a parameter. Thus we
provide a theoretical justification for using neural networks for
nonparametric regression in a Bayesian framework.
Keywords: Bayesian statistics, Asymptotic consistency, Posterior
approximation, Nonparametric regression, Sieve Asymptotics, Hellinger
distance, Bracketing entropy