Nonparametric Methods

Non-parametric (or distribution-free) inferential statistical methods are mathematical procedures for statistical hypothesis testing which, unlike parametric statistics, make no assumptions about the probability distributions of the variables being assessed.

There are currently no projects for this area of research.

ANOVA for diffusions

The paper examines the relationship among Ito processes from the angle of quadratic variation. The proposed methodology, "ANOVA for diffusions", allows drawing inference for a time interval, rather than for single time points. One of its applications is in fitness of modeling a stochastic process, it also helps quantify and characterize the trading (hedging) error in the case of financial applications.

The reason why the ANOVA permits conclusions over a time interval is that the asymptotic errors of the residual quadratic variation converge as a process (in time). A main conceptual finding was the clear cut effect of the two sources behind the asymptotics. The variation component (mixed Gaussian) comes only from the discretization error (in time discrete sampling). On the other hand, the bias depends only on the choice of estimator of the quadratic variation. This two-sources principle carries over to other criteria of goodness of fit, for example, the coefficient of determination.

Bayesian Empirical Likelihood

Empirical likelihood has been suggested as a data-based, nonparametric alternative to the usual likelihood function. Research has shown that empirical likelihood tests have many of the same asymptotic properties as those derived from parametric likelihoods. This leads naturally to the possibility of using empirical likelihood as the basis for Bayesian inference. Different ways in which this goal might be accomplished are considered. The validity of the resultant posterior inferences is examined, as are frequentist properties of the Bayesian empirical likelihood intervals.

Bayesian Time Series Modelling with Long-Range Dependence

We present a class of models for trend plus stationary component time series, in which the spectral densities of stationary components are represented via non-parametric smoothness priors combined with long-range dependence components. We discuss model fitting and computational issues underlying Bayesian inference under such models, and provide illustration in studies of a climatological time series. These models are of interest to address the questions of existence and extent of apparent long-range effects in time series arising in specific scientific applications.

CATS: Clustering After Transformation and Smoothing

CATS - Clustering After Transformation and Smoothing - is a technique for nonparametrically estimating and clustering a large number of curves. Our motivating example is a genetic microarray experiment but the method is very general. The method includes: transformation and smoothing multiple curves, multiple nonparametric testing for trends, clustering curves with similar shape, and nonparametrically inferring the misclustering rate.

Consistency of Posterior Distributions for Neural Networks

In this paper we show that the posterior distribution for feedforward neural networks is asymptotically consistent. This paper extends earlier results on universal approximation properties of neural networks to the Bayesian setting. The proof of consistency embeds the problem in a density estimation problem, then uses bounds on the bracketing entropy to show that the posterior is consistent over Hellinger neighborhoods. It then relates this result back to the regression setting. We show consistency in both the setting of the number of hidden nodes growing with the sample size, and in the case where the number of hidden nodes is treated as a parameter. Thus we provide a theoretical justification for using neural networks for nonparametric regression in a Bayesian framework.


We present a method for constructing nonparametric confidence sets for density functions based on an approach due to Beran and Dümbgen (1998). We expand the density in an appropriate basis and we estimate the basis coefficients by using linear shrinkage methods. We then find the limiting distribution of an asymptotic pivot based on the quadratic loss function. Inverting this pivot yields a confidence ball for the density.

(Revised 10/04)

Nonparametric Density Estimation and Clustering in Astronomical Sky Surveys

We present a nonparametric method for galaxy clustering in astronomical sky surveys. We show that the cosmological definition of clusters of galaxies is equivalent to density contour clusters (Hartigan, 1975) \(S_c = \{ f > c \}\) where \(f\) is a probability density function. The plug-in estimator \(\hat S_c =\{ \hat f > c \}\) is used to estimate \(S_c\) where \(\hat f\) is the multivariate kernel density estimator. To choose the optimal smoothing parameter, we use cross-validation and the plug-in method and show that cross-validation method outperforms the plug-in method in our case. A new cluster catalogue based on the plug-in estimator is compared to existing cluster catalogs, the Abell and EDCCI. Our result is more consistent with the EDCCI than with the Abell, which is the most widely used catalogue. We present a practical algorithm for local smoothing and use the smoothed bootstrap to asses the validity of clustering results.

(Revised 10/04)

Nonparametric Inference in Astrophysics

We discuss nonparametric density estimation and regression for astrophysics problems. In particular, we show how to compute nonparametric confidence intervals for the location and size of peaks of a function. We illustrate these ideas with recent data on the Cosmic Microwave Background. We also briefly discuss nonparametric Bayesian inference.

Rates of Convergence of Posterior Distributions

We compute the rate at which the posterior distribution concentrates around the true parameter value. The spaces we work in are quite general and include infinite dimensional cases. The rates are driven by two quantities: the size of the space, as measure by metric entropy or bracketing entropy, and the degree to which the prior concentrates in a small ball around the true parameter. We apply the results to several examples. In some cases, natural priors give sub-optimal rates of convergence and better rates can be obtained by using sieve-based priors.

(Revised 08/98)

Rodeo: Sparse Nonparametric Regression in High Dimensions

We present a method for simultaneously performing bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth in directions where the gradient of the estimator with respect to bandwidth is large. When the unknown function satisfies a sparsity condition, the approach avoids the curse of dimensionality. The method - called rodeo (regularization of derivative expectation operator) - conducts a sequence of hypothesis tests, and is easy to implement. A modified version that replaces testing with soft thresholding may be viewed as solving a sequence of lasso problems. When applied in one dimension, the rodeo yields a method for choosing the locally optimal bandwidth.

Sparse Nonparametric Graphical Models

We present some nonparametric methods for graphical modeling. In the discrete case, where the data are binary or drawn from a finite alphabet, Markov random fields are already essentially nonparametric, since the cliques can take only a finite number of values. Continuous data are different. The Gaussian graphical model is the standard parametric model for continuous data, but it makes distributional assumptions that are often unrealistic. We discuss two approaches to building more flexible graphical models. One allows arbitrary graphs and a nonparametric extension of the Gaussian; the other uses kernel density estimation and restricts the graphs to trees and forests. Examples of both methods are presented. We also discuss possible future research directions for nonparametric graphical modeling.

The Consistency of Posterior Distribtions in Nonparametric Problems

We give conditions that guarantee that the posterior probability of every Hellinger neighborhood of the true density tends to 1 almost surely. The conditions are (i) a smoothness condition on the prior and (ii) a requirement that the prior put positive mass in appropriate neighborhoods of the true density. The results are based on the idea of approximating the set of densities with a finite dimensional set of densities and then computing the Hellinger bracketing metric entropy of the approximating set. We apply the results to some examples.

There are currently no lab groups for this area of research.