Imagine being shown $N$ samples of random variables drawn independently from the same distribution. What can you say about the distribution? In general, of course, the answer is nothing, unless we have some prior notions about what to expect. From a Bayesian point of view we need an a priori distribution on the space of possible probability distributions, which defines a scalar field theory. In one dimension, free field theory with a constraint provides a tractable formulation of the problem, and we also discus generalizations to higher dimensions.

The traditional kernel density estimator of an unknown density is by construction completely nonparametric in the sense that it has no preferences and will work reasonably well for all shapes. The present paper develops a class of semiparametric methods that are designed to work better than the kernel estimator in a broad nonparametric neighbourhood of a given parametric class of densities, for example, the normal, while not losing much in precision when the true density is far from the parametric class. The idea is to multiply an initial parametric density estimate with a kernel-type estimate of the necessary correction factor. This works well in cases where the correction factor function is less rough than the original density itself. Extensive comparisons with the kernel estimator are carried out, including exact analysis for the class of all normal mixtures. The new method, with a normal start, wins quite often, even in many cases where the true density is far from normal. Procedures for choosing the smoothing parameter of the estimator are also discussed. The new estimator should be particularly useful in higher dimensions, where the usual nonparametric methods have problems. The idea is also spelled out for nonparametric regression.

An exact and easily computable expression for the mean integrated squared error (MISE) for the kernel estimator of a general normal mixture density, is given for Gaussian kernels of arbitrary order. This provides a powerful new way of understanding density estimation which complements the usual tools of simulation and asymptotic analysis. The family of normal mixture densities is very flexible and the formulae derived allow simple exact analysis for a wide variety of density shapes. A number of applications of this method giving important new insights into kernel density estimation are presented. Among these is the discovery that the usual asymptotic approximations to the MISE can be quite inaccurate, especially when the underlying density contains substantial fine structure and also strong evidence that the practical importance of higher order kernels is surprisingly small for moderate sample sizes.

We wish to estimate the probability density g y. that produced an observed random sample of vectors y1, y2 , . . . , yn. Estimates of g y. are traditionally constructed in two quite different ways: by maximum likelihood fitting within some parametric family such as the normal or by nonparametric methods such as kernel density estimation. These two methods can be combined by putting an exponential family ‘‘through’’ a kernel estimator. These are the specially designed exponential families mentioned in the title. Poisson regression methods play a major role in calculations concerning such families.

A fundamental problem of statistical data analysis, distribution density estimation by experimental data, is considered. A new method with optimal asymptotic behavior, the root density estimator, is developed. The method proposed may be applied to its full extent to solve the statistical inverse problem of quantum mechanics, namely, estimating the psi function on the basis of the results of mutually complementing experiments.

Probability density functions are estimated by the method of maximum likelihood in sequences of regular exponential families. This method is also familiar as entropy maximization subject to empirical constraints. The approximating families of log-densities that we consider are polynomials, splines and trigonometric series. Bounds on the relative entropy (Kullback-Leibler distance) between the true density and the estimator are obtained and rates of convergence are established for log-density functions assumed to have square integrable derivatives.

We describe and illustrate Bayesian inference in models for density estimation using mixtures of Dirichlet processes. These models provide natural settings for density estimation and are exemplified by special cases where data are modeled as a sample from mixtures of normal distributions. Efficient simulation methods are used to approximate various prior, posterior, and predictive distributions. This allows for direct inference on a variety of practical issues, including problems of local versus global smoothing, uncertainty about density estimates, assessment of modality, and the inference on the numbers of components. Also, convergence results are established for a general class of normal mixture models.

The maximum-entropy approach to the solution of underdetermined inverse problems is studied in detail in the context of the classical moment problem. In important special cases, such as the Hausdorff moment problem, we establish necessary and sufficient conditions for the existence of a maximum-entropy solution and examine the convergence of the resulting sequence of approximations. A number of explicit illustrations are presented. In addition to some elementary examples, we analyze the maximum-entropy reconstruction of the density of states in harmonic solids and of dynamic correlation functions in quantum spin systems. We also briefly indicate possible applications to the Lee�Yang theory of Ising models, to the summation of divergent series, and so on. The general conclusion is that maximum entropy provides a valuable approximation scheme, a serious competitor of traditional Pad�-like procedures. Journal of Mathematical Physics is copyrighted by The American Institute of Physics.