Expansions for posterior distributions

Abstract

: Suppose that X n is a sample of size n with log likelihood nl ( θ ), where θ is an unknown parameter in R p having a prior distribution ξ ( θ ). We need not assume that the sample values are independent or even stationary. Let (cid:98) θ be the maximum likelihood estimate (MLE). We show that θ | X n is asymptotically normal with mean (cid:98) θ and covariance − n − 1 l (cid:5) , (cid:5) (cid:16)(cid:98) θ (cid:17) − 1 , where l (cid:5) , (cid:5) ( θ ) = ∂ 2 l ( θ ) /∂θ∂θ ′ . In contrast (cid:98) θ | θ is asymptotically normal with mean θ and covariance n − 1 [ I ( θ )] − 1 , where I ( θ ) = − E (cid:104) l (cid:5) , (cid:5) (cid:16)(cid:98) θ (cid:17) | θ (cid:105) is Fisher’s information. So, frequentist inference conditional on θ cannot be used to approximate Bayesian inference, except for exponential families. However, under mild conditions − l (cid:5) , (cid:5) (cid:16)(cid:98) θ (cid:17) | θ → I ( θ ) in probability. So, Bayesian inference (that is, conditional on X n ) can be used to approximate frequentist inference. For t ( θ ) any smooth function, we obtain posterior cumulant expansions, posterior Edgeworth-Cornish-Fisher (ECF) expansions and posterior tilted Edgeworth expansions for L t ( θ ) | X n , as well as confidence regions for t ( θ ) | X n of high accuracy. We also give expansions for the Bayes estimate (estimator) of t ( θ ) about t (cid:16)(cid:98) θ (cid:17) , and for the maximum a posteriori estimate about (cid:98) θ , as well as their relative efficiencies with respect to squared error loss.