Prediction with Confidence – General Framework for Predictive Inference

Fri, Sep 27, 2019, 12:00 pm
Corwin Hall, Room 127
Center for Statistics and Machine Learning
Department of Politics, Quantitative Social Science Colloquium


We propose a general framework for prediction in which a prediction is in the form of a distribution function, called ‘predictive distribution function’. This predictive distribution function is well suited for prescribing the notion of confidence under the frequentist interpretation and  providing meaningful answers for prediction-related questions. Its very form of a distribution function also lends itself as a useful tool for quantifying uncertainty in prediction. A general approach under this framework is formulated and illustrated using the so-called confidence distributions (CDs). This CD-based prediction approach inherits many desirable properties of CD, including its capacity to serve as a common platform for directly connecting the existing procedures of predictive inference in Bayesian, fiducial and frequentist paradigms. We discuss the theory underlying the CD-based predictive distribution and related efficiency and optimality. We also propose a simple yet broadly applicable Monte-Carlo algorithm for implementing the proposed approach. This concrete algorithm together with the proposed definition and associated theoretical development provide a comprehensive statistical inference framework for prediction. Finally, the approach is demonstrated by simulation studies and a real project on predicting the volume of application submissions to a government agency. The latter shows the applicability of the proposed approach to even dependent data settings.

This is joint work with Jieli Shen, Goldman Sachs, and Minge Xie, Rutgers University


I joined Rutgers Statistics faculty after receiving my Ph.D. from Columbia. My research interests include resampling, data depth, multivariate nonparametric statistics, more recently on fusion learning and collaborative research with the FAA on aviation risk analysis. I am currently a co-editor of JASA T&M and the IMS President-Elect.

Link to CV: