Abstract: We study deep neural networks and their use in semiparametric inference. We prove valid inference after first-step estimation with deep learning, a result new to the literature. We provide new rates of convergence for deep feedforward neural nets and, because our rates are sufficiently fast (in some cases minimax optimal), obtain valid semiparametric inference. Our estimation rates and semiparametric inference results handle the current standard architecture: fully connected feedforward neural networks (multi-layer perceptrons), with the now-common rectified linear unit activation function and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed-width, very deep networks. We establish nonasymptotic bounds for these deep nets for nonparametric regression, covering the standard least squares and logistic losses in particular. We then apply our theory to develop semiparametric inference, focusing on treatment effects, expected welfare, and decomposition effects for concreteness. Inference in many other semiparametric contexts can be readily obtained. We demonstrate the effectiveness of deep learning with a Monte Carlo analysis and an empirical application to direct mail marketing.
Max H. Farrell studies econometric theory and applied econometrics. His research focus on model selection, high-dimensional data, and robust semiparametric methods, with a focus on increasing reliability and implementability in data analysis. His publications appear in the Journal of Econometrics and Advances in Econometrics, as well as a variety of healthcare and medical journals. Farrell earned a Ph.D. in economics from the University of Michigan, as well as an M.A. in statistics. Farrell pursued undergraduate studies at the Massachusetts Institute of Technology where he earned S.B. degrees in mathematics and economics. He has experience teaching statistics and econometrics at the undergraduate and graduate level. Prior to his graduate studies, Farrell worked at the Center for Research on Health Care at the University of Pittsburgh and at Analysis Group, Inc, where he worked on a variety of statistical and economic consulting issues.