Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators provides a uniquely broad compendium of the key mathematical concepts and results that are relevant for the theoretical development of functional data analysis (FDA). The self–contained treatment of selected topics of functional analysis and operator theory includes reproducing kernel Hilbert spaces, singular value decomposition of compact operators on Hilbert spaces and perturbation theory for both self–adjoint and non self–adjoint operators. The probabilistic foundation for FDA is described from the perspective of random elements in Hilbert spaces as well as from the viewpoint of continuous time stochastic processes. Nonparametric estimation approaches including kernel and regularized smoothing are also introduced. These tools are then used to investigate the properties of estimators for the mean element, covariance operators, principal components, regression function and canonical correlations. A general treatment of canonical correlations in Hilbert spaces naturally leads to FDA formulations of factor analysis, regression, MANOVA and discriminant analysis. This book will provide a valuable reference for statisticians and other researchers interested in developing or understanding the mathematical aspects of FDA. It is also suitable for a graduate level special topics course.

Table of Contents:
Preface xi 1 Introduction 1 1.1 Multivariate analysis in a nutshell 2 1.2 The path that lies ahead 13 2 Vector and function spaces 15 2.1 Metric spaces 16 2.2 Vector and normed spaces 20 2.3 Banach and Lp spaces 26 2.4 Inner Product and Hilbert spaces 31 2.5 The projection theorem and orthogonal decomposition 38 2.6 Vector integrals 40 2.7 Reproducing kernel Hilbert spaces 46 2.8 Sobolev spaces 55 3 Linear operator and functionals 61 3.1 Operators 62 3.2 Linear functionals 66 3.3 Adjoint operator 71 3.4 Nonnegative, square-root, and projection operators 74 3.5 Operator inverses 77 3.6 Fréchet and Gâteaux derivatives 83 3.7 Generalized Gram–Schmidt decompositions 87 4 Compact operators and singular value decomposition 91 4.1 Compact operators 92 4.2 Eigenvalues of compact operators 96 4.3 The singular value decomposition 103 4.4 Hilbert–Schmidt operators 107 4.5 Trace class operators 113 4.6 Integral operators and Mercer’s Theorem 116 4.7 Operators on an RKHS 123 4.8 Simultaneous diagonalization of two nonnegative definite operators 126 5 Perturbation theory 129 5.1 Perturbation of self-adjoint compact operators 129 5.2 Perturbation of general compact operators 140 6 Smoothing and regularization 147 6.1 Functional linear model 147 6.2 Penalized least squares estimators 150 6.3 Bias and variance 157 6.4 A computational formula 158 6.5 Regularization parameter selection 161 6.6 Splines 165 7 Random elements in a Hilbert space 175 7.1 Probability measures on a Hilbert space 176 7.2 Mean and covariance of a random element of a Hilbert space 178 7.3 Mean-square continuous processes and the Karhunen–Lòeve Theorem 184 7.4 Mean-square continuous processes in L2 (E,B(E), mu) 190 7.5 RKHS valued processes 195 7.6 The closed span of a process 198 7.7 Large sample theory 203 8 Mean and covariance estimation 211 8.1 Sample mean and covariance operator 212 8.2 Local linear estimation 214 8.3 Penalized least-squares estimation 231 9 Principal components analysis 251 9.1 Estimation via the sample covariance operator 253 9.2 Estimation via local linear smoothing 255 9.3 Estimation via penalized least squares 261 10 Canonical correlation analysis 265 10.1 CCA for random elements of a Hilbert space 267 10.2 Estimation 274 10.3 Prediction and regression 281 10.4 Factor analysis 284 10.5 MANOVA and discriminant analysis 288 10.6 Orthogonal subspaces and partial cca 294 11 Regression 305 11.1 A functional regression model 305 11.2 Asymptotic theory 308 11.3 Minimax optimality 318 11.4 Discretely sampled data 321 References 327 Index 331 Notation Index 334