For courses in Probability and Random Processes. This book is a comprehensive treatment of probability and random processes that, more than any other available source, combines rigor with accessibility. Beginning with the fundamentals of probability theory and requiring only college-level calculus, the book develops all the tools needed to understand more advanced topics such as random sequences (Chapter 6), continuous-time random processes (Chapter 7), and statistical signal processing (Chapter 9). The book progresses at a leisurely pace, never assuming more knowledge than contained in the material already covered. Rigor is established by developing all results from the basic axioms (Chapters 1,2) and carefully defining and discussing such advanced notions as stochastic convergence, stochastic integrals and resolution of stochastic processes (Chapter 8). The 3rd Edition has a large number of new topics, not present in the 2nd Edition, including additional material on basic probability (Appendix B, Section 1.8, Section 1.11), statistics (chi-square and Student-t in Section 2.4, Section 4.1), misuses of probability (Sec. 1.3), and signal processing (all of Chapter 9).

Table of Contents:
(NOTE: Each chapter concludes with a Summary, Problems, and References.) 1. Introduction to Probability. Introduction: Why Study Probability? The Different Kinds of Probability. Misuses, Miscalculations, and Paradoxes in Probability. Sets, Fields, and Events. Axiomatic Definition of Probability. Joint, Conditional, and Total Probabilities; Independence. Bayes' Theorem and Applications. Combinatorics. Bernoulli Trials—Binomial and Multinomial Laws. Asymptotic Behavior of the Binomial Law: The Poisson Law. Normal Approximation to the Binomial Law. 2. Random Variables. Introduction. Definition of a Random Variable. Probability Distribution Function. Probability Density Function. Continuous, Discrete and Mixed Random Variables. Conditional and Joint Distributions and Densities. Failure Rates. 3. Functions of Random Variables. Introduction. Solving Problems of the Type Y=g(X). Solving Problems of the Type Z=g(X,Y). Solving Problems of the Type V=g(X,Y), W=h(X,Y). Additional Examples. 4. Expectation and Introduction to Estimation. Expected Value of a Random Variable. Conditional Expectation. Moments. Chebyshev and Schwarz Inequalities. Moment Generating Functions. Chernoff Bound. Characteristic Functions. Estimators for the Mean and Variance of the Normal Law. 5. Random Vectors and Parameter Estimation. Joint Distributions and Densities. Multiple Transformation of Random Variables. Expectation Vectors and Covariance Matrices. Properties of Covariance Matrices. Simultaneous Diagonalization of Two Covariance Matrices and Applications in Pattern Recognition. The Multidimensional Gaussian Law. Characteristic Functions of Random Vectors. Parameter Estimation. Estimation of Vector Means and Covariance Matrices. Maximum Likelihood Estimators. Linear Estimation of Vector Parameters. 6. Random Sequences. Basic Concepts. Basic Principles of Discrete-Time Linear Systems. Random Sequences and Linear Systems. WSS Random Sequence. Markov Random Sequences. Vector Random Sequences and State Equations. Convergence of Random Sequences. Laws of Large Numbers. 7. Random Processes. Basic Definitions. Some Important Random Processes. Continuous-Time Linear Systems with Random Inputs. Some Useful Classification of Random Processes. Wide-Sense Stationary Processes and LSI Systems. Periodic and Cyclostationary Processes. Vector Processes and State Equations. 8. Advanced Topics in Random Processes. Mean-Square (m.s.) Calculus. m-s Stochastic Integrals. m-s Stochastic Differential Equations. Ergodicity. Karhunen-Loève Expansion. Representation of Bandlimited and Periodic Processes. 9. Applications to Statistical Signal Processing. Estimation of Random Variables. Innovation Sequences and Kalman Filtering. Wiener Filter for Random Sequence. Expectation-Maximization Algorithm. Hidden Markov Models (HMM). Spectral Estimation. Simulated Annealing. Appendices. Appendix A: Review of Relevant Mathematics. Basic Mathematics. Continuous Mathematics. Residue Method for Inverse Fourier Transform. Mathematical Induction <091>A-4<093>. Appendix B: Gamma and Delta Functions. Gamma Function. Dirac Delta Function. Appendix C: Functional Transformations and Jacobians. Introduction. Jacobians for n = 2. Jacobian for General n. Appendix D: Measure and Probability. Introduction and Basic Ideas. Application of Measure Theory to Probability. Appendix E: Sampled Analog Waveforms and Discrete-time Signals.