ECE 730 Modern Probability Theory and Stochastic Processes, Lec. 1, Fall 2005

9/6	Tuesday.  Started Section 1.2 Review of Set Notation, pp. 5-8.
	Covered Functions, Countable & Uncountable Sets, pp. 8-11.
	Assigned HW #1 Ch 1: 7(d), 8, 9, 27, 12, 14, 29, 30, 35,
	43 DUE THU 9/15.

9/8	Section 1.4 Axioms and Properties of Probability, pp. 14-16.
	Note 1 on sigma-algebras, pp. 26-27.

9/13	Section 1.5 Conditional Probability, Law of Total Probability &
	Bayes' Rule.  Section 1.6 Independence.  Chapter 2 Introduction to
	Discrete RVs: through Max and Min Problems.

9/15	Geometric RVs.  You should read the material on Joint PMFs (not
	covered in class).  Section 2.4 Expectation, skipped derivation of
	LOTUS, Expectations of Functions of Products, Correlation and
	Covariance, Correlation Coefficient.
	Assigned HW #2 Ch 1: 16, 46; Ch 2: 9, 38, 43, 44, 48 DUE THU 9/22.

9/20	Examples 2.39 and 2.40.  Definition of covariance.  For uncorrelated
	RVs, the variance of the sum is the sum of the variances.  Chapter 3:
	Sections 3.1-3.3.  Briefly discussed Section 3.4, law of total
	probability, substitution law.

9/22	Worked Examples 3.17 and 3.21.  Started Chapter 4.  Introduced
	continuous RVs, uniform, exp, Laplace, Cauchy, and normal densities.
	Discussed location and scale parameters and the gamma densities.
	Noted special cases of Erlang and chi-squared.  Worked Example 4.11
	to compute the moments of the N(0,1) RV.  Introduced the double
	factorial notation.  Derived LOTUS for Y=g(X) when g takes finitely
	many distinct values.  For general RVs, defined expectation as a limit.
	Assigned HW #3 Ch 3: 12, 13, 14, 20, 30, 32, 42 DUE THU 9/29.

9/27	Derive general version of LOTUS for continuous RVs.  Section 4.3 Transform
	methods.  Worked Examples 4.23-4.25 in Section 4.4.  Started derivation
	of the Chernoff bound.

9/29	Finished Chernoff bound.  Covered Section 5.5 Properties of CDFs.
	Discussed Problems 11, 12, 19 in Chapter 5.  Also briefly talked
	about Problems 22, 23, and 25.
	Assigned HW #4 Ch 4: 14(d), 15(read only) 16, 17, 30(b), 44(a),
	46, 47, 55, 65 DUE THU 10/6.

10/4	Quick overview of Ch 7: Joint cdfs p. 177, Marginal cdfs (7.3)-(7.4),
	Independent RVs p. 179, Jointly Continuous RVs p. 180, Joint and
	Marginal Densities etc. through the end of Section 7.2.
	Section 7.3 Conditional Probability and Expectation: Conditional
	densities, law of total probability, substitution.  Worked
	Example 7.14; recommend you work through Example 7.15.
	Mentioned that the two RVs that are continuous but not jointly
	continuous are also uncorrelated but not independent (Problem 20).
	Introduced Craig's Formula (Problem 22) and showed how it could
	be used to compute E[Q(sqrt(X))].

10/6	Worked Example 7.24.  Started Chapter 8.  Read Section 8.1 for review
	of vector and matrix algebra.  Started Section 8.2 Random Vectors and
	Random Matrices.  Partially covered Decorrelation and the Karhunen-
	Loeve expansion.
	Assigned HW #5 Ch 7: 40, 42, 59; Ch 8: 20 DUE THU 10/13.

10/11	Finished Decorrelation & Karhunen-Loeve Expansion.  Briefly
	covered Section 8.3 Transformations of Random Vectors; please
	read the examples there.  Covered Section 8.4 Linear Estimation of
	Random Vectors through Example 8.15.

10/13	Derived Linear MMSE Estimator.  Started Ch 9 Gaussian Random Vectors.
	Covered Section 9.2.  Covered Section 9.3 up to beginning of Example 9.6.
	Assigned HW #6 Ch 8: 24, 25, 28, 30, 32; Ch 9: 3, 8 DUE THU 10/20.

10/18	Finished Sections 9.3, 9.4, and 9.5.  We will not cover Section 9.6.

10/20	Did a fast-paced survey of the first 5 sections of Chapter 10.
	Our main focus will be on WSS processes and their processing by
	LTI systems.
	Suggested Review Problems:
	Old midterms from 2003, 2004, and 2005 (but skip problem on the Wiener process).
	Ch 1: 17, 18, 19, 20, 45(b), 47
	Ch 2: 45
	Ch 3: 11, 41
	Ch 4: 31, 58, 64
	Ch 7: 31, 41
	Ch 8: 29, 33, 35, 37, 39
	Ch 5: 13, 14
	

10/25	Answered questions in preparation for the exam.

10/26	WEDNESDAY: Evening Exam.  5:15-6:45.  Room 376 Mechanical Engineering.
	You may bring to the exam one 8.5 in. x 11 in. sheet of paper
	with any formulas you want.  Since the following tables
	of pmfs, pdfs, transforms, and series will be included on the
	exam, you don't need to write these down.

10/27	Went over Exam 1.  Discussed nth order strict stationarity and strict
	stationarity on p. 241.  Worked Examples 10.13 and 10.14.  Read
	pargraph at top of rhs of p. 241.  Worked Example 10.19.  Showed S_X(f)
	is real and even at top of p. 244.  Worked Example 10.22.  You may also
	want to work Example 10.21 yourself.
	Assigned HW #7 Ch 9: 13, 14, 16, 17; Ch 10: 1, 4, 11, 14 DUE THU 11/3.

11/1	Returned Exams.  Worked Examples 10.23, 10.25, 10.26, and 10.28.
	Briefly discussed the first part of Section 10.6 Characterization
	of Correlation Functions.  Started Section 11.1 The Poisson Process.

11/3	Showed that the arrival times of a Poisson process are Erlang.
	Pointed out that the interarrival times are i.i.d. exp(lambda).
	Derived Poisson probabilities starting from limit assumptions.
	Introduced Marked Poisson process and shot-noise processes.
	Mentioned Renewal Processes (Section 11.2).  Started Section 11.3
	The Wiener Process.  Worked Example 11.6.  Defined the Wiener Process.
	Assigned HW #8 Ch 10: 36, 37, 38, 39, 42, 50 DUE THU 11/10.

11/8	Introduced the Wiener integral.  Skipped Random Walk Approximation
	of the Wiener Process.  Started Section 11.4 Specifications of Random
	Processes.  Worked Examples 11.9.  Briefly mentioned Example 11.10;
	you should read this on your own.

11/10	Worked Problems 37-39 in Ch. 11.  Started Chapter 13.  Worked Examples
	13.1-13.3, 13.5
	Assigned HW #9 Ch 11: 8, 9, 11, 12, 27, 29, 30, 31 DUE THU 11/17.

11/15	Worked Examples 13.7 and 13.8.  Discussed mean-square continuity and
	its relation to continuity of the correlation function R(t,s).  Started
	Section 13.2 Normed Vector Spaces of Random Variables.  Introduced
	the L^p norm, triangle inequality, Cauchy sequences of real numbers
	and Cauchy sequences of random variables, inner product on L^2,
	Cauchy-Schwarz inequality.  Worked Example 13.10.
	Some optional problems for review:
	Conditional Probability: Ch 7: 33(a)-(e), 34, 38(part (a) is tricky), 39(b).
	Conditional Expectation: Ch 7: 54-58.
	Est. of random vectors:  Ch 8: 38, 46(MMSE est. only).

11/17	Worked Example 13.11, discussed interchanging expectation and
	infinite sums.  Skipped Mean-Square Integrals.  Skipped Section 13.3
	The Karhunen-Loeve Expansion.  Discussed Section 13.4 The Wiener
	Integral (Again).  Started Section 13.5 Projections, Orthogonality
	Principle, Projection Theorem.
	Assigned HW #10 Ch 13: 1, 2, 8, 9, 12, 16, 21, 22 DUE THU 12/1.

11/22	Finished Section 13.5.  Started Section 13.6 Conditional Expectation
	and Probability.  Discussed Example 13.18.  Skip Examples 13.19 and
	13.20.  Worked Examples 13.21 and 13.22.  Skip Example 13.23.  Started
	The Smoothing Property.

11/24	THANKSGIVING -- No class.

11/29	Computed E[X|Z] where the joint density of X, Y, and Z is given in
	Problem 11 of Chapter 8.  Worked Example 13.24.  Skip Example 13.25.
	Covered Section 14.1 Convergence in Probability.

12/1	Discussed Section 14.2 Convergence in Distribution.  Worked Example
	14.4.  Skipped derivation that convergence in probability implies
	convergence in distribution.  Skipped derivation that if X_n
	converges in distribution to a constant, then X_n also converges
	in probability to that constant.  Gave example in which X_n
	converges in distribution to X, but not in probability.  Worked
	Example 14.5.  Skipped sketch of proof that convergence in
	distribution implies E[g(X_n)] converges to E[g(X)] for all
	bounded, continuous functions g(x).  Covered Examples 14.6-14.8.
	Assigned HW #11 Ch 13: 38, 39, 43, 46, 53, 54 DUE THU 12/8.

12/2	Optional Review Problems:
	Ch  9: 11, 12, 18
	Ch 10: 23-29, 40, 43
	Ch 11:  7, 10, 13, 25, 26, 32, 33, 34, 46
	Ch 13: 11, 14, 19, 24, 25, 26, 44, 47, 55, 57, 59-64

	Old finals from 2002, 2003, and 2004.

12/6	Section 5.6  The Central Limit Theorem.  See also discussion on p. 348
	in Chapter 14.  Worked Example 14.10.  Skipped Example 14.11 Slutsky's
	Theorem.

12/8	Started Section 14.3 Almost Sure Convergence.  Worked Examples 14.12
	and 14.13.
	Recommended Problems: Ch 14: 1, 3, 4, 6, 10, 11, 12, 14, 15, 16,
	17, 18, 21, 25, 26, 32, 38, 44, 47.

12/13	Worked Examples 14.14 and 14.15.  You should look over Example 14.16.
	Sketched pictures to illustrate Example 14.17.  Distributed
	teaching evaluations.

12/15	Last Class Day.  Worked requested problems.

12/17	SATURDAY: FINAL EXAM 7:45-9:45 AM.
	You may bring to the exam two 8.5 in. x 11 in. sheets of paper
	with any formulas you want.  Since the following tables
	of pmfs, pdfs, transforms, and series will be included on the
	exam, you don't need to write these down or memorize them.