ECE 730 Modern Probability Theory and Stochastic Processes, Lec. 1, Fall 2003

9/3     Wednesday.  Worked Example 1.4, pp. 9-10.  Covered subsection
	on Functions, pp. 11-13.  Covered subsection on Countable and
	Uncountable sets, pp. 13-15.
	Assigned HW #1 Ch 1: 3(d), 4, 5, 8, 10, 20, 22, 23, 28, 35 DUE WED 9/10.

9/5	Covered Section 1.3.  Also covered Note 1 at the end of the chapter.

9/8	Worked Problems 26, 27, 30, and 31 in Chapter 1.
	Covered Sections 1.4 and 1.5 very quickly; highlights only.
	Worked Example 1.24.

9/10	Worked Problem 32.  Discussed Problem 38.  Defined random variables,
	shorthand notation in Chapter 2.  Please also read Note 1 at the end
	of Chapter 2.

9/12	Worked Example 1.25.  Worked Problem 76 in Chapter 2 and therefore
	reviewed Poisson pmf, mean, and variance and law of total probability.
	Covered Note 1 in Chapter 2.  Started Chapter 3. Covered Section 3.1
	emphasizing the subsection on Location and Scale Parameters and the
	Gamma densities.
	Assigned HW #2 Ch 3: 12, 13(d), 15(b), 16, 18, 19, 32, 43, 44, 56,
	Ch 6: 36, 38, 39 DUE WED 9/17.

9/15	Section 3.2 Expectation.  Emphasized Derivation of LOTUS pp. 137-138.
	Derived the Markov and Chebyshev inequalities, pp. 81-82.  Derived
	the Chernoff bound, pp. 148. Covered Section 4.5, Properties of CDFs.
	Derived property (vi).  Section 6.2: Defined jointly continuous RVs
	on p. 264.  Jumped to p. 271 for an example of two continuous RVs
	that are not jointly continuous.  Mentioned Law of Total Probability.
	Examples 6.14, 6.15, and 6.18 are recommended reading.

9/17	Covered Section 7.1, 7.2 up to WSS.
	Assigned HW #3 Ch 7: 1, 2, 4, 7, 9, 12, 17, 18 DUE WED 9/24.

9/19	WSS processes in Section 7.2.  Section 7.3, WSS Processes through
	LTI Systems.  Time-Domain Analysis: Note the definition of
	joint wide-sense stationarity from bottom of p. 309 and top of p. 310.
	Frequency-Domain Analysis: eqs. (7.17) and (7.18) are crucial.

9/22	Section 7.4, Power Spectral Densities for WSS Processes.  Three ways
	to express the power in a process (7.19).  White Noise.  Examples.
	Section 7.5, Characterization of Correlation Functions.  Started
	Chapter 8.  Covered Section 8.1 through definition of positive
	semidefinite and positive definite matrices.

9/24	Covered Example 8.3.  Covered  subsection on Decorrelation, Data
	Reduction, and the Karhunen-Loeve Expansion.  Covered subsection on
	Characteristic Functions.  Section 8.3 Transformations of Random
	Vectors through Example 8.7.
	Assigned HW #4 Ch 8: 1, 10, 11, 14, 23, 24, 25 DUE WED 10/1.

9/26	Example 8.8.  Section 8.3 The Multivariate Gaussian to Example 8.11.

9/29	Examples 8.11, 8.12, 8.13.  Covered the Transformation Derivation
	of the N(m,C) joint density.  Skipped Inverse Characteristic Function
	derivation of N(m,C) density.  Briefly discussed level sets.

10/1	Section 8.4 Estimation of Random Vectors.  Focused on Linear Estimation.
	Assigned HW #5 Ch 8: 22, 26, 29, 30, 32, 36, Ch 3: 48 DUE WED 10/8.

10/3	Continued Section 8.4.  Worked Examples 8.14, 8.15.  Covered Minimum
	MSE Estimation.  Started Section 9.1, The Poisson Process.

10/6	Covered Section 9.1, The Poisson Process.  Skipped Example 9.2, but
	you should go over it yourself.  Covered Derivation of the Poisson
	probabilities.  *Briefly* discussed marked Poisson processes and
	shot noise.

10/8	Covered Section 9.2, Renewal Processes.  Started Section 9.3 The
	Wiener Process; got to the bottom of p. 397.
	Assigned HW #6 Ch 9: 6, 9, 11, 16, 18, 19 DUE WED 10/15.

10/9	TA finished Section 9.3 covering The Wiener Integral and the
	Random Walk Approximation of the Wiener Process.  Did TA evaluations.

10/13	Discussed Problems 20-21 in Chapter 9.  Started Section 9.4 Specification
	of Random Processes, Finitely Many Random Variables, Infinite
	Sequences (Discrete Time).  Kolmogorov's Consistency Theorem.

10/15	Finished Chapter 9.  Also showed that the Wiener process is a Gaussian
	process.  Distributed two review questions for Exam 1 (corrected 10/20/03).
	(pdf version)
	Suggested Review Problems:
	Ch 1:  6, 14, 15, 16, 55
	Ch 3: 47, 53, 55
	Ch 6: 37, 52
	Ch 7: 13, 19, 35
	Ch 8: 27
	Ch 9: 5, 7, 24.

10/17	Answered questions about review problems.

10/20	Answered questions about review problems.

10/21	TUESDAY: Night Exam.  5:15-6:45, Room 3345 Engr. Hall.
	You may bring one 8.5 x 11 in sheet of paper with any formulas you
	think are important.

10/22	Went over exam.  Talked about Problems 28, 30, and 31 in Chapter 9.
	Assigned HW #7 Ch 9: 29, 37; Ch 11: 1, 2, 9, 10, 11, 12 DUE WED 10/29.

10/24	Started Chapter 11, pp. 451-455, but skipped solutions of Examples 11.4
	and 11.6.

10/27	Continuity in Mean of Order p, pp. 456-457.  Started Section 11.2
	Normed Vector Spaces of RVs pp. 457-458.
	
10/29	pp. 458-460: Discussed Cauchy sequences of real numbers, inner product
	spaces, parallelogram law, Cauchy-Schwarz inequality.  Example 11.9.
	Assigned HW #8 Ch 11: 15, 16, 21, 22, 23, 24, 25 DUE WED 11/5.

10/31	Justified pushing expectation through infinite sums that converge in
	L^p, pp. 460-461.  Skipped Mean-Square Integrals.  Skipped Section 11.3
	The Karhunen-Loeve expansion.  Covered Section 11.4 The Wiener Integral
	(Again).

11/3	Started Section 11.5 Projections, Orthogonality Principle, Projection
	Theorem, pp. 469-top of 471.

11/5	Proved the Projection Theorem.  Started Section 11.6 Conditional
	Expectation and Probability.
	Assigned HW #9 Ch 11: 40, 44, 45, 46, 51, 52, 55, 56, 59 DUE WED 11/12.

11/7	Gave abstract definition of conditional expectation, (11.23), and
	showed that it exists for X in L^2.  Skipped proof that conditional
	expectation exists for X in L^1.  Introduced standard definition (11.26).
	Worked Examples 11.18 an 11.19.

11/10	Worked Examples 11.20, 11.21.  Briefly discussed conditional probability.
	Briefly covered Section 11.7 The Spectral Representation.  Started Chapter 12.
	Started Section 12.1 Convergence in Probability.  Stated definition.
	Showed convergence in mean of order p implies convergence in probability,
	and gave an example that shows you can have convergence in probability
	without convergence in mean of order p for all p >= 1.

11/12	Worked Example 12.2. Started Section 12.2 Convergence in Distribution.
	Discussed Example 12.4.  Mentioned several important properties of
	convergence in distribution.
	Assigned HW #10 Ch 12: 1, 2, 3, 4, 10, 11, 12, 13 DUE WED 11/19.

11/14	Covered derivations of results on convergence in distribution, pp. 502-505,
	including Examples 12.7 and 12.8.

11/17	Discussed Central Limit Theorem (CLT) in terms of convergence in
	distribution, pp. 506-607.  Worked Example 12.10.  Covered new
	notes (pp. 196-202) on the CLT; click here for PostScript file (10 pages).
	(pdf version)

11/19	Derived Cental Limit Theorem.  Started Section 12.3 Almost Sure
	Convergence, including Examples 12.12, 12.13, 12.14, and 12/15.
	Assigned HW #11 Ch 12: 14, 18, 19, 20, 23, 25 DUE WED 11/26.

11/21	Finished Example 12.15.  Worked Example 12.16, drew pictures of g_n(x)
	for Example 12.17.  Worked Problems 30, 32, and 27 in Ch 12.

11/24	Started Chapter 10, Markov Chains.  Covered Examples 10.1, 10.2.
	Defined state space, transition probabilities, state transition
	diagram.  Discussed random walk, random walk with a barrier.
	Reflecting and absorbing barriers.

11/26	Discussed various MCs pp. 421-423.  Stationary Distributions,
	n-step transition probabilities, Chapman-Kolmogorov eq. p. 424.
	Assigned HW #12 Ch 12: 28, 29, 31, 33, 34, 39, 41, 43 DUE WED 12/3.

11/28	NO CLASS --- Thanksgiving Recess.

12/1	Worked Examples 10.3 and 10.4.  Gave alternative proofs using the
	smoothing property of the Chapman-Kolmogorov eq. and of the stationarity
	of the n-step transition probabilities.

12/3	Started Section 10.2 on Limit Distributions, pp. 429-432.
	Assigned HW #13 Ch 10: 3, 7, 8, 10 DUE WED 12/10.
	You may also want to work problems 1, 2, 4, 5, 6.

12/5	Continued discussion of Markov chains, pp. 432-434.
	Got side-tracked to discussion of specification of joint
	probability mass functions.  Click here for related PostScript handout.
	Click here for pdf version.
	Suggested review problems: Ch 11: 4, 13, 14, 20, 37, 58, 60, 61,
	and Ch 12: 36.

12/8	Started Other Characterizations of Transience and Recurrence, p. 434.
	Answered questions.
12/10	Answered questions.
12/12	Last Class Day.  Answered questions.