ECE 730 Modern Probability Theory and Stochastic Processes, Lec. 1, Fall 2002
Last Modified: Tue 15 Oct 2019, 01:45 PM, CDT
Instructor Office Hours:
Java
Demos for Probability and Statistics
(by
Prof.
Stanton,
Cal. State Univ., San Bernardino)
Syllabus
Class Schedule for Fall 2002
9/4 Wednesday. Section 1.3 Axioms and Properties of Prob.
Also sigma-algebra of events (Note 1 in Chapter 1). Emphasis
on the limit properties of probability, eqs. (1.2)-(1.4).
Worked Example 1.15. Definition of random variable, Borel
sigma-algebra (Note 1 in Chapter 2). Use of limit properties
of probability to derive properties of cdfs (Section 4.6),
and joint cdfs (Note 2 in Chapter 5).
Assigned HW #1 Ch 1: 11, 12, 13, 14; Ch 2: 12, 41, 45, 63 DUE WED 9/11.
9/6 Finished going over Problem 10 in Ch. 1. Worked max/min Example 2.7
and first occurrence Example 2.8. Discussed Law of Total Probability
as on p. 21 in Ch. 1. Applied to random variables in Example 2.32
(Signal in Additive Noise). Defined continuous random variables
(p. 85). Started developing expectation for arbitrary random
variables (Section 3.2).
9/9 Discussed the Law of Total Probability and the Substitution Law
for discrete RVs, pp. 70-71. Recommended you read Example 2.36.
Finished development of expectation for arbitrary RVs (Section 3.2).
Derived expectation formula (4.2) on pp. 123-134. Derived the
Central Limit Theorem (stated on pp. 137-138; derivation of the
general case starts on the second half of p. 140).
9/11 Ch 3: Discussed the gamma function and gamma density as in Problem 11.
Introduced a scale parameter (see Problem 12) leading to the Erlang
and chi-squared RVs. Worked Problem 13 concerning the beta density
and beta function. Pointed out some simple consequences of
formula (3.6). Started discussing Problem 54 on the noncentral
chi-squared density.
Assigned HW #2 Ch 3: 11(a)(b)(read c), 14+Read Rmk, 15, 16,
17+Read Rmk, 19, 29, 30(b), 31, 40, 41, 42, 46.
Ch 4: 33, 44, 45 DUE WED 9/18.
9/13 Finished discussion of Problem 54 in Ch 3. In Ch 4, discussed
Problems 13 and 17.
9/16 Distributed handout with solution of Problem 17(a)(c) in Ch 4.
Defined jointly continuous RVs pp. 158-159. Discussed "Continuous
RVs That Are not Jointly Continuous" on p. 164. Discussed conditional
expection for jointly continuous RVs pp. 164-165. You may want to
read Examples 5.7, 5.8, and 5.9. Worked Problems 22, 23 in Ch 5.
9/18 Worked Problem 24 in Ch 5. Started Ch 6 on Random Processes.
Discussed "What is a random process?" Section 6.1: Introduced the mean
function, the correlation and covariance functions. Worked Example 6.1.
Section 6.2: WSS, strict-sense stationarity. Worked Example 6.2.
Assigned HW #3 Ch 6: 2, 10, 15, 21, 22 DUE WED 9/25.
9/20 Finished Section 6.2. Skipped examples in class, but you should read
them on your own. Covered Section 6.3: WSS Processes through LTI
Systems. Started Section 6.4: The Matched Filter.
9/23 Finished deriving the matched filter. Derived the Wiener filter
(Section 6.5).
9/25 Causal Wiener filters, pp. 203-206. Skipped Example 6.11, but you
should read it yourself. Section 6.6 Expected Time-Average Power
and the Wiener-Khinchin Theorem, pp. 206-208. Started Mean-Square
Law of Large Numbers for WSS Processes.
Assigned HW #4: Ch 6: 28, 29, 33, 34, 38, 39 DUE WED 10/2.
9/27 Distributed handout on Mean-Square Law of Large Numbers and lectured
from it. Started Ch 7. Covered Section 7.1. Started Section 7.2:
The Multivariate Gaussian, pp. 226-227.
9/30 Covered the characteristic function of a Gaussian random vector, p. 227.
Showed that for Gaussian random vectors, uncorrelated implies
independent, p. 228. Worked Example 7.5. Derived density of Gaussian
random vector, pp. 229-230.
10/2 Section 7.3: Estimation of Random Vectors.
Assigned HW #5 Ch 7: 1, 5(b)(c), 6, 9, 10, 11, 12, 13 DUE WED 10/9.
10/4 Section 7.4: Transformations of Random Vectors.
Highlights from Section 7.5: Complex Random Variables and Vectors.
10/7 Worked Problem 20 in Ch 7. Started Ch 8, Section 8.1: The Poisson
Process. Skipped Examples 8.1 and 8.2 and computation of E[N_t]
and cov(N_t,N_s) on p. 251 --- You should read these on your own.
Began subsection "Derivation of the Poisson Probabilities," pp. 253-255.
10/9 Finished derivation of the Poisson probabilities. Briefly discussed
marked Poisson processes, p. 255, shot noise, p. 256, and Section 8.2:
Renewal Processes, pp. 256-257. Started Section 8.3: The Wiener Process.
Assigned HW #6 Ch 7: 14, 17, 21, 23, 29 DUE WED 10/16.
10/11 Finished Section 8.3, including The Problem with White Noise, The Wiener
Integral, and the Random Walk Approximation of the Wiener Process.
10/14 Discussed Section 8.4: Specification of Random Processes. Skipped
discussion on second half of p. 267 through Example 8.4 on p. 269,
but you are welcome to read them on your own. Briefly discussed
continuous-time random processes on p. 269.
10/16 Worked Problems 26, 27, and discussed 18, 19, 20 in Ch 8. Mentioned
stochastic differential equations.
Assigned HW #7 Ch 8: 6, 9, 10, 11, 14, 16, 17 DUE WED 10/23.
10/18 Finshed Problem 27. Started Chapter 9, Intro to Markov Chains. Started
Section 9.1 Discrete-Time MCs. Worked Examples 9.1 and 9.2. Discussed
State transition diagrams, transition matrix, etc. on pp. 281-283.
10/21 Discussed n-step transition probabilities, Chapman-Kolmogorov eq.,
and stationary distributions of discrete-time MCs. Worked Example 9.3.
Skipped derivation of discrete-time Chapman-Kolmogorov eq. and
stationarity of n-step transition probabilities on pp. 287-288.
Started Section 9.2 Continuous-Time MCs. Worked Example 9.4 showing
that a Poisson process is a continuous-time MC with stationary
transition probabilities. Derived the Chapman-Kolmogorov eq. for
continuous-time MCs.
10/23 Gave heuristic derivations of Kolmogorov's forward and backward
differential equations. Skipped rigorous derivation of backward
equation. Started Chapter 10, Section 10.1 Convergence in Mean of
Order p. Worked Examples 10.1 and 10.2.
Suggested Review Problems. Ch 6: 11, 27, 36.
Ch 7: 22(note typo; should be V = sqrt(-2 ln X) sin(2 pi Y)),
Repeat Example 7.7 if instead of X and Y being independent,
E[ X Y ] = rho. Find F_{R Theta} and F_{Theta}.
Let Y = q_1 X_1 + ... + q_n X_n + W, where W, X_1, ... , X_n
are jointly normal. Find E[ X_1 | Y=y ].
Ch 8: 5, 8, 15, 21, 22, 24.
10/25 Worked Examples 10.3, 10.4, 10.5, and 10.6. Started Section 10.2:
Normed Vector Spaces of Random Variables.
10/28 Answered questions/Exam review.
10/30 Exam 1. Starts early at 1pm in usual classroom.
11/1 Went over Exam 1. Showed that the set of RVs with finite pth
moment is closed under linear combinations. Defined the p norm.
11/4 Defined Cauchy sequences of real numbers and random variables in L^p.
Defined complete space. Fact: The real numbers are complete, and
L^p is complete. Defined inner product of random variables. Stated
parallelogram law, Cauchy-Schwarz inequality. Worked Example 10.7.
In all, covered pp. 303-306.
11/6 Section 10.3: The Wiener Integral (Again). Section 10.4: Projections,
Orthogonality Principle, Projection Theorem. Covered pp. 308-309 upto,
but not including Example 10.9.
Assigned HW #8 Ch 9: 2, 3, 4, 7, 9, 10; Ch 10: 1, 2 DUE WED 10/13.
11/8 Worked Problems in Ch 10: 6, 9, 10, 15, 16, 18, 25.
11/11 Worked Problem 12 in Ch 10. Derived the Projection Theorem and used it
to derive the formula for the projection of a RV X onto the closed
subspace of Wiener integrals.
11/13 Covered Section 10.5: Conditional Expectation. Covered derivations for
L^2 RVs. For L^1 RVs, covered results but not derivations. Started
Section 10.6: The Spectral Representation, pp. 313-314.
Assigned HW #9 Ch 10: 7, 8, 11, 13, 17, 22, 23 DUE WED 11/20.
11/15 Finished Spectral Representation. Added simple proof that for a zero-mean
WSS process the sample mean converges in mean square zero.
11/18 Finished discussion of convergence in probability. Began discussion of
Section 11.2: Convergence in Distribution.
11/20 Continued discussion of convergence in distribution. Almost finished
solution of Example 11.8.
Assigned HW #10 Ch 10: 32; Ch 11: 2, 3, 5, 10, 11, 12,
13(note missing sigma^2; use Example 11.6, not 11.5), 14 DUE WED 11/27.
Announced Exam 2 will be an evening exam on Wed. 12/11.
11/22 Finished Example 11.8. Worked Problem 31 in Ch 10. Showed that
conditional expectation is linear. Worked Problem 9 in Ch 11.
11/25 When X and Y are independent, showed that the conditional expectation
of X given Y is E[X]. Worked Problem 15 in Ch 11. Started
Section 11.3: Almost Sure Convergence, pp. 330-331.
11/27 Worked problems from HW #10. Showed that almost-sure convergence
implies convergence in probability (Ex. 11.10). Worked Ex. 11.11
giving a sufficient condition for almost-sure convergence.
Assigned HW #11 Ch 11: 16, 18, 20, 22(typo, should be F(z/c)),
25, 26, 27, 29, 30 DUE WED 12/4.
11/29 No Class --- Thanksgiving Recess.
12/2 Worked Example 11.12 proving the strong law of large numbers (SLLN) for
i.i.d. RVs with finite 4th moment. Discussed the general SLLN.
Discussed the WLLN for i.i.d. RVs with only finite first moment.
12/4 Did teaching evaluations. Worked Problem 4 in Ch 11 and Problem 29
in Ch 10.
Review Problems: Old HW & old review problems;
Ch 7: 4; Ch 9: 11, 12, 13; Ch 10: 19, 21, 27;
Ch 11: 7, 23, 24, 28, 35, 36, 37, 38.
12/6 Worked review problems.
12/9 Worked Example similar to Examples 11.11 and 11.12 in the notes.
Worked Problem 12 in Ch 9, Problems 20, 27 in Ch 10, Problems
36, 37 in Ch 11.
12/11 Worked review problems. Discussed laws of large numbers.
Also Exam 2 in 159 Mech Engr, 5-7 pm
You may bring 2 sheets of 8.5 x 11 in. paper with any formulas
you believe may be helpful.
12/13 Last Class Day.
Web Page Contact: John (dot) Gubner (at) wisc (dot) edu