ECE 735 Signal Synthesis and Recovery Techniques, Fall 2002

9/4     Wednesday.  Course overview.  Discussed span and linear
	independence.  Worked Problem 2-4.  Talked about sinc functions
	and the fact that the only function that is both time limited and
	bandlimited is the zero function.
	Assigned Reading: Section 1, pp. 1-3 (you may skip the proof
	of Lemma 1.5); Section 2, pp. 3-5.
	Assigned HW #1: Problems 2-5, 2-6, 2-7, 2-8 DUE WED 9/11.

9/6	Distributed syllabus.  Finished discussion of sampling theorem
	and approximately bandlimited functions.  Briefly discussed proof
	of Lemma 2.8.  Covered Definition 1.8 and Lemma 1.9.
	Distributed handout on Piecewise Polynomial Functions and covered
	p. 1.

9/9	Continued discussion of Piecewise Polynomial Functions handout.
	Discussed Section 1.2 Smooth Subspaces; Section 2.1 Knot Sequences;
	Section 2.2 Another Basis (the B-splines).  Look over pictures of
	B-splines on pp. 5-6 of handout.

9/11	Section 3: Introduced the least upper bound axiom, supremum,
	and infimum.  Stated the Bolzano-Weierstrass Theorem.
	Defined inner product space (beginning of Section 6), normed
	vector space (beginning of Section 5).  Started discussing
	metric spaces (Section 4).
	Assigned HW #2: Download the file xyfile.txt into the directory where
	you will use Matlab.  In Matlab, use the commands
		load xyfile.txt
		splinetool(x,y)
	Use the spline tool with least squares approximation selected and
	knots displayed.  Click on "Edit" and select "add knot" to add internal
	knots until you get a good fit.  When you are happy with the result,
	click on "File" and select "Print to Figure" to create an ordinary
	Matlab figure of what is shown in the spline tool.  From there you
	can print out your results.  Turn in your printout by WED 9/18.
	Also do problems 4-2, 4-4, and 4-9 DUE WED 9/18.

9/13	Said a few words reviewing splines.  Covered Section 4.1: Open and
	Closed Sets and Section 4.2: Some General Results.  You may skip
	Def. 4.8 through Proof of Lemma 4.9.  You may also skip the proofs
	of Theorems 4.13 and 4.14.

9/16	Convered Section 4.3 up through Definition 4.24 (Complete Space).
	Skipped proofs of Theorems 4.19 and 4.20.

9/18	Proved R and R^d are complete (Th 4.26, Cor. 4.27).  Showed that
	closed and bounded subsets of R^d are sequentially compact (Th 4.28).
	Section 4.4: Continuous Functions.  Skip all material following
	Def. 4.32 up to Th 4.34.  Stated Th 4.34 (f is cont. iff it is
	conv. preserving).  Proved that a continuous fcn on a seq. compact
	set is bounded and achieves its max and min values (Th 4.35).
	Assigned HW #3: Problems 4-12, 4-13, 4-14 (=> only) DUE WED 9/25.

9/20	Covered selected parts of Section 4.5: Compact Sets.  Discussed
	Defs. 4.36 and 4.37.  Skipped to Def 4.46 (Uniform Continuity)
	and proved Th 4.47 (Continuous fcn on a compact set is uniformly
	cont.)  Stated Th 4.44 (In a metric space a set is compact if and
	only if it is sequentially compact).  Started Section 5: Normed
	Vector Spaces.  Covered introductory material and Section 5.1:
	Projections --- Introduction.

9/23	Section 5.2: Finite Dim Subspaces.  Proved Lemma 5.4.  Discussed
	Problems 5-5, 5-6, 5-7, 5-8.  Stated Lemma 5.7.  Stated and discussed
	Theorem 5.8.  Proved Th 5.9 and Th 5.10.  Skipped Example 5.11.
	Stated Th 6.7 (Orthogonality Principle).

9/25	Derived the OP (but you should read the uniqueness proof on p. 20.)
	Derived the Cauchy-Schwarz inequality.  Derived (6.7), (6.8), (6.9),
	(6.11), and Bessel's inequality (6.14).  Section 6.2: Projections onto
	Infinite-Dimensional Spaces.
	Assigned HW #4: Problems 5-5, 5-6, 5-7, 5-8, 6-3, 6-5, 6-6, 6-10.
	DUE WED 10/2.

9/27	Section 6.3: Def of convex set.  Stated and proved the Projection Thm.
	Sums and direct sums of subspaces.  Section 6.4: Applications of the
	Projection Thm: Prop. 6.13 generalizes OP to convex sets.

9/30	Worked Problems 6-15, 6-16, 6-26.  Stated and proved the Strict
	Separation Th., 6.14.

10/2	Proved Farkas' Lemma 6.15.  Discussed Problem 6-23, and the following
	Remark on theorem's of the alternative.  Discussed the Karhunen-Loeve
	expansion and related eigenvalue problem for integral operators.
	Discussed ill-posed equations of the form Ax=y.
	Assigned HW #5: Problems 6-18, 6-20, 6-21, 6-24, 6-25 DUE WED 10/9.

10/4	Section 7.1: Linear Functionals.  Section 7.2: Linear Operators through
	proof of Proposition 7.12.

10/7	Went over solutions to Problems 5-5, 5-6, 5-7, 5-8.  Covered Def 7.13,
	Example 7.14, Def 7.15.  Worked Problem 7-17.  Started Section 7.3:
	Adjoint Operators.

10/9	Worked Problem 7-21 computing an adjoint operator.  Derived Prop. 7.17.
	Discussed Problem 7-25 and example following it.  Skipped Lemmas 7.18
	(Farkas) and 7.19 (Motzkin).  Proved Th 7.20.  Discussed Problem 7-29.
	Proved Lemmas 7.21 and 7.22.  Announced Exam 1 will be on 10/23.
	Assigned HW #6: Problems 7-3, 7-9, 7-13, 7-22, 7-29, 7-30, 7-31 DUE WED 10/16.

10/11	Discussed Problems 7-30 and 7-31.  Covered Section 7.3.1 Ellipses,
	Section 7.3.2 Projections onto range A, Started Section 7.4 Solving
	Linear Equations.

10/14	Finished Section 7.4.  Discussed the problem of estimating the
	unknown deterministic x from the observation Y = Fx + W.  Discussed
	Section 7.5: The Pseudoinverse.

10/16	Mentioned important results we have covered.
	Surveyed results we will cover in Sections 8 & 9.
	Review Problems: 4-10, 4-16, 6-4, 6-11(b), 6-17, 6-22, 6-28, 6-29,
	6-31, 6-32, 7-33, 7-34, 2 problems on board.

10/18	Worked Problems 7-33, 6-28, 6-29.  Talked a bit about Section 8 and
	developed the notion of a compact operator and working Problem 8-4,
	which equates a compact operator to the fact that the closure of the
	image of the unit sphere is sequentially compact.

10/21	Worked some of the review problems.

10/23	Exam 1 in class in 2317 EH.

10/25	Went over Exam 1.  Proved Lemmas 8.2 and 8.3.

10/28	Proved the invariance Lemma 8.4 and the Spectral Theorem.  Also
	discussed the Example on p. 37.

10/30	Mentioned Matlab commands for finding eigenvalues and eigenvectors
	of matrices, p. 37.  Discussed square roots of positive-semidefinite
	operators and matrices.  Discussed Fredholm equations of the 2nd
	kind when A is self adjoint, p. 38.  Discussed a special class of
	positive definite operators, p. 39.

11/1	Discussed Problems 8-1, 8-2, and 8-3.  Proved Theorem 8.6.

11/4	Section 8.2 The Singular-Value Decomposition (SVD), application to
	Fredholm equations of the 1st kind, ill-posedness of 1st-kind eqs.
	Assigned HW #7: Copy the 5 files main.m, Ggen.m, integrand.m,
	kernel.m, data.txt.  Start Matlab, and run the script file main.m,
	and print out resulting figure.  Now choose your own k(t,tau) and
	x(tau) and compute y(t) theoretically.  Do not use k(t,tau) of the form
	k(t,tau)=f(t)g(tau).  Do not use k(t,tau) and x(tau) so that y(t) is
	a polynomial or a piecewise polynomial.  Using your formula for y(t),
	write a script to generate vectors of t_i and y(t_i).  To check your
	work, use the command plot(t,y(t)) and print out your figure.  For
	plotting purposes, you may want to use vectors of length 100.  However,
	when using the data to recover x, you should start with only a few
	samples, say 5, until everything runs without errors.  Then you
	can increase the amount of data to try to get a better reconstruction.	
	Write your own version of kernel.m corresponding to your function
	k(t,tau).  Use the Matlab routines main.m, Ggen.m, integrand.m, and
	your version of kernel.m to reconstruct x(t).  You may also want to
	change the initial number of knots (set to 5) in main.m.  But start
	small, say 5, until everything runs without errors.  This number
	controls the number of basis functions.  Also Problems 8-3, 8-8(a)(b),
	8-9, 8-11 DUE WED 11/13.

11/6	Well-posedness of 2nd-kind Fredholm eqs.  Section 8.3 Regularization.
	Skipped Section 8.4.  Started Section 9: Optimization Using Lagrange
	Multipliers.  Section 9.1 Sufficient Conditions for Arbitrary Functions.
	Theorem 9.1.

11/8	Covered Section 9.2: Convex Functions.  Skipped Section 9.3.  Covered
	Section 9.4: One-Sided Gateaux Derivatives, Section 9.5: Kuhn-Tucker
	Sufficiency Theorems.

11/11	Section 9.6: More on Gateaux Derivatives.  Skipped Section 9.7.
	Section 9.8: Still More on the Gateaux Derivative.  Skipped Lemmas
	9.25 and 9.26, at least for now.  Prove Mean-Value Theorem 9.27.
	Worked Problem 9-14.

11/13	Section 9.9: The Frechet Derivative.  Skipped Section 9.10.
	Section 9.11: The Chain Rule and the Product Rule.  Discussed, but
	did not derive them.
	Assigned HW #8: Problems 9-1, 9-4, 9-6, 9-8, 9-9 DUE WED 11/20.

11/15	Section 9.12: Applications.

11/18	Discussed part of Frank Kelly's paper
	Charging and Rate Control for Elastic Traffic.

11/20	Discussed regularization for last Matlab HW.  Discussed Matlab
	matrix-vector operations for computing the SVD series representation
	and the regularized x_alpha in eq. (8.23).  Worked some examples
	for directly computing Frechet derivatives.
	Assigned HW #9: 9-12, 9-13, 9-15, 9-19.  Also modify the last Matlab HW to
	use regularization, as discussed today in class, to try to improve
	your results.  DUE WED 11/27.
	Announced Exam 2 will be an evening exam on Wed. 12/4.

11/22	Worked another example of computing a Frechet derivative.  Started
	Section 9.13 Functions Concentrated in Time and Frequency thru (9.40).

11/25	Continued Section 9.13 through Proof of Lemma 9.42.

11/27	Concluded discussion of Section 9.13.  Derived the Contraction
	Mapping Theorem and discussed some of its applications.
	Review Problems: 8-6, 8-7, 8-8(c)(d), 8-10, 8-13, 8-15, 9-2, 9-10,
	9-11, 9-20.

11/29	No Class --- Thanksgiving Recess.

12/2	Worked some of the Review problems.

12/4	Worked more of the Review problems.
	Also Exam 2 in 3534 EH, 5-7 pm
	You may bring 2 sheets of 8.5 x 11 in. paper with any formulas
	you believe may be helpful.

12/6	Discussed entropy, log inequality, the forward part of Shannon's
	source coding theorem.

12/9	Went over Exam 2 solutions and covered handout on Lloyd-Max
	quantizer design.

12/11	Started discussion of Arimoto-Blahut Algorithm for computing
	channel capacity.

12/13	Last Class Day.