EE4C03 Statistical digital signal processing and modeling
IntroductionThis is a second course in discrete-time signal processing, with a focus on random signals. It provides a comprehensive treatment of signal processing algorithms for modeling discrete-time signals, designing optimum filters, estimation of the power spectrum of a random process, and implementing adaptive filters. These are important topics that are frequently encountered in professional engineering, and major applications such as digital communication, array processing, biomedical signal processing and multimedia (speech and audio processing, image processing).
The course provides a framework that connects signal models to
filter structures, formulates filter design as an optimization problem,
solved in turn via linear algebra techniques applied to structured matrices.
The connections between these topics are strong, and provide insights that
can also be used in other disciplines.
The course treats:
- Brief refresher on DSP, linear algebra and random processes;
- Linear prediction, parametric methods such as Pade approximation, Prony's method and ARMA models;
- The Yule-Walker equations;
- Wiener and Kalman filtering;
- Spectrum estimation (nonparametric and parametric), frequency estimation (Pisarenko, MUSIC algorithm);
- Adaptive filtering (LMS, RLS).
This is a common course for the MSc EE (all tracks).
Preliminary knowledgeTo follow the course with profit, you will need the background knowledge provided by an elementary course in Signals and Systems, in particular you need to know what is a discrete-time Fourier transform (DTFT), a z-transform, and their properties. This can be found, e.g., in J.G. Proakis and D.G. Manolakis (Prentice Hall, 2007), chapters 2-4. You may consult the course EE2S11 Signals and Systems ) (video lectures in english available). In addition, you need basic notions of random signals, as shown in the course EE2S31 Signal Processing, and of Linear Algebra.
ExamThe exam is written, open-book. For the exam, you can bring the book (or a print-out of the pdf) and copies of the slides. No written notes or other materials are allowed.
The lab assignment is completed with a compact report. Passing the lab assignment is compulsory for the exam grade to become valid. Moreover, the assignment is graded and counts for 25% (+ 5% peer review) of your final grade.
Lab assignmentsThe course contains a compulsory lab assignment worth 1 EC (28 hours, 25% of your final grade, + 5% peer review). The assignment is done in groups of 2 students. Several track-dependent assignments are offered, which can be found on Brightspace. Indicate your selection via the 'group enroll' button in Brightspace.
Each assignment typically consists of a problem description and a data set. Using the tools offered in the course, develop matlab algorithms to 'solve' the problem. The assignment is concluded with a short lab report. (If you wish, you could probably extend this report into a more fancy essay for EE4C01 Introduction to EE.) General remarks are:
- Make sure that the report has a connection to the course. The title of the course has 'statistical' and 'modeling', and you are expected to link the practical problem to the modeling and analysis techniques covered by the course. I.e., to have a structured approach.
- Make sure to describe the data model. Given a data set, how do you model it? What are the random parameters, what are the model assumptions? (Also, define/explain notation, and use it consistently.)
- Given the data model, what are possible approaches to estimate its parameters? This is linked to the properties of the data model. Often, you will be able to find algorithms related to the specified problem in literature, but take a step back and think of alternatives.
- When using an algorithm, consider also its properties. E.g., when using an FFT, what is the expected resolution and the variance of the spectrum estimate, and is that sufficient for its purpose? What are the parameters in the algorithm and how did you select them? Are the matlab graphs consistent with theory (= the model)? Do you need to apply windowing?
- Use a clear and concise reporting style: clear explanations, consistent use of notation, explanation of the graphs (what is shown, what do you observe, what do you conclude), conclusions and reference list.
Monson H. Hayes, "Statistical digital signal processing and modeling", John Wiley and Sons, New York, 1996. ISBN: 0-471 59431-8
CollegeramaThe course has been videotaped in 2015 for viewing in Collegerama. You can view old episodes by using the direct links in the table below.
(The first recording EE4C03_01 has failed to record some of the slides. The recording EE4C03_07 skipped the first minutes due to video recording problems. The recording of EE4C03_10 was made in 2016. In 2017 we inserted an additional introductory class on random processes that has not been videotaped; there may be overlap with EE4C03_03.)
Instructorsdr.ir. Justin Dauwels (JD), prof.dr.ir. Geert Leus (GL), prof.dr.ir. Alle-Jan van der Veen (AJ).
ScheduleThe schedule for 2021 is as follows. Classes are on Wednesdays 10:45-12:30 and Thursdays 13:45-15:30. The slides as well as the latest recordings can be found on Brightspace.
|0.||Wed 1 Sep||JD & GL||Introduction to the course||
|1.||Thu 2 Sep||GL||Background: z-transform, DTFT principles, (matrix algebra, complex gradients)||Ch.2|
|2.a||Wed 8 Sep||JD||Refresher linear algebra||Ch.2||
|2.b||Wed 8 Sep||JD||Computer exercises: linear algebra|
|3.||Thu 9 Sep||JD||Refresher random processes; power spectra, spectral factorization, Yule-Walker equations||Ch.3||EE4C03_02|
|4.a||Wed 15 Sep||GL||Signal modeling (deterministic): Pade, Prony||Ch.4.1-4.4, 4.6||EE4C03_03|
|4.b||Wed 15 Sep||JD||Computer exercises: basics of signals and filters|
|5.||Thu 16 Sep||GL||Signal modeling (stochastic): all-pole modeling, ARMA models||Ch.4.7||EE4C03_04|
|6.||Wed 22 Sep||GL||Examples and exercises||
|7.||Thu 23 Sep||JD||Computer exercises: Pade, Prony, ARMA|
|8.||Wed 29 Sep||JD||Nonparametric spectrum estimation||Ch.8.2 (skip 8.2.6)||EE4C03_08|
|9.||Thu 30 Sep||JD||Minimum variance spectrum estimation, Parametric spectrum estimation, Frequency estimation: Pisarenko, MUSIC||Ch.8.3, 8.5, 8.6||
|10.||Wed 6 Oct||GL||Computer exercises: nonparametric spectrum estimation, minimum variance spectrum estimation, MUSIC|
|11.||Thu 7 Oct||GL||Optimal FIR filtering: The Wiener filter, prediction, deconvolution, ...||Ch.7 (skip 7.4)||EE4C03_10|
|12.||Wed 13 Oct||GL||Adaptive filters: LMS||Ch.9.1, 9.2 (skip 9.2.7, 9.2.8)||EE4C03_11|
|13.||Thu 14 Oct||GL||Adaptive filters: RLS, the Kalman filter||Ch.9.4; Ch.7.4||EE4C03_12|
|14.||Wed 20 Oct||JD||Computer exercises: LMS, RLS, Wiener/Kalman filter|
|15.||Thu 21 Oct||Examples and exercises||EE4C03_13|
Previous examsNote that the exams are open book, but you must be very familiar with the material to be able to solve the questions in time. There are 3 types of questions: (1) applying the theory directly, e.g. compute a covariance sequence; (2) show more in-depth insight into one aspect of the theory, e.g. prove a certain property; (3) extend or apply the theory to a new situation. Train by solving many exercise questions from the book.
Exam and solutions of January 2021.
ExercisesThe book contains many exercises. Below is a list of suggested problems. A pdf of the Solutions Manual can probably be found on the internet.
|Chapter 3:||3.2; 3.3; 3.8; 3.11; 3.13; 3.25|
|Chapter 4:||4.1; 4.2; 4.4; 4.5; 4.12; 4.14; 4.18; 4.20; 4.23|
|Chapter 5:||5.5; 5.6; 5.8; 5.11; 5.14; 5.18; 5.20|
|Chapter 7:||7.2; 7.5; (7.7; 7.12); 7.15; 7.17; 7.18 ; 7.20|
|Chapter 8:||8.1; 8.2; 8.3; 8.5; 8.22 (b), (c)|
|Chapter 9:||9.1; 9.3; 9.7; 9.8; 9.10; 9.11; 9.16; 9.17; 9.19|