EE4C03 Statistical digital signal processing and modeling
Introduction
This is a second course in discretetime signal processing, with a focus on random signals. It provides a comprehensive treatment of signal processing algorithms for modeling discretetime signals, designing optimum filters, estimation of the power spectrum of a random process, and implementing adaptive filters. These are important topics that are frequently encountered in professional engineering, and major applications such as digital communication, array processing, biomedical signal processing and multimedia (speech and audio processing, image processing).
The course provides a framework that connects signal models to
filter structures, formulates filter design as an optimization problem,
solved in turn via linear algebra techniques applied to structured matrices.
The connections between these topics are strong, and provide insights that
can also be used in other disciplines.
The course treats:
 Brief refresher on DSP, linear algebra and random processes;
 Linear prediction, parametric methods such as Pade approximation, Prony's method and ARMA models;
 The YuleWalker equations;
 Wiener and Kalman filtering;
 Spectrum estimation (nonparametric and parametric), frequency estimation (Pisarenko, MUSIC algorithm);
 Adaptive filtering (LMS, RLS).
The course complements ET4386 Estimation and detection and ET 4147 Signal Processing for Communications.
This is a common course for the MSc EE (all tracks).
Preliminary knowledge
To follow the course with profit, you will need the background knowledge provided by an elementary course in Signals and Systems, in particular you need to know what is a discretetime Fourier transform (DTFT), a ztransform, and their properties. This can be found, e.g., in J.G. Proakis and D.G. Manolakis (Prentice Hall, 2007), chapters 24. You may consult the course EE2S11 Signals and Systems ) (video lectures in english available). In addition, you need basic notions of random signals, as shown in the course EE2S31 Signal Processing, and of Linear Algebra.
Exam
The exam is written, openbook. For the exam, you can bring the book (or a printout of the pdf) and copies of the slides. No written notes or other materials are allowed.The lab assignment is completed with a compact report. Passing the lab assignment is compulsory for the exam grade to become valid. Moreover, the assignment is graded and counts for 25% (+ 5% peer review) of your final grade.
Lab assignments
The course contains a compulsory lab assignment worth 1 EC (28 hours, 25% of your final grade, + 5% peer review). The assignment is done in groups of 2 students. Several trackdependent assignments are offered, which can be found on Brightspace. Indicate your selection via the 'group enroll' button in Brightspace.Each assignment typically consists of a problem description and a data set. Using the tools offered in the course, develop matlab algorithms to 'solve' the problem. The assignment is concluded with a short lab report. (If you wish, you could probably extend this report into a more fancy essay for EE4C01 Introduction to EE.) General remarks are:
 Make sure that the report has a connection to the course. The title of the course has 'statistical' and 'modeling', and you are expected to link the practical problem to the modeling and analysis techniques covered by the course. I.e., to have a structured approach.
 Make sure to describe the data model. Given a data set, how do you model it? What are the random parameters, what are the model assumptions? (Also, define/explain notation, and use it consistently.)
 Given the data model, what are possible approaches to estimate its parameters? This is linked to the properties of the data model. Often, you will be able to find algorithms related to the specified problem in literature, but take a step back and think of alternatives.
 When using an algorithm, consider also its properties. E.g., when using an FFT, what is the expected resolution and the variance of the spectrum estimate, and is that sufficient for its purpose? What are the parameters in the algorithm and how did you select them? Are the matlab graphs consistent with theory (= the model)? Do you need to apply windowing?
 Use a clear and concise reporting style: clear explanations, consistent use of notation, explanation of the graphs (what is shown, what do you observe, what do you conclude), conclusions and reference list.
Book
Monson H. Hayes, "Statistical digital signal processing and modeling", John Wiley and Sons, New York, 1996. ISBN: 0471 594318
Collegerama
The course has been videotaped in 2015 for viewing in Collegerama. You can view old episodes by using the direct links in the table below.(The first recording EE4C03_01 has failed to record some of the slides. The recording EE4C03_07 skipped the first minutes due to video recording problems. The recording of EE4C03_10 was made in 2016. In 2017 we inserted an additional introductory class on random processes that has not been videotaped; there may be overlap with EE4C03_03.)
Instructors
dr.ir. Justin Dauwels (JD), prof.dr.ir. Geert Leus (GL), prof.dr.ir. AlleJan van der Veen (AJ).
Schedule
The schedule for 2021 is as follows. Classes are on Wednesdays 10:4512:30 and Thursdays 13:4515:30. The slides as well as the latest recordings can be found on Brightspace.
Date  Book  Collegerama  

0.  Wed 1 Sep  JD & GL  Introduction to the course 
EE4C03_01
2020_01 

1.  Thu 2 Sep  GL  Background: ztransform, DTFT principles, (matrix algebra, complex gradients)  Ch.2  
2.a  Wed 8 Sep  JD  Refresher linear algebra  Ch.2 
EE4C03_02
2020_02 
2.b  Wed 8 Sep  JD  Computer exercises: linear algebra  
3.  Thu 9 Sep  JD  Refresher random processes; power spectra, spectral factorization, YuleWalker equations  Ch.3  EE4C03_02 
4.a  Wed 15 Sep  GL  Signal modeling (deterministic): Pade, Prony  Ch.4.14.4, 4.6  EE4C03_03 
4.b  Wed 15 Sep  JD  Computer exercises: basics of signals and filters  
5.  Thu 16 Sep  GL  Signal modeling (stochastic): allpole modeling, ARMA models  Ch.4.7  EE4C03_04 
6.  Wed 22 Sep  GL  Examples and exercises 
EE4C03_05
2020_05 

7.  Thu 23 Sep  JD  Computer exercises: Pade, Prony, ARMA  
8.  Wed 29 Sep  JD  Nonparametric spectrum estimation  Ch.8.2 (skip 8.2.6)  EE4C03_08 
9.  Thu 30 Sep  JD  Minimum variance spectrum estimation, Parametric spectrum estimation, Frequency estimation: Pisarenko, MUSIC  Ch.8.3, 8.5, 8.6 
EE4C03_09
2020_09 
10.  Wed 6 Oct  GL  Computer exercises: nonparametric spectrum estimation, minimum variance spectrum estimation, MUSIC  
11.  Thu 7 Oct  GL  Optimal FIR filtering: The Wiener filter, prediction, deconvolution, ...  Ch.7 (skip 7.4)  EE4C03_10 
12.  Wed 13 Oct  GL  Adaptive filters: LMS  Ch.9.1, 9.2 (skip 9.2.7, 9.2.8)  EE4C03_11 
13.  Thu 14 Oct  GL  Adaptive filters: RLS, the Kalman filter  Ch.9.4; Ch.7.4  EE4C03_12 
14.  Wed 20 Oct  JD  Computer exercises: LMS, RLS, Wiener/Kalman filter  
15.  Thu 21 Oct  Examples and exercises  EE4C03_13 
Previous exams
Note that the exams are open book, but you must be very familiar with the material to be able to solve the questions in time. There are 3 types of questions: (1) applying the theory directly, e.g. compute a covariance sequence; (2) show more indepth insight into one aspect of the theory, e.g. prove a certain property; (3) extend or apply the theory to a new situation. Train by solving many exercise questions from the book.Exam and solutions of January 2021.
Exam and solutions of November 2020.
Exam and solutions of January 2020.
Exam and solutions of November 2019.
Exam and solutions of January 2019.
Exam and solutions of November 2018.
Exam and solutions of January 2018.
Exam and solutions of November 2017.
Exam and solutions of January 2017.
Exam and solutions of November 2016.
Exercises
The book contains many exercises. Below is a list of suggested problems. A pdf of the Solutions Manual can probably be found on the internet.Chapter 3:  3.2; 3.3; 3.8; 3.11; 3.13; 3.25 
Chapter 4:  4.1; 4.2; 4.4; 4.5; 4.12; 4.14; 4.18; 4.20; 4.23 
Chapter 5:  5.5; 5.6; 5.8; 5.11; 5.14; 5.18; 5.20 
Chapter 7:  7.2; 7.5; (7.7; 7.12); 7.15; 7.17; 7.18 ; 7.20 
Chapter 8:  8.1; 8.2; 8.3; 8.5; 8.22 (b), (c) 
Chapter 9:  9.1; 9.3; 9.7; 9.8; 9.10; 9.11; 9.16; 9.17; 9.19 