40719 Deep Learning

Hamid Beigy, Sharif University of Technology, Spring Semester 2020-21.

Course Description

The course helps to understand the fundamentals of Deep Learning. The course starts off gradually with multi-layer preceptrons and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. This course also covers other models of deep learning such as convolutional neural networks, recurrent neural networks, deep generative models such as autoregressive, GAN, VAE, NFM, representation learning, and deep reinforcement learning methods. We use frameworks such as PyTorch and Tensorflow, which are very important for implementing deep Learning models.

Course Information

Instructor: Hamid Beigy
Teaching Assistants: Fariba Lotfi, Fatemeh Karimkhani, Aryan Sadeghi, Fatemeh Sarshar Tehrani, Amirhossein Noohian, Amirhossein Youssefi, Amirhossein Sohrabbeig, Fatemeh Sarhsar Tehrani, Maryam Afshari, Maryam Riyahi Madvar, Mostafa Norouzi, Paria Kashani, Yasaman Boreshban
Time and Location: Saturday/Monday: 10:30 am -12:00 am, virtual class
Contact: Please contact the teaching staffs on Piazza site.

Required Texts

[GYC] Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016.
[DY] Li Deng and Dong Yu, Foundations and Trends in Signal Processing, Vol. 7, No. 3–4, pp 197-387, 2014.
[SB] Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, Second edition. The MIT Press, 2018.

Grading Policy

15%: Mid-term exam 1 (1400/01/19).
15%: Mid-term exam 2 (1400/02/22).
20%: Final exam (1400/04/07).
30%: Homeworks.
15%: Quiz.
5%: Paper & Explore a theoretical or empirical question and present it. Deadline for choosing paper: 1400/02/25.

Lecture Schedule

Lecture	Date	Topics	Related Readings and Links	Homework & Assignments
1	1399-11-25	Introduction: what is deep learning?	Chapter 1 of GYC Chapter 1 of DY
2 3	1399-11-27 1399-12-02	Machine learning overview	Chapter 4 of GYC
4 5	1399-12-04 1399-12-09	Feedforward deep networks & backpropagation	Chapter 6 of GYC
6 7	1398-12-11 1398-12-16	Optimization and regularization	Chapters 7 & 8 of GYC Papers given in the slides
8 9 10	1399-12-18 1399-12-23 1399-12-25	Convolutional networks	Chapter 9 of GYC Papers given in the slides	Homework 1 (Due date: 1400-01-06) Quiz 1
	1400-01-06			Homework 2 (Due date: 1400-01-27)
11 12	1400-01-14 1400-01-16	Recurrent neural networks	Chapter 10 of GYC Papers given in the slides
	1400-01-19	The first exam		At 13-15
13 14	1400-01-21 1400-01-23	Recurrent neural networks	Chapter 10 of GYC Papers given in the slides	Quiz 2
15 16 17 18	1400-01-28 1400-01-30 1400-02-04 1400-02-06	Representation learning	Chapter 14 of GYC & Papers given in the slides	Homework 3 (Due date: 1400-02-18) Quiz 3
19 20	1400-02-11 1400-02-13	Attention models	Papers given in the slides
21	1400-02-18	Dual Learning Sum-product networks	Papers given in the slides
	1400-02-22	The second exam		At 13:30-15:30
22 23 24 25 26	1400-02-20 1400-02-25 1400-02-27 1400-03-01 1400-03-03	Deep generative models	Chapter 20 of GYC Papers given in the slides	Quiz 4
	1400-03-06			Homework 4 (Due date: 1400-03-18)
27	1400-03-08	Deep generative models Deep reinforcement learning	Papers given in the slides Chapters 1 to 6 and 13 of SB
28 29	1400-03-10 1400-03-15	Deep reinforcement learning	Chapters 1 to 6 and 13 of SB Papers given in the slides	Quiz 5
30	1400-03-17	Grap neural networks	Papers given in the slides
	1400-03-19			Homework 5 (Due date: 1400-04-25)
	1400-04-07	The final exam		At 9-12

Problem sets

Problem set 1 Due 1400-01-06
Problem set 2 Due 1400-01-27
Problem set 3 Due 1400-02-18
Problem set 4 Due 1400-03-18
Problem set 5 Due 1400-04-25