伯克利深度学习专题课程：对抗生成网络创始人首次剖析训练实例（44PDF下载）

　　 Lecture 1

　　 8/31/16

　　 何恺明: Deep Learning Gets Way Deeper [P PT]

　　 主要阅读材料：

1. Deep Learning, by Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.

　　 Lecture 2

　　 9/7/16

　　 Ilya Sutskever : Why does deep learning actually work?

　　 主要阅读材料：

1. Explaining and Harnessing Adversarial Examples, by Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy.

2. Qualitatively Characterizing Neural Network Optimization Problems, by Ian J. Goodfellow, Oriol Vinyals, and Andrew M. Saxe.

　　 背景阅读：

1. Algorithmic probability, Wikipedia

2. Does Algorithmic Probability Solve the Problem of Induction?, by Ray Solomonoff

3. A Theory of the Learnable, by Leslie Valiant

4. On Small Depth Threshold Circuits, by Alexander A. Razborov (read table on last page)

　　 Lecture 3

　　 9/14/16

　　 Wojciech Zaremba: Turing-complete neural-network based models

　　 主要阅读材料：

1. Extensions and Limitations of the Neural GPU, by Eric Price, Wojciech Zaremba, and Ilya Sutskever

2. Learning Simple Algorithms from Examples, by Wojciech Zaremba, Tomas Mikolov, Armand Joulin, and Rob Fergus

　　 背景阅读：

1. Learning Efficient Algorithms with Hierarchical Attentive Memory, by Marcin Andrychowicz and Karol Kurach

2. Neural Random-Access Machines, by Karol Kurach, Marcin Andrychowicz, and Ilya Sutskever

3. Neural Programmer-Interpreters, by Scott Reed and Nando de Freitas

　　 Lecture 4

　　 9/21/16

　　 Kunal Talwar: Deep Learning with Differential Privacy

　　 主要阅读材料：

1. Deep Learning with Differential Privacy by Abadi, Chu, Goofellow, McMahan, Mironov, Talwar and Zhang

2. Chapters 1 and 2 of The Algorithmic Foundations of Differential Privacy by Cynthia Dwork and Aaron Roth

　　 背景阅读：

1. Chapter 3 of The Algorithmic Foundations of Differential Privacy

2. Differentially Private Empirical Risk Minimization by Bassily, Smith and Thakurta (Sections 1 and 2)

3. Analyze Gauss: Optimal bounds for privacy-preserving PCA by Dwork, Talwar, Thakurta and Zhang

　　 Lecture 5

　　 9/27/16

　　 (Special date)

　　 Yoshua Bengio

　　 Location: Wozniak Lounge (430-8 Soda Hall)

　　 Time: 12:30 pm

　　 Lecture 6 ? ?

　　 9/28/16

　　 Samy Bengio: Sequence models

　　 主要阅读材料：

1. Order Matters: Sequence to sequence for sets, by Oriol Vinyals, Samy Bengio, and Manjunath Kudlur

2. Reward Augmented Maximum Likelihood for Neural Structured Prediction, by Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, and Dale Schuurmans

　　 背景阅读：

1. Sequence to Sequence Learning with Neural Networks, by Ilya Sutskever, Oriol Vinyals, and Quoc V. Le

2. Pointer Networks, by Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly

3. Sequence Level Training with Recurrent Neural Networks, by Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba

　　 10/5/16

　　 Ian Goodfellow: Adversarial Examples and Adversarial Training

　　 主要阅读材料：

1. Explaining and Harnessing Adversarial Examples, by Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy

2. Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples, by Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow

　　 背景阅读：

1. Intriguing properties of neural networks, by Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus

　　 Lecture 8

　　 10/12/16

　　 Oriol Vinyals : Sequences and one-shot learning

　　 主要阅读材料：

1. Matching Networks for One Shot Learning, by Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra

2. One-shot Learning with Memory-Augmented Neural Networks, by Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap

　　 Lecture 9

　　 10/19/16

　　 Bill Dally: Hardware for Deep Learning

　　 主要阅读材料：

1. EIE: Efficient Inference Engine on Compressed Deep Neural Network, by Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A. Horowitz, and William J. Dally

2. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding, by Song Han, Huizi Mao, and William J. Dally

　　 背景阅读：

1. Deep learning with COTS HPC systems, by Adam Coates, Brody Huval, Tao Wang, David J. Wu, Andrew Y. Ng, and Bryan Cantanzaro

　　 Lecture 10

　　 10/26/16

　　 Hugo Larochelle: Autoregressive Generative Models with Deep Learning

　　 主要阅读材料：

1. Neural Autoregressive Distribution Estimation, by Benigno Uria, Marc-Alexandre Côté, Karol Gregor, Iain Murray, and Hugo Larochelle

　　 背景阅读：

1. Deep Learning, Chapter 20 (Generative Models)

2. Sections 20.1 to 20.4 (background on undirected graphical models)

3. Sections 20.9 and 20.10.1 to 20.10.5 (background on directed graphical models)

　　 Lecture 11

　　 11/2/16

　　 Lecture 12

　　 11/9/16

　　 Lecture 13

　　 11/16/16

　　 Lecture 14

　　 11/23/16

　　 Lecture 15

　　 11/30/16