Dive into Deep Learning¶

Dive into Deep Learning
Interactive deep learning book with code, math, and discussions
Implemented with NumPy/MXNet, PyTorch, and TensorFlow
Adopted at 175 universities from 40 countries
Announcements
- [Jan 2021] Check out the brand-new Chapter: Attention Mechanisms. We have also completed PyTorch implementations. To keep track of the latest updates, please follow D2L's open-source project.
- [Oct 2020] We have added TensorFlow implementations up to Chapter 7 (Modern CNNs).
- [Apr 2020] We have revamped Chapter: NLP pretraining and Chapter: NLP applications, and added sections of BERT and natural language inference.
- [Jul 2019] The Chinese version is the No. 1 best seller of new books in "Computers and Internet" at the largest Chinese online bookstore.
- [May 2019] Slides, Jupyter notebooks, assignments, and videos of the Berkeley course can be found at the syllabus page.
Each section is an executable Jupyter notebook
You can modify the code and tune hyperparameters to get instant feedback to accumulate practical experiences in deep learning.

Mathematics + Figures + Code
We offer an interactive learning experience with mathematics, figures, code, text, and discussions, where concepts and techniques are illustrated and implemented with experiments on real data sets.




Active community support
You can discuss and learn with thousands of peers in the community through the link provided in each section.
D2L as a textbook or a reference book






Alexandria University
Amity University
Ateneo de Naga University
Australian National University
Birla Institute of Technology and Science, Hyderabad
Brown University
Cairo University
Carnegie Mellon University
College of Engineering Pune
Columbia University
Duke University
Durban University of Technology
Emory University
Eötvös Loránd University
Federal University Lokoja
Fudan University
Gayatri Vidya Parishad College of Engineering (Autonomous)
Gazi Üniversitesi
George Mason University
Georgia Institute of Technology
Golden Gate University
Habib University
Hangzhou Dianzi University
Hankuk University of Foreign Studies
Harbin Institute of Technology
Hasso-Plattner-Institut
Heinrich-Heine-Universität Düsseldorf
Hertie School
Hiroshima University
Ho Chi Minh City University of Foreign Languages and Information Technology
Hochschule Bremen
Hochschule für Technik und Wirtschaft
Hong Kong University of Science and Technology
Huazhong University of Science and Technology
Imperial College London
IMT Mines Alès
Indian Institute of Technology Bombay
Indian Institute of Technology Jodhpur
Indian Institute of Technology Kanpur
Indian Institute of Technology Kharagpur
Indian Institute of Technology Mandi
Indian Institute of Technology Ropar
Indira Gandhi National Open University
Indraprastha Institute of Information Technology, Delhi
Institut catholique d'arts et métiers (ICAM)
Institut de recherche en informatique de Toulouse
Institut Supérieur d'Informatique et des Techniques de Communication
Institut Supérieur De L'electronique Et Du Numérique
Instituto Tecnológico Autónomo de México
İstanbul Teknik Üniversitesi
IT-Universitetet i København
King Abdullah University of Science and Technology
Kongu Engineering College
KPR Institute of Engineering and Technology
Kyungpook National University
Lancaster University
Leibniz Universität Hannover
Leuphana University of Lüneburg
London School of Economics & Political Science
Make School
Massachusetts Institute of Technology
McGill University
Milwaukee School of Engineering
Minia University
Monash University
Multimedia University
National Chung Hsing University
National Institute of Technical Teachers Training&Research
National Institute of Technology, Warangal
National Sun Yat-sen University
National Taiwan University
National Technical University of Athens
National United University
National University of Singapore
Nazarbayev University
New York University
Newman University
North Ossetian State University
Northeastern University
Ohio University
Pakuan University
Peking University
Pohang University of Science and Technology
Politecnico di Milano
Pontificia Universidad Católica de Chile
Portland State University
Purdue University
Queen's University
Radboud Universiteit
Rowan University
Rutgers, The State University of New Jersey
Sapienza Università di Roma
Shanghai Jiao Tong University
Shanghai University of Finance and Economics
Shenzhen University
Sogang University
Southern New Hampshire University
St. Pölten University of Applied Sciences
Stanford University
Stevens Institute of Technology
Sungkyunkwan University
Technische Universiteit Delft
Tekirdağ Namık Kemal Üniversitesi
Texas A&M University
Thapar Institute of Engineering and Technology
The State University of New York at Binghamton
The University of Texas at Austin
Tsinghua University
Universidad Carlos III de Madrid
Universidad de Zaragoza
Universidad Militar Nueva Granada
Universidad Nacional Agraria La Molina
Universidad Nacional de Colombia Sede Manizales
Universidad Tecnológica de Pereira
Universidade Católica de Brasília
Universidade Estadual de Campinas
Universidade Federal de Minas Gerais
Universidade Federal de Ouro Preto
Universidade Federal do Rio Grande
Universidade NOVA de Lisboa
Universidade Presbiteriana Mackenzie
Università degli Studi di Bari Aldo Moro
Università degli Studi di Brescia
Università degli Studi di Catania
Università degli Studi di Padova
Universitas Andalas, Padang
Universitas Udayana
Universität Heidelberg
Universitat Politècnica de Catalunya
Universitatea Babeș-Bolyai
Universitatea de Vest din Timișoara
Université de technologie de Compiègne
Université Paris-Saclay
University of Arkansas
University of Augsburg
University of California, Santa Cruz
University of California, Berkeley
University of California, Los Angeles
University of California, San Diego
University of California, Santa Barbara
University of Cambridge
University of Canberra
University of Cincinnati
University of Florida
University of Iceland
University of Idaho
University of Illinois at Urbana-Champaign
University of International Business and Economics
University of Liège
University of Maryland
University of Minnesota, Twin Cities
University of New Hampshire
University of North Carolina at Chapel Hill
University of North Texas
University of Northern Philippines
University of Pennsylvania
University of São Paulo
University of Science and Technology of China
University of Southern Maine
University of St Andrews
University of Technology Sydney
University of Warsaw
University of Washington
University of Waterloo
University of Wisconsin Madison
Univerzita Komenského v Bratislave
Vardhaman College of Engineering
Vardhman Mahaveer Open University
Vietnamese-German University
Wageningen University
West Virginia University
Western University
Xavier University Bhubaneswar
Xiamen University
Yeshiva University
Yunnan University
Zhejiang University
Amity University
Ateneo de Naga University
Australian National University
Birla Institute of Technology and Science, Hyderabad
Brown University
Cairo University
Carnegie Mellon University
College of Engineering Pune
Columbia University
Duke University
Durban University of Technology
Emory University
Eötvös Loránd University
Federal University Lokoja
Fudan University
Gayatri Vidya Parishad College of Engineering (Autonomous)
Gazi Üniversitesi
George Mason University
Georgia Institute of Technology
Golden Gate University
Habib University
Hangzhou Dianzi University
Hankuk University of Foreign Studies
Harbin Institute of Technology
Hasso-Plattner-Institut
Heinrich-Heine-Universität Düsseldorf
Hertie School
Hiroshima University
Ho Chi Minh City University of Foreign Languages and Information Technology
Hochschule Bremen
Hochschule für Technik und Wirtschaft
Hong Kong University of Science and Technology
Huazhong University of Science and Technology
Imperial College London
IMT Mines Alès
Indian Institute of Technology Bombay
Indian Institute of Technology Jodhpur
Indian Institute of Technology Kanpur
Indian Institute of Technology Kharagpur
Indian Institute of Technology Mandi
Indian Institute of Technology Ropar
Indira Gandhi National Open University
Indraprastha Institute of Information Technology, Delhi
Institut catholique d'arts et métiers (ICAM)
Institut de recherche en informatique de Toulouse
Institut Supérieur d'Informatique et des Techniques de Communication
Institut Supérieur De L'electronique Et Du Numérique
Instituto Tecnológico Autónomo de México
İstanbul Teknik Üniversitesi
IT-Universitetet i København
King Abdullah University of Science and Technology
Kongu Engineering College
KPR Institute of Engineering and Technology
Kyungpook National University
Lancaster University
Leibniz Universität Hannover
Leuphana University of Lüneburg
London School of Economics & Political Science
Make School
Massachusetts Institute of Technology
McGill University
Milwaukee School of Engineering
Minia University
Monash University
Multimedia University
National Chung Hsing University
National Institute of Technical Teachers Training&Research
National Institute of Technology, Warangal
National Sun Yat-sen University
National Taiwan University
National Technical University of Athens
National United University
National University of Singapore
Nazarbayev University
New York University
Newman University
North Ossetian State University
Northeastern University
Ohio University
Pakuan University
Peking University
Pohang University of Science and Technology
Politecnico di Milano
Pontificia Universidad Católica de Chile
Portland State University
Purdue University
Queen's University
Radboud Universiteit
Rowan University
Rutgers, The State University of New Jersey
Sapienza Università di Roma
Shanghai Jiao Tong University
Shanghai University of Finance and Economics
Shenzhen University
Sogang University
Southern New Hampshire University
St. Pölten University of Applied Sciences
Stanford University
Stevens Institute of Technology
Sungkyunkwan University
Technische Universiteit Delft
Tekirdağ Namık Kemal Üniversitesi
Texas A&M University
Thapar Institute of Engineering and Technology
The State University of New York at Binghamton
The University of Texas at Austin
Tsinghua University
Universidad Carlos III de Madrid
Universidad de Zaragoza
Universidad Militar Nueva Granada
Universidad Nacional Agraria La Molina
Universidad Nacional de Colombia Sede Manizales
Universidad Tecnológica de Pereira
Universidade Católica de Brasília
Universidade Estadual de Campinas
Universidade Federal de Minas Gerais
Universidade Federal de Ouro Preto
Universidade Federal do Rio Grande
Universidade NOVA de Lisboa
Universidade Presbiteriana Mackenzie
Università degli Studi di Bari Aldo Moro
Università degli Studi di Brescia
Università degli Studi di Catania
Università degli Studi di Padova
Universitas Andalas, Padang
Universitas Udayana
Universität Heidelberg
Universitat Politècnica de Catalunya
Universitatea Babeș-Bolyai
Universitatea de Vest din Timișoara
Université de technologie de Compiègne
Université Paris-Saclay
University of Arkansas
University of Augsburg
University of California, Santa Cruz
University of California, Berkeley
University of California, Los Angeles
University of California, San Diego
University of California, Santa Barbara
University of Cambridge
University of Canberra
University of Cincinnati
University of Florida
University of Iceland
University of Idaho
University of Illinois at Urbana-Champaign
University of International Business and Economics
University of Liège
University of Maryland
University of Minnesota, Twin Cities
University of New Hampshire
University of North Carolina at Chapel Hill
University of North Texas
University of Northern Philippines
University of Pennsylvania
University of São Paulo
University of Science and Technology of China
University of Southern Maine
University of St Andrews
University of Technology Sydney
University of Warsaw
University of Washington
University of Waterloo
University of Wisconsin Madison
Univerzita Komenského v Bratislave
Vardhaman College of Engineering
Vardhman Mahaveer Open University
Vietnamese-German University
Wageningen University
West Virginia University
Western University
Xavier University Bhubaneswar
Xiamen University
Yeshiva University
Yunnan University
Zhejiang University
If you use D2L to teach (or plan to) and would like to receive a free hardcopy, please contact us.
BibTeX entry for citing the book
@book{zhang2020dive,
title={Dive into Deep Learning},
author={Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola},
note={\url{https://d2l.ai}},
year={2020}
}
Table of contents
- 1. Introduction
- 2. Preliminaries
- 3. Linear Neural Networks
- 4. Multilayer Perceptrons
- 4.1. Multilayer Perceptrons
- 4.2. Implementation of Multilayer Perceptrons from Scratch
- 4.3. Concise Implementation of Multilayer Perceptrons
- 4.4. Model Selection, Underfitting, and Overfitting
- 4.5. Weight Decay
- 4.6. Dropout
- 4.7. Forward Propagation, Backward Propagation, and Computational Graphs
- 4.8. Numerical Stability and Initialization
- 4.9. Environment and Distribution Shift
- 4.10. Predicting House Prices on Kaggle
- 5. Deep Learning Computation
- 6. Convolutional Neural Networks
- 7. Modern Convolutional Neural Networks
- 8. Recurrent Neural Networks
- 9. Modern Recurrent Neural Networks
- 10. Attention Mechanisms
- 11. Optimization Algorithms
- 12. Computational Performance
- 13. Computer Vision
- 13.1. Image Augmentation
- 13.2. Fine-Tuning
- 13.3. Object Detection and Bounding Boxes
- 13.4. Anchor Boxes
- 13.5. Multiscale Object Detection
- 13.6. The Object Detection Dataset
- 13.7. Single Shot Multibox Detection (SSD)
- 13.8. Region-based CNNs (R-CNNs)
- 13.9. Semantic Segmentation and the Dataset
- 13.10. Transposed Convolution
- 13.11. Fully Convolutional Networks (FCN)
- 13.12. Neural Style Transfer
- 13.13. Image Classification (CIFAR-10) on Kaggle
- 13.14. Dog Breed Identification (ImageNet Dogs) on Kaggle
- 14. Natural Language Processing: Pretraining
- 14.1. Word Embedding (word2vec)
- 14.2. Approximate Training
- 14.3. The Dataset for Pretraining Word Embedding
- 14.4. Pretraining word2vec
- 14.5. Word Embedding with Global Vectors (GloVe)
- 14.6. Subword Embedding
- 14.7. Finding Synonyms and Analogies
- 14.8. Bidirectional Encoder Representations from Transformers (BERT)
- 14.9. The Dataset for Pretraining BERT
- 14.10. Pretraining BERT
- 15. Natural Language Processing: Applications
- 15.1. Sentiment Analysis and the Dataset
- 15.2. Sentiment Analysis: Using Recurrent Neural Networks
- 15.3. Sentiment Analysis: Using Convolutional Neural Networks
- 15.4. Natural Language Inference and the Dataset
- 15.5. Natural Language Inference: Using Attention
- 15.6. Fine-Tuning BERT for Sequence-Level and Token-Level Applications
- 15.7. Natural Language Inference: Fine-Tuning BERT
- 16. Recommender Systems
- 16.1. Overview of Recommender Systems
- 16.2. The MovieLens Dataset
- 16.3. Matrix Factorization
- 16.4. AutoRec: Rating Prediction with Autoencoders
- 16.5. Personalized Ranking for Recommender Systems
- 16.6. Neural Collaborative Filtering for Personalized Ranking
- 16.7. Sequence-Aware Recommender Systems
- 16.8. Feature-Rich Recommender Systems
- 16.9. Factorization Machines
- 16.10. Deep Factorization Machines
- 17. Generative Adversarial Networks
- 18. Appendix: Mathematics for Deep Learning
- 19. Appendix: Tools for Deep Learning