Enrollment Closed
Starts in 7 days
You can also start immediately after joining!
This exclusive course is part of the program:
Creative Applications of Deep Learning with TensorFlow - Enrollment Closed
Coming Soon
Go at your own pace
4 Sessions / 15 hours of work per session
Included w/ premium membership ($20/month)
Skill Level
Video Transcripts
English, Japanese, Spanish; Castilian, Russian, Chinese, Portuguese
Generative audio, deep generative networks, generative adversarial networks, sketch to photo, neural doodle, style net
Open for Enrollment

Creative Applications of Deep Learning with TensorFlow III

Coming Soon

Would you like to enroll?

Enrollment for this course has closed. But you can enroll in a future offering (please select)

Enrollment has closed

Go at your own pace
4 Sessions / 15 hours of work per session
Included w/ premium membership ($20/month)
Skill Level
Video Transcripts
English, Japanese, Spanish; Castilian, Russian, Chinese, Portuguese
Generative audio, deep generative networks, generative adversarial networks, sketch to photo, neural doodle, style net
Course Sponsor

Filmed with exclusive content featuring Google Magenta

TensorFlow logo and any related marks are trademarks of Google Inc.

Course Description

This course extends our existing background in Deep Learning to state of the art techniques in audio, image and text modeling. We'll see how dilated convolutions can be used to model long term temporal dependencies efficiently using a model called WaveNet. We'll also see how to inspect the representations in deep networks using a deep generator network, leading to some of the strongest insights into deep networks and the representations they learn. We'll then switch gears to one of the most exciting directions in Deep Learning thus far: Reinforcement Learning. We'll take a brief tour of this fascinating topic and explore toolkits released by OpenAI, DeepMind, and Microsoft. Finally, we're teaming up with Google Brain's Magenta Lab for our last session on Music and Art Generation. We'll explore Magenta's libraries using RNNs and Reinforcement Learning to create generative and improvised music.

What Students Are Saying:

"This course lets you explore things like audio synthesis, music generation and natural language processing using the Tensorflow skills learned in the previous two courses. It is very open-ended. Parag and the Google Magenta team give some great overviews and then set you free to explore each space further. Highly recommended!"

This course is in adaptive mode and is open for enrollment. Learn more about adaptive courses here.

Session 1: Modeling Music and Art: Google Brain’s Magenta Lab (March 10, 2021)
We're teaming up with the Google Brain lab, Magenta to explore the generative creation of Music and Art! We'll explore their libraries which use RNNs and Reinforcement Learning to compose, generate, improvise, and even create duets of music.
20 lessons
1. Introduction to Magenta w/ Douglas Eck
2. Magenta Installation
3. MIDI Setup
4. Introduction to MIDI w/ Adam Roberts
5. Melody RNN: Pre-trained Model w/ Harry Potter
6. MIDI Processing with Magenta
7. Melody RNN: Preprocessing The Legend of Zelda
8. Melody RNN: Training the Legend of Zelda
9. Polyphony RNN: Introduction w/ Curtis Hawthorne
10. Drums and Improv RNN: Introduction w/ Ian Simon
11. Drums RNN: Jam w/ Adam Roberts and Ian Simoq
12. Drums RNN: Setup and Training
13. Magenta MIDI: Introduction w/ Adam Roberts
14. Magenta MIDI: Setup
15. AI Duet
16. Magenta NIPS Demo: Max + Ableton Live Set - Demo
17. Sageev Oore Introduction
18. Magenta Jam: Sageev Oore
19. Magenta Jam w/ Doug Eck, Adam Roberts, and Sageev Oore
20. Closing Thoughts and Homework
Session 2: Modeling Language: Natural Language Processing (March 17, 2021)
This session develops an understanding in natural language processing covering word2vec, glove, seq2seq and attention mechanisms.
17 lessons
1. Introduction
2. Count-Based Methods
3. Modeling Sequences with N-Grams
4. Predict-Based Methods
5. Noise Contrastive Estimation
6. Word2Vec Implementation and Considerations
7. GloVe: Global Vectors - Overview
8. GloVe: Global Vectors - Pre-Trained Model Exploration
9. RNN Language Model: Seq2Seq
10. Seq2Seq: Overview
11. Seq2Seq: Special Tokens, Buckets, Dynamic Unrolling
12. Seq2Seq: Training Data
13. Seq2Seq: Preprocessing w/NLTK Part I
14. Seq2Seq: Preprocessing w/NLTK Part II
15. Seq2Seq: Making Training Pairs
16. Dynamic RNN Seq2Seq Model w/ Attention
17. Homework
Session 3: Autoregressive Image Modeling w/ PixelCNN (March 24, 2021)
This session covers an advanced technique for synthesizing objects resembling deep dream techniques. We show how this can be used to much more clearly understand the representations in deep networks.
9 lessons
1. Introduction
2. Introduction to Pixel RNN
3. Pixel RNN Models
4. PixelRNN Versus PixelCNN
5. LSTM Recap
6. Modeling LSTMs with Convolution
7. Extensions: Conditional Generation, Queues, Residuals, Skip Connections, and Gates Convolution
8. Conditional PixelCNN Implementation
9. Homework
Session 4: Modeling Audio w/ Wavenet and NSynth (March 31, 2021)
This session covers new work in generative modeling of images, sound, and text using masked and dilated convolution operations. We describe what these are and how they can be used to model various media types very efficiently.
14 lessons
1. Introduction to WaveNet
2. Understanding Audio, Samples, and Sample Rates
3. Bit Depth and Mu Law Encoding
4. Dilated Convolution and Receptive Field Sizes
5. A note on Skip Connections, Residual Connections, and Gated Convolution
6. WaveNet Code
7. Fast WaveNet Generation
8. Introduction to Magenta w/Jesse Engel
9. Motivations for NSynth w/ Jesse Engel
10. Introduction to NSynth w/ Jesse Engel
11. NSynth Albeton Live Sampler w/ Jesse Engel
12. NSynth Training Code and AI Experiment Sampler
13. NSynth Pre-trained Model, Encoding and Decoding, and Fast Generation
14. Homework
Learning Outcomes

Below you will find an overview of the Learning Outcomes you will achieve as you complete this course.

Instructors And Guests
What You Need to Take This Course
Additional Information

Some knowledge of basic python programming is assumed, including how to start a python session, working with jupyter (ipython) notebook (for homework submissions), numpy basics including how to manipulate arrays and images, how to draw images with matplotlib, and how to work with files using the os package. You should also have completed the first course in the CADL program before taking this second course.

If a student signs up for the Creative Applications of Deep Learning program, it is recommended that these courses are taken sequentially.

Peer Assessment Code of Conduct: Part of what makes Kadenze a great place to learn is our community of students. While you are completing your Peer Assessments, we ask that you help us maintain the quality of our community. Please:

  • Be Polite. Show your fellow students courtesy. No one wants to feel attacked - ever. For this reason, insults, condescension, or abuse will not be tolerated.
  • Show Respect. Kadenze is a global community. Our students are from many different cultures and backgrounds. Please be patient, kind, and open-minded when discussing topics such as race, religion, gender, sexual orientation, or other potentially controversial subjects.
  • Post Appropriate Content. We believe that expression is a human right and we would never censor our students. With that in mind, please be sensitive of what you post in a Peer Assessment. Only post content where and when it is appropriate to do so.

Please understand that posts which violate this Code of Conduct harm our community and may be deleted or made invisible to other students by course moderators. Students who repeatedly break these rules may be removed from the course and/or may lose access to Kadenze.

Students with Disabilities: Students who have documented disabilities and who want to request accommodations should refer to the student help article via the Kadenze support center. Kadenze is committed to making sure that our site is accessible to everyone. Configure your accessibility settings in your Kadenze Account Settings.