Open for Enrollment

You can also start immediately after joining!
Join Now

Would you like to enroll?

Enrollment for this course has closed. But you can enroll in a future offering (please select)

Enrollment has closed

Go at your own pace
7 Sessions / 8 hours of work per session
Included w/ premium membership ($20/month)
Skill Level
Video Transcripts
Machine Learning, Wekinator, Gaming Controllers, Interactive Art
Open for Enrollment

Machine Learning for Musicians and Artists

Open for Enrollment
You can also start immediately after joining!

Would you like to enroll?

Enrollment for this course has closed. But you can enroll in a future offering (please select)

Enrollment has closed

Go at your own pace
7 Sessions / 8 hours of work per session
Included w/ premium membership ($20/month)
Skill Level
Video Transcripts
Machine Learning, Wekinator, Gaming Controllers, Interactive Art
Course Description

Have you ever wanted to build a new musical instrument that responded to your gestures by making sound? Or create live visuals to accompany a dancer? Or create an interactive art installation that reacts to the movements or actions of an audience? If so, take this course!

In this course, students will learn fundamental machine learning techniques that can be used to make sense of human gesture, musical audio, and other real-time data. The focus will be on learning about algorithms, software tools, and best practices that can be immediately employed in creating new real-time systems in the arts.

Specific topics of discussion include:

• What is machine learning?

• Common types of machine learning for making sense of human actions and sensor data, with a focus on classification, regression, and segmentation

• The “machine learning pipeline”: understanding how signals, features, algorithms, and models fit together, and how to select and configure each part of this pipeline to get good analysis results

• Off-the-shelf tools for machine learning (e.g., Wekinator, Weka, GestureFollower)

• Feature extraction and analysis techniques that are well-suited for music, dance, gaming, and visual art, especially for human motion analysis and audio analysis

• How to connect your machine learning tools to common digital arts tools such as Max/MSP, PD, ChucK, Processing, Unity 3D, SuperCollider, OpenFrameworks

• Introduction to cheap & easy sensing technologies that can be used as inputs to machine learning systems (e.g., Kinect, computer vision, hardware sensors, gaming controllers)


This course is in adaptive mode and is open for enrollment. Learn more about adaptive courses here.

Session 1: Introduction (January 25, 2022)
What is machine learning? And what is it good for? We’ll introduce a variety of artistic, musical, and interactive applications in which machine learning can help you create new things.
8 lessons
1. Welcome
2. About the class: Philosophy and learning outcomes
3. Who is the course for?
4. Getting started building interactive systems
5. The machine learning pipeline
6. The Wekinator
7. Connecting inputs and outputs using Open Sound Control (Premium Exclusive)
8. Wrap-up
Session 2: Classification, Part I (February 1, 2022)
In this session, we’ll cover the basics of classification, which can be used to make sense of complex data in a meaningful way. We’ll look at two classification algorithms: nearest-neighbor and decision stumps. You’ll be introduced to the Wekinator, a free software tool for using machine learning in real-time applications.
11 lessons
1. What is classification?
2. Building a simple classifier
3. Nearest-neighbor and decision stump algorithms
4. Decision boundaries and comparing classifiers
5. Working with multiple classes; Decision trees
6. Artistic applications of classification (Premium Exclusive)
7. Features
8. Feature selection
9. Anatomy of a musical classification system: Blinky (Premium Exclusive)
10. Practical tips for building classifiers with Wekinator (Premium Exclusive)
11. Conclusion
Session 3: Regression (February 8, 2022)
We will discuss the fundamentals of regression, which can be used for creating continuous mapping and controls. We’ll explore the use of linear regression, polynomial regression, and neural networks to create new types of interactions. You’ll gain hands-on practice exploring regression algorithms and starting to apply them to build your own systems.
12 lessons
1. What is regression?
2. What is a regression model?
3. Polynomial regression
4. Which regression model is best?
5. Introduction to neural networks for regression
6. Using neural networks
7. Mapping and the creation of new expressive interfaces
8. The Blotar comes alive
9. Training as optimization: Linear and polynomial regression (Premium Exclusive)
10. Training neural networks (Premium Exclusive)
11. Practical tips for using regression with Wekinator (Premium Exclusive)
12. Conclusion
Session 4: Classification, Part II; Design Considerations (February 15, 2022)
In this session, we’ll take a deeper look at what it means to build a good classifier, and we’ll explore some common and powerful classification algorithms, including decision trees, Naive Bayes, AdaBoost, and support vector machines. We’ll also dig deeper into an exploration of how learning algorithms can be integrated into your own work most easily to achieve your desired outcomes. You’ll get a chance to explore these new algorithms and continue to work them into your own projects.
10 lessons
1. Overview
2. What is a "good" classifier?
3. K-nearest neighbor; Reasoning about feature spaces
4. Naive Bayes
5. Decision stumps and decision trees
6. AdaBoost
7. Support vector machines
8. Evaluating classifiers (Premium Exclusive)
9. Using a probability distribution over classes (Premium Exclusive)
10. Using more than one classifier at once (Premium Exclusive)
Session 5: Sensors and Features: Generating Useful Inputs for Machine Learning (February 22, 2022)
Machine learning makes it easier and more fun to work with all sorts of real-time sources of data, including real-time audio, video, game controllers, sensors, and more! We’ll talk about good strategies for making sense of the data you’ll get from different inputs, and for designing feature extractors that make machine learning easier. We’ll be encouraging students to develop their own feature extractors and share them with each other!
8 lessons
1. Intro
2. Simple features
3. Feature processing fundamentals
4. Working with audio input: Common audio features
5. Audio-driven instrument building; using average and standard deviation (Premium Exclusive)
6. Smoothing and filtering: Arduino demo
7. Video features (Premium Exclusive)
8. Features: How fast to send them? (Premium Exclusive)
Session 6: Working with Time (March 1, 2022)
In this session, we’ll talk about algorithms that have been specifically designed to help you make sense of changes in data over time. Rebecca will dive into dynamic time warping, and guest lecturer Baptiste Caramiaux will discuss Gesture Variation Follower, an algorithm designed with the arts in mind. You’ll continue to get plenty of opportunities to apply temporal modeling algorithms to real-time data analysis.
16 lessons
1. Welcome: Capturing change over time
2. Simple approach: Features that encode change
3. Motivating other modelling approaches
4. Dynamic time warping
5. How DTW works
6. DTW for music and speech analysis (Premium Exclusive)
7. DTW in Wekinator: Practical tips (Premium Exclusive)
8. HMMs
9. Baptiste Caramiaux - Introduction
10. Gesture Follower (GF): Temporal modeling with real-time alignment
11. How gesture follower works
12. Gesture Variation Follower
13. How does GVF work?
14. Sonic interaction with GVF
15. What these models have in common
16. Designing custom algorithms for music
Session 7: Developing a Machine Learning Practice; Wrap-up (March 8, 2022)
Guest lecturer Laetitia Sonami will give a masterclass in which she discusses the way machine learning fits into her own work building new musical instruments, and Rebecca will discuss practical tools, boos, and resources you can access for furthering your work in this field.
12 lessons
1. Overview
2. Resources
3. Other topics: Unsupervised learning, computational creativity, deep learning
4. How is machine learning in the arts different?
5. Guest Lecture - Introduction
6. Guest Lecture - Performance with Spring Spyre
7. The instrument: Description
8. The instrument: Advanced techniques (Premium Exclusive)
9. The Instrument: Evolution (Premium Exclusive)
10. Performance with Bird (Premium Exclusive)
11. Guest lecture - Conclusion (Premium Exclusive)
12. Rebecca's Final Advice
Show off your Certificate of Accomplishment

Verify Your Achievements
Whenever you complete a course as a premium member, you can earn a verified Certificate of Accomplishment. These certificates are proof that you completed an online course on our platform.

Easily Shareable
Using its unique link, you can share your certificate with everyone from future employers and schools, to friends, family, and colleagues. It's the perfect tool to help you land that new job or promotion, apply to college, or simply share your achievements with the world.

Learning Outcomes

Below you will find an overview of the Learning Outcomes you will achieve as you complete this course.

Instructors And Guests
What You Need to Take This Course
  • Materials: Ideally some sensors (could be as simple as a joystick, a webcam, a microphone, or a smartphone with TouchOSC installed, etc.)
  • Equipment: Computer with installation privileges and ability to run Java 7 or higher
  • Software: Wekinator++

Some text-based or visual programming background (e.g., an introduction to ChucK, Processing, Max/MSP, PD, or some other environment) is strongly recommended; without this, students will be much more limited in their ability to experiment with the course material. No prior knowledge of machine learning, mathematics, or other topics is required.

Additional Information

Please note: Taking part in a Kadenze course as a Premium Member, does not affirm that the student has been enrolled or accepted for enrollment by Goldsmiths University of London. 

Peer Assessment Code of Conduct: Part of what makes Kadenze a great place to learn is our community of students. While you are completing your Peer Assessments, we ask that you help us maintain the quality of our community. Please:

  • Be Polite. Show your fellow students courtesy. No one wants to feel attacked - ever. For this reason, insults, condescension, or abuse will not be tolerated.

  • Show Respect. Kadenze is a global community. Our students are from many different cultures and backgrounds. Please be patient, kind, and open-minded when discussing topics such as race, religion, gender, sexual orientation, or other potentially controversial subjects.

  • Post Appropriate Content. We believe that expression is a human right and we would never censor our students. With that in mind, please be sensitive of what you post in a Peer Assessment. Only post content where and when it is appropriate to do so.

Please understand that posts which violate this Code of Conduct harm our community and may be deleted or made invisible to other students by course moderators. Students who repeatedly break these rules may be removed from the course and/or may lose access to Kadenze.

Students with Disabilities: Students who have documented disabilities and who want to request accommodations should refer to the student help article via the Kadenze support center.  Kadenze is committed to making sure that our site is accessible to everyone. Configure your accessibility settings in your Kadenze Account Settings.