Open for Enrollment

You can also start immediately after joining!
Join Now

Would you like to enroll?

Enrollment for this course has closed. But you can enroll in a future offering (please select)

Enrollment has closed

Go at your own pace
7 Sessions / 8 hours of work per session
Price
Premium membership $20/month (Preview session 1 free)
Included w/ premium membership ($20/month)
Skill Level
Intermediate
Video Transcripts
English
Topics
Machine Learning, Wekinator, Gaming Controllers, Interactive Art
Open for Enrollment

Machine Learning for Musicians and Artists

Open for Enrollment
You can also start immediately after joining!

Would you like to enroll?

Enrollment for this course has closed. But you can enroll in a future offering (please select)

Enrollment has closed

Go at your own pace
7 Sessions / 8 hours of work per session
Price
Premium membership $20/month (Preview session 1 free)
Included w/ premium membership ($20/month)
Skill Level
Intermediate
Video Transcripts
English
Topics
Machine Learning, Wekinator, Gaming Controllers, Interactive Art
Course Description

Have you ever wanted to build a new musical instrument that responded to your gestures by making sound? Or create live visuals to accompany a dancer? Or create an interactive art installation that reacts to the movements or actions of an audience? If so, take this course!

In this course, students will learn fundamental machine learning techniques that can be used to make sense of human gesture, musical audio, and other real-time data. The focus will be on learning about algorithms, software tools, and best practices that can be immediately employed in creating new real-time systems in the arts.

Specific topics of discussion include:

• What is machine learning?

• Common types of machine learning for making sense of human actions and sensor data, with a focus on classification, regression, and segmentation

• The “machine learning pipeline”: understanding how signals, features, algorithms, and models fit together, and how to select and configure each part of this pipeline to get good analysis results

• Off-the-shelf tools for machine learning (e.g., Wekinator, Weka, GestureFollower)

• Feature extraction and analysis techniques that are well-suited for music, dance, gaming, and visual art, especially for human motion analysis and audio analysis

• How to connect your machine learning tools to common digital arts tools such as Max/MSP, PD, ChucK, Processing, Unity 3D, SuperCollider, OpenFrameworks

• Introduction to cheap & easy sensing technologies that can be used as inputs to machine learning systems (e.g., Kinect, computer vision, hardware sensors, gaming controllers)

Reviews
schedule

This course is in adaptive mode and is open for enrollment. Learn more about adaptive courses here.

Session 1: Introduction (July 5, 2022)
What is machine learning? And what is it good for? We’ll introduce a variety of artistic, musical, and interactive applications in which machine learning can help you create new things.
8 lessons
1. Welcome
2. About the class: Philosophy and learning outcomes
3. Who is the course for?
4. Getting started building interactive systems
5. The machine learning pipeline
6. The Wekinator
7. Connecting inputs and outputs using Open Sound Control
8. Wrap-up
Session 2: Classification, Part I (July 12, 2022)
In this session, we’ll cover the basics of classification, which can be used to make sense of complex data in a meaningful way. We’ll look at two classification algorithms: nearest-neighbor and decision stumps. You’ll be introduced to the Wekinator, a free software tool for using machine learning in real-time applications.
11 lessons
1. What is classification? (Premium Exclusive)
2. Building a simple classifier (Premium Exclusive)
3. Nearest-neighbor and decision stump algorithms (Premium Exclusive)
4. Decision boundaries and comparing classifiers (Premium Exclusive)
5. Working with multiple classes; Decision trees (Premium Exclusive)
6. Artistic applications of classification (Premium Exclusive)
7. Features (Premium Exclusive)
8. Feature selection (Premium Exclusive)
9. Anatomy of a musical classification system: Blinky (Premium Exclusive)
10. Practical tips for building classifiers with Wekinator (Premium Exclusive)
11. Conclusion (Premium Exclusive)
Session 3: Regression (July 19, 2022)
We will discuss the fundamentals of regression, which can be used for creating continuous mapping and controls. We’ll explore the use of linear regression, polynomial regression, and neural networks to create new types of interactions. You’ll gain hands-on practice exploring regression algorithms and starting to apply them to build your own systems.
12 lessons
1. What is regression? (Premium Exclusive)
2. What is a regression model? (Premium Exclusive)
3. Polynomial regression (Premium Exclusive)
4. Which regression model is best? (Premium Exclusive)
5. Introduction to neural networks for regression (Premium Exclusive)
6. Using neural networks (Premium Exclusive)
7. Mapping and the creation of new expressive interfaces (Premium Exclusive)
8. The Blotar comes alive (Premium Exclusive)
9. Training as optimization: Linear and polynomial regression (Premium Exclusive)
10. Training neural networks (Premium Exclusive)
11. Practical tips for using regression with Wekinator (Premium Exclusive)
12. Conclusion (Premium Exclusive)
Session 4: Classification, Part II; Design Considerations (July 26, 2022)
In this session, we’ll take a deeper look at what it means to build a good classifier, and we’ll explore some common and powerful classification algorithms, including decision trees, Naive Bayes, AdaBoost, and support vector machines. We’ll also dig deeper into an exploration of how learning algorithms can be integrated into your own work most easily to achieve your desired outcomes. You’ll get a chance to explore these new algorithms and continue to work them into your own projects.
10 lessons
1. Overview (Premium Exclusive)
2. What is a "good" classifier? (Premium Exclusive)
3. K-nearest neighbor; Reasoning about feature spaces (Premium Exclusive)
4. Naive Bayes (Premium Exclusive)
5. Decision stumps and decision trees (Premium Exclusive)
6. AdaBoost (Premium Exclusive)
7. Support vector machines (Premium Exclusive)
8. Evaluating classifiers (Premium Exclusive)
9. Using a probability distribution over classes (Premium Exclusive)
10. Using more than one classifier at once (Premium Exclusive)
Session 5: Sensors and Features: Generating Useful Inputs for Machine Learning (August 2, 2022)
Machine learning makes it easier and more fun to work with all sorts of real-time sources of data, including real-time audio, video, game controllers, sensors, and more! We’ll talk about good strategies for making sense of the data you’ll get from different inputs, and for designing feature extractors that make machine learning easier. We’ll be encouraging students to develop their own feature extractors and share them with each other!
8 lessons
1. Intro (Premium Exclusive)
2. Simple features (Premium Exclusive)
3. Feature processing fundamentals (Premium Exclusive)
4. Working with audio input: Common audio features (Premium Exclusive)
5. Audio-driven instrument building; using average and standard deviation (Premium Exclusive)
6. Smoothing and filtering: Arduino demo (Premium Exclusive)
7. Video features (Premium Exclusive)
8. Features: How fast to send them? (Premium Exclusive)
Session 6: Working with Time (August 9, 2022)
In this session, we’ll talk about algorithms that have been specifically designed to help you make sense of changes in data over time. Rebecca will dive into dynamic time warping, and guest lecturer Baptiste Caramiaux will discuss Gesture Variation Follower, an algorithm designed with the arts in mind. You’ll continue to get plenty of opportunities to apply temporal modeling algorithms to real-time data analysis.
16 lessons
1. Welcome: Capturing change over time (Premium Exclusive)
2. Simple approach: Features that encode change (Premium Exclusive)
3. Motivating other modelling approaches (Premium Exclusive)
4. Dynamic time warping (Premium Exclusive)
5. How DTW works (Premium Exclusive)
6. DTW for music and speech analysis (Premium Exclusive)
7. DTW in Wekinator: Practical tips (Premium Exclusive)
8. HMMs (Premium Exclusive)
9. Baptiste Caramiaux - Introduction (Premium Exclusive)
10. Gesture Follower (GF): Temporal modeling with real-time alignment (Premium Exclusive)
11. How gesture follower works (Premium Exclusive)
12. Gesture Variation Follower (Premium Exclusive)
13. How does GVF work? (Premium Exclusive)
14. Sonic interaction with GVF (Premium Exclusive)
15. What these models have in common (Premium Exclusive)
16. Designing custom algorithms for music (Premium Exclusive)
Session 7: Developing a Machine Learning Practice; Wrap-up (August 16, 2022)
Guest lecturer Laetitia Sonami will give a masterclass in which she discusses the way machine learning fits into her own work building new musical instruments, and Rebecca will discuss practical tools, boos, and resources you can access for furthering your work in this field.
12 lessons
1. Overview (Premium Exclusive)
2. Resources (Premium Exclusive)
3. Other topics: Unsupervised learning, computational creativity, deep learning (Premium Exclusive)
4. How is machine learning in the arts different? (Premium Exclusive)
5. Guest Lecture - Introduction (Premium Exclusive)
6. Guest Lecture - Performance with Spring Spyre (Premium Exclusive)
7. The instrument: Description (Premium Exclusive)
8. The instrument: Advanced techniques (Premium Exclusive)
9. The Instrument: Evolution (Premium Exclusive)
10. Performance with Bird (Premium Exclusive)
11. Guest lecture - Conclusion (Premium Exclusive)
12. Rebecca's Final Advice (Premium Exclusive)
Show off your Certificate of Accomplishment

Verify Your Achievements
Whenever you complete a course as a premium member, you can earn a verified Certificate of Accomplishment. These certificates are proof that you completed an online course on our platform.

Easily Shareable
Using its unique link, you can share your certificate with everyone from future employers and schools, to friends, family, and colleagues. It's the perfect tool to help you land that new job or promotion, apply to college, or simply share your achievements with the world.

Learning Outcomes

Below you will find an overview of the Learning Outcomes you will achieve as you complete this course.

Instructors And Guests
What You Need to Take This Course
  • Materials: Ideally some sensors (could be as simple as a joystick, a webcam, a microphone, or a smartphone with TouchOSC installed, etc.)
  • Equipment: Computer with installation privileges and ability to run Java 7 or higher
  • Software: Wekinator++

Some text-based or visual programming background (e.g., an introduction to ChucK, Processing, Max/MSP, PD, or some other environment) is strongly recommended; without this, students will be much more limited in their ability to experiment with the course material. No prior knowledge of machine learning, mathematics, or other topics is required.

Additional Information

Please note: Taking part in a Kadenze course as a Premium Member, does not affirm that the student has been enrolled or accepted for enrollment by Goldsmiths University of London. 

Peer Assessment Code of Conduct: Part of what makes Kadenze a great place to learn is our community of students. While you are completing your Peer Assessments, we ask that you help us maintain the quality of our community. Please:

  • Be Polite. Show your fellow students courtesy. No one wants to feel attacked - ever. For this reason, insults, condescension, or abuse will not be tolerated.

  • Show Respect. Kadenze is a global community. Our students are from many different cultures and backgrounds. Please be patient, kind, and open-minded when discussing topics such as race, religion, gender, sexual orientation, or other potentially controversial subjects.

  • Post Appropriate Content. We believe that expression is a human right and we would never censor our students. With that in mind, please be sensitive of what you post in a Peer Assessment. Only post content where and when it is appropriate to do so.

Please understand that posts which violate this Code of Conduct harm our community and may be deleted or made invisible to other students by course moderators. Students who repeatedly break these rules may be removed from the course and/or may lose access to Kadenze.

Students with Disabilities: Students who have documented disabilities and who want to request accommodations should refer to the student help article via the Kadenze support center.  Kadenze is committed to making sure that our site is accessible to everyone. Configure your accessibility settings in your Kadenze Account Settings.