Open For Enrollment

Starts in 2 days

Join Now

Would you like to enroll?

Enrollment for this course has closed. But you can enroll in a future offering (please select)

Enrollment has closed

Enrollment for this course is currently closed, but the next offering will be available shortly. Check back soon!

Go at your own pace
7 Sessions / 8 hours of work per session
Price
Free
Included w/ premium membership ($20/month)
Skill Level
Intermediate
Video Transcripts
English
Topics
Machine Learning, Wekinator, Gaming Controllers, Interactive Art
Open For Enrollment

Machine Learning for Musicians and Artists

Starts in 2 days

Would you like to enroll?

Enrollment for this course has closed. But you can enroll in a future offering (please select)

Enrollment has closed

Enrollment for this course is currently closed, but the next offering will be available shortly. Check back soon!

Go at your own pace
7 Sessions / 8 hours of work per session
Price
Free
Included w/ premium membership ($20/month)
Skill Level
Intermediate
Video Transcripts
English
Topics
Machine Learning, Wekinator, Gaming Controllers, Interactive Art
Course Description

Have you ever wanted to build a new musical instrument that responded to your gestures by making sound? Or create live visuals to accompany a dancer? Or create an interactive art installation that reacts to the movements or actions of an audience? If so, take this course!

In this course, students will learn fundamental machine learning techniques that can be used to make sense of human gesture, musical audio, and other real-time data. The focus will be on learning about algorithms, software tools, and best practices that can be immediately employed in creating new real-time systems in the arts.

Specific topics of discussion include:

• What is machine learning?

• Common types of machine learning for making sense of human actions and sensor data, with a focus on classification, regression, and segmentation

• The “machine learning pipeline”: understanding how signals, features, algorithms, and models fit together, and how to select and configure each part of this pipeline to get good analysis results

• Off-the-shelf tools for machine learning (e.g., Wekinator, Weka, GestureFollower)

• Feature extraction and analysis techniques that are well-suited for music, dance, gaming, and visual art, especially for human motion analysis and audio analysis

• How to connect your machine learning tools to common digital arts tools such as Max/MSP, PD, ChucK, Processing, Unity 3D, SuperCollider, OpenFrameworks

• Introduction to cheap & easy sensing technologies that can be used as inputs to machine learning systems (e.g., Kinect, computer vision, hardware sensors, gaming controllers)

schedule

This course is in adaptive mode and is open for enrollment. Learn more about adaptive courses here.

Session 1: Introduction (September 25, 2018)
What is machine learning? And what is it good for? We’ll introduce a variety of artistic, musical, and interactive applications in which machine learning can help you create new things.
8 lessons
1. Welcome
2. About the class: Philosophy and learning outcomes
3. Who is the course for?
4. Getting started building interactive systems
5. The machine learning pipeline
6. The Wekinator
7. Connecting inputs and outputs using Open Sound Control
8. Wrap-up
Session 2: Classification, Part I (October 2, 2018)
In this session, we’ll cover the basics of classification, which can be used to make sense of complex data in a meaningful way. We’ll look at two classification algorithms: nearest-neighbor and decision stumps. You’ll be introduced to the Wekinator, a free software tool for using machine learning in real-time applications.
11 lessons
1. What is classification?
2. Building a simple classifier
3. Nearest-neighbor and decision stump algorithms
4. Decision boundaries and comparing classifiers
5. Working with multiple classes; Decision trees
6. Artistic applications of classification
7. Features
8. Feature selection
9. Anatomy of a musical classification system: Blinky
10. Practical tips for building classifiers with Wekinator
11. Conclusion
Session 3: Regression (October 9, 2018)
We will discuss the fundamentals of regression, which can be used for creating continuous mapping and controls. We’ll explore the use of linear regression, polynomial regression, and neural networks to create new types of interactions. You’ll gain hands-on practice exploring regression algorithms and starting to apply them to build your own systems.
12 lessons
1. What is regression?
2. What is a regression model?
3. Polynomial regression
4. Which regression model is best?
5. Introduction to neural networks for regression
6. Using neural networks
7. Mapping and the creation of new expressive interfaces
8. The Blotar comes alive
9. Training as optimization: Linear and polynomial regression
10. Training neural networks
11. Practical tips for using regression with Wekinator
12. Conclusion
Session 4: Classification, Part II; Design considerations (October 16, 2018)
In this session, we’ll take a deeper look at what it means to build a good classifier, and we’ll explore some common and powerful classification algorithms, including decision trees, Naive Bayes, AdaBoost, and support vector machines. We’ll also dig deeper into an exploration of how learning algorithms can be integrated into your own work most easily to achieve your desired outcomes. You’ll get a chance to explore these new algorithms and continue to work them into your own projects.
10 lessons
1. Overview
2. What is a "good" classifier?
3. K-nearest neighbor; Reasoning about feature spaces
4. Naive Bayes
5. Decision stumps and decision trees
6. AdaBoost
7. Support vector machines
8. Evaluating classifiers
9. Using a probability distribution over classes
10. Using more than one classifier at once
Session 5: Sensors and features: Generating useful inputs for machine learning (October 23, 2018)
Machine learning makes it easier and more fun to work with all sorts of real-time sources of data, including real-time audio, video, game controllers, sensors, and more! We’ll talk about good strategies for making sense of the data you’ll get from different inputs, and for designing feature extractors that make machine learning easier. We’ll be encouraging students to develop their own feature extractors and share them with each other!
8 lessons
1. Intro
2. Simple features
3. Feature processing fundamentals
4. Working with audio input: Common audio features
5. Audio-driven instrument building; using average and standard deviation
6. Smoothing and filtering: Arduino demo
7. Video features
8. Features: How fast to send them?
Session 6: Working with time (October 30, 2018)
In this session, we’ll talk about algorithms that have been specifically designed to help you make sense of changes in data over time. Rebecca will dive into dynamic time warping, and guest lecturer Baptiste Caramiaux will discuss Gesture Variation Follower, an algorithm designed with the arts in mind. You’ll continue to get plenty of opportunities to apply temporal modeling algorithms to real-time data analysis.
16 lessons
1. Welcome: Capturing change over time
2. Simple approach: Features that encode change
3. Motivating other modelling approaches
4. Dynamic time warping
5. How DTW works
6. DTW for music and speech analysis
7. DTW in Wekinator: Practical tips
8. HMMs
9. Baptiste Caramiaux - Introduction
10. Gesture Follower (GF): Temporal modeling with real-time alignment
11. How gesture follower works
12. Gesture Variation Follower
13. How does GVF work?
14. Sonic interaction with GVF
15. What these models have in common
16. Designing custom algorithms for music
Session 7: Developing a practice with machine learning; Wrap-up (November 6, 2018)
Guest lecturer Laetitia Sonami will give a masterclass in which she discusses the way machine learning fits into her own work building new musical instruments, and Rebecca will discuss practical tools, boos, and resources you can access for furthering your work in this field.
12 lessons
1. Overview
2. Resources
3. Other topics: Unsupervised learning, computational creativity, deep learning
4. How is machine learning in the arts different?
5. Guest Lecture - Introduction
6. Guest Lecture - Performance with Spring Spyre
7. The instrument: Description
8. The instrument: Advanced techniques
9. The Instrument: Evolution
10. Performance with Bird
11. Guest lecture - Conclusion
12. Rebecca's Final Advice
Reviews
Show off your Certificate of Accomplishment

Verify Your Achievements
Whenever you complete a course as a premium member, you can earn a verified Certificate of Accomplishment. These certificates are proof that you completed an online course on our platform.

Easily Shareable
Using its unique link, you can share your certificate with everyone from future employers and schools, to friends, family, and colleagues. It's the perfect tool to help you land that new job or promotion, apply to college, or simply share your achievements with the world.

Learning Outcomes

Below you will find an overview of the Learning Outcomes you will achieve as you complete this course.

Instructors & Guests
What You Need to Take This Course
  • Materials: Ideally some sensors (could be as simple as a joystick, a webcam, a microphone, or a smartphone with TouchOSC installed, etc.)
  • Equipment: Computer with installation privileges and ability to run Java 7 or higher
  • Software: Wekinator++

Some text-based or visual programming background (e.g., an introduction to ChucK, Processing, Max/MSP, PD, or some other environment) is strongly recommended; without this, students will be much more limited in their ability to experiment with the course material. No prior knowledge of machine learning, mathematics, or other topics is required.

Additional Information

Please note: Taking part in a Kadenze course as a Premium Member, does not affirm that the student has been enrolled or accepted for enrollment by Goldsmiths University of London. 

Peer Assessment Code of Conduct: Part of what makes Kadenze a great place to learn is our community of students. While you are completing your Peer Assessments, we ask that you help us maintain the quality of our community. Please:

  • Be Polite. Show your fellow students courtesy. No one wants to feel attacked - ever. For this reason, insults, condescension, or abuse will not be tolerated.

  • Show Respect. Kadenze is a global community. Our students are from many different cultures and backgrounds. Please be patient, kind, and open-minded when discussing topics such as race, religion, gender, sexual orientation, or other potentially controversial subjects.

  • Post Appropriate Content. We believe that expression is a human right and we would never censor our students. With that in mind, please be sensitive of what you post in a Peer Assessment. Only post content where and when it is appropriate to do so.

Please understand that posts which violate this Code of Conduct harm our community and may be deleted or made invisible to other students by course moderators. Students who repeatedly break these rules may be removed from the course and/or may lose access to Kadenze.

Students with Disabilities: Students who have documented disabilities and who want to request accommodations should refer to the student help article via the Kadenze support center.  Kadenze is committed to making sure that our site is accessible to everyone. Configure your accessibility settings in your Kadenze Account Settings.