Published on August 29, 2022 (over 1 year ago)

Developer spotlight: interactive video music lessons with middy.com

Dave Kiss
By Dave Kiss8 min readCustomers

Building software products isn’t all that different from building with Those Colorful Children’s Construction Blocks That Our Lawyer Doesn’t Want Us To Name™.

The individual bricks on their own aren’t all that interesting — it’s the creative ways you stack and snap them together that really bring your vision to life.

In this way, Mux products could be considered the construction blocks, Video Kit edition. They’re great tools to add to your toolkit, but more importantly, they can empower you to dream up and deploy applications that have the ability to change the world with video.

In this series, you’ll hear stories from the trenches of builders, founders, creators, and developers who are powering parts of their business with the help of Mux products. We hope you gain inspiration, ideas, and a greater sense of community by reading these stories straight from our customers’ desks.

LinkEpisode 1: Stephen from Middy.com

I met Stephen down at React Miami while staffing the Mux video guestbook. He shared that he was already a Mux customer and was using Mux’s video delivery APIs to help music educators deliver video lessons through his platform at middy.com.

Sounded like an awesome story, so I had to learn more. Let’s dig in.

LinkWho are you, and what are you working on?

Hey, I'm Stephen, the founder and lead developer of Middy.com, a website where anyone can create and sell web-based interactive piano courses that are compatible with an electronic keyboard (MIDI keyboard).

I’ve been involved in other successful web-based platforms (notably www.represent.com) as a marketer, but this is my first time in the developer seat. I left Represent determined to contribute to the product side of a web company. A software bootcamp and many years later, I put myself in a position to launch my own platform around my passion: music.

My mission with Middy is to make interactive learning part of every music teacher's toolkit. Other tech companies in the music education space provide accessible ways to learn piano by using software to offer “your own personal music teacher” — essentially replacing teachers with software.

Our goal is to put the power back in the hands of the teachers by providing them with software that will help enhance their teaching abilities as well as their business. This approach has benefits for the students as well: they get a greater depth and breadth of curriculum offerings as opposed to being held to a specific curriculum that one company believes they should learn. The teacher and student are both given the autonomy to choose what curriculum and teaching style best suits their needs.

LinkHow was the idea for Middy born?

They say if you want to learn how to code, you should work on something you're passionate about. After leaving a coding bootcamp, I wanted to take a crack at building something. I reflected on one part of my experience at a music school that frustrated me: piano class. This was my favorite class; however, I was really frustrated by how much time teachers wasted holding students accountable for their homework.

At the beginning of class, teachers invited each student to the front of the room to perform their piano homework and receive a grade on the spot while others waited. In a class of 20, this resulted in losing the first 30 minutes of class. I later learned that the online program required students to upload large video files of themselves playing piano to be manually graded by a teacher.

I knew schools could benefit from a solution where students could perform their piano homework using an electronic keyboard and receive a grade outside of class, saving valuable class time for the teacher to share their expertise. Additionally, I could capture new analytics about the students if they also used that application to practice (e.g., correlation between practice times and test scores), providing a feedback system to help the school refine their curriculum.

Many iterations later, under the guidance of the Director of Online Education at Icon Collective, a premier music school in Los Angeles, Middy was eventually put to use. Today, Icon Collective has a Middy-dedicated “Keys Lab” on campus that powers their keyboard curriculum.

LinkWhat makes Middy unique?

Middy is the only site where you can create your own web-based interactive piano course that’s compatible with a MIDI. What makes it especially attractive to schools is that it can be embedded and integrated with a learning management system (LMS).

Schools want all of their grades in one place; they don’t want to have to worry about extracting grades from a bunch of third-party services, and they want to control the student experience. They’d rather embed something into their existing system that has their custom branding than send their students off to a third-party website. We want to continue to expand our offering to more white-label service.

From a community standpoint, we are attracting the next generation of online educators who are developing new approaches to teaching online. For instance, take Elijah Fox, a multi-instrumentalist, singer, and producer who has recorded and produced for the likes of Schoolboy Q, Dency Curry, and J Cole. On Middy, you can get access to a unique course where you can learn his process for making music and play along to many of his compositions using Middy’s interactive software.

Likewise, Noah Kellman, an award-winning composer and jazz pianist who has amassed a huge following teaching jazz piano online, uses Middy to offer interactive versions of his jazz courses as bonus content.

LinkWhat tech stack is Middy built on?

Middy is a React and Typescript app that’s built on top of a few Google cloud services, primarily Firebase. As a small dev team, we wanted to focus all of our efforts on the front end. Anything outside the scope of our unique domain, we wanted to hand to a third-party service.

The piano learning tool is just React and CSS. It was very challenging to create a rich experience purely with HTML/CSS, and a large source of frustration was how limited we felt with trying to maximize DOM performance. We are looking into other technologies to refactor our piano learning tool to enable us to build a richer gameplay experience.

We chose services that had simple APIs, an active developer community, and a lot of flexibility in case we needed to make drastic changes to the platform. For example, Firebase provided us with an amazing suite of services that are all really well integrated: authentication, NoSQL database, and serverless functions.

The video content on our site is all run through Mux, which is an easy-to-implement solution that has flexible privacy controls beyond white-listing domains. We looked at alternatives like Vimeo to host videos, but they were more limited and lacked developer experience compared to Mux.

LinkHow are you integrating Mux into your app?

Integrating Mux was super easy with our stack and straightforward thanks to its simple API and great documentation. We built a simple interface that enables a course creator to upload a video from their computer.

Once a user selects the video they want to upload to Middy, Middy calls a cloud function to generate an upload URL and leverages UpChunk to upload the video in chunks from the front end.

Middy then creates a Firestore document that will store the video data, including the upload status, and has a real-time listener on the front end that will let the course creator know when the video was successfully uploaded. Another cloud function is set up to receive webhook events from Mux and update the document containing video data and upload status. When the video asset is ready, this document also receives a playback ID that we use to playback the video.

To playback the video, we grab the playback ID from our video document and generate a signed playback using a Cloud Function. We use the playback ID to construct the streaming URL and then pass the URL into React Player. We are looking into implementing Mux Player since it has other great features, like thumbnail previews on the timeline.

Thanks for reading! Be sure to check out Middy.com. Don’t forget your MIDI keyboard :P

Written By

Dave Kiss

Dave Kiss – Senior Community Engineering Lead

Fake programmer. Likes bikes, trees, and cookie dough custard. Collects wood scraps. Lots of spiders live in his basement.

Leave your wallet where it is

No credit card required to get started.