He is now on a journey to build games that interact with notes on scores, like colorize all the Cs, click on the notes that are wrong vs what you hear etc.
He is building everything from scratch, with the scores, the notes, and I'm wondering if there are some tools or libraries out there that provide much flexibility to build stuff for music theory.
Libs that helped:
- https://github.com/saebekassebil/teoria for scales, chords etc
- https://github.com/0xfe/vexflow for notation
- https://github.com/goldfire/howler.js for playing piano samples. I used to DIY (because I love DIY), not even using WebAudio but recently tried howler and it's abstracting a few things I don't need to worry about
- https://github.com/omnibrain/svguitar/ draw guitar fretboard. I recently did my first guitar-specific exercise and this was good. I even filed a feature request and the dev did it
I'd be happy to help you and your prof if you need anything, lmk
Thanks for sharing.
I've used it to build an open source vocal range detector: https://www.makeartwithpython.com/blog/vocal-range-python-mu...
https://humandotlearning.github.io/guitar_visualiser/
Feedback welcome!
Very Tangental: I had another idea for a 'music' game. Years ago, I used to mumble. People couldn't always understand me. So I randomly made a collection of phrases I struggled to say crispy to practice them 10 times each per morning. It improved my articulation a lot. I sometimes do the same (as a form of vocal exercise) but I made it way more fun by putting on one of my favourite rap tunes and cranking up the playback rate. For example chance the rapper's 'no problems' or 'angel' at 1.5 speed is a great vocal exercise. Perhaps this could be gamified. (I guess technically, it already is, as it's already a fun way of doing a boring task).
https://xenharmlib.readthedocs.io/en/latest/
If he wants to generate audio, Tone.js can be a useful higher level abstraction for WebAudio.
I've also made a "music theory interpreter" in Rust that goes a bit further. The source files for music theory & the chord database might prove especially interesting. https://github.com/tiniuclx/harmony-explorer
Finally, Ian Ring's website is a very fascinating place to learn about scales in particular: https://ianring.com/musictheory/scales/
I remember using WebMIDI for the interface, TonalJS for chord/theory.
From another comment, https://github.com/0xfe/vexflow looks cool, I could use it together with PIXI.js for the rendering.
If you also look for the later, game engines like Godot might also be worth a consideration. With it you got a lot of flexibility to build what you want while also making the coding part slightly easier.
I wonder how teaching will look like than.
It has built-in functions for chords and scales and is pretty easy to make catchy loops.
I recently helped a tutor using BespokeSynth to create audio explainers, wiring together frequencies and interactive oscillators and waveform viewers to explain resonant frequencies (or something, I didn't completely get it)
https://static.everythingability.opalstacked.com/bespoke_syn...
One approach that seems promising is combining spaced repetition with knowledge graphs. The graph helps map knowledge areas and dependencies. For instance:
- Notes depend on Pitch.
- Intervals depend on Notes.
- Chords depend on Intervals.
...etc. The system would ensure concepts are mastered in the right order before progressing and use spaced repetition [1] to determine the optimal intervals for quizzing the student.
A site discussed on HN previously outlines how it applies this approach to teaching math [2].
Also, check out Hookpad [3], which offers some innovative tools for learning music theory.
--
1: https://en.wikipedia.org/wiki/Spaced_repetition
2: https://mathacademy.com/how-it-works
3: https://www.hooktheory.com/
[0] https://youtu.be/qeS8txkoUH4?si=PF0jH8VlTcbQpchn