Creatability is a collection of experiments built collaboratively with the disability community that use Chrome + AI to explore how creative tools can be more accessible. Keyboard, Sound Canvas, Body Synth, Seeing Music, Clarion Lite, Word Synth, Sampler all explore unique modalities for creation.
One of my central contributions was an open-sourced library of Web Components solving many complex accessibility issues and providing novel modes of interaction such as using PoseNet with Tensorflow.js to enable anyone to control a cursor with their nose, wrist or many other body parts with nothing other than a basic webcam.
Creatability won the Cannes Grand Prix for Design in 2019, the Webby Awards Gold for Technical Achievements and has been exhibited across the world.
The collaborative nature of these projects were amongst some of the most rewarding for me. I will never forget the first time I watched one of our blind partners drawing with Sound Canvas or when I attended a Flaming Lips concert with Jay Allen Zimmerman and watched him pull out Seeing Music from his bag to enjoy the show.
We created these open-source components in order to encourage others to create more accessible creative experiences a couple of my favorites came from Luisa Pereira:
- Nose theremins and light painters
- Making Your Own Musical Instruments with P5.js, Arduino, and WebMIDI
My role on this project was Lead Creative Technologist, I created the components used on all experiments as well as invented the novel user interface elements alongside my Creative Director Alexander Chen and fellow technologist Yotam Mann.
