INTERACTION DESIGN
Resonance
A real-time visual that responds to both music and the movement of your hand.
September 2025
— What if you could conduct light the way a conductor shapes music? Built in TouchDesigner and connected to an Arduino with an ultrasonic sensor, this piece listens to the music and watches your hand at the same time, so the visual is always shaped by two performers: the song and you.
ROLE
CONCEPT, TOUCHDESIGNER, INTERACTION DESIGN, ARDUINO INTEGRATION, ENCLOSURE DESIGN
Project Overview
Light that listens and moves with you.
Most audio-reactive visuals only listen. They pulse with the beat, but the viewer just watches. I wanted to hand some of that control back to a person. In this piece, the music drives the rhythm of the visual while your hand shapes it: moving closer to the sensor transforms the form as the sound plays through it.
The result is a mirrored, glowing figure, part wings and part mask, that never looks the same twice, because it depends on both what you're hearing and what you're doing.
Approach
Fashion shooting in the studio
The visual is built in TouchDesigner, a node-based tool for real-time graphics. The music is analysed as it plays, and its energy feeds into the form, so louder, busier moments make it flare and expand. At the same time, an Arduino reads the distance of your hand from an ultrasonic sensor and sends it to TouchDesigner, where it becomes a live control.
Hand distance controls size, shape, brightness, and colour all at once. As your hand moves closer, the form grows, shifts shape, brightens, and warms from cool green to glowing orange, so a single gesture transforms the whole visual.
Process
Two inputs, one visual. The challenge was making both feel meaningful without fighting each other.
Sensor readings can be jumpy, and music changes constantly, so raw values made the visual flicker and twitch. I smoothed the incoming data so every change feels fluid rather than glitchy, then balanced the two inputs: the music sets the pulse, and the hand sets the shape. The mirrored symmetry keeps the composition readable, so even at its wildest, the form feels intentional.
The music sets the pulse. Your hand shapes the light.
Final Design
Creative Coding
Interaction Design
creative technology
TouchDesigner
The final piece runs in real time, with the sensor sitting beside the screen like a small instrument. Anyone can walk up, move their hand, and watch the light respond immediately, with no instructions needed.
Future versions could project the visual at room scale, add more sensors for different parts of the form, or pair it with AeroSynth, so the same gesture shapes both the sound and the light.
Product Images