How to Play Gesture Synth

Short answer: press Begin, allow the camera, raise one to four fingers on your left hand to play chords I to IV, and move your right hand up for volume. That is enough to play a song. The rest of this page explains every control of the gesture synth and how to get a clean signal from your webcam.

1. Set up in thirty seconds

1. Open the gesture synth in Chrome, Edge or Firefox on a laptop. Safari on Mac works too.
2. Press Begin and accept the camera prompt. The tracking model (about 8 MB) loads once and is cached.
3. Sit about an arm's length from the camera with a plain background if you can. Face a window or lamp rather than having it behind you.
4. Raise your left hand with the palm facing the camera. A skeleton overlay appears on it and the readout shows the chord.

2. The chord hand (left)

Chords take their natural quality from the key, so ii, iii and vi are minor by themselves. To flip the quality, lean the hand toward the centre of the screen by about twenty degrees: I becomes Cm, vi becomes A major. Lean it back to return. Full details are in the chord guide.

3. The expression hand (right)

• Height = volume. Hand at the top of the frame is loud, near the bottom is quiet. Dropping it all the way makes a natural fade.
• Pinch = brightness. Thumb and index together closes the filter for a dark tone; spread them for a bright one.
• Fingers = voicing. One finger is root position, two is first inversion, three is second inversion, four adds the seventh.
• Thumb out = octave down. Useful for a bass-heavy verse.
• Left to right = stereo position. Subtle, but it helps when recording.

4. Sounds, keys and the beat

Four sounds are built in. Warm is a soft triangle pad and the easiest to listen to for long sessions. Bright is a saw lead. Retro is a square with a short release for stabs. Strings has a slow attack that hides small tracking jitters, so it is the friendliest sound for beginners.

Turn on the Beat for a click at the chosen tempo, or tick Drums for a kick and hi-hat pattern. Four dots on the stage show the beat so you can change chords on the bar.

5. Your first progression

Set the key to C, Strings sound, Beat at 80 BPM. Play one finger for a bar, open hand for a bar, index plus pinky for a bar, four fingers for a bar. That is I, V, vi, IV, the progression behind hundreds of pop songs. Once it feels steady, raise your right hand on the V chord and drop it on the IV.

6. Recording and MIDI

Press Record before you play and Stop when done; a WAV file downloads. To drive a DAW instead, press MIDI out in Chrome or Edge, pick a virtual port, and arm a software instrument on that port. Chords, pitch bend and volume arrive as ordinary MIDI.

7. No camera, or tracking is slow?

Switch Input to Mouse / touch: drag across the stage, left to right for chords I to VII, upward for volume, Shift or a second finger for minor. Or choose Keyboard: number keys 1 to 7 are the chords, Shift is minor, Q W E R are voicings. Every mode makes the same sound, so nothing is lost. If the camera fails to start, the troubleshooting guide lists the fixes by symptom, and the mobile guide covers phones.