◢◣ the MUSE suite

◢◣ MUSE SUITE

an audio-visual instrument set — one visualiser, one gesture controller, one bridge

bridge · probing…
◢◣ MUSEALL v3.3
music-reactive visualiser
  • AI-generated scenes with RIFE-interpolated style morphs — ACID · CV SCAN · BIOLUM · SIGIL
  • 12 canvas FX + 6 generative 3D slides, incl. audio-warped GLB objects
  • 14 audio-reactive overlays on a composited canvas layer
  • Trigger bindings with adaptive peak detection — bind bass → kaleido, beat → glitch
  • Recording — canvas + audio capture with overlays baked in
▶ LAUNCH checking…
⟠ CONDUCTOR v1.0.0
gesture instrument
  • Webcam hand-tracking — MediaPipe hands → 50 normalised control sources
  • Pinch, per-finger curl, spread, palm roll — your hands are the knobs
  • Voice analysis + mic FX as extra bindable sources
  • Broadcasts everything to MUSEALL over the muse-bridge protocol
  • A control surface, not a visualiser — the consumer decides what the numbers mean
▶ LAUNCH checking…
◇ HOW THEY LINK
01
Open both
Launch MUSEALL and CONDUCTOR in two tabs or windows of the same browser — the bridge is a BroadcastChannel, so both pages must share an origin.
02
Camera on
Hit START CAMERA in CONDUCTOR. Hands appear on the tracking stage; ~50 sources start streaming at up to 30 Hz.
03
Bind in MUSEALL
Open MUSEALL's ⇄ MAP panel — the catalog-driven wizard lists every gesture/voice source. Bind pinch_R → kaleido, and watch the ⟠ EXT pill in the header flip green.

no pairing, no config — CONDUCTOR announces itself on muse-bridge-v1 and MUSEALL's ⟠ EXT pill lights up the moment a conductor is live. Sources bind exactly like audio bands.

◢ UNDER THE HOOD
scene images
Z-Image-Turbo (8-step distilled diffusion) run locally via ComfyUI — seed-locked style variants
morph tweens
RIFE v4.26 optical-flow interpolation — 8 tween frames per direction, blended live by the audio-driven morph engine
hand tracking
MediaPipe tasks-vision hands — pinch aperture, finger curl, spread, palm roll → 50 normalised sources
audio analysis
Web Audio API — 2048-pt FFT, 7 bands + beat/drop detection at ~60 fps
bridge
BroadcastChannel muse-bridge-v1 — catalog + source values streamed at ~30 Hz, ping/pong discovery
engine
Vanilla JS ES modules, zero framework — Canvas 2D FX pipeline + Three.js for 3D slides