synesthesia

synesthesia · hybrid · schema v1

Audio→video event processor modeled on the LZX Sensory Translator — two independent copies (A/B), each switchable between AUDIO and VIDEO mode. In AUDIO mode a copy splits its mono input into 4 MUSICAL spectral bands (bass 20–200 / low-mid 200–1k / high-mid 1k–4k / treble 4k+ Hz) so a drum kit lands cleanly across the bands (kick→band1, snare→band2/3, hats→band4). In VIDEO mode the 4 lanes become the R/G/B/Luma channels of a patched video frame (a_video_in / b_video_in cross-domain inputs): the card averages the frame to per-channel 0..1 levels (solid red maxes R, solid white maxes all incl. luma). In BOTH modes each lane derives a gained audio/CV tap, slow (500 ms) + fast (50 ms) envelope-follower CV (real ~2/40 ms attack so a kick onset hits the band CV hard + locked to the transient, without strobing video), boosted by a per-band CV makeup gain so a STRONG kick drives the bass CV to (near) full scale — patch it into a continuous CV input (e.g. OUTLINES rotation) and a strong kick runs the whole range. A hysteresis gate (keyed off the un-boosted envelope, so its timing + the gibribbon game feel are unchanged), a per-band BEAT TRIGGER (a ~10 ms pulse from a spectral-flux onset detector with an adaptive threshold + 80 ms debounce — fires once per kick/snare/hat, NOT continuously on a sustained tone), a 10-bar green→red VU meter, and a mono-video raster. Master gain (0.5–1.5×) sets the floor; per-band gain (1–2×) adds on top. A per-band ENV-OUTPUT DEPTH knob (8 = 2 copies × 4 bands, range 0..2, default 1× = unchanged) scales BOTH that band's env CV outputs (env_slow + env_fast) together — the source-side modulation-depth control: turn it down (0 = silenced) to tame a band's envelopes, or up (toward 2×, clamped at the 0..1 CV ceiling) so even a weak band reaches full modulation depth at the SOURCE. It only touches the two env CV outputs — gate / beat-trigger / band-audio / VU are unaffected. Patch a band trigger into a video switch/flash to cut video on the beat; patch the slow envelopes for smooth colour modulation.

SYNESTHESIA is a dual 4-band audio analyser + envelope/gate/trigger generator — an audio-reactive modulation source. It holds TWO independent copies (A and B); each takes a mono input and splits it into four MUSICAL bands (bass / low-mid / high-mid / treble). For every band of every copy it emits a rich fan of outputs: the isolated band audio, a SLOW (~500 ms) and a FAST (~50 ms) envelope-follower CV, a hysteresis GATE that opens while the band is loud, a beat TRIGGER fired on each spectral-flux onset in that band, and a mono-video RASTER of the band's waveform — 4 bands × 6 kinds × 2 copies = 48 outputs. Each copy can instead run in VIDEO mode, where the 4 lanes become R/G/B/Luma of a patched video frame (sampled card-side) and flow through the same envelope/gate/meter stage. Per-band GAIN, per-copy MASTER, an env-output DEPTH per band, and a UNIPOLAR/BIPOLAR polarity switch shape the modulation; a 10-bar VU meter per band is drawn on the card.

the faceplate

4 inputs · 48 outputs · 22 params

about this module

Hover a control, or open the patch panel and hover a jack.

inputs

id	cable	what it does
`a_in`	audio	Copy A audio input — the mono signal copy A splits into its 4 spectral bands (in AUDIO mode). audio signal
`b_in`	audio	Copy B audio input — the mono signal copy B splits into its 4 spectral bands (in AUDIO mode). audio signal
`a_video_in`	video	Copy A video input — used only when copy A is in VIDEO mode: the card reads this frame's pixels, averages them to R/G/B/Luma levels, and feeds those through copy A's 4 lanes (the frame handoff happens card-side, not as an audio edge). RGB video stream
`b_video_in`	video	Copy B video input — the VIDEO-mode frame source for copy B (same card-side handoff as copy A). RGB video stream

outputs

id	cable	what it does
`a_band1_audio`	audio	Copy A band 1 AUDIO — the isolated bass (20–200 Hz) band of copy A's input (post gain). Patch it as a band-split audio signal. audio signal
`a_band1_env_slow`	cv	Copy A band 1 SLOW envelope CV — a ~500 ms envelope-follower tracking the bass (20–200 Hz) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band1_env_fast`	cv	Copy A band 1 FAST envelope CV — a ~50 ms envelope-follower on the bass (20–200 Hz) band; snappier modulation that tracks transients. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band1_gate`	gate	Copy A band 1 GATE — goes high while the bass (20–200 Hz) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`a_band1_trig`	gate	Copy A band 1 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the bass (20–200 Hz) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`a_band1_raster`	mono-video	Copy A band 1 RASTER — a mono-video output painting the bass (20–200 Hz) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream
`a_band2_audio`	audio	Copy A band 2 AUDIO — the isolated low-mid (200 Hz–1 kHz) band of copy A's input (post gain). Patch it as a band-split audio signal. audio signal
`a_band2_env_slow`	cv	Copy A band 2 SLOW envelope CV — a ~500 ms envelope-follower tracking the low-mid (200 Hz–1 kHz) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band2_env_fast`	cv	Copy A band 2 FAST envelope CV — a ~50 ms envelope-follower on the low-mid (200 Hz–1 kHz) band; snappier modulation that tracks transients. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band2_gate`	gate	Copy A band 2 GATE — goes high while the low-mid (200 Hz–1 kHz) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`a_band2_trig`	gate	Copy A band 2 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the low-mid (200 Hz–1 kHz) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`a_band2_raster`	mono-video	Copy A band 2 RASTER — a mono-video output painting the low-mid (200 Hz–1 kHz) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream
`a_band3_audio`	audio	Copy A band 3 AUDIO — the isolated high-mid (1–4 kHz) band of copy A's input (post gain). Patch it as a band-split audio signal. audio signal
`a_band3_env_slow`	cv	Copy A band 3 SLOW envelope CV — a ~500 ms envelope-follower tracking the high-mid (1–4 kHz) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band3_env_fast`	cv	Copy A band 3 FAST envelope CV — a ~50 ms envelope-follower on the high-mid (1–4 kHz) band; snappier modulation that tracks transients. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band3_gate`	gate	Copy A band 3 GATE — goes high while the high-mid (1–4 kHz) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`a_band3_trig`	gate	Copy A band 3 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the high-mid (1–4 kHz) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`a_band3_raster`	mono-video	Copy A band 3 RASTER — a mono-video output painting the high-mid (1–4 kHz) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream
`a_band4_audio`	audio	Copy A band 4 AUDIO — the isolated treble (4 kHz+) band of copy A's input (post gain). Patch it as a band-split audio signal. audio signal
`a_band4_env_slow`	cv	Copy A band 4 SLOW envelope CV — a ~500 ms envelope-follower tracking the treble (4 kHz+) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band4_env_fast`	cv	Copy A band 4 FAST envelope CV — a ~50 ms envelope-follower on the treble (4 kHz+) band; snappier modulation that tracks transients. Polarity set by copy A's POLARITY. control voltage (CV)
`a_band4_gate`	gate	Copy A band 4 GATE — goes high while the treble (4 kHz+) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`a_band4_trig`	gate	Copy A band 4 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the treble (4 kHz+) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`a_band4_raster`	mono-video	Copy A band 4 RASTER — a mono-video output painting the treble (4 kHz+) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream
`b_band1_audio`	audio	Copy B band 1 AUDIO — the isolated bass (20–200 Hz) band of copy B's input (post gain). Patch it as a band-split audio signal. audio signal
`b_band1_env_slow`	cv	Copy B band 1 SLOW envelope CV — a ~500 ms envelope-follower tracking the bass (20–200 Hz) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band1_env_fast`	cv	Copy B band 1 FAST envelope CV — a ~50 ms envelope-follower on the bass (20–200 Hz) band; snappier modulation that tracks transients. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band1_gate`	gate	Copy B band 1 GATE — goes high while the bass (20–200 Hz) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`b_band1_trig`	gate	Copy B band 1 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the bass (20–200 Hz) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`b_band1_raster`	mono-video	Copy B band 1 RASTER — a mono-video output painting the bass (20–200 Hz) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream
`b_band2_audio`	audio	Copy B band 2 AUDIO — the isolated low-mid (200 Hz–1 kHz) band of copy B's input (post gain). Patch it as a band-split audio signal. audio signal
`b_band2_env_slow`	cv	Copy B band 2 SLOW envelope CV — a ~500 ms envelope-follower tracking the low-mid (200 Hz–1 kHz) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band2_env_fast`	cv	Copy B band 2 FAST envelope CV — a ~50 ms envelope-follower on the low-mid (200 Hz–1 kHz) band; snappier modulation that tracks transients. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band2_gate`	gate	Copy B band 2 GATE — goes high while the low-mid (200 Hz–1 kHz) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`b_band2_trig`	gate	Copy B band 2 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the low-mid (200 Hz–1 kHz) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`b_band2_raster`	mono-video	Copy B band 2 RASTER — a mono-video output painting the low-mid (200 Hz–1 kHz) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream
`b_band3_audio`	audio	Copy B band 3 AUDIO — the isolated high-mid (1–4 kHz) band of copy B's input (post gain). Patch it as a band-split audio signal. audio signal
`b_band3_env_slow`	cv	Copy B band 3 SLOW envelope CV — a ~500 ms envelope-follower tracking the high-mid (1–4 kHz) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band3_env_fast`	cv	Copy B band 3 FAST envelope CV — a ~50 ms envelope-follower on the high-mid (1–4 kHz) band; snappier modulation that tracks transients. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band3_gate`	gate	Copy B band 3 GATE — goes high while the high-mid (1–4 kHz) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`b_band3_trig`	gate	Copy B band 3 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the high-mid (1–4 kHz) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`b_band3_raster`	mono-video	Copy B band 3 RASTER — a mono-video output painting the high-mid (1–4 kHz) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream
`b_band4_audio`	audio	Copy B band 4 AUDIO — the isolated treble (4 kHz+) band of copy B's input (post gain). Patch it as a band-split audio signal. audio signal
`b_band4_env_slow`	cv	Copy B band 4 SLOW envelope CV — a ~500 ms envelope-follower tracking the treble (4 kHz+) band's level; smooth modulation that rides the band's overall energy. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band4_env_fast`	cv	Copy B band 4 FAST envelope CV — a ~50 ms envelope-follower on the treble (4 kHz+) band; snappier modulation that tracks transients. Polarity set by copy B's POLARITY. control voltage (CV)
`b_band4_gate`	gate	Copy B band 4 GATE — goes high while the treble (4 kHz+) band's level is above a hysteresis threshold and low when it falls below; a level-sensitive gate that follows energy in that band. gate / trigger
`b_band4_trig`	gate	Copy B band 4 TRIGGER — a short ~10 ms pulse on each spectral-flux onset (beat) detected in the treble (4 kHz+) band (LZX-Sensory-Translator style). Patch into envelopes/drum voices to fire on that band's hits. gate / trigger
`b_band4_raster`	mono-video	Copy B band 4 RASTER — a mono-video output painting the treble (4 kHz+) band's live waveform as a raster (audio→video), for patching into video destinations. mono video stream

params

id	label	range	default	curve
`a_mode`	A Mode	0..1	0	discrete
`b_mode`	B Mode	0..1	0	discrete
`a_bipolar`	A Polarity	0..1	0	discrete
`b_bipolar`	B Polarity	0..1	0	discrete
`a_master`	A Mas	0.5..1.5	1	linear
`b_master`	B Mas	0.5..1.5	1	linear
`a_gain1`	A1	1..2	1	linear
`a_gain2`	A2	1..2	1	linear
`a_gain3`	A3	1..2	1	linear
`a_gain4`	A4	1..2	1	linear
`b_gain1`	B1	1..2	1	linear
`b_gain2`	B2	1..2	1	linear
`b_gain3`	B3	1..2	1	linear
`b_gain4`	B4	1..2	1	linear
`a_envdepth1`	a1 dpt	0..2	1	linear
`a_envdepth2`	a2 dpt	0..2	1	linear
`a_envdepth3`	a3 dpt	0..2	1	linear
`a_envdepth4`	a4 dpt	0..2	1	linear
`b_envdepth1`	b1 dpt	0..2	1	linear
`b_envdepth2`	b2 dpt	0..2	1	linear
`b_envdepth3`	b3 dpt	0..2	1	linear
`b_envdepth4`	b4 dpt	0..2	1	linear

controls

control	what it does
A Polarity	Copy A env POLARITY — UNIPOLAR (env CV outputs run 0..1, the default) vs BIPOLAR (−1..+1, so a strong onset sweeps a destination's FULL range through a knob-centred CV→video bridge). Toggle on the card.
a1 dpt	Copy A band 1 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy A's bass (20–200 Hz) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
a2 dpt	Copy A band 2 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy A's low-mid (200 Hz–1 kHz) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
a3 dpt	Copy A band 3 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy A's high-mid (1–4 kHz) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
a4 dpt	Copy A band 4 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy A's treble (4 kHz+) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
A1	Copy A band 1 GAIN (1×..2×) — boosts the bass (20–200 Hz) band's level for copy A (affects its audio tap + how hard it drives the envelopes/gate/meter).
A2	Copy A band 2 GAIN (1×..2×) — boosts the low-mid (200 Hz–1 kHz) band's level for copy A (affects its audio tap + how hard it drives the envelopes/gate/meter).
A3	Copy A band 3 GAIN (1×..2×) — boosts the high-mid (1–4 kHz) band's level for copy A (affects its audio tap + how hard it drives the envelopes/gate/meter).
A4	Copy A band 4 GAIN (1×..2×) — boosts the treble (4 kHz+) band's level for copy A (affects its audio tap + how hard it drives the envelopes/gate/meter).
A Mas	Copy A MASTER gain (0.5×..1.5×, unity at noon) — raises or lowers the floor of all four of copy A's bands together.
A Mode	Copy A MODE — AUDIO (analyse the audio input into 4 spectral bands) vs VIDEO (the 4 lanes become R / G / B / Luma of the patched a_video_in frame, sampled card-side). Toggle on the card.
B Polarity	Copy B env POLARITY — UNIPOLAR (env CV outputs run 0..1, the default) vs BIPOLAR (−1..+1, so a strong onset sweeps a destination's FULL range through a knob-centred CV→video bridge). Toggle on the card.
b1 dpt	Copy B band 1 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy B's bass (20–200 Hz) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
b2 dpt	Copy B band 2 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy B's low-mid (200 Hz–1 kHz) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
b3 dpt	Copy B band 3 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy B's high-mid (1–4 kHz) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
b4 dpt	Copy B band 4 ENV DEPTH (0×..2×, unity at noon) — scales BOTH env CV outputs (slow + fast) for copy B's treble (4 kHz+) band; 0 silences that band's modulation, 2 doubles it (clamped to the CV ceiling).
B1	Copy B band 1 GAIN (1×..2×) — boosts the bass (20–200 Hz) band's level for copy B (affects its audio tap + how hard it drives the envelopes/gate/meter).
B2	Copy B band 2 GAIN (1×..2×) — boosts the low-mid (200 Hz–1 kHz) band's level for copy B (affects its audio tap + how hard it drives the envelopes/gate/meter).
B3	Copy B band 3 GAIN (1×..2×) — boosts the high-mid (1–4 kHz) band's level for copy B (affects its audio tap + how hard it drives the envelopes/gate/meter).
B4	Copy B band 4 GAIN (1×..2×) — boosts the treble (4 kHz+) band's level for copy B (affects its audio tap + how hard it drives the envelopes/gate/meter).
B Mas	Copy B MASTER gain (0.5×..1.5×, unity at noon) — raises or lowers the floor of all four of copy B's bands together.
B Mode	Copy B MODE — AUDIO (analyse the audio input into 4 spectral bands) vs VIDEO (the 4 lanes become R / G / B / Luma of the patched b_video_in frame, sampled card-side). Toggle on the card.

source

synesthesia.ts on GitHub.