warren's visions singleton

warrensvisions · video-effects

The visual analogue of WARREN'S SPECTRUM — a 2D spectral video resynthesizer. Every SLICE frames it downsamples VIDEO IN to a 128×128 luma plane, runs a 2D FFT, and picks the strongest peaks in the WAVEVECTOR plane: each peak is a sine grating whose radius is how fine the pattern is, whose angle is its orientation, whose magnitude is contrast and whose phase is where it sits. Peaks are ranked by SALIENCE, snapped toward a detected lattice comb by LOCK, matched frame to frame as tracks, and everything unclaimed is measured in 16 log-spaced RINGS and replayed as procedural texture (the SMS residual — grain, foliage, smoke). The bank is summed back through an inverse FFT and composited against the source. COHERENCE is the control the module exists for and has no audio counterpart: at 1 each component's phase is re-seated on the phase measured in the incoming frame, giving a recognisable sparse reconstruction; at 0 phase is seated at track birth and free-runs — the literal audio behaviour — and the picture drifts off into a moving interference painting with the source's spectrum and none of its geometry. COMPONENTS (1-256) sets density and CPU; FLOOR/STABILITY gate which peaks earn a slot; SLEW smears contrast; DRIFT gives each component its own phase velocity so the picture boils; SHAPE morphs every grating sine→saw→square by harmonic injection; CENTER transposes every wavevector, zooming the reconstruction; FREEZE (control or GATE) holds the bank as a still; MIX crossfades against the source. Resynthesizes LUMA and keeps the source's chroma. An EFFECT — nothing patched in means black — and capped at ONE instance. v1 defers MASSPASS, the editable post-filterbank, independent RGB tracking and a 256² grid.

warren's visions is the visual analogue of warren's spectrum: a spectral resynthesizer that takes a picture apart into a bank of 2D sine gratings and rebuilds it out of them. Every SLICE frames it downsamples whatever is patched into VIDEO IN to a 128x128 luma plane, runs a 2D FFT over it, and picks the strongest peaks in the wavevector plane. A peak is a grating: its distance from the origin is how FINE the pattern is, its angle is the pattern's ORIENTATION, its magnitude is CONTRAST, and its phase is WHERE the pattern sits. Peaks are ranked by SALIENCE, matched frame to frame as tracks (so a slowly-changing pattern is one component that moves rather than a new one every frame), and everything the tracker does NOT claim is measured as energy in 16 log-spaced rings and replayed as procedural texture — the residual that puts the grain, foliage and smoke back. The whole bank is then summed back into a picture and composited against the source. COHERENCE is the control the module exists for: at 1 every component's phase is re-seated on the phase measured in the incoming frame, so the sum is a genuine sparse reconstruction and you see a recognisable, band-limited version of the source; at 0 phase is seated once when a component is born and then free-runs, which is exactly what the audio module's oscillators do and which drifts the picture off into a moving interference painting with the source's spectrum and none of its geometry; in between the picture assembles and melts. COMPONENTS sets how many gratings the bank has (1-256) and therefore how much of the picture survives; FLOOR and STABILITY gate which peaks earn a slot; SLEW smears contrast in time; DRIFT gives each component its own phase velocity so the reconstruction boils; SHAPE morphs every grating sine->saw->square by injecting its harmonics; CENTER transposes every wavevector, zooming the whole reconstruction in or out; FREEZE (control or GATE input) holds the current components as a still. It resynthesizes LUMA and keeps the source's colour, so chroma follows the input even when the geometry does not. It is an EFFECT: with nothing patched into VIDEO IN the output is black. One instance at a time. Usage: patch a camera or clip in, leave COHERENCE at 1 and pull COMPONENTS down until the picture is as abstracted as you want it, then bring COHERENCE down to let it come apart.

the faceplate

9 inputs · 1 outputs · 12 params

about this module

Hover a control, or open the patch panel and hover a jack.

inputs

id	cable	what it does
`video_in`	video	The frame under analysis. Downsampled to a 128x128 luma plane every SLICE frames; its chroma is kept and recombined with the resynthesized luminance. With nothing patched here the output is black — this is an effect, not a source. RGB video stream
`gate`	gate	FREEZE while HIGH. Holds the current component bank so the picture drones on the last analysed frame; the bank keeps slewing, drifting and rendering, only the analysis stops. Level-sensitive, not edge-triggered, so a held gate holds the freeze and releasing it resumes from the live frame. gate / trigger; modulates engineFreeze (summed directly (the destination DSP scales it)); gate — acts while the level is high (reacts to both edges)
`components_cv`	cv	CV input that modulates Components — how many gratings the bank rebuilds the picture from. Linear-scaled across 1..256 centred on the knob. control voltage (CV); modulates visionsComponents (additive offset — ±1 CV sweeps the full range, centered on the knob)
`coherence_cv`	cv	CV input that modulates Coherence — how hard each component is pulled back onto the phase measured in the incoming frame. Linear-scaled into 0..1; sweeping it is the module's main gesture, from resynthesized camera to free-running interference. control voltage (CV); modulates visionsCoherence (additive offset — ±1 CV sweeps the full range, centered on the knob)
`residual_cv`	cv	CV input that modulates Residual — the level of the procedural texture rebuilt from the energy the component bank did not claim. Linear-scaled into 0..2. control voltage (CV); modulates visionsResidual (additive offset — ±1 CV sweeps the full range, centered on the knob)
`shape_cv`	cv	CV input that modulates Shape — the sine/saw/square morph applied to every grating by harmonic injection. Linear-scaled into 0..1. control voltage (CV); modulates visionsShape (additive offset — ±1 CV sweeps the full range, centered on the knob)
`center_cv`	cv	CV input that modulates Center — the transposition applied to every wavevector, in cents, zooming the reconstruction. Linear-scaled across -3600..3600. control voltage (CV); modulates visionsCenter (additive offset — ±1 CV sweeps the full range, centered on the knob)
`drift_cv`	cv	CV input that modulates Drift — each component's own phase velocity. Linear-scaled into 0..1. control voltage (CV); modulates visionsDrift (additive offset — ±1 CV sweeps the full range, centered on the knob)
`mix_cv`	cv	CV input that modulates Mix — the crossfade between the source and the resynthesis. Linear-scaled into 0..1. control voltage (CV); modulates visionsMix (additive offset — ±1 CV sweeps the full range, centered on the knob)

outputs

id	cable	what it does
`out`	video	The composited frame: resynthesized luminance recombined with the source chroma, crossfaded against the source by MIX. A normal downstream video texture. RGB video stream

params

id	label	range	default	curve
`visionsCoherence`	Coherence	?..?	—	linear
`visionsComponents`	Components	?..?	—	discrete
`visionsFloor`	Floor	?..?dB	—	linear
`visionsStability`	Stability	?..?fr	—	discrete
`visionsSlew`	Slew	?..?s	—	log
`visionsSlice`	Slice	?..?fr	—	discrete
`visionsResidual`	Residual	?..?	—	linear
`visionsShape`	Shape	?..?	—	linear
`visionsCenter`	Center	?..?ct	—	linear
`visionsDrift`	Drift	?..?	—	linear
`visionsMix`	Mix	?..?	—	linear
`engineFreeze`	Freeze	?..?	—	discrete

controls

control	what it does
Freeze	Freeze (LIVE / FREEZE, default LIVE): FREEZE stops the analysis, so the component bank holds whatever it last measured and the picture drones on it. Slew, drift and rendering all keep running, and releasing it resumes from the live frame rather than a stale one. The GATE input does the same thing while held HIGH.
Center	Center (-3600..3600 cents, default 0): transposes every wavevector by one ratio, zooming the whole reconstruction. Positive makes every pattern finer (the picture appears to shrink into itself); negative makes it coarser. Components pushed past the grid limit fade out on a ramp rather than folding back as low-frequency ghosts.
Coherence	Coherence (0..1, default 1): how much of the incoming frame's PHASE each component adopts on every analysis commit. 1 re-seats phase exactly, so the bank is a sparse reconstruction and the picture is recognisable and correctly positioned. 0 never touches phase after a component is born, so the bank keeps the source's spatial frequencies and loses its geometry — the free-running behaviour the audio oscillator bank has. Values between pull part of the way each commit, so structure forms over several frames and decays between them. This is the control that decides whether the module is a camera or an instrument.
Components	Components (1..256, default 64): how many tracked gratings the bank holds. Low counts collapse the picture toward its coarsest structure (image spectra fall off with spatial frequency, so the survivors are the large shapes); high counts approach a band-limited copy of the source. Also the module's main CPU control — a full 256-component bank costs about 1.2 ms of CPU per analysed frame.
Drift	Drift (0..1, default 0): gives each component a phase velocity proportional to its own spatial frequency, so fine detail boils fast and coarse structure moves slowly — the literal analogue of an audio oscillator running at its own pitch. The residual rings churn at the same rate. At 0 phase only moves when COHERENCE moves it, so the picture is still.
Floor	Floor (-90..-20 dB, default -42): the peak-detection threshold, RELATIVE to the strongest component in the frame. Lower admits faint structure and fills the bank from a flat image; higher keeps only the boldest gratings. Everything it rejects becomes residual rather than disappearing.
Mix	Mix (0..1, default 1): crossfades the composited resynthesis against the untouched source. At 0 the module is a passthrough.
Residual	Residual (0..2, default 0.5): the level of the texture rebuilt from unclaimed energy — the 16 log-spaced rings of the spectrum the component bank did not take. This is the difference between a reconstruction and a wireframe: it puts back grain, foliage, smoke and compression fizz. Scaled by the cube root of the component count, so thinning the bank also cleans up the texture. At 0 it is silent. It is band-limited to the analysis grid like everything else, so it renders coarse and mid texture and cannot render fine film grain.
Shape	Shape (0..1, default 0 = SINE): morphs every grating from a sine (0) through a saw (0.5) to a square (1) by injecting the component's harmonics into the spectrum. Harmonics past the grid limit are simply not written, so it band-limits exactly rather than aliasing. The fundamental is never rescaled, so the morph does not change contrast.
Slew	Slew (0.02..4 s, default 0.25): the time constant on every component's CONTRAST and on the residual ring envelopes. Short tracks the source tightly; long smears the picture in time so it dissolves between states rather than cutting. Wavevectors themselves are not slewed — a grating that slid across the frame would tear the geometry.
Slice	Slice (1..16 frames, default 2): the analysis period, in RENDERED FRAMES rather than milliseconds. The source only changes when a frame is drawn, and frames are the unit the CPU budget is spent in, so a wall-clock period would mean different behaviour on every renderer. 1 analyses every frame and costs the most; higher values hold each analysis longer, which is both cheaper and a usable stutter.
Stability	Stability (1..16 frames, default 3): how many consecutive commits a component must be matched on before it reaches full contrast, ramping in over that span. 1 lets every flicker through; high values hold the picture to the structure that persists, which is what stops a noisy source from boiling.

source

warrensvisions.ts on GitHub.