frametable

frametable · effects

FRAMETABLE — a video WAVETABLE oscillator. It continuously records the last 60 rendered input frames into a GPU frame ring (a TEXTURE_2D_ARRAY, one layer per frame) and treats that 60-frame history like the single-cycle waves of a wavetable synth: MORPH scans a centre point through the table and SPREAD sets how wide a window around that centre each output pixel may draw from. For EVERY output pixel the shader draws exactly ONE source frame — a whole-pixel dither/mosaic, NEVER an alpha-average across frames — chosen probabilistically from a bell distribution over the window (centre frame most likely, periphery least). The frame index is picked in O(1) by an analytic inverse-CDF (one sqrt, one branch, one texture-array fetch — never a 60-frame loop). The per-pixel choice is fixed in SCREEN space by a static per-pixel threshold with NO time term, so a STILL input yields a stable image (no TV-static shimmer) even while the ring keeps refreshing, while MOVING content becomes a coherent, morph-scannable time-smear mosaic that scanning MORPH slides through like a soft crossfade. SPREAD 1 collapses to a single frame (a delta — crisp playback of one moment); wider SPREAD blends more of the history into the bell, centre frame still most likely. SHAPE morphs the bell from triangular (default, compact) to gaussian (smooth). SHIMMER (default 0 = fully static) animates the threshold along a golden-ratio sequence for a gentle living grain on moving content while the time-averaged distribution stays exactly the target bell. FREEZE (a toggle button, plus a freeze gate that holds the ring frozen while the gate is held high) stops the ring from advancing so you can scrub a held 60-frame window with MORPH/SPREAD; SAVE (a momentary button, also a rising edge on the SAVE trigger) snapshots the current 60-frame ring into an in-GPU slot for later recall (and to feed a future video Cube — the snapshot + live ring are readable through the handle). Beyond that in-GPU slot, FRAMETABLE saves + loads REAL FILES, mirroring the wavetable file workflow: SAVE FILE writes the whole 60-frame ring to a lossless .frametable.png sprite-sheet atlas (a fixed 10×6 = 60-tile contact sheet, no video codec) via the save-file picker (with a download fallback); LOAD reads a .frametable.png back into the ring, freezing it so the loaded table can be morphed/scanned. The multi-megabyte frame bytes persist PER-BROWSER in IndexedDB (never the Y.Doc — only a tiny file descriptor syncs), so a loaded/saved table survives a reload on the same machine. Rendered at half engine resolution (SwiftShader/CI budget); an unpatched input renders black. The selection math (triangular + gaussian inverse-CDF, ring wrap, head→layer mapping, freeze/save reducers) is a 1:1 CPU mirror unit-tested in $lib/video/frametable-core. All ports live on the yellow drill-down PATCH PANEL (no raw side jacks). Look-affecting new WebGL shader — held for owner visual preview.

FRAMETABLE is a video WAVETABLE oscillator. It continuously records the last 60 rendered input frames into a GPU frame ring (a TEXTURE_2D_ARRAY, one layer per frame) and treats that 60-frame history like the single-cycle waves of a wavetable synth: MORPH scans a centre point through the table and SPREAD sets how wide a window around it each output draws from. A faceplate MODE selector picks one of THREE render engines. SMOOTH (the default) paints a smooth 2D field of temporal sample-centres from two morphable waveforms (one per screen axis) and outputs a CAPPED WEIGHTED TEMPORAL AVERAGE — a blend favouring the peak frame, not a single-frame pick — so the result is a flowing, liquid, recognizable-waveform distortion (sub-frame temporal positions are interpolated manually for a buttery result). MORPH is the smoothest possible cross-dissolve: a spatially-uniform, pop-free scan between temporal positions using a periodic raised-cosine (Hann) reconstruction kernel that is C¹ at its window edges AND N-periodic across the 59→0 wrap seam, so scanning MORPH loops seamlessly. CHAOS is the original per-pixel stochastic look — for every pixel the shader draws exactly ONE source frame (a whole-pixel dither/mosaic) chosen in O(1) by an analytic inverse-CDF from a static screen-space threshold, so a still input yields a stable image while moving content becomes a coherent morph-scannable time-smear. LAG MODEL: lag = (mode ≠ CHAOS) && !LIVE. CHAOS is always real-time (no lag). SMOOTH and MORPH auto-engage a ~2-second lag (they read a trailing window of the buffer) unless the LIVE control forces real-time. On the first real input frame the whole 60-layer ring is filled with that frame (buffer instantly full = a still image), then real frames wash in over ~2s — so there is no black warmup and a lagged read always hits a full buffer. LIVE (a faceplate switch OR its gate) forces real-time / no-lag in any mode; CHAOS (a momentary switch OR its gate) overrides the selector to the real-time CHAOS render while held. SPREAD is the temporal-average window (SMOOTH) / cross-dissolve width (MORPH) / bell window (CHAOS). SHIMMER is flow speed (SMOOTH field drift) / auto-scan drift (MORPH) / threshold dither (CHAOS). SHAPE morphs the CHAOS bell triangular↔gaussian (idle in SMOOTH/MORPH, which use a fixed gaussian/Hann). The X/Y waveform controls (freq, amt, shape per axis) sculpt SMOOTH's field. FREEZE (a toggle button, plus a freeze gate that holds the ring frozen while high) stops the ring from advancing so you can scrub a held 60-frame window; SAVE (a momentary button, also a rising edge on the save trigger) snapshots the ring into an in-GPU slot for later recall (and to feed a future video Cube). FRAMETABLE also SAVES + LOADS real FILES (the wavetable file workflow): a Save-file control writes the whole 60-frame ring to a lossless .frametable.png sprite-sheet atlas (a fixed 10x6 = 60-tile contact sheet, no video codec) on disk, and a Load control reads a .frametable.png back into the ring, freezing it so the loaded table can be morphed/scanned; the multi-megabyte frame bytes persist per-browser in IndexedDB (never the patch / Y.Doc — only a tiny file descriptor syncs), so a loaded or saved table survives a reload. Rendered at half engine resolution (SwiftShader/CI budget); an unpatched input renders black. The mode/lag dispatch, morphable-waveform field, weighted-average blend, Hann kernel, inverse-CDF and freeze/save/first-fill reducers are a 1:1 CPU mirror unit-tested in $lib/video/frametable-core. All ports live on the yellow drill-down PATCH PANEL (no raw side jacks).

the faceplate

frametablevideo_invideomorph_cvcvspread_cvcvshimmer_cvcvweightShape_cvcvwaveFreqX_cvcvwaveAmtX_cvcvwaveShapeX_cvcvwaveFreqY_cvcvwaveAmtY_cvcvwaveShapeY_cvcvfreeze_gategatesave_triggatechaos_gategatelive_gategatevideo_outvideoaudiocvgatepitch
15 inputs · 1 outputs · 18 params

inputs

idcablewhat it does
video_invideoThe source video recorded, frame by frame, into the 60-frame ring. Unpatched, the output is black.
RGB video stream
morph_cvcvCV that modulates MORPH (the centre point scanned through the 60-frame history), swept linearly over 0..1 (wraps at the ring seam).
control voltage (CV); modulates morph (additive offset — ±1 CV sweeps the full range, centered on the knob)
spread_cvcvCV that modulates SPREAD (the window width in frames — temporal-average width / dissolve width / bell window depending on mode), swept linearly over 1..60.
control voltage (CV); modulates spread (additive offset — ±1 CV sweeps the full range, centered on the knob)
shimmer_cvcvCV that modulates SHIMMER (flow speed in SMOOTH / auto-scan drift in MORPH / threshold dither in CHAOS), swept linearly over 0..1.
control voltage (CV); modulates shimmer (additive offset — ±1 CV sweeps the full range, centered on the knob)
weightShape_cvcvCV that modulates SHAPE (the CHAOS selection bell, triangular↔gaussian), swept linearly over 0..1.
control voltage (CV); modulates weightShape (additive offset — ±1 CV sweeps the full range, centered on the knob)
waveFreqX_cvcvCV that modulates X FREQ (cycles of the SMOOTH X-axis waveform across the screen), swept linearly over 0..8.
control voltage (CV); modulates waveFreqX (additive offset — ±1 CV sweeps the full range, centered on the knob)
waveAmtX_cvcvCV that modulates X AMT (the SMOOTH X-axis temporal-displacement amount), swept linearly over 0..1.
control voltage (CV); modulates waveAmtX (additive offset — ±1 CV sweeps the full range, centered on the knob)
waveShapeX_cvcvCV that modulates X SHAPE (the SMOOTH X-axis waveform morph, sine→tri→saw→square), swept linearly over 0..1.
control voltage (CV); modulates waveShapeX (additive offset — ±1 CV sweeps the full range, centered on the knob)
waveFreqY_cvcvCV that modulates Y FREQ (cycles of the SMOOTH Y-axis waveform), swept linearly over 0..8.
control voltage (CV); modulates waveFreqY (additive offset — ±1 CV sweeps the full range, centered on the knob)
waveAmtY_cvcvCV that modulates Y AMT (the SMOOTH Y-axis temporal-displacement amount), swept linearly over 0..1.
control voltage (CV); modulates waveAmtY (additive offset — ±1 CV sweeps the full range, centered on the knob)
waveShapeY_cvcvCV that modulates Y SHAPE (the SMOOTH Y-axis waveform morph), swept linearly over 0..1.
control voltage (CV); modulates waveShapeY (additive offset — ±1 CV sweeps the full range, centered on the knob)
freeze_gategateFREEZE gate. WHILE the gate is HELD HIGH (level >= 0.5) the ring is frozen — a momentary hold, with the output staying live over the held 60 frames — and it resumes the instant the gate drops low. OR-combined with the FREEZE toggle button, so either can freeze the ring independently.
gate / trigger; modulates freezeGate (summed directly (the destination DSP scales it)); gate — acts while the level is high (reacts to both edges)
save_triggateSAVE trigger. A RISING edge fires ONCE, snapshotting the current 60-frame ring into an in-GPU slot (idempotent per edge — held high does not re-snapshot).
gate / trigger; modulates saveTrig (summed directly (the destination DSP scales it)); trigger — fires once per rising edge
chaos_gategateCHAOS gate. WHILE held HIGH (level >= 0.5) it momentarily forces the real-time per-pixel CHAOS render, overriding the MODE selector; it drops back to the selected mode the instant the gate goes low. OR-combined with the momentary CHAOS switch.
gate / trigger; modulates chaosGate (summed directly (the destination DSP scales it)); gate — acts while the level is high (reacts to both edges)
live_gategateLIVE gate. WHILE held HIGH (level >= 0.5) it forces the real-time / no-lag read in ANY mode (SMOOTH and MORPH stop lagging and track the live input); it re-engages the ~2-second lag the instant the gate goes low. OR-combined with the LIVE switch.
gate / trigger; modulates liveGate (summed directly (the destination DSP scales it)); gate — acts while the level is high (reacts to both edges)

outputs

idcablewhat it does
video_outvideoThe rendered frame: the SMOOTH weighted-average field, the MORPH cross-dissolve, or the CHAOS selected-frame mosaic, depending on the active mode. The card preview shows this output.
RGB video stream

params

idlabelrangedefaultcurve
modemode0..2discrete
livelive0..1discrete
chaoschaos0..1discrete
morphmorph0..1linear
spreadspread1..60linear
shimmershimmer0..1linear
weightShapeshape0..1linear
freezefreeze0..1discrete
waveFreqXx freq0..8linear
waveAmtXx amt0..1linear
waveShapeXx shape0..1linear
waveFreqYy freq0..8linear
waveAmtYy amt0..1linear
waveShapeYy shape0..1linear
liveGatelive gate0..1linear
chaosGatechaos gate0..1linear
freezeGatefrz gate0..1linear
saveTrigsave0..1discrete

controls

controlwhat it does
chaosCHAOS (0/1, default 0): momentarily overrides the MODE selector to the real-time CHAOS render while engaged, then reverts to the selected mode. Latched by the switch; the chaos gate additionally forces it while that gate is high.
chaos gateHidden synthetic param the chaos-gate CV bridge writes each frame with the chaos gate LEVEL; while it is HIGH (>= 0.5) the CHAOS render is forced, overriding the MODE selector (OR-combined with the momentary CHAOS switch). Exposed only as the chaos gate jack, not as a knob.
Frametable file input {n}Load table - imports a `.frametable.png` atlas file (a lossless 10x6 = 60-tile contact sheet, no video codec) from disk and detiles it straight into the 60-frame ring, FREEZING the ring so the loaded table can be scanned with MORPH and SPREAD without live frames washing it away. A status line reports the frame count and tile size, or the reason a file was rejected (a PNG whose dimensions are not a 10x6 grid is not a frametable atlas). The multi-megabyte frame bytes are kept in THIS browser's IndexedDB and only a ~120-byte descriptor is saved with the patch, so a loaded table survives a reload for you but a peer without the local copy sees the loader rather than your table.
Frametable save file {n}Save table - reads all 60 ring layers back from the GPU, tiles them into a lossless `.frametable.png` atlas and writes it to disk through the file picker (or a plain download where no picker exists). FREEZE first if you want a specific 60 frames: the ring keeps advancing while you click otherwise. Unlike the SAVE pad above — which snapshots into GPU memory and dies on reload — this produces a real file you can keep, re-load into any FRAMETABLE, and pass to someone else. It also stores a copy in this browser and stamps the descriptor onto the node, so the saved table is what a reload restores.
freezeFREEZE (0/1, default 0): stops the ring from advancing so the held 60-frame window can be scrubbed with MORPH/SPREAD (the render pass keeps running, so the controls stay live over the frozen frames). The toggle button latches it; the freeze gate additionally holds it frozen while that gate is high.
frz gateHidden synthetic param the freeze-gate CV bridge writes each frame with the freeze gate LEVEL; while it is HIGH (>= 0.5) the ring is held frozen (OR-combined with the FREEZE toggle, so the per-frame level never stomps the button's latched state). Exposed only as the freeze gate jack, not as a knob.
liveLIVE (0/1, default 0): forces the REAL-TIME / no-lag read in any mode. Off (default) leaves SMOOTH and MORPH auto-lagged by ~2 seconds so they read a full trailing buffer; on makes every mode track the live input. Latched by the switch; the live gate additionally forces it while that gate is high.
live gateHidden synthetic param the live-gate CV bridge writes each frame with the live gate LEVEL; while it is HIGH (>= 0.5) the real-time / no-lag read is forced in any mode (OR-combined with the LIVE switch, so the per-frame level never stomps the button's latched state). Exposed only as the live gate jack, not as a knob.
modeMODE (0..2, default 0 = SMOOTH): the render engine. 0 = SMOOTH (flowing 2D temporal-average field from two morphable waveforms, auto-lagged), 1 = MORPH (smoothest cross-dissolve scan of the table, auto-lagged), 2 = CHAOS (the original per-pixel inverse-CDF single-frame pick, always real-time). A faceplate selector; no CV (discrete).
morphMORPH (0..1, default 0): scans the centre point through the 60-frame ring (wraps at the seam). SMOOTH → the field DC / scan depth; MORPH → the temporal position being dissolved to; CHAOS → the centre lag of the bell.
saveSAVE (0/1, default 0): a MOMENTARY press-pad. Its RISING edge snapshots the current 60-frame ring into an in-GPU slot (idempotent per edge — held down does not re-snapshot), which is the slot a VideoCube reads and the one the save trigger jack also fires. The snapshot lives in GPU memory and dies on reload; the FILE controls below are the ones that survive it. Releasing returns the pad to rest, so nothing is left latched in the patch.
shimmerSHIMMER (0..1, default 0): SMOOTH → flow SPEED (the field drifts, incommensurate X/Y rates so it never loops = liquid); MORPH → a gentle auto-scan drift of the dissolve centre; CHAOS → temporal dither of the static per-pixel threshold. 0 = fully static.
spreadSPREAD (1..60, default 12): the window width in frames. SMOOTH → the temporal-average window; MORPH → the cross-dissolve blend width (1 ≈ a crisp single moment, wider = a longer buttery dissolve); CHAOS → the bell window (1 = a single-frame delta).
x amtX AMT (0..1, default 0.35): how far the SMOOTH X-axis waveform displaces the per-pixel temporal centre (mapped to frames = amt·N/2). 0 = flat; higher = more of the history spread across the X axis. SMOOTH mode only.
y amtY AMT (0..1, default 0.35): the SMOOTH Y-axis temporal-displacement amount (mapped to frames = amt·N/2). SMOOTH mode only.
x freqX FREQ (0..8, default 1): cycles of the SMOOTH X-axis morphable waveform across the screen. More cycles = finer horizontal temporal ripples in the field. SMOOTH mode only.
y freqY FREQ (0..8, default 1): cycles of the SMOOTH Y-axis morphable waveform across the screen. SMOOTH mode only.
x shapeX SHAPE (0..1, default 0): morphs the SMOOTH X-axis waveform sine→triangle→saw→square (zero-crossing-aligned so blending never phase-cancels). SMOOTH mode only.
y shapeY SHAPE (0..1, default 0): morphs the SMOOTH Y-axis waveform sine→triangle→saw→square. SMOOTH mode only.
shapeSHAPE (0..1, default 0): morphs the CHAOS selection bell from triangular (0, compact) to gaussian (1, smooth). Idle in SMOOTH/MORPH (which use a fixed gaussian / Hann kernel).

source

frametable.ts on GitHub.

Generated from packages/web/src/lib/{audio,video}/module-registry.ts · repo