grains of vision

GRAINS OF VISION — a granular VIDEO synthesizer. It shatters the incoming picture into a swarm of tiny windowed patches (grains) sampled from source A at jittered positions AND jittered moments in time (from a short frame-history ring, so grains have a real TEMPORAL axis — the novel part of granular VIDEO), re-scatters them into a new frame, then runs that through a FEEDBACK block and a video REVERB block on ONE fixed linear chain (video -> granular engine -> feedback -> reverb -> out) — NOT a node graph. The grain engine is a single fullscreen pass that is O(1) per pixel regardless of density: the frame is a jittered grid of cells (DENSITY = cells across), each cell hosts one hashed grain, and every pixel gathers its 3x3 cell neighbourhood, window-weights each grain and normalises — so density changes grain SIZE, not the cost. Knobs map the audio-granular vocabulary onto the image: SIZE (window radius), WINDOW (hard chip -> soft gaussian), SPRAY (position scatter of the grain + source-read; 0 reconstructs the picture faithfully, rising clouds it into abstraction), RATE (how far back in the frame history grains read — the pitch/rate analog), T-SPRAY (temporal scatter for a shimmering time-smear), ORIENT (per-grain rotation). The FEEDBACK block mixes the transformed previous OUTPUT back in (FB amount / decay / per-pass zoom + rotate that compound into tunnels and spirals); FB amount 0 (or FB Dry) is a transparent dry passthrough. The REVERB block is a true VIDEO reverb — the image analogue of an FDN/Schroeder tail: a decaying, spatially-diffused temporal accumulator (rev = blur(feedbackImg + decay*prevRev)), so each bright grain blooms outward (Rev Size = room size, Rev Diffuse = tap scatter) and lingers, fading over frames (Rev Decay = tail length), mixed dry/wet by Rev Mix; Rev Mix 0 (or Rev Dry) is a transparent dry passthrough (its blur passes skipped). Patch a second source into B and pick a COMPOSITE mode to have B modulate A grains region-by-region: density-map (B brightness thins/thickens grains), displace (B chroma warps where grains read from), size-map (B scales grain size), rate-map (B scrubs the per-region history time); with only A patched it runs mono-source (composite off). TWO video outputs: OUT (the full chain) and GRAINS (the raw granular scatter tap, pre-feedback/pre-reverb, resolved via read outputTexture:grains — a clean grains-only output to key/mix elsewhere, optional). Rendered at half engine resolution (SwiftShader/CI budget). Like every video processor an unpatched input renders black; the defaults are tuned so any patched source is immediately alive. All ports live on the yellow drill-down PATCH PANEL (no raw side jacks). The grain window / hash / temporal-blend / feedback-UV / reverb-accumulate / composite-mode math is a 1:1 CPU mirror unit-tested in $lib/video/modules/grainsOfVision.

grains of vision is a granular VIDEO synthesizer: it shatters the incoming picture into a swarm of tiny windowed patches (grains) and re-scatters them into a new frame, then runs that through a feedback block and a video-reverb block on ONE fixed linear chain (video -> granular engine -> feedback -> reverb -> out) — not a node graph. A grain is a small windowed patch of source A sampled at a jittered position AND a jittered moment in time (from a short history of recent frames), so grains have a real temporal axis — the novel part of GRANULAR video. Density sets how many grains pack the frame; Size sets each grain's radius (>=1 cell = overlapping, blended); Window morphs the grain edge from a hard chip to a soft gaussian bloom; Spray scatters where each grain sits and where in the source it grabs from (at 0 it reconstructs the picture faithfully, rising it clouds into abstraction); Rate sets how far back in the frame-history grains reach and T-Spray scatters them across time for a shimmering time-smear; Orient tumbles each patch. The FEEDBACK block mixes the previous OUTPUT back in, zoomed/rotated a touch each pass so it compounds into tunnels and trails (FB amount 0 = a transparent dry passthrough). The REVERB block is a true video reverb — a decaying, spatially-diffused accumulator (the image analogue of a reverb tail): each bright grain blooms outward (Rev Size = room size, Rev Diffuse = how it scatters) and lingers, fading over frames (Rev Decay = tail length), mixed dry/wet by Rev Mix (0 = a transparent dry passthrough). Patch a second source into B and pick a COMPOSITE mode to have B modulate A's grains region-by-region: density-map (B brightness thins/thickens grains), displace (B warps where grains grab from), size-map (B scales grain size), rate-map (B scrubs the per-region time). With only A patched it runs mono-source (composite off). Like every video processor an unpatched input renders black; the defaults are tuned so any patched source is immediately alive — a mid-density overlapping grain field with a little temporal smear, gentle feedback trails and a soft reverb bloom.

the faceplate

inputs

outputs

params

controls

source

grainsOfVision.ts on GitHub.