The Java video pipeline library under OpenJVID — ingest, digest, egress, NDI5, QOYV, and a 60fps flow controller.

jVid is a flow-based video and audio processing library written in Java with JNI/native C pieces. It is the engine that OpenJVID wires up in a graph UI: capture, transform, and ship frames across the local machine or a LAN.

Repo: github.com/danschirripa/jVid

If OpenJVID is the rack you patch cables into, jVid is the DSP cards behind the panels.

Relation to OpenJVID

Layer Role
OpenJVID Desktop app — component palette, property dialogs, diagnostic web UI, peripheral discovery
jNodeGraph Swing node-graph front end OpenJVID embeds
jVid Pipeline library — VideoIngestor / VideoDigestor / VideoEgress, FlowController, codecs, NDI, OpenCV

OpenJVID does not reimplement capture or codecs. Its graph components map onto jVid processors: NDI Ingest/Egress, FFMPEG Ingest, QOYV Ingest/Egress, Scaling Digestor, Multiview, Face Detector, Audio Injector/Extractor (JACK), and so on. The peripheral path’s QOYV subset also bottoms out in jVid’s streaming types.

The custom video format story (QOI → QOY → QOYV) is spelled out in the QOYV post. jVid owns the native encode/decode and the Java stream ingress/egress; OpenJVID exposes them as graph nodes.

Pipeline model

Everything is a directed flow of frames:

flowchart LR
  ingest[VideoIngestor] --> digest[VideoDigestor]
  digest --> egress[VideoEgress]
  fc[FlowController] -.->|~60fps drive| ingest
  • VideoProcessor — shared contract: processFrame, open / close, resolution, properties
  • VideoIngestor — sources (NDI, FFmpeg, GStreamer, OpenCV camera, QOI/QOY streams, screen capture, test patterns)
  • VideoDigestor — transforms (scale, crop, multiview, matrix route, face detect/paint, auto-track, audio inject/extract, picture adjust)
  • VideoEgress — sinks (NDI, QOY/QOI streams, local/SDL windows, GStreamer, file/series, UVC, serial/PTZ helpers)
  • FlowNode / FlowController — graph wiring and the global clock that walks ingestors and propagates frames

Target cadence is about 60fps (~16ms per tick), with drop-rather-than-block behaviour when the graph falls behind.

What ships in the library

Ingest

NDI 5 receive, FFmpeg (URL / file / device / raw string), GStreamer, OpenCV cameras, local screen capture, QOI and QOYV TCP streams, raw video/audio, and test patterns.

Digest

Scaling (Java, OpenCV, FFmpeg variants), crop, multiview composite, matrix routing, face detection (Haar / DNN), face-set painting, auto-framing and PTZ auto-tracking, background removal, picture adjustment, frame sync, and JACK-backed audio inject/extract.

Egress

NDI 5 publish (optional audio), QOYV / QOI stream senders, local and accelerated/SDL windows, fullscreen and PTZ-button previews, GStreamer, image series / raw dumps, UVC, and experimental paths (e.g. AV1 stream experiments).

Native / platform bits

JNI libraries are packaged under assets/ for amd64, aarch64, and darwin, including QOY/QOI codecs and NDI bindings. JACK and OpenCV availability still follow the host OS.

V2: timestamped frames

Legacy processors pass bare BufferedImages — fine for many live graphs, weak for A/V sync. Newer VideoProcessorV2 / VideoIngestorV2 APIs carry a VideoFrame with presentation timestamps so PTS from NDI or FFmpeg can survive the graph. Legacy nodes can be wrapped with adapters while the stack migrates.

That work sits alongside digestor/runtime hardening (multiview behaviour, picture-adjustment digestors, operational test suites in-tree).

How you’d use it without OpenJVID

jVid is a Maven library (JVid:JVid), not a product UI. A host process builds a graph of processors, registers them with FlowController, and runs. OpenJVID is the primary productized host; the same library can back other tools or headless nodes for distributed capture → encode → process → NDI/QOYV layouts.

Further reading