video-processor — FaceKom v2 repo reference

video-processor

Browser-side TypeScript library that performs real-time virtual background / background-blur on a webcam MediaStream using Google MediaPipe’s ImageSegmenter (DeepLab v3 model). It is built into a single ES-module bundle (build/video-processor.js) and copied into consuming FaceKom front-end projects — it is not a standalone server or service.

• npm name: @techteamer/video-processor, version 1.0.0 (package.json:2-3)
• description: "Video stream processor" (package.json:4)
• author TechTeamer, license MIT, repo https://github.com/TechTeamer/video-processor (package.json:18,26,29)
• HEAD commit: cc2aaa2 chore: [fkqa-327]: maintenance/master (#6) (base branch master)

Note

Distribution is copy-paste, not npm install. README.md:11-20 says: build with yarn build, then copy build/video-processor.js and build/video-processor.d.ts into “the relevant project”. See architecture-overview for where this plugs into the FaceKom front-end.

Purpose & role in FaceKom

Provides a client-side VideoProcessor class (see Key modules) that:

1. Takes a MediaStream (typically from getUserMedia) — VideoProcessor.ts:69.
2. Segments each frame into person/background using MediaPipe ImageSegmenter with the bundled deeplab_v3.tflite model — VideoProcessor.ts:116-131.
3. Replaces the background pixels with a still background image and returns a new processed MediaStream via canvas.captureStream() — VideoProcessor.ts:193-202, callbackForVideo VideoProcessor.ts:214-259.

The processed stream is what a consuming app would feed into a WebRTC video element / call. The bundled example/ app demonstrates exactly this flow (example/index.js:13-27).

Note

Role within FaceKom is inferred from the library’s own code + README (“Copy the artifacts also to the relevant project”). This repo contains no RabbitMQ, HTTP, DB, or other FaceKom-service integration code — see Dependencies on other FaceKom services.

Tech stack & runtime

Aspect	Value	Source
Language	TypeScript (ES modules, "type": "module")	package.json:31, src/*.ts
Target runtime	Browser (uses window, document, HTMLVideoElement, HTMLCanvasElement, Worker, requestAnimationFrame)	VideoProcessor.ts:5,99,134,172; src/index.ts:5
Node version (build/tooling)	>=20.8.1	package.json:32-34 (engines)
TS compile target	es2022, module: esnext, libs esnext/dom/WebWorker	tsconfig.json:3-4,12-16
Bundler	Rollup 4 (esbuild transform + terser + web-worker inline)	rollup.config.js, package.json:51
Package manager	Yarn (classic 1.x — yarn.lock present, yarn audit v1.22.22 in logs)	yarn.lock, security/2026-05-04.md
Test runner	Vitest 1.6 + c8/coveralls	package.json:22-23
Lint	ESLint 8 (standard + n + @typescript-eslint/stylistic)	.eslintrc.json:3

Runtime dependencies (only two — package.json:58-61):

• @mediapipe/tasks-vision ^0.10.17 — the vision segmentation engine.
• ua-parser-js ^1.0.39 — used to detect Firefox-on-non-Windows to pick CPU vs GPU delegate (VideoProcessor.ts:111-114,125).

Warning

rollup.config.js:32 sets external: [], so all deps (including @mediapipe/tasks-vision) are bundled into build/video-processor.js. The output .js is large and self-contained. The web worker is inlined too (webWorkerLoader({ target: 'browser', inline: true }), rollup.config.js:24).

Build & run

Entrypoints:

• main / module → build/video-processor.js (package.json:5-6) — generated, not in the source tree (.gitignore:18-20 ignores /build/).
• exports["."] maps import/require/default → ./build/video-processor.js, types → ./build/video-processor.d.ts (package.json:7-13).
• Source entry for the bundle: src/index.ts (rollup.config.js:17,35).
• No bin field, no CLI, no server start command. There is no Dockerfile in the repo (verified via git ls-tree — see Top-level structure).

npm/yarn scripts — verbatim with effect (package.json:19-25)

Script	Command	What it does
build	rollup -c	Runs Rollup with rollup.config.js: produces build/video-processor.js (ESM, minified, inline worker, sourcemap) and build/video-processor.d.ts (type bundle via rollup-plugin-dts).
lint	eslint ./src	Lints the src/ TypeScript.
unit	vitest run	Runs Vitest once (no watch).
test	eslint . && c8 --temp-directory=./test/coverage/tmp vitest run && c8 report --reporter=text-lcov --report-dir=./test/coverage/ --reporter=lcovonly | coveralls	Lints everything, runs Vitest under c8 coverage, then pipes an lcov report to coveralls.
tsc	tsc	Type-checks with tsconfig.json. Note tsconfig.json:29 sets "noEmit": true, so this is check-only (it does not emit the build — Rollup does).

To run locally (inferred from README.md:15-20 + example/): yarn build, then serve the repo root over HTTP and open example/index.html (it imports ../build/video-processor.js and loads model from /model/wasm). The example needs camera permission and the build/ to exist first.

Warning

The test script depends on a coveralls binary and a c8 binary that are not in package.json dependencies/devDependencies (only @vitest/coverage-v8 is). Running yarn test as written may fail without c8/coveralls installed globally or added. yarn unit (just vitest run) is the safer local run.

Top-level structure

Verified via git ls-tree -r HEAD. Meaningful entries:

Path	One-liner
src/	TypeScript source (5 files): the library. See .
example/	Standalone browser demo: index.html, index.js, plus remote-video-background.jpg (the default background).
model/wasm/	MediaPipe runtime assets committed into the repo: deeplab_v3.tflite (segmentation model) + 4 WASM/JS files (SIMD + non-SIMD vision runtime). See callout below.
test/	Only test/vitest.config.js — Vitest config. No spec files exist. See .
maintenance/	Dated dependency-maintenance logs (2026-04-07.md, 2026-05-04.md) — yarn outdated output + upgrade notes.
security/	Dated yarn audit logs (2026-04-07.md, 2026-05-04.md) — known advisory tracking.
rollup.config.js	Rollup build config (two outputs: JS bundle + dts).
tsconfig.json	TS compiler config (browser libs, strict, noEmit).
workers.d.ts	Ambient module decl for web-worker:* imports (workers.d.ts:1-4).
.eslintrc.json / .editorconfig / .gitignore / .npmignore	Tooling config.
package.json / yarn.lock	Manifest + lockfile.
README.md	Short usage/update/deploy notes.

Warning

No Git-LFS in this repo. There is no .gitattributes at HEAD (verified: git cat-file -p HEAD:.gitattributes → “path does not exist”). The model/wasm/ binaries are committed directly as ordinary git blobs, not LFS pointers:

• deeplab_v3.tflite — 2,780,176 bytes (header magic TFL3 confirmed → real TFLite model)
• vision_wasm_internal.wasm — 9,514,390 bytes
• vision_wasm_nosimd_internal.wasm — 9,379,090 bytes
• vision_wasm_internal.js / vision_wasm_nosimd_internal.js — ~204 KB each Sizes from git ls-tree -r -l HEAD model/. Do not open the .wasm/.tflite blobs as text.

Note

README.md:5-11 (“Updating”): when @mediapipe/tasks-vision is upgraded, the wasm files must be manually re-copied from node_modules into model/, and a new TFLite model added when the model updates. The model/ dir is hand-maintained, not generated by the build.

First-party bin scripts

None. There is no first-party bin/ directory and no bin field in package.json (verified via git ls-tree -r HEAD — no bin/ path exists; the only executables referenced are tool binaries rollup/eslint/vitest/tsc/c8/coveralls from devDependencies). Nothing to enumerate.

Key modules

All under src/. Read in full.

src/index.ts (5 lines)

Barrel/entry. Exports VideoProcessor and assigns it to window.VideoProcessor (src/index.ts:5) — i.e. the bundle also exposes the class as a global for non-module consumers.

src/VideoProcessor.ts (260 lines) — the core class

• VideoProcessorOptions interface (:9-20): slidingWindowSize, assets.{url, modelLocation}, video.{width,height,frameRate}.
• defaultOptions (:36-47): sliding window 3, video 640×480@30, model from https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision/wasm, background '/example/remote-video-background.jpg'.
• Private constructor (:52-67): spawns the inlined web worker (new MyWorker()), deepMerges options, parses UA, reads the input track’s real width/height (throws 'Input MediaStream does not have a valid video track' if missing), creates an offscreen output canvas with willReadFrequently: true.
• static async create(mediaStream, options) (:69-79): the public factory. Builds the instance then sends an init message to the worker (width/height/bufferSize) and awaits ack.
• sendMessageToWorker() (:81-92): Promise wrapper around worker.postMessage / onmessage; rejects on event.data.error.
• createImageSegmenter() (:116-131): FilesetResolver.forVisionTasks(modelLocation) then ImageSegmenter.createFromOptions with modelAssetPath = ${modelLocation}/deeplab_v3.tflite, runningMode: 'VIDEO', outputCategoryMask: true. Delegate is CPU on Firefox-non-Windows, else GPU (:125, via isFirefox() :111-114).
• attach(element?) (:150-165): creates hidden <video> + <canvas>, appends to the given element (or document.body), builds the segmenter, loads the background image into bgImgData.
• startProcessing() (:193-202): sets video.srcObject = externalStream, plays, hooks loadeddata → predictWebcam, returns canvas.captureStream(frameRate). Throws if attach() wasn’t called.
• stopProcessing() (:204-212): stops all tracks of the processed stream. Throws if not attached / not started.
• predictWebcam() (:167-183): per-frame loop; skips duplicate frame times; draws video to the output canvas; calls imageSegmenter.segmentForVideo(...) with performance.now() timestamp and callbackForVideo.
• callbackForVideo(result) (:214-259): the compositing core. Reads the raw mask (result.categoryMask.getAsUint8Array()), pushes it to the worker (frame), requests the smoothed mask back (mask), then per-pixel blends source vs background using alphaTable = [0, 0.8, 0.9, 1] (:34) indexed by the smoothed mask value; puts the result onto the visible canvas; schedules the next frame via setTimeout(..., 1000/frameRate).

Warning

Verified bug in callbackForVideo (VideoProcessor.ts:243-244): the blue channel blend mistakenly uses the green source again — imageData[j+2] = imageData[j+2]*alpha + this.bgImgData[j+1]*(1-alpha) (should be bgImgData[j+2]). This shifts background color in the blended (edge) region. Present at HEAD.

Note

The file contains profanity in a comment at VideoProcessor.ts:231 (// FUCK OFF ESLINT) and several commented-out perf-timing lines (:215,256-257). Harmless but noted for anyone grepping.

src/videoWorker.ts (50 lines) — the web worker

Runs off the main thread. Owns a SlidingWindowBuffer. Message protocol (discriminated union VideoProcessorWorkerMessage, :4-20):

• init → creates new SlidingWindowBuffer(bufferSize, width*height), replies {result:'ok'}.
• frame → slidingWindowBuffer.push(mask).
• mask → posts back slidingWindowBuffer.getMaskData() (the temporally smoothed mask). Wraps handling in try/catch and posts {error} on failure (:45-47). Exports self as default for the worker-loader (:50).

src/SlidingWindowBuffer.ts (39 lines)

A fixed-size circular buffer of Uint8Array masks (:1-15). push() overwrites the oldest (:21-24). getMaskData() (:30-38) sums, per pixel, how many of the buffered frames had category value 15 (the person class in DeepLab) — producing a count in [0, bufferSize] that indexes alphaTable on the main thread. This is the temporal anti-flicker smoothing.

src/utils.ts (14 lines)

Single export deepMerge(target, source) — recursive object merge used for options (VideoProcessor.ts:54, videoWorker.ts:38).

workers.d.ts

Ambient declaration so import MyWorker from 'web-worker:./videoWorker.ts' type-checks (VideoProcessor.ts:5). Registered via tsconfig.json:28 "types": ["node", "./workers.d.ts"].

Configuration

This is a build-time-configured library; there are no runtime env vars and no getconfig/AJV schemas (none present in the tree). “Configuration” is:

1. Caller-supplied options — the Partial<VideoProcessorOptions> passed to VideoProcessor.create() (VideoProcessor.ts:9-47). Key knobs: slidingWindowSize, video.frameRate, assets.url (background image), assets.modelLocation (where the .tflite/wasm are served from). The example overrides modelLocation: '/model/wasm' and assets.url (example/index.js:14-20).
2. Build config: rollup.config.js (bundle + dts, target es2022, inline worker, terser), tsconfig.json (browser libs, strict, noEmit).
3. Tooling config: .eslintrc.json, .editorconfig (2-space, LF, max line 240), .npmignore (publishes only build/ per package.json:15-17; .npmignore additionally excludes test, .eslintrc.json, yarn.lock, etc.), .gitignore (ignores node_modules/, /build/, dist/, test/coverage/).
4. Test config: test/vitest.config.js (:4-16) — excludes node_modules/, web/, yarn-offline-cache/; v8 coverage → test/coverage/vitest; 30 s timeout.

Note

test/vitest.config.js:11 references a customization/test coverage-exclude path that does not exist in this repo — likely copied from a sibling FaceKom repo template (cf. customization-architecture). Harmless leftover.

Tests

• Framework: Vitest (vitest run, package.json:22), coverage via @vitest/coverage-v8 / c8, reported to Coveralls in test (package.json:23).
• Config: test/vitest.config.js.
• There are currently no test/spec files. git ls-tree -r HEAD shows the only file under test/ is vitest.config.js. So yarn unit / yarn test would run zero specs at HEAD.
• Run: yarn unit (specs only) or yarn test (lint + coverage + coveralls — see the Build & run warning about missing c8/coveralls deps).

Dependencies on other FaceKom services

None evidenced in code. This repo has:

• No RabbitMQ / amqp usage (no such import or queue name anywhere in src/).
• No HTTP server or outbound FaceKom HTTP calls. The only network fetch is MediaPipe loading model/wasm from assets.modelLocation (default jsDelivr CDN, or /model/wasm when self-hosted) — VideoProcessor.ts:117-124.
• No DB / Redis client.

Its only coupling to the rest of FaceKom is build-output reuse: the compiled video-processor.js / .d.ts are copied into consuming front-end projects (README.md:11-20). See architecture-overview and vuer_oss for the larger system; this library is a leaf front-end asset.

Verified gotchas (summary)

Warning

• No LFS / binaries committed raw: model/wasm/* (~21 MB total: 9.5 MB + 9.4 MB wasm, 2.8 MB tflite) are plain git blobs. No .gitattributes. Cloning pulls the full binaries.
• build/ is gitignored and not published-from-source: consumers must run yarn build (or receive a copied artifact) — there is no committed build/. package.json main points at a path that does not exist until built.
• Blue-channel compositing bug at VideoProcessor.ts:244 (uses bgImgData[j+1] for the blue channel).
• test script needs c8 + coveralls that aren’t declared as deps.
• Empty test suite at HEAD (only vitest.config.js).
• Manual model/wasm update step is required on every @mediapipe/tasks-vision upgrade (README.md:5-11); easy to forget and ship a stale model.
• Firefox path uses CPU delegate (slower) by design (VideoProcessor.ts:125); other browsers use GPU.

Unverified / gaps

• I did not open the binary blobs (*.wasm, *.tflite, remote-video-background.jpg) — recorded by size/magic only, per task rules. Their internal contents are third-party MediaPipe artifacts.
• The maintenance/*.md and security/*.md files were read in part (dependency yarn outdated/yarn audit logs); I captured that they are dated upgrade/advisory logs (e.g. security tracks an esbuild moderate advisory via vitest > vite > esbuild, patched >=0.25.0). I did not transcribe every advisory line.
• I did not read yarn.lock line-by-line (142 KB lockfile) — surveyed structurally; dependency versions cited come from package.json and the maintenance logs.
• The precise FaceKom front-end repo that consumes this artifact is not identifiable from this repo alone (README says only “the relevant project”); cross-repo wiring is out of scope here — see architecture-overview.

Sources

Files read (all under /Users/levander/coding/facekom-v2-clones/video-processor):

• package.json, README.md, rollup.config.js, tsconfig.json, workers.d.ts
• src/index.ts, src/VideoProcessor.ts, src/videoWorker.ts, src/SlidingWindowBuffer.ts, src/utils.ts
• example/index.html, example/index.js
• test/vitest.config.js
• .eslintrc.json, .editorconfig, .gitignore, .npmignore
• maintenance/2026-05-04.md, maintenance/2026-04-07.md (partial), security/2026-05-04.md, security/2026-04-07.md (partial)
• Git inspection (read-only): git -C … branch -a, log --oneline, ls-tree -r [-l] HEAD, cat-file -p HEAD:.gitattributes (absent), cat-file -p HEAD:model/wasm/deeplab_v3.tflite (magic/size only)
• Binaries recorded by metadata only (not opened): model/wasm/deeplab_v3.tflite, model/wasm/vision_wasm_internal.{js,wasm}, model/wasm/vision_wasm_nosimd_internal.{js,wasm}, example/remote-video-background.jpg