Levandor

Face Comparison: which code paths actually persist a row

9 min read

Which vuer_oss code paths write a faceComparisons row, which ones compute a face match without persisting anything, and why compareFaceWith is not the universal switch earlier vault notes implied. Verified against vuer_oss at tag raiffeisen-1.9.11.100 (worktree /Users/levander/coding/facekom/vuer_oss-rel100, git describe = raiffeisen-1.9.11.99-7-g350d3e626b).

This note CORRECTS a half-truth in older vault notes

face-comparison-different-face-db-query and face-comparison-data-verdict-threshold-model state that liveness comparisons “only persist if the proto sets compareFaceWith”. That is true only for the liveness-check-v2 handler. There are three liveness handlers; two of them never persist a comparison at all, and compareFaceWith is inert on those. Following the old advice sends you to add compareFaceWith to a proto where it does nothing — which is exactly what happened during SLARAFIPI-84 triage on 2026-08-28.

For Agents

Decision rule when asked “why is comparison X missing from the DB / the export?”:

  1. Find the task’s options.step.type. If it is liveness-check or liveness-check-v1 → no row is possible, full stop. Do not suggest compareFaceWith.
  2. If it is liveness-check-v2 → row requires recognitionOptions.compareFaceWith and a successful CV face extract.
  3. If it is a photo/info task with recognitionOptions → core FlowService path; row requires both sides to have a FaceRecognition with status:'success' AND getFaceCount() > 0.
  4. Partner customization/ may compute a match in memory and never persist (Raiffeisen/myra does this twice). Always grep customization/flow/**/*.handler.js and customization/listeners/*.js before concluding.

TL;DR

  • There is exactly ONE writer: FaceRecognitionService.createFaceComparisonModel → FaceComparison.create (server/service/FaceRecognitionService.js:152).
  • Exactly three call sites reach it. Not four — the older “4 call sites” tables double-count self-service v1.
  • status is always 'success' in practice. A failed comparison is structurally unrepresentable.
  • A face match can be computed without any row existing — three separate mechanisms do this.
  • faceRecognitions.imageCategory is not a setting; it is a copy of the task’s screenshotCategory.

1. The single writer

server/service/FaceRecognitionService.js:

async compareFaceDetections (recognitionFrom, recognitionTo, faceCompareModelParams) {   // :213
  const distance = this.calculateCosineDistance(recognitionTo.convertDescriptors(), recognitionFrom.convertDescriptors())
  const faceRecognition = await this.createFaceComparisonModel(Object.assign({}, faceCompareModelParams, {
    distance, recognitionFromId: recognitionFrom.id, recognitionToId: recognitionTo.id
  }))                                                                                    // :216
  return faceRecognition
}
  • No threshold gate. The distance is persisted whatever its value. A “different face” result is not filtered out here — see face-comparison-distance-thresholds for why the verdict is read-time only.
  • createFaceComparisonModel (:140-160) destructures status = 'success' as a default.

status is effectively a constant — failures cannot be recorded

The model enum is {created, failed, success}, but of the three call sites one passes status:'success' explicitly and the other two pass no status at all (so the default applies). Nothing in vuer_oss ever writes status:'failed' to faceComparisons. Consequence: every guard upstream of compareFaceDetections fails by writing no row, not by writing a failed row. Any report built on this table therefore shows only successes, and “the export is missing the failures” is a category error — the failures were never rows. This is the root of complaint F1 in SLARAFIPI-84.

2. The three call sites

#Sitefile:lineGates (ALL must pass)FK written
1Core flow / portrait / ID-docserver/flow/FlowService.js:2918-2968 (handleTaskRecognitionOptions, from submitTaskPhoto:2905)config.faceRecognition truthy · recognitionOptions.compareFaceWith set · ≥1 named compare task exists · recognitionFrom has status:'success' and getFaceCount()>0 · same for recognitionToselfServiceRoomId
2Liveness v2 onlyserver/service/SelfServiceV2Service.js:1416 gate → :1431 call, inside saveLivenessCheckV2Messages:1349CV result has status.checkGroups and face.encodings · task.options.recognitionOptions.compareFaceWith set · compare task has data.attachmentId · a matching FaceRecognition existsselfServiceRoomId
3Videochat close hookserver/faceRecognition/faceRecognitionHooks.js:41, hook videochat:close (:25-49)deployment config faceRecognition.comparisonPairs yields a pair · both sides found via findBestScoreRecognitionForRoom · both getFaceCount()>0roomId

Correcting the "4 call sites" count

Older notes list a 4th entry, “Self-service V1 — via the same compareFaceDetections”. At raiffeisen-1.9.11.100 there is no such distinct site: a repo-wide grep for compareFaceDetections returns exactly the three call sites above plus the definition. Treat the count as 3.

Every guard on path 1 is a silent skip

FlowService.js:2918-2968 — the path that writes the overwhelming majority of self-service rows:

  • :2921 — if (!config.faceRecognition || !recognitionOptions || !recognitionOptions.compareFaceWith) return
  • :2933 — compare task(s) not found → logger.warn + return
  • :2937-2947 — recognitionFrom missing / status !== 'success' / zero faces → logger.warn + return
  • :2954-2966 — recognitionTo missing or zero faces → the if simply doesn’t fire; the loop continues with no log at all

So a missing row can mean any of five different upstream conditions, and only two of them leave a log line.

3. Liveness has THREE handlers — only one can persist

This is the fact that makes the old compareFaceWith advice wrong. Dispatch is by task.options.step.type in server/queue/rpc_server/SelfServiceV2.js:208-219:

step.typeHandlerPersists a comparison?Does compareFaceWith do anything?
liveness-checkSelfServiceV2Service.handleLivenessCheck:1199NoNo — inert
liveness-check-v1handleLivenessCheckV1:1459 → mergeRecognitions:1310NoNo — inert
liveness-check-v2handleLivenessCheckV2:2114 → saveLivenessCheckV2Messages:1349Yes, if gated openYes

handleLivenessCheckV1 calls mergeRecognitions on both its success and failure branches and never touches compareFaceDetections, createFaceComparisonModel, or createFaceRecognitionModel. The comparison-writing block lives only inside saveLivenessCheckV2Messages, which is called only from handleLivenessCheckV2 (:2146 success, :2181 failure).

Raiffeisen / myra runs liveness-check-v1

customization/flow/myra-self-service-v2-phase-1/myra-self-service-v2-phase-1.flow.proto.js:123-133 declares the liveness task as name: 'liveness-check-v1', step.type: 'liveness-check-v1', and gives it no recognitionOptions at all. ⇒ It dispatches to handleLivenessCheckV1. Adding compareFaceWith to that task would change nothing — there is no reader for it on that code path. Delivering persisted liveness comparisons for Raiffeisen is feature work (either migrate the flow to liveness-check-v2, or add persistence to the v1 handler), not a config or script fix.

4. Three ways a face match happens with NO row

MechanismWhereWhat it does instead
Raiffeisen liveness → vuer_cvcustomization/listeners/self-service-v2.js, hook self-service:v2:liveness-check-v1:initLoads the customer-portrait FaceRecognition and passes faceRecognition.faceDetection.descriptor into a CV LivenessTask. The match is computed inside vuer_cv and comes back as a CV recognition result. vuer_oss never sees a distance to persist.
Raiffeisen portrait checkmyra-...flow.handler.js::_isSameFace:290-361, called from onTaskRecognitionSubmit:149Computes the cosine distance in memory, gets a verdict from getFaceComparisonResult(), returns a details object used for flow control (actions.recognitionValid). Never calls compareFaceDetections.
Any guard failing on paths 1–3see §2Silent skip; no row, not even status:'failed'.

liveness-reference-face FaceRecognitions are not comparisons

FaceRecognition rows with imageCategory: 'liveness-reference-face' do exist for Raiffeisen. They are reference descriptors fed to CV, not evidence that a faceComparisons row was written. Do not infer persistence from their presence.

5. imageCategory is not a setting — and it lies about which side is which

faceRecognitions.imageCategory is a per-recognition column copied verbatim from the task’s screenshotCategory. It is not configurable anywhere in the OSS UI (a customer asked exactly this on SLARAFIPI-84).

Write sites:

  • server/service/RecognitionService.js:235 and :363 — imageCategory: ids.screenshotCategory
  • server/service/SelfServiceV2Service.js:1405 — imageCategory: task.options.screenshotCategory

FaceRecognitionService.js:194 is a READ, not a write

:194 (imageCategory: screenshotCategory) sits in the where clause of findBestScoreRecognitionForRoom — it is a query filter. It is good evidence that the codebase treats the two names as interchangeable, but cite RecognitionService.js:235,363 / SelfServiceV2Service.js:1405 for the actual assignment.

The myra mislabelling trap

In the myra proto both sides of the eMRTD↔selfie comparison carry the same category:

  • emrtd task (:80-96) → screenshotCategory: 'customer-portrait'
  • customer-portrait task (:100-121) → screenshotCategory: 'customer-portrait', recognitionOptions.compareFaceWith: ['emrtd']

⇒ Every resulting row is stored as customer-portrait ↔ customer-portrait. The comparison is correct; the labelling is not. Since faceComparisons has no step/task column, imageCategory on the joined recognitions is the only discriminator — and here it discriminates nothing.

Do not build reports that key on imageCategory

For myra it cannot tell the chip photo from the selfie. Join through recognitionFromId/recognitionToId → sourceAttachmentId → Task.data.attachmentId to recover which task each side came from.

6. Fail-open in the myra portrait check (partially UNVERIFIED)

myra-...flow.handler.js::_isSameFace:290-361 selects the largest distance across all compareFaceWith targets:

const compareTo = { score: 0, id: null }                          // :319-322
for (const compareWithTask of compareWithTasks) {
  const compareAttachmentId = compareWithTask?.data?.attachmentId
  if (!compareAttachmentId) { continue }
  const recognitionTo = await ...FaceRecognition.findOne({ where: { status: 'success', sourceAttachmentId: compareAttachmentId } })
  if (recognitionTo?.getFaceCount()) {                            // :337
    const eucledianDistance = ...calculateCosineDistance(recognitionFrom, recognitionTo.convertDescriptors())
    if (eucledianDistance > compareTo.score) { compareTo.score = eucledianDistance; ... }   // :339-343
  }
}
const faceComparisonResult = await ...getFaceComparisonResult({ euclideanDistance: compareTo.score })   // :347

Taking the max is the conservative choice when several targets exist — the worst match wins. But the identity element is wrong:

Max-reduction over an empty set yields the BEST possible score

compareTo.score starts at 0 and is only ever written inside if (recognitionTo?.getFaceCount()). If no compareFaceWith task yields an attachment with a successful recognition, the score stays 0 → getFaceComparisonResult({euclideanDistance: 0}) → 0 <= perfect → CHECK_SUCCESS. Missing comparison data reads as a perfect match. In a distance metric the neutral element for a max-reduction is Infinity (or an explicit “no target” failure), never 0. The failure is also silent: _logCVError (:357, defined :397) is gated behind config.get('raiffeisen.debug.cv', false) — off by default. This holds for any numeric perfect >= 0, so it does not depend on the deployment’s threshold value.

Contrast — the no-face branch fails CLOSED. At :303-305, if no faceEncoding is found, _isSameFace returns the boolean false. The caller then evaluates faceCompareResult.success → undefined → isFaceMissmatch = true → recognitionValid = false. So a missing source face is rejected correctly; only a missing target fails open.

Verified vs unverified

ClaimStatus
compareTo.score initialised to 0, only updated inside the getFaceCount() guardVERIFIED (:319-343)
getFaceComparisonResult({euclideanDistance: 0}) returns CHECK_SUCCESSVERIFIED — SelfServiceCheckerService.js:132-152, 0 <= perfect with <= operators
_logCVError off by defaultVERIFIED — config.get('raiffeisen.debug.cv', false)
myra emrtd step is required: true (mitigating factor)VERIFIED — proto :86
Can a failed/skipped NFC read still reach _isSameFace with no target?UNVERIFIED — trace this before escalating externally. Same family as the eMRTD fail-open discussed on SLARAFIPI-53.
Raiffeisen production perfect threshold = 0.55UNVERIFIED as a repo fact. Code default is 0.5 (SelfServiceCheckerService.js:34-38). Raiffeisen overrides perfect from deployment config raiffeisen.customerPortrait.threshold via the self-service:v2:updated-settings hook (customization/listeners/self-service-v2.js:114-127) — the value is not in the repo. 0.55 appears only in a test fixture (RaiffeisenFaceComparisonExportService.test.js:69) and in the customer’s own observation.

_isSameFace never gets per-room thresholds

It calls getFaceComparisonResult(faceComparison) with no flow argument, so flow defaults to {} and flow.selfServiceRoomId is undefined. getSettingsFromSelfServiceRoomConfigState then calls getActivityLog(null, 'selfService:v2:config:state', undefined); the helper’s default parameter (selfServiceRoomId = null, server/db/helpers.js:44) turns that into where: {roomId: null, selfServiceRoomId: null} — which matches no room’s config state, so it always falls back to the global Setting. The in-memory myra check and the per-room threshold machinery described in face-comparison-data-verdict-threshold-model are therefore not using the same numbers. (Fallback path VERIFIED; whether the global Setting happens to equal raiffeisen.customerPortrait.threshold is UNVERIFIED.)

Graph View

Backlinks