Levandor

BE-2272 Flatten Datadog Log Fields to Root

3 min read

Flatten Datadog log JSON: hoist span fields from the span.* namespace to the document root so DD facets/dashboards can reference them as @flow.exec_id instead of @span.flow.exec_id.

Goal

Eliminate the span.* JSON namespace in mando-lib dd_formatter output; emit flow.exec_id, flow.context, step.name, step.connection at the document root alongside error.*, http.*, etc.

Why

  • DD facets/dashboards work best when correlated fields live at a single, predictable path. Splitting span context under @span.* while events live at root forces dual-path queries.
  • Symmetric layout at root (error, http, flow, step) matches how the rest of the JSON envelope is structured and how downstream filters were built post-BE-1842 Datadog Observability.
  • Keeps wire format aligned with the typed error.kind work from MR !481 so existing dashboards only need column renames, not query rewrites.

Implementation

Single-file refactor in mando-lib/src/app/dd_formatter.rs (+295 / -16):

  • Dropped serializer.serialize_entry("span", &span_fields) — the wrapper object is gone.
  • Added MapVisitor implementing tracing::field::Visit to collect event fields into a serde_json::Map<String, Value> (previously emitted directly into the serializer mid-stream).
  • Merge order: span fields first (from collect_span_fields walking ctx.event_scope() root-to-leaf), then event fields, so event-level overrides win — explicit, documented precedence.
  • Removed the magic name injection in collect_span_fields (was the outermost span’s name; not used in any DD dashboard and conflicted with flattened root layout).

Diff stats: +295 / -16, scope strictly contained to the formatter — no callsite or span-definition changes required (continuation of BE-1842 Datadog Observability field plumbing already in place).

Tests

11 unit tests added with a capture harness:

  • Pattern: tracing::subscriber::with_default(subscriber, || { ... }) with a custom MakeWriter that pushes lines into a Mutex<Vec<u8>>; tests then serde_json::from_slice the captured bytes and assert on the Value tree.
  • Covered cases: root-level presence of each flattened field, event-overrides-span precedence, missing-span fallback, error fields untouched, nested span-scope walk, non-string field types, empty-event passthrough, name-injection regression guard, and ordering invariants.
  • Harness is reusable for future dd_formatter changes.

Workspace: 262 tests pass, 0 regressions.

Out of Scope / Follow-ups

  • DD dashboard column migration: @span.flow.exec_id → @flow.exec_id, @span.step.name → @step.name, etc. (ops/dashboards repo, not mando).
  • Naming unification: execution.id (used in some HTTP/error paths) vs flow.exec_id (used in span context) — pick one and migrate.
  • Dead-code cleanup: unused ErrorCode derive arms in mando-lib-macro left over from MR !481 redesign.

State

  • Branch feature/BE-2272 (renamed from prior bugfix/BE-2023 working branch — same arc, different ticket).
  • Local-only, uncommitted at time of writing.
  • BE-1842 Datadog Observability — prerequisite scope walking + field plumbing this builds on.
  • mando-lib — crate containing dd_formatter.
  • MR !481 — error handling redesign (BE-2023) that established typed error.kind at root; this aligns span fields to the same convention.
  • LOG — daily activity log.

References

  • Plan in repo: docs/superpowers/plans/2026-05-05-flatten-log-fields-to-root.md
  • Source: mando-lib/src/app/dd_formatter.rs

Graph View