Levandor

BESS AM (BE-2262) — mando-bess-am

7 min read

BESS AM (BE-2262) — mando-bess-am

BESS AM = BESS Asset Management: a headless Kinesis stream processor that ingests per-second WAGO battery-telemetry events, aggregates them to 1-minute windows, and forwards the result into the mando-bess ingestion stream. A POC tracked under ticket BE-2262, authored almost exclusively by Gergely Vászon (ext, gergely.vaszon@alpiq.com).

Status — in flight, NOT merged

As of 2026-06-24 nothing BESS AM has merged to develop. The core MR !512 is OPEN with detailed_merge_status = requested_changes (blocked on review). All paths, crate names, schema names, and stream names below describe the proposed state on the feature branches, not what is live on develop. Treat as forward-looking reference until !512 merges.

For Agents

BESS AM is a newer workstream distinct from the existing flow engine. The flow-engine list (Trading / Manual Schedule / Auction / Intraday / Data Update / AFRR — see Flow Implementations) does NOT include AM. AM introduces a new Event-typed data-point class (real-time telemetry) that is separate from the time-series flows. If asked “where does real-time SoC/SoH come from”, the answer is this pipeline, not a flow.

What it does

WAGO boxes mounted on the physical battery (the FI / Valkeakoski / Beskar asset) emit per-second telemetry — real-time State of Charge (SoC) and State of Health (SoH) — over AWS IoT Core. mando-bess-am is the headless service that consumes the raw stream, persists it, down-samples it to 1-minute aggregates, and hands it off to mando-bess.

Pipeline

WAGO Box
  → AWS IoT Core
    → Kinesis (bess-am-events)        # raw per-second telemetry
      → mando-bess-am                 # this service: persist raw + 1-min aggregate
        → Kinesis (bess-os-events)    # 1-min aggregated results
          → mando-bess                # ingests into the platform

Processing steps (inside mando-bess-am)

  1. Consume the raw bess-am-events Kinesis stream.
  2. Persist raw per-second events to the bess_am Postgres schema (event table).
  3. Aggregate to 1-minute windows, computing: Mean / Max / Min / Last / StdDev / Count / Sum.
  4. Close windows via a windowing + grace-period closure mechanism (a window is finalized only after its grace period elapses, to tolerate late/out-of-order per-second events).
  5. Forward the aggregated results to the bess-os-events Kinesis stream that mando-bess ingests.

Code & infrastructure fingerprints

Where to look in the mando repo

These are the files/modules introduced or touched by the BE-2262 work. Use them to locate the implementation when the branch is checked out.

New crate — mando-bess-am/ (mando_bess_am)

ModuleResponsibility
pipelineStream wiring (consume → aggregate → forward)
service/aggregation1-minute window aggregation (Mean/Max/Min/Last/StdDev/Count/Sum)
service/forwardPush aggregated results to bess-os-events
service/window_closureWindowing + grace-period closure logic
startupService bootstrap
metricsService metrics

Crate split not yet its own MR

Branch feature/BE-2262-bess-am-poc-build-test + commit 5a739baa (“feat: separate out bess-am”) splits AM into its own mando-bess-am crate/image, but has NOT been raised as its own MR — it is a downstream branch of !512.

Supporting changes elsewhere in mando

PathPurpose
mando-lib/src/adapter/kinesis/Kinesis adapter: consumer / producer / pipeline / checkpoint / config
mando-core/src/model/event.rsNew Event-typed data-point model (real-time telemetry)
mando-core/src/model/wago.rsWAGO event model
mando-bess/src/kinesis.rsmando-bess side of the Kinesis wiring (ingest of bess-os-events)
mando-bess/config/parts/battery_online.yamlDefines the Event-typed “Online” datapoints (see below)
container.bess-am.DockerfileContainer image build for the AM service
config/bess-am.yamlAM service config

battery_online.yaml — Event-typed datapoints

Defines Event-typed datapoints under the path Asset/FI/Valkeakoski/Beskar/Battery/.../Online, e.g.:

  • BATTERY_SOC_ONLINE
  • BATTERY_SOH_ONLINE

These are keyed to Kinesis external IDs [bess-am-events, OnlineSOC/OnlineSOH]. This is the first place the new Event data-point class (vs. the existing TimeSeriesDouble / TimeSeriesDoubleMatrix / StaticData enumerated in DataPointType) shows up in config.

Database & infra

ItemValue
Postgres schemabess_am
Migrationscreate_bess_am_schema, create_event_table, create_kinesis_checkpoint_table, add_event_latest_historization
Env prefixBESS_AM_*
Kinesis streamsbess-am-events (raw in), bess-os-events (aggregated out)
CI job”Publish BESS-AM Docker Dev” (ECR tag prefix bess-am-<version>)

Local-dev artifacts

  • doc/kinesis-local-setup.md — local Kinesis setup guide.
  • MiniStack (ministackorg/ministack) — used to stand up local AWS-like services for development.
TicketRelationship
BE-2262The BESS AM POC itself (this note). Core MR is !512.
BE-2341Generic Kinesis producer in mando-lib — the reusable producer half that AM forwarding depends on. No standalone MR; folded into the BE-2262 MR.
BE-2132Interval / Event datapoint groundwork (basic interval functionality) that AM builds on. MRs !524 (current) and !500 (superseded draft).

Merge-request landscape (GitLab, as of 2026-06-24)

Repo

sales-and-origination/flexible-assets/bess/poc/mando on GitLab. Main branch is develop.

MR !512 — THE core BESS AM MR

MR !524 — interval/Event groundwork (AM-adjacent)

  • Title: feat: add basic interval functionality
  • Branch: feature/BE-2132-interval-poc-2 → develop
  • State: OPEN, not_approved, no reviewers. Closes BE-2132.

MR !500 — superseded draft

  • Title: Draft: feat: add basic interval functionality
  • Branch: feature/BE-2132-interval-poc → develop
  • State: OPEN / DRAFT, stale (last update 2026-05-06). Superseded by !524.

Unreleased crate-split branch

  • Branch feature/BE-2262-bess-am-poc-build-test + commit 5a739baa (“feat: separate out bess-am”) splits AM into its own mando-bess-am crate/image. Not raised as its own MR — downstream of !512.

Update 2026-06-24

Two things verified directly from the mando repo today.

develop merged into the MR branch (!512)

origin/develop was merged into feature/BE-2262-bess-am-poc (the !512 branch), which had drifted 46 commits behind. Result pushed as a fast-forward f9a1f81d..80a17e78 (merge commit 80a17e78).

Conflicts resolved during the merge:

FileResolution
mando-lib/src/repo_postgres.rsKept both new DataSchemaPostgres trait default methods — the branch’s has_override() and develop’s latest_pk_columns().
py-mando/bess-csv/defaults.csvKept the 3 BESS AM Event rows; took develop’s newer EUR/MW/h FINGRID units.
generated domain.rs filesRegenerated via build.rs; converged CRLF → LF to match develop’s line-ending convention (the branch had committed them as CRLF).

Post-merge state: compiles clean (cargo check, 0 errors), 428 lib unit tests pass.

FIXED (2026-06-25) — codegen python_field casing for multi-segment Named variants

Was: mando-codegen’s expand_variants mis-cased the python_field for multi-segment Named variants — e.g. a Min1/Mean variant emitted market.da.price.min1.Mean instead of the all-lowercase market.da.price.min1.mean (it lowercased the first path segment but not the value segment). This affected the BESS AM 1-minute aggregate datapoint names (Min1/Mean, Min1/Max, Min1/StdDev, …) — see Processing steps (inside mando-bess-am).

  • Fix: mando-codegen/src/util.rs:41 now lowercases the value segment — format!("{pf}.{name_lower}.{}", v.to_lowercase()).

  • Commit: cc4720b2 “fix: lowercase value segment of variant python_field” on branch feature/BE-2262-bess-am-poc, pushed to !512.

  • Verified: cargo test -p mando_codegen now 51/51 passing (was 50/1). The previously-failing util::tests::expand_variants::named_variant_generates_datapoint test now passes.

  • Origin (for the record): the bug came from the branch’s own feat: separate out bess-am commit ([[#unreleased-crate-split-branch|5a739baa]]), not the develop merge — mando-codegen was byte-identical before and after that merge.

Residual (unrelated to this fix) pre-existing rustfmt drifts remain in mando-codegen/src/util.rs at lines ~50 and ~390 (the author's code), left untouched. Flagged by the project's nightly rustfmt; unrelated to this bug or the develop merge.

Two

Update 2026-07-14: the POC’s dev migrations now bite every develop-based deploy

The May feat deploy of this branch to dev left two applied migrations in dev’s refinery_schema_history: U202605131400__create_event_table and U202605141900__create_kinesis_checkpoint_table. The files exist only on the unmerged BE-2262 branches, so any image built from develop lacks them and crash-loops at boot with MigrationError::PatchApplicationFailed (refinery validates applied migrations against the image’s filesystem). Hit and diagnosed during the Dev Deploy 2026-07-14 (error telemetry); worked around there by restoring both files byte-exact (single canonical git blob f45b6dd4 across all source commits, so refinery’s checksums pass).

Resolved 2026-07-24 (no action needed)

The two migration files landed on origin/develop with the bess-am POC itself: commit 3ebcc897 “feat: add bess-am POC” (merge 6fae64a5, 2026-07-23). develop’s mando-bess/database/ set is now byte-identical to the throwaway dev-deploy commit 6bc418cd (git ls-tree diff, verified 2026-07-24), so develop-based dev deploys no longer need the restored files. Details: Dev Deploy 2026-07-14 (error telemetry); repo docs/superpowers/HANDOVER-error-handling-2026-07-24.md item 4.

Graph View

Backlinks