mando-scrt

Rust-based Python runtime for the Mando workspace. Replaces gunicorn/uvicorn with a single Rust binary that hosts Python algorithms via PyO3, providing memory safety, zero-copy Arrow data exchange, and virtual module injection.

Status

Initial reference implementation (“one nighter”). Core virtual module system and Arrow FFI streaming work. Next: algo integration, watch mode, cross-module call capture.

Purpose

Replace pyo3-polars (memory safety issues) and eliminate transitive dependency problems by running Python algorithms inside a Rust-managed runtime. Goals:

  1. Centralized runtime — replace gunicorn/uvicorn with a single Rust binary
  2. Memory safety — eliminate pyo3-polars issues, monitor memory via memray
  3. Virtual modules — inject mock/redirect/algo modules into Python’s import system
  4. Arrow FFI — zero-copy data exchange between Rust and Python via __arrow_c_stream__
  5. Cross-module call capture — virtual boundaries for testing/debugging (planned)

Structure

mando-scrt/
├── Cargo.toml
├── README.md                    # Design doc with roadmap
├── src/
│   ├── lib.rs                   # Exports: MandoPythonIsolate, MandoVirtualModule, safe_ffi_stream
│   ├── main.rs                  # Test binary: 9 integration tests
│   ├── ctx.rs                   # MandoPythonIsolate — Python import hook registry
│   ├── virtmodule.rs            # MandoVirtualModule trait definition
│   ├── safe_ffi_stream.rs       # Arrow FFI: produce_to_python / consume_from_python
│   └── virtualized/
│       ├── mod.rs
│       ├── tracked_stub.rs      # Mock module with call tracking + JSON returns
│       ├── redirected.rs        # Redirect imports to external .py files
│       └── algo.rs              # VirtualizedAlgoConfig (WIP — Arrow streaming + gRPC)

Key Abstractions

MandoVirtualModule (trait)

pub trait MandoVirtualModule: Send + Sync + 'static {
    type Config;
    fn get_name(&self) -> String;
    fn init(c: Self::Config) -> Self;
    fn module_init(&self, gil: Python, delegate: &Bound<PyModule>) -> PyResult<()>;
}

Implementations:

  • MandoTrackedStubVirtualModule — creates mock Python modules with stub methods (returns None, JSON, or throws). Tracks call counts via AtomicUsize.
  • MandoRedirectorVirtualModule — redirects Python import foo to load from an arbitrary .py file on disk.

MandoPythonIsolate

Central registry that hooks into Python’s sys.meta_path:

let isolate = MandoPythonIsolate::new();
isolate.register_virt::<MandoTrackedStubVirtualModule>(config);
isolate.install()?;  // injects into sys.meta_path

Arrow FFI Streaming

Zero-copy Rust ←> Python data exchange via __arrow_c_stream__ ABI:

// Rust -> Python
produce_to_python(py, schema, batches) -> PyObject
 
// Python -> Rust
consume_from_python(py_reader) -> (SchemaRef, Vec<RecordBatch>)

Compatible with arro3, pyarrow, polars — any Arrow C Stream consumer.

Test Suite (main.rs)

TestDescription
1Import and call tracked stubs
2Stub returning JSON data
3User script using virtual modules
4Redirected module from .py file
5Rust→Python Arrow stream (produce)
6Python→Rust Arrow stream (consume)
7Rust→Python→Rust roundtrip with transform
8JSON mock algo service (Axum HTTP, 8x8760 rows)
9Python calling JSON algo endpoint

Optional: MANDO_MEMRAY=1 enables memray profiling with flamegraph generation.

Dependencies

CratePurpose
pyo3 0.25.1Python FFI (auto-initialize, not extension-module)
pyo3-arrow 0.11.0Arrow C Stream bridge
arrow / arrow-array 56.2.0Columnar data
axumHTTP mock algo service
tokioAsync runtime
serde / serde_jsonJSON serialization
reqwestHTTP client for tests

PyO3 Mode

Uses auto-initialize feature (not extension-module like py-mando). This means mando-scrt is a standalone binary that embeds Python, not a Python extension.

Feature Flags

FeaturePurpose
pythonEnables PyO3 + pyo3-arrow (required for runtime)
appReserved (unused)
codegenReserved (unused)
workflowReserved (unused)

Running

# Setup Python venv with dependencies
python3 -m venv /tmp/mando-scrt-venv
/tmp/mando-scrt-venv/bin/pip install arro3-core numpy memray
 
# Run tests
cargo run -p mando_scrt --features python
 
# With memory profiling
MANDO_MEMRAY=1 cargo run -p mando_scrt --features python

Roadmap (from README)

  • Rewrite each Python algo to new virtual module API
  • Load balancer for safe testing without wrecking env
  • Intuitive updates without downtime/SSH/volumes
  • Watch mode on dev and local
  • Cross-module call capture with virtual boundaries (HMR-style)
  • Mirror Python dependency resolver via filter query
  • Memory usage monitoring
  • Core dumps and advanced debugging
  • Automatic stub generation
  • Teach non-devs to use it (integrate into local CLI)
  • Route /[path]/... domains to Python via config/OpenAPI stubs
  • Arrow: PyCapsule high-level API, gRPC algo calls
  • Mando — parent workspace
  • py-mando — existing PyO3 extension module (complementary, not replacement)
  • mando-lib — core library
  • Alpiq BESS — project overview