A central relief order with atomic multi-office execution

The scenario

A central relief office in a Southern Ming-style setting issues a grain relief order. The treasury releases 600 units of grain and a county grain office records the local execution. The central office may read both offices’ records but may only write its own, and it needs to confirm that both offices carried out the same order with the expected result.

The question the example answers: how do several plugins, each owning its own state, execute one order together so that either every part commits or the whole step rolls back?

The office names are labels in the example code. Canwu has no built-in court, ministry, or county types; a host application models its own institutions.

What this example shows

• Three simulation plugins (SimulationPlugin), each owning one typed domain record (DomainRecordType).
• Boundary systems (BoundarySystemContract) running in settlement phases 7, 10, and 12 of one boundary.
• Plugin-to-plugin work sent through canonical ingress with BoundaryDirective::SchedulePluginIngress.
• Atomic commit: a failing system rolls back every write in its boundary.
• Snapshot restore and exact replay with the same plugins.

Run it

cargo run -p canwu-api --example governance_transition

Output:

relief_order=relief-order-1646 treasury_grain=600 county_grain=600 exact_replay=ok

The line prints only after every assertion in main passes: two ingress items generated by the first boundary, two records committed by the second, and a restored snapshot and a replayed run that both equal the original.

How the order flows

Sequence of the central relief order across two boundaries. View diagram source.

Rendering diagram…

View diagram source

sequenceDiagram
  participant Host as Host application
  participant Central as case-relief-central
  participant Treasury as case-relief-treasury
  participant County as case-relief-county
  Host->>Central: issue-relief-order
  Note over Central: boundary 1, phase 7: publish-order writes the ReliefOrder manifest
  Central->>Treasury: execute-relief-order
  Central->>County: execute-relief-order
  Note over Treasury,County: boundary 2, phase 10: each office creates its own ReliefAction
  Note over Central: boundary 2, phase 12: audit-order checks both records
  Note over Central,County: all three pass: commit. Any error: the boundary rolls back.

Walkthrough

All code is in governance_transition.rs and uses only canwu-api.

1. Three plugins, three record types

Plugin	Record it owns	Boundary systems
case-relief-central	ReliefOrder manifest (case.relief / order)	publish-order, audit-order
case-relief-treasury	Treasury ReliefAction (case.relief.treasury / action)	prepare-treasury
case-relief-county	County ReliefAction (case.relief.county / action)	prepare-county

Each plugin registers its record schema in register with registrar.register_record_schema. The manifest lists, for each office, the system that must act, the record version it must produce, a disposition, and the hash of the expected payload:

struct ReliefOrder {
    order_id: String,
    issued_by: String,
    treasury_system: String,
    treasury_version: u64,
    treasury_disposition: String,
    treasury_hash: String,
    county_system: String,
    county_version: u64,
    county_disposition: String,
    county_hash: String,
}

action_hash computes each hash with canonical_hash over the expected ReliefAction payload (status: "committed", grain_units: 600).

2. Issue the order

main creates the simulation with all three plugins, queues the issue packet, and settles the first boundary:

canwu.enqueue_plugin_ingress(canwu_api::PluginIngressRequest::new(
    CENTRAL_PLUGIN,
    ISSUE_INGRESS,
    SimTime::EPOCH,
    json!({"order_id": ORDER_ID}),
))?;

let first = canwu.settle_boundary(BoundaryRequest::at(SimTime::EPOCH))?;
assert_eq!(first.generated_ingress.len(), 2);

3. Publish the manifest (boundary 1, phase 7)

publish_order runs in BoundaryPhase::DomainDeltaProposal. It creates the manifest with DomainRecordMutation::Create and schedules one execute-relief-order packet for each office:

BoundaryDirective::SchedulePluginIngress {
    target_plugin: TREASURY_PLUGIN.to_owned(),
    after: SimDuration::ZERO,
    packet_type: EXECUTE_INGRESS.to_owned(),
    priority: 0,
    payload: json!({"order_id": ORDER_ID}),
    affected: Vec::new(),
},

The system contract lists both targets in plugin_ingress_targets. A packet scheduled with zero delay is admitted at the next boundary, so after boundary 1 the manifest exists and neither office record does.

4. Each office writes its own record (boundary 2, phase 10)

prepare_treasury and prepare_county both call prepare_owner, which runs in BoundaryPhase::HistoricalCandidateEvaluation. It reads only the packet addressed to its own plugin (owned_order_ingress), checks the manifest with validate_order, including the hash of the payload it is about to write, and creates its record. Its contract declares StateVisibility::SameBoundary, so the phase-12 audit in the same boundary can read the new record.

5. The center audits (boundary 2, phase 12)

audit_order runs in BoundaryPhase::StrategicAggregation. Its contract declares reads and no writes. For each office it loads the record and returns an error if the record is missing, is not at version 1, is inactive, has the wrong owner, or hashes differently from the manifest:

if record.version != 1
    || !record.is_active()
    || record.owner != expected_plugin
    || actual_hash != expected_hash
    || record.payload != json!({"status": "committed", "grain_units": 600})
{
    return Err(CanwuError::new(
        ErrorCode::InvalidBoundary,
        format!("central audit found an invalid {label} disposition"),
    ));
}

An error from any system fails the boundary, and both office records roll back with it. In boundary 1 the audit sees the issue packet and returns early, because the offices have not acted yet.

6. Restore and replay

let snapshot = canwu.snapshot_json()?;
let restored = Canwu::from_snapshot_json_with_plugins(&snapshot, &plugins)?;
let replayed = Canwu::replay_from_journal(&plugins, &canwu.replay_journal())?;
assert_eq!(restored.snapshot(), canwu.snapshot());
assert_eq!(replayed.snapshot(), canwu.snapshot());

Restore and replay both take the same plugin list, because the snapshot records which plugin versions produced the state.

What to notice

• Each office writes only the record its plugin owns. The central office coordinates through the manifest and packets, and reads the results.
• Work scheduled with zero delay lands in the next boundary. The order takes two boundaries: publish, then execute and audit.
• The audit makes the step all-or-nothing: if one office fails or skips its part, neither record commits.

Try changing

• A wrong amount. In prepare_county, pass &action_payload(500). The hash no longer matches county_hash, validate_order returns an error, and the second settle_boundary call fails, so main exits with an error.
• A silent office. Make prepare_county return Ok(BoundaryProposal::default()). The audit fails with central audit is missing the county relief disposition, and the treasury record rolls back too.
• Your own institutions. Replace ReliefOrder with your own office or decree schema, and connect the issue packet to an actor’s authority and actor knowledge.

Beyond the example: transition manifests

The example’s hand-written manifest and audit are an ordinary application pattern. Canwu also provides the same guarantee as a built-in transition manifest (TransitionManifest):

• The coordinating plugin registers the manifest with BoundaryDirective::RegisterTransitionManifest from a phase-7, phase-10, or phase-12 system. Here lineage_id would be the order ID, the participants the treasury and county plugins, and each participant lists the record versions it expects before (expected_pre) and after (expected_post) in place of a payload hash.
• In phase 10 of the ready boundary, each participant stages its own writes with BoundaryDirective::StageTransitionWrite.
• In phase 11, the simulation core audits each ready manifest before committing it. If every participant staged and the versions match, it commits. If only some staged, or a version differs, the whole boundary rolls back. If none staged, the manifest expires and the coordinator can register a new attempt.
• Phase-12 systems read the result as a TransitionAuditRecord.

A manifest that no participant staged expires and the boundary still commits, so a missing office is caught only when another participant staged. Give the manifest at least two participants.

Source

Open the runnable example

Read the conformance tests

Read the repository case note