Last updated: 2026-09-19

U
Undergraduate level

Schemas and Scenarios: A Working Z/BDD Hybrid

Formal Methods covers a Z schema: a universally-quantified invariant, checkable independently of any one concrete case. BDD as Specification covers the opposite kind of claim: a Given/When/Then scenario is one concrete, witnessed example, useful precisely because it’s concrete, not despite it. This page connects them for real: a Z-style schema states a general invariant, and a set of Gherkin-style scenarios serve as its concrete witnesses, checked against each other rather than written and trusted independently. Every mechanism described below is built, tested, and runnable — the live examples further down run in this page, not on a build machine somewhere else.

Prior Art

This isn’t unclaimed territory, and it would be dishonest to write about it as though it were. Bowen Liu’s research at the University of Waikato converts BDD-style behavioural specifications into first-order-logic predicates and checks them for consistency against formal models1; the follow-on PhD work, supervised by Judy Bowen, Jessica Turner, and Steve Reeves, does the same specifically against Z specifications, including checking that the consistency survives a Z specification’s own refinement steps — the harder, more general version of the same problem, aimed at safety-critical interactive systems2.

What follows differs in scope, not in the underlying idea. Liu and Bowen’s work treats the behavioural specification and the Z specification as two separate documents, reconciled by an external consistency checker — a defensible choice for safety-critical systems where a formal model and a requirements document may genuinely come from different processes. What’s built here instead folds the same idea into one language and one executable artifact: PatLang schemas and scenarios live in the same source, checked by the same tool, in one pass. The schema/scenario pairing itself is Liu and Bowen’s.

Two Kinds of Claim, Briefly

A Z schema states what must hold for every input satisfying its precondition — a BorrowBook schema states that any title already in books and not already in borrowedBy leads to a specific update, for every such title and member, not just one. A BDD scenario states one specific, concrete case; BDD as Specification is explicit that this is deliberate — a scenario is training data and a pass/fail oracle precisely because it’s concrete. Neither substitutes for the other. That’s exactly why keeping them consistent with each other is worth doing rather than assuming it happens for free.

From Sketch to Implementation: What Changed

An earlier version of this page sketched a Schema:/Operation: block extension to PatLang’s Feature-file syntax, with require/ensure clauses written as inline text. Building it for real, rather than just imagining it, forced two corrections — both found by reading the actual PatLang source rather than trusting the sketch’s own assumptions:

  • require/ensure/assert are fatal on violation — confirmed directly in rust-runtime/src/ir/hosts.rs: a failed contract check aborts the running program. That rules out the sketch’s inline text clauses outright; a schema check has to report a diagnosis and keep running, not kill the process on the first violation. The real implementation represents a schema’s invariant, precondition, and postcondition as ordinary, separately-compiled PatLang functions returning bool, invoked by name through PatLang’s existing apply() primitive — the same mechanism self_hosting/lib/primitive_registry.patlang already uses for named contract predicates elsewhere in the codebase, not new machinery.
  • apply() has no argument-spread form. A checking function written once, generic over any schema’s own number of state variables, can’t pass one positional argument per variable — it has no way to know in advance how many there’ll be. State and inputs are therefore always bundled as a single list argument: invariant_fn(state_list), require_fn(state_list, input_list), ensure_fn(before_list, after_list, input_list).

Set and Map types didn’t exist either, as the sketch already suspected — self_hosting/lib/pset.patlang and pmap.patlang are new, small, association-list-based libraries built to fill exactly that gap, following the same list-plus-linear-scan idiom already used elsewhere in the codebase rather than inventing a new primitive.

Try It Yourself: The Mechanism, Live

Both examples below run entirely in this page — the same self-hosted PatLang compiler and interpreter compiled to WebAssembly that powers every other live demo on this site, with the new pset/pmap/schema_bdd libraries inlined directly into the source (there’s no real filesystem in the browser sandbox, so include isn’t available here — everything the example needs is in the one block of text below). Pick an example from the dropdown, edit it if you like, and press Run.

(not run yet)

The Negative Case: Catching a Scenario Before It Runs

Run the first example (one operation, one violation) and look at its second scenario: Given already states that “Dune” is borrowed by S. Okonkwo, then When has A. Diallo try to borrow it too. Checked against BorrowBook’s schema rather than run against any implementation, this fails before any code executes at all — its own Given already violates the operation’s precondition. That’s the mechanism’s whole point: schema_check returns a tagged diagnosis, ["precondition_violated", [op, state_before, inputs]], and schema_format_diagnosis turns it into a real, specific question rather than a bare pass/fail — mirroring the same tagged-diagnosis-plus-question pattern BDD as Specification’s induction engine already uses for a scenario a rule can’t satisfy:

> schema_check("LibraryLoans", "BorrowBook")
["precondition_violated", ["BorrowBook", [...], ["Dune", "A. Diallo"]]]

> schema_format_diagnosis("LibraryLoans", "BorrowBook", diagnosis)
["Scenario claims BorrowBook can run from this Given, but BorrowBook's own
 precondition returned false for these inputs. Is the Given wrong, or is
 BorrowBook's precondition too strict -- or is this scenario meant to test a
 rejection path, which needs its own operation schema rather than
 BorrowBook's?"]

That question is the actual payoff, not a rhetorical flourish. It distinguishes two mistakes that look identical from the outside — a scenario reporting an outcome the current implementation doesn’t produce — but need entirely different fixes: an operation modelled with the wrong precondition, versus a scenario silently testing a code path no schema was ever written to cover. A plain Given/When/Then runner can’t tell these apart; a schema, checked independently of any implementation, can.

A Fuller Worked Example: Three Operations and a Real Invariant

The second dropdown option scales the same idea up: three pieces of interacting state (books, borrowedBy, and a per-member loanCount), two operations (BorrowBook, ReturnBook), and a genuine cross-cutting invariant — no member may hold more than three books at once, a rule that isn’t derivable from the book-tracking half of the state at all. Run it and its seven scenarios exercise every diagnosis schema_check can produce: an available title accepted, a title someone else has out rejected, a member at the loan limit rejected, a corrupted loan record caught by the invariant before the operation is even considered, a scenario whose own claimed effect doesn’t match what its postcondition demands, and a clean return accepted and a spurious one rejected. Full source: self_hosting/examples/library_loans_schema_demo.patlang.

Beyond Hand-Written Scenarios: Synthesis Integration

A schema doesn’t only check scenarios a person wrote by hand. The same schema_check_values core also plugs into two of PatLang’s existing program-synthesis mechanisms as a stronger correctness oracle — catching an artifact that’s internally consistent with its own training data or its own plan, but still wrong by an independent standard. Both examples below need real subprocess execution or a native-only host function, so they’re shown as source plus a real captured run rather than live in this page — see the callouts on each for exactly why.

Inductive Synthesis: A Logically Valid Rule, Still Policy-Violating

BDD as Specification covers PatLang’s inductive-logic-programming engine deriving grandparent(X) :- parent(X,Y), parent(Y,Z) from training facts and examples. That derivation is provably correct with respect to its own training data — but the engine has no way to know about a constraint from a completely different part of a system, e.g. a policy that names specific people. self_hosting/lib/schema_synthesis_bridge.patlang asserts an already-induced rule into PatLang’s real logic engine, enumerates every value it proves true, and checks each one against a schema. Not runnable in this page’s sandbox: the induction engine’s own diagnosis step (synth5_inducesynth2_diagnose_hypothesis) verifies its candidate by spawning a real subprocess — confirmed directly by running it through the exact WASM binary above, which fails with exec_capture: ... operation not supported on this platform. That’s a real, structural limit of any browser sandbox, not a bug to fix.

  ok: induction succeeds on the training examples
  ok: induced rule is the expected 2-hop parent/parent chain
  ok: the induced rule is caught violating an unrelated schema policy
  ok: the flagged candidate is the restricted name, not the other one
tests: 4 passed, 0 failed
ALL TESTS PASSED

Two people, “alice” and “dave”, both satisfy the induced rule — the training data treats them identically. A GrandparentPolicy schema with an independent restricted-names list catches “dave” specifically, without touching the induction engine’s own logic at all. Full source: self_hosting/schema_synthesis_bridge_selftest.patlang.

GOAP Planning: Checking Real State, Not a Parsed Label

PatLang’s pre-existing GOAP contract system (goap_verify_contracts) can only check a contract against a string-parsed action-label binding, like extracting X=5 out of the text "scale(X=5)" — it has no way to express “and the book must not also still be at the origin branch,” because that needs the plan’s full resulting state, not one action’s own parameter. A new native function, plan_with_state, exposes that real resulting state directly — implemented three times over, honestly: as a Rust interpreter host function, in the Rust-to-native codegen path, and, canonically, as genuine self-hosted PatLang in self_hosting/lib/x64_runtime.patlang, compiled and run through the real patc1.exe --x64 production toolchain, not just the interpreter. Not runnable in this page’s sandbox: plan_with_state is brand new and isn’t in the WASM module embedded on this page yet.

  ok: the planner finds the full three-hop route
  ok: step 1 packs the book for transit
  ok: step 2 ships it to the depot
  ok: step 3 ships it on to the destination branch
  ok: the real resulting state has the book at branch_b
  ok: the real resulting state no longer has it at branch_a
  ok: the real resulting state has no dangling in-transit record
  ok: the full transfer plan satisfies the transfer policy
  ok: a destination with no shipping route is reported as no_plan_found
tests: 9 passed, 0 failed
ALL TESTS PASSED

A rare book moves from one branch to another through a real three-step GOAP plan (pack for transit, ship to the depot, ship on to the destination); the schema checks the plan’s actual resulting facts — the book present at the destination and genuinely absent from the origin, not inferred from the last action’s own label text. A destination with no shipping route is correctly reported as no_plan_found, distinct from a schema violation. Full source: self_hosting/examples/interlibrary_transfer_goap_demo.patlang.

The Other Direction: Schema Suggesting Scenarios

Checking existing scenarios against a schema is the easier direction, and it’s the one that’s built. The harder, more useful direction runs the other way: since a schema states a universally-quantified property rather than one instance, a schema-aware tool could in principle enumerate concrete cases the current scenario set doesn’t cover and propose them as new scenarios, rather than waiting for someone to think of the edge case by hand. This isn’t a new idea either — it’s what property-based testing already does, generating concrete test cases from a stated property instead of a human enumerating them by hand3. This direction remains unbuilt: require_fn/ensure_fn are always named, separately-compiled, introspectable functions rather than inlined text, specifically so a future generator could enumerate the operation registry without any rework to what’s here now.

What Was Actually Needed

PieceStatus
Feature:/Scenario:/Given/When/ThenPre-existing PatLang syntax, unchanged
Set<T>/Map<K,V>Built: pset.patlang, pmap.patlang
Schema/operation declaration + bindingBuilt: schema_bdd.patlang (schema_define, schema_operation, schema_bind_state, schema_bind_input)
The witness check itselfBuilt: schema_check/schema_check_values, four-stage diagnosis
Self-healing synthesis hookBuilt: an opt-in schema check in green_phase, zero effect unless registered
Inductive-synthesis bridgeBuilt: schema_synthesis_bridge.patlang
GOAP state exposure (plan_with_state)Built three times over: Rust interpreter, Rust codegen, and canonically as native self-hosted PatLang
Schema-to-scenario generatorStill not built — see above

Open Questions and Limits

The checking direction — does this one scenario satisfy the schema — is cheap: substitute concrete values into a predicate and evaluate it, confirmed directly by every example on this page running well under the length of a page load. The harder question — does a feature’s full set of scenarios, taken together across a whole sequence of operations, ever drive the state into something the invariant forbids — is a state-space exploration problem, not a per-scenario evaluation, and it inherits exactly the scaling limit Formal Methods already names for Z on its own: a schema doesn’t check itself past a certain size without tool support. That’s precisely the harder problem Liu and Bowen’s PhD work spends its own length on, under Z refinement specifically. A narrower, now-resolved limit: an absence-sentinel ambiguity in pmap_get (a missing key and a genuinely falsy stored value both read back the same way) is a documented hard rule — pmap_has is the only authoritative presence check — rather than a live bug, since every schema built so far uses string- or list-typed state where it doesn’t bite.

References


  1. Liu, B. (2019). Using Behavioural Specifications to Support Model-Checking [Master’s thesis, University of Waikato].

  2. Liu, B. (2024). Integrating Behavioural and Formal Specifications [PhD thesis, University of Waikato]. https://hdl.handle.net/10289/17330

  3. Claessen, K., & Hughes, J. (2000). QuickCheck: A lightweight tool for random testing of Haskell programs. ACM SIGPLAN Notices, 35(9), 268–279. https://doi.org/10.1145/357766.351266