Last updated: 2026-11-22

U
Undergraduate level
EPH
Ephemeral / Tooling โ€” Knowledge that evolves in months to a year โ€” check for updates

Issues Found in the PatLang 2.0 Gates

For new readers

The gates are every check that must pass before a PatLang change is pushed: the compiler's fixpoint, the bootstrap readers, and each suite on each engine. Between 16 and 22 November 2026 they grew from 229 units to 965 and began running in parallel, and much of the work in that period went into making a pass mean what it says. This page lists the issues in the test machinery itself, under four headings: results that passed without testing anything, checks that failed only under load, tests that interfered with each other, and measurements that misled. The v2 toolchain describes how the gates run.

Size of a full run Ephemeral / ToolingKnowledge that evolves in months to a year โ€” check for updates

DateUnitsFull runChange behind it
15 Nov146 + Part 12 h 36 min (4 h 52 min one after another)Parallel units within free cores and memory
16 Nov229โ€”Every selftest gated
16 Nov3352 h 13 minFeatures over ten minutes split into one unit per scenario
16 Nov33577 minBlock-model suite retired
19 Nov534, then 621โ€”Package specs gated; v2.0.0
20 Nov859โ€”Linux units; v2.1.0
22 Nov965โ€”TLS, sha512 and curve25519 package suites

Passes that tested nothing FoundationalKnowledge that endures for decades โ€” core principles

IssueExampleCheck now in place
A unit exited 0 while reporting failuresinterp_run_selftest passed with two checks still expecting get to give falseA unit fails when its last tests: N passed, M failed line has failures, whatever its exit code
A suite listed no tests and passedlanguage_linux, with its runner's arguments in the wrong placeA runner that lists nothing gives a failing unit, SUITE/listing
Selftests that nothing ranSix of the 29 synthesis selftests were missing from their runnerA repository check: every selftest is gated, or listed as retired with a reason (21 are)
Fixtures that nothing ranrt_aslr and rt_hostsA repository check fails on a fixture no scenario names
Steps that did nothingLanguage scenarios whose programs lived in separate fixturesEach scenario carries its program in a doc string; no-op steps removed (#269)
A pass that relied on the old enginesSynthesis selftests that ran pat.exe or patc1.exeThey run on v2; a repository check refuses programs that start a retired tool
Grepping old logs for FAILLogs from earlier runs and hand reruns sat beside the current run'sEach run first moves everything in v2_gates to v2_gates/previous
Numbers quoted from a stale buildThe first v2 commit quoted block-model and language counts from runs against a stale x64_runtime.obj, corrected in the next commitCorrected in the next commit's message

Checks that failed only under load Applied / MethodologicalKnowledge with a 5โ€“10 year half-life โ€” stable practice

Running every core at once changed what timing-based checks measured. A check that passed alone and failed in the gates was a fault in the check until shown otherwise.

CheckWhy it failed under loadChange
Scaling checks on clock_ms()The clock counts time a unit spends waiting for a processorThey count cpu_cycles(), this thread's own cycles
Scaling checks on cyclescpu_cycles still moves by about a third between runs; vi_bytes compared sizes 2.5 times apart against a bound of 3Sizes four times apart against a bound of eight: linear work gives four, quadratic sixteen, each a factor of two from the bound
Budgeted-loop overheadOne run of a few millisecondsBest of three runs, at twenty times the work
Signals round tripA peer whose message arrived more than 250ย ms after the accept was droppedA peer part way through a message is parked and read on at the next poll; a scenario delays a peer 600ย ms
Idle-daemon scenarioThe test had to find the daemon before its one idle second ran outThe test checks the exit code only the idle path gives, which does not depend on winning the race
Build failures that did not recurSeen only under heavy load, with no output to diagnose themA failed build prints the compiler's output, its exit status and the phases it finished

Tests that interfered with each other Applied / MethodologicalKnowledge with a 5โ€“10 year half-life โ€” stable practice

  • Fixed ports. Signals scenarios claimed a fixed port another program could hold. Fixtures that listen claim port 0. Where a fixed port is part of what is tested, as port 9410 is for the build daemon, the features that use it share a lane and never run together.
  • Engines in parallel. The Linux language suite first had a lane of its own, and its build-daemon scenario ran beside the Windows suite's, sharing the port, the task registry and the daemon's history file. Lanes are now per feature file, shared by every language suite.
  • Shared steps between features. The build-daemon feature backed up its history file in a step of the signals feature, so it could not run alone; it takes the backup itself. A step registration that class_inheritance.feature shared with other features was lost when that feature was converted, and was restored.
  • Scratch paths. Runners wrote to fixed paths in the tree. Each unit now writes only under its own directory, given as PATLANG_SPEC_OUT and read through spec_out(). The package specs wrote to packages/*/build/, which only a hand run had created; spec_out() now makes its directory when missing, and a repository check holds package specs to it.
  • A daemon left running. A lost quit signal left the build daemon holding its port for every later run. build_daemon_demo --idle-ms exits with 3 when nothing signals it.

Measurements that misled Applied / MethodologicalKnowledge with a 5โ€“10 year half-life โ€” stable practice

Each speed-up in v2 was measured before and after. The issues below were found while measuring, and each is now a step in the project's change checklist.

IssueRule
A trial build meant to change generated code changed nothing, and timed the same as the baseline, which read as "this costs nothing"Show a trial differs (its --ir output against the baseline's) before timing it
Two compilers built in the edited tree built the same runtime, since a compiled program reads the runtime from the working directoryBuild a runtime baseline from a checkout of the old sources, and check it lacks the change
A trial compiler with a different cache key rebuilds everything on its first runSay which runs are comparable
A cost that grows quadratically can still be a negligible part of a workload's timeTake a suspected cost away in a trial and time that before fixing it
Linux peak memory varied by about 15% with address randomisation onMeasure Linux peaks with it off (setarch -R)
A compiler built before an edit reused code compiled by the old sourcesThe compiler's cache key is the one recorded when it was built (The v2 toolchain)

Gaps in what the suites covered Applied / MethodologicalKnowledge with a 5โ€“10 year half-life โ€” stable practice

  • Engine agreement. Scenarios compare the engines on programs someone wrote. lib/engine_fuzz.patlang generates programs from seeds, around the calls two review findings were about, and checks that both engines print the same.
  • Undeclared names. A program calling a function v2 lacks compiles, and stops only when the call is reached. The names suite runs v2.exe --names over every demo and gated selftest.
  • Untested primitives. The code review listed functions no test called (bool_array_get, sb_len, fact_in, query_in and others) and paths no test took (a write to a peer that has gone, quoting a child's arguments). Each has a test.
  • Failure paths. The WebAssembly host can fail the Nth call to a named system function, so error handling in file reads is tested.
  • Twin implementations. The compiler's value checks have copies in the two Python bootstrap readers. The gates' WebAssembly unit found ir_wasm.py's copy missing a new value form; the checklist now names the Python twins for every new primitive.

The rule behind the changes FoundationalKnowledge that endures for decades โ€” core principles

The project's working notes state one rule for all of the above: what goes wrong twice becomes a check, a tool, a rule or a checklist step. Many rows on this page are repository checks or checklist steps that came from a retrospective after the issue was fixed.

See also

The v2 toolchain describes units, keys, lanes and the planner the gates use. Porting to PatLang 2.0: issues by stage covers the move of programs and content onto v2. PatLang 2.0 beyond Windows covers the Linux units. Risk management: lessons from testing real systems draws teaching material from earlier PatLang test failures.

Graduate Attributes

Modelled on this page

  • Researcher: Failures in the test machinery are categorised by type and traced to their root causes.
  • Reflective: Past errors are converted into permanent checks and checklist steps through retrospective analysis.
  • Digitally Fluent: Technical documentation distinguishes between compiler fixpoints, bootstrap readers and engine suites.

University of Reading Graduate Attributes framework.