Last updated: 2026-11-15

U
Undergraduate level
EPH
Ephemeral / Tooling โ€” Knowledge that evolves in months to a year โ€” check for updates

The v2 Toolchain: Compiler, Interpreter and Gates

For new readers

PatLang 2.0 is three programs, all written in PatLang and all built by the v2 compiler: the compiler itself, an interpreter, and a test-gate runner that decides what needs retesting after a change. This page is the working reference for each: the commands and options, what the compiler does in what order and how it reuses earlier work, how the interpreter runs programs and records traces for the browser debugger, and how the gates run hours of tests in parallel while leaving the machine usable.

The compiler Applied / MethodologicalKnowledge with a 5โ€“10 year half-life โ€” stable practice

v2.exe SRC.patlang OUT.exe [--workers N] [--shuffle SEED] [--no-cache]
v2.exe SRC.patlang OUT.ir --ir
v2.exe SRC.patlang OUT.wasm --wasm
v2.exe SRC.patlang --constant-time
v2.exe --evict KEEP_MB

v2.exe is self_hosting/v2/compile.patlang built by itself, at self_hosting/build/v2.exe, and runs from the repository's root. A build with errors exits 1, and a usage error 2.

OptionEffect
--workers NRun the per-function jobs on N threads (1 by default)
--shuffle SEEDRun the jobs in a shuffled order; the executable is byte-identical whatever the order
--no-cacheBuild every job, the runtime's included, without reading stored artefacts
--irWrite the program's IR as text, and nothing else (The v2 IR)
--wasmWrite a WebAssembly module for the browser host
--constant-timeBuild nothing; report where time can depend on a secret (Constant-time arithmetic)
--evict KEEP_MBRemove stored jobs, least recently used first, until the store is no larger than KEEP_MB

Stages

  1. Front end. Read the program and the runtime, expand syntax definitions, take each file once by its normalised path, and parse, with the lexer and parser shared with the rest of the toolchain.
  2. Lowering (lower.patlang). Each declaration's syntax tree becomes IR. Lowering records the program facts as it goes: the events handled, the functions reached by name, the rules. It also fits each named call to its declaration and refuses a wrong argument count.
  3. One job per function (jobs.patlang): the ownership stage (own.patlang), code generation (x64.patlang) and assembly (lib/x64_asm.patlang), giving the function's chunk of machine code.
  4. Linking (lib/x64_pe_link.patlang): the chunks, in declaration order, into a PE executable.

Each stage is a function of its inputs and touches no shared mutable state, which is what makes the parallel and the sequential build give the same bytes. The swarm compiler explains the design in full.the graph of jobs is the build's heart

The cache

A job's key hashes the function's IR, the program facts its code generation reads, the ownership summary of every function it calls, and the compiler's own key. Its artefacts are stored under self_hosting/build/v2_artefacts and reused by any later build, of any program, with the same key. So a second program reuses every runtime job, and changing one function's body rebuilds that function alone, unless its summary changes too.

  • The compiler's key. When v2 builds a compiler, it writes OUT.key beside it: a hash of every source the compiler was built from, as they were read. The compiler reads the key beside itself and mixes it into every job key. An earlier version hashed the compiler's sources as they stood on disk, so after an edit to the code generator a compiler built before the edit reused code from the old one. A compiler with no key builds without the cache and says so.
  • Complete or absent. A job's record gives the length of each artefact and ends with a marker line. An artefact shorter than that, cut short by an interrupted build or by another build writing it at that moment, is built again. Two builds can share a store with no lock: a job's artefacts are the same bytes whoever writes them.
  • Eviction happens only when asked, with --evict. A build marks each job it reuses, so the least recently used go first.

Build times Ephemeral / ToolingKnowledge that evolves in months to a year โ€” check for updates

Measured in November 2026 on a 12-core, 20-thread machine:

BuildTime
Warm build of a one-line programAbout 12.3 s: front end 2.2, keys 1.05, jobs 2.2, link 6.7
Cold build of the compiler on 8 workersAbout 7.8 minutes
The compiler's own warm link22 s

Most of the fixed cost is linking the runtime's few hundred stored functions, and it does not run in parallel. More workers help most where there are many jobs: a cold build of the compiler went from 995 seconds on one worker to 386 on eight and 326 on sixteen, while a one-line program gained little. One worker, the default, is the slowest setting in every case measured, and eight give most of the gain.

Much of v2's compile speed came from finding quadratic costs in the compiler's own PatLang: a list push that copied the list when its value read the list, a linker that gave every name index 4,096 buckets, an assembler that found each byte of a value by repeated subtraction, and an artefact reader that turned each byte into a one-character string. Each fix kept the executables byte-identical, which the fixpoint check confirmed.

The interpreter Applied / MethodologicalKnowledge with a 5โ€“10 year half-life โ€” stable practice

v2_interp.exe SRC.patlang [args...]

The interpreter is self_hosting/v2/interp.patlang, a command over the library self_hosting/v2/interpreter.patlang, built by v2 as self_hosting/build/v2_interp.exe. It finds the runtime's source from its own executable's path, so it must stay in self_hosting/build/, and a program gives the same result whichever directory it is started in.

It lowers the program with the compiler's own front end, against the runtime's declarations alone, and calls the runtime's compiled functions directly, so only the program's own code is interpreted. Each function is translated once into a list of integers read word by word, with each operand held as its byte offset in the frame and each jump as its target's byte offset, and its constants in a second list. Frames come from one block the interpreter allocates, not a list per call, and two ints add directly when no handler is registered. Those changes took bench_fib from 104.9 seconds to 8.9; startup takes about 1.7 seconds.frame is a block of memory for local variables

Traces and the debugger

Asked to, the lowering compiles a program to report each step: it calls __trace_step before each statement with its line, and __trace_enter and __trace_leave around each function's body, with the function and its locals. A traced run gives those names closures that record into a trace. The IDE's debugger steps through such a trace after the program has run, showing the line, the function and its locals at each step.

In the browser

The playground, the REPL, the IDE and the dynamic-syntax demo all run lib/interp_run.patlang's world backend, built for the browser by v2. A page gives it the runtime's files as a read-only bundle, and it compiles the user's text with the front end and runs it in a fresh world on the interpreter. Every demo card in the portfolio is built by v2, natively for its transcript and as WebAssembly for its page.

The gates Applied / MethodologicalKnowledge with a 5โ€“10 year half-life โ€” stable practice

self_hosting/v2/tools/gates.sh [--all] [--plan] [--cap N] [--only PREFIX]

The gates are every check that must pass before a change is pushed: the compiler and its fixpoint, the bootstrap reader, the WebAssembly writers, the interpreter, and each suite's feature files on each engine. Run one after another, they took 4 hours 52 minutes. gates.sh builds the first v2.exe from the IR when there is none, uses it to build the gates tool, self_hosting/v2/tools/gates.patlang, and runs the tool with its own arguments.

OptionEffect
(none)Run the units that are stale, in parallel
--allRun every unit, fresh or not, for before a push and to check the record against reality
--planSay what would run, and run nothing
--cap NGive the units N cores whatever else the machine is doing
--only PREFIXKeep as goals the units whose names start with PREFIX; what they need still runs when it must

Units, keys and the plan

A unit is one build or one feature file of one suite on one engine. Each declares the units it needs, the files and directories its result depends on (a runner's source with every file it includes, and the fixtures and directories its steps read), the cores it keeps busy, and a lane: units in one lane never run together, as the language suite's two engines share ports and files. A unit's key hashes its inputs' contents and the keys of the units it needs. A unit that passed with its current key is fresh and is not run again; a unit with no declared run-time inputs is always stale, so a missing declaration costs time and never gives a wrong verdict.

The tool uses PatLang's own logic engine for this. The units, their needs and inputs are facts, and staleness is a rule. The stale units are the planner's goals, its actions are the units, and each action needs what the units it depends on produce, so a build that no stale unit needs, and whose product is present, does not run. Running the gates this way found the old planner's habit of answering "no plan" for problems that had one, and led to the A* planner.stale means: run me again

Sharing the machine

  • Cores. Units get at most the logical cores less two, by weight. Before starting one, the tool reads cpu_times; cores busy beyond what its own units account for belong to someone else, and the budget shrinks by them, never below one unit at a time.
  • Memory. A unit starts only when memory_status shows room for its peak from last time (or 2 GB) plus the peaks of the units running and a 4 GB reserve for the user. Peaks are measured with exec_measured, which runs the command in a Windows job object, so a suite's compiles and child programs count.
  • Isolation. Each unit writes only under self_hosting/build/gates/out/UNIT, given to the runner as PATLANG_SPEC_OUT, and fixtures that listen on the network take port 0, so two units never collide.
  • Status. Progress goes to self_hosting/build/gates/status, and a status signal on the port that file names gives the same. A quit signal starts no more units and lets the running ones finish; each result is recorded as it comes, so the next run takes up the rest.

Results Ephemeral / ToolingKnowledge that evolves in months to a year โ€” check for updates

RunTime
Every gate, one after another4 h 52 min
The other 146 units in parallel, with Part 1 already fresh2 h 36 min
Part 1 on both engines, in parallel, from nothingAbout 28 min, against 83 sequentially
A second run with nothing changed4 s, running nothing
After a blank line added to text.featureExactly that feature's two units

Every suite gave its sequential count in the parallel run: Part 2 136, the language suite 297 on each engine, match 26, the block model 210, engines agree 62, the interpreter-run selftest 54 and the isolate selftest 65, with the bootstrap and WebAssembly checks identical. Two feature files now bound a full run, each longer than its whole suite took alone, since everything else competes with them for cores; they start first so the rest fits inside them.

Rust is out of the loop Ephemeral / ToolingKnowledge that evolves in months to a year โ€” check for updates

At the pre-v2 tag, three things depended on Rust: pat --ir-run ran the compiler, the suite runners and everyday development; WebAssembly went through generated Rust and rustc; and a clean checkout started from a Rust build. All three are gone. Since 14 November 2026 no step of any suite starts pat; the site builder and the portfolio builder run compiled by v2; the browser gets its modules from v2's own writer; and a clean checkout starts from the IR and a Python reader. The Patbuild demo page, which built its targets with rustc, was retired with it.v2 compiles itself, bootstrapping the toolchain

The old compiler's runtime and fixtures still keep their own global state, and leave with it when it is retired. When the 418 programs that existed at the tag were surveyed a second time, on 6 November, 385 compiled on v2. Of the 33 that did not, 16 stopped on the name unit, which has been fixed since; the rest were include paths that no longer exist, three parse errors, examples written to test the Rust runtime itself, and one other.

See also

The swarm compiler covers the job graph and the cache in depth. Six self-hosted dev tools and the BDD framework describe the tools the suites are written with. What changed since v1 summarises the release.

Graduate Attributes

Modelled on this page

  • Subject Specialist: Technical details of compiler stages, caching keys and interpreter frames are presented with precise implementation specifics.
  • Researcher: Performance improvements are substantiated with specific benchmark timings before and after optimisations.
  • Critical Thinker: Build time data is analysed to identify diminishing returns on parallelism and justify default worker settings.

University of Reading Graduate Attributes framework.