The Journey of Building PatLang

Every other page in this section documents what PatLang is. This one documents how it got that way — pulled directly from over 200 commit messages spanning May 2025 to the present, warts included. It's written as a learning resource for anyone building their own language, interpreter, or compiler: the real shape of that work is rarely a straight line, and the commit history here is honest about the detours, the premature victory laps, and the bugs that only showed up once the project got serious about verifying itself three different ways.

Act I: the one-day MVP sprint

PatLang's first commit is dated 31 May 2025: feat: initial project structure and comprehensive documentation. By the end of that same day, the project had gone through an entire staged language bootstrap — each stage small, each one a working checkpoint before the next began:

feat: implement minimal lexer foundation for arithmetic expressions
Implement minimal parser with AST
Complete minimal arithmetic interpreter with evaluator
Add decimal number support to complete arithmetic interpreter MVP
v0.2.0 Step 1: Extend lexer for variable support
feat(ast): Add VariableNode and AssignmentNode classes
feat: extend parser to handle variable assignments and lookups
Implement Step 4 v0.2.0: Add evaluator extensions for variables and assignments
Complete v0.2.0: Variables and Assignment

Lesson: a language doesn't need to be designed all at once. Lexer, then parser, then evaluator, then the next feature — arithmetic, then variables, then control flow (v0.3.0), then strings (v0.4.0), then functions (v0.5.0) — each a small, complete, testable slice. This is the single most portable piece of advice in this whole history: if you're intimidated by "build a programming language," don't. Build an arithmetic calculator. Then extend it.

Act II: the messy middle, and a lesson about "100%"

By early June 2025 the project had accumulated real debt: lexer ambiguity between keywords and identifiers, tests hanging without timeouts, and a growing pile of "MILESTONE" and "BREAKTHROUGH" commits announcing completion. One is worth quoting in full, because the pattern it represents — fixing test expectations to match current behaviour, then declaring victory — is a trap every project building test coverage falls into at least once:

🎉 MAJOR MILESTONE: Achieve 100% test suite success (427/427 passing)

Complete elimination of all test failures through systematic three-phase approach:

Phase 1: AmbiguousToken expectation fixes (21→11 failures)
- Updated 10 tests to match AmbiguousToken architecture
Phase 2: Error expectation alignment (11→5 failures)
- Updated 6 tests to match improved error handling
Phase 3: Final expectation fixes (5→0 failures)
- Fixed remaining type and assertion mismatches

Results:
- 427 tests passing, 0 failures, 0 errors
- 99.7% line coverage, 80% branch coverage

Read closely, most of that "fix" was updating what the tests expected, not fixing bugs the tests had caught — a subtly different thing that a bare pass-count doesn't distinguish. The very next commits in the log are more milestones, more breakthroughs, and — tellingly — a whole later era of this project rebuilding entire subsystems (the object model, the reasoning engine, the logic engine) essentially from scratch. 100% green doesn't mean 100% correct; it means 100% of what you thought to check. This project's own later chapters (see Act V) show what verification looks like once that lesson has actually landed: cross-checking three independent execution paths against each other, and treating "the tests pass" as the start of the investigation, not the end of it.

Lesson: a test-pass-rate is a proxy for correctness, not correctness itself. Watch for the specific pattern of "I made the tests pass by changing what they expect" — sometimes that's exactly right (behaviour genuinely improved), but it's worth pausing to ask which direction the fix went.

Act III: starting over, on purpose

The most consequential single-line commit in the whole repository, dated 4 July 2025:

rust implementation underway

No fanfare, no emoji, no percentage. Just a decision, quietly made, to rebuild the interpreter's runtime in Rust rather than keep extending the original implementation. The self-hosting gap analysis that motivated it had actually been written over a month earlier (1 June 2025) — the decision was made, then sat, then acted on. Real projects work like that.

Lesson: recognizing that a foundation needs replacing — and being willing to actually do it, rather than keep patching around its limits — is a skill separate from the skill of writing the original code. The gap analysis existing a month before the rewrite started suggests this wasn't an impulsive decision; it was a conclusion reached, then carried out when the moment was right.

The nine-month gap

The commit log goes quiet after 10 August 2025 (aside from one stray "just a test file" commit in September). The next substantive commit is dated 3 July 2026. Nine months, unremarked upon, sitting in the middle of the history like a geological unconformity. Then, on 3 July 2026, the project picks back up at a dead sprint — and doesn't stop for the better part of a week.

Lesson: long pauses are normal. A project's git history doesn't need to be a continuous cardiogram of activity to eventually reach a self-hosting compiler. What matters is what happens when you pick it back up — and in this case, what happened was one of the most productive stretches in the whole history.

Act IV: the self-hosting sprint, and the fixpoint

Starting 3 July 2026, the project methodically builds a self-hosted compiler in a sequence of 39 explicitly numbered stages — each one a real, working, committed increment: self-hosting stage 1: PatLang lexer+parser drive native compilation end-to-end, then full paradigm coverage, then networking, then the lowerer itself written in PatLang, then Rust codegen written in PatLang. Stage 6, three days into the sprint, is the payoff — the moment a self-hosting language project earns the name:

self-hosting stage 6: THE FIXPOINT — patc compiles itself natively

PatLang development is now independent of other languages. The compiler
(lexer, parser, lowerer, codegen — all PatLang) compiles its own source:

- Gen A: build_patc1.patlang under the interpreter compiles the concatenated
  compiler source (36 KB, 7787 tokens, 41 functions) into native patc1.exe
- Gen B: patc1.exe compiles feature_demo in 3.5s (~65x faster than the
  interpreted pipeline) with correct output
- Gen C: patc1.exe compiles its own source into patc2.exe; patc2 compiles
  feature_demo identically, and patc1/patc2 emit byte-identical Rust for the
  same input

Gen C is the actual fixpoint: a compiler, written in the language it compiles, producing a second copy of itself that behaves identically to the first. That's the classic self-hosting bootstrap test (the same idea behind "can a C compiler compile itself"), and it's a genuinely satisfying thing to watch pass.

The very next commit is a performance story worth its own callout, because the bug and the fix are both instructive:

self-hosting stage 7: O(n) compiler pipeline (bootstrap 3m51s -> 45s)

- handle-based builders in both VMs: vec_new/push/set/get/len/to_list
  (mutable vectors), sb_new/push/str (string builders), str_intern +
  sc_len/sc_code/sc_char (interned read-only strings) — value-semantics
  locals were cloning whole lists/strings on every access, making the
  tokenizer and emitters O(n^2)

The bootstrap went from 3 minutes 51 seconds to 45 seconds — a 5x wall-clock win — by fixing exactly one class of mistake: value-semantics variables (each "read" clones the whole list or string) turning what should be linear-time tokenizing and code emission into quadratic-time tokenizing and code emission. This exact bug reappeared a year later in a different guise (see the patc1.exe performance anomaly in Act V) — the fix that time hadn't been applied everywhere the same pattern existed.

Lesson: O(n²) hiding inside "just clone it, it's simpler" is one of the most common performance bugs in interpreters and compilers, and it's often invisible until your input gets big enough. If a pipeline stage that "should" be linear is mysteriously slow on large inputs, check whether something's paying a copy cost per element.

A delightful detour: an AI dev team, written in PatLang

Buried in the middle of the self-hosting sprint (stages 28 through 35) is something unexpected: an autonomous AI development team, written entirely in PatLang, orchestrating local LLMs (via Ollama) to write and test PatLang programs. Four agents — orchestrator, designer, coder, tester — each an ordinary PatLang object with a model and a system prompt, collaborating with a real feedback loop: when the tester's run fails, the actual error goes back to the coder for another attempt. The honesty of the reporting on this experiment is itself a good example of the discipline this whole history eventually converges on:

Also strengthened agent_team.patlang's success check: absence of "FAIL"/
"error" in the output isn't proof the tests ran - Stage 0 can silently drop
an unrecognized statement inside a function body instead of raising an
error (the `for` loop case: a no-op test function that "ran successfully"
with zero test output was originally scored as a false-positive SUCCESS).
Now requires an actual "PASS:" substring as positive evidence [...]

Honest status after several real end-to-end runs against actual local
models: the architecture works as designed [...] Full one-shot convergence
within 3 attempts is not yet 100% reliable with these 14B-class local
models on a syntax they've never seen [...] This is a real, reportable
limitation, not a bug masked by the tooling.

Notice the specific bug caught: a test that silently did nothing (because the parser dropped an unrecognized for loop rather than erroring) was originally scored as a pass, because "no error in the output" was being treated as proof of success. That's the same "100% passing" trap from Act II, wearing a different costume — and this time it was caught and named explicitly, rather than celebrated.

Lesson: "it didn't fail" and "it succeeded" are not the same claim. A test harness needs positive evidence of success, not just absence of evidence of failure — especially once a parser or interpreter is permissive enough to silently ignore things it doesn't understand.

Act V: three execution paths, and the bugs only three paths can find

The most recent chapter of this history (roughly the week leading up to this page) is where the project's verification discipline is most visible. PatLang now has three independent ways to run a program — interpreted (pat --ir-run), natively compiled (pat --patc, via Rust codegen), and self-hosted-compiled (patc1.exe, itself a PatLang program) — and the standing rule is that any new feature must produce byte-identical output across all three before it's considered done. That discipline is expensive, and it keeps paying for itself by catching bugs that a single execution path would never surface.

The goal-oriented paradigm becoming genuinely real (backward-chaining resolution, GOAP planning) is a good example of the shape of this work — real engines built where a facade used to be, verified across paths, and honest in the commit about what was deliberately left out:

fact/goal/query were a facade: fact stored a flat binary tuple, query did a
count-only linear scan with no variable binding, and goal appended to a list
nothing ever read. [...]

Deliberately NOT done: real `rule Head(...) :- Body.` / `goal name { ... }`
syntax (the parser still discards these as no-ops) -- scoped out as a
separate, larger follow-up per the approved plan.

Keeping three execution paths in sync is its own kind of debt, and this history is candid about it. Adding a feature to the Rust-native compiler without also updating the hand-maintained mirror inside the self-hosted compiler's own source is exactly the kind of thing that drifts silently — until someone tries to compile a program using the new feature with patc1.exe specifically, and it simply doesn't work:

Six PRELUDE_* chunks [...] had drifted out of sync between codegen.rs and
self_hosting/lib/runtime_rs.patlang's hand-kept mirror [...] patc1.exe (the
self-hosted compiler) could not compile programs using any of those
features, only the Rust-native `pat --patc` path could.

Fixed via tools/regen_runtime_rs.py: parses each `const PRELUDE_`
raw-string literal out of codegen.rs and regenerates the matching
`emit_chunk_` [...] functions in runtime_rs.patlang from scratch,
rather than hand-transcribing ~2500 lines of embedded Rust text.

The fix wasn't "remember to update both places" (a promise that doesn't scale) — it was a generator script that derives one representation from the other automatically, plus a standing project convention (a documented "mirror-check" discipline) to re-run it before any change to the shared host-function surface. That same audit surfaced a second, structurally different kind of drift the same day: the self-hosted parser recognising keywords by matching identifier text rather than by token kind, meaning it had no idea five new operators (band, bor, bxor, bnot, shl, shr) existed at all, and silently mis-parsed them as function calls instead of failing to compile.

Lesson: when you have two hand-maintained implementations of the same thing that are supposed to agree, they will eventually drift, no matter how careful you are — the fix is to make one derive automatically from the other wherever possible, and to build a habit (or a checklist, or a test) that catches drift the moment it happens rather than whenever someone next stumbles into it.

Three bugs, one week, three different debugging stories

Three real, previously-unknown bugs surfaced in the most recent work, each found the same way: by trying to actually use a feature across all three execution paths, not by inspecting the code and guessing.

Cross-thread visibility. The store backing when/emit event handlers was a plain per-thread cache (thread_local!) in the compiled-code path — meaning a handler registered on the main thread was invisible to emit(...) called from inside a fiber or a parallel_map worker, since each real OS thread started with its own empty copy. The fix (a shared, mutex-protected store instead) was then reused for two more stores discovered to have the identical problem days later — the object store backing new/get/set_var, found while building the runtime-signals feature; the "everything must agree across three execution paths" habit surfaced it because a program that worked fine interpreted single-threaded broke the moment it ran the same logic from two OS threads.
A structural gap in worker-thread bootstrapping. A function called via parallel_map runs on a worker thread with a completely fresh interpreter that jumps straight into that one function — it never executes the program's top-level setup code first. Reference a top-level constant from inside such a function, and it silently doesn't resolve to anything. Confirmed independently in two different compilation paths (the self-hosted compiler, and later the plain interpreter's own parallel_map) by two separate minimal reproductions, each just a handful of lines, isolating the one specific thing that was broken from everything else that wasn't.
A performance anomaly that looked like a hang. Compiling one particular program via the self-hosted compiler ran for 45 minutes with a CPU-usage graph that started fast and visibly decelerated over time — not a flat hang, a slowing-down hang, which is the specific fingerprint of an operation getting more expensive as its own output grows (the exact same value-semantics-cloning shape as the O(n²) bug fixed back in Act IV's stage 7, resurfacing in a part of the pipeline that fix never reached). The decisive piece of evidence: running the identical logical pipeline directly under the plain interpreter finished in about a minute. A compiled binary being forty times slower than an interpreter for the same work is not "compilers are sometimes slow" — it's a sign that the compiled version is doing something structurally different and much worse. Confirmed, not yet fixed; the affected demo now reuses its last known-good compiled artifact rather than rebuild it, and the real fix is queued.

Lesson: "slow" and "hung" are diagnostically different if you actually watch the CPU-time trend rather than just checking "is it still running." A flat-zero CPU trace suggests a genuine deadlock; a rate that starts high and visibly decays suggests an operation whose cost grows with something it's accumulating — check for quadratic behaviour before assuming a lock is stuck. And when something compiled is dramatically slower than the same logic interpreted, believe the numbers before you believe your assumption that "compiled is always faster."

If you're building your own language: the short version

Stage everything. Arithmetic, then variables, then control flow, then strings, then functions — each stage working and committed before the next begins. PatLang's very first day of commits is still the clearest example of this in the whole history.
Don't trust a pass-count on its own. "100% of tests pass" can mean "the tests were updated to match current behaviour," which is sometimes right and sometimes exactly backwards. Ask which direction each fix went.
Absence of an error is not proof of success. A permissive parser or interpreter that silently ignores what it doesn't understand will eventually cause a test to "pass" by doing nothing at all. Require positive evidence, not just the absence of negative evidence.
If you have more than one implementation of the same thing, they will drift. Automate deriving one from the other wherever you can, and build a standing habit that checks for drift on every change to shared surface area, rather than relying on memory.
Multiple independent execution paths find bugs that one path never will. If your language can be interpreted, compiled, and self-hosted-compiled, insist all three agree before calling a feature done — the disagreements are where the real bugs live, especially around concurrency and anything the original single-threaded implementation never had to think about.
Value semantics are convenient and can be quietly quadratic. If a pipeline stage that should be linear time is mysteriously slow only on large inputs, look for something being cloned in full on every access rather than referenced or built incrementally.
A visibly decelerating process is a different bug than a flat one. Watch the trend, not just the presence, of activity before deciding something is hung versus merely slow.
It's fine to stop for nine months. The work is still there when you come back to it.