PatLang: A Primer on Triangulation, Evidence, and Process

ReadRead this before whichever "Journey" instalment you've been given. This version is the most directly useful one for your own final-year project — it's built from the same site's own project-guidance pages, applied to a real, much bigger project than yours.

Why this lens matters more than the others

You'veYou've probably already read (or will read) this site's pages on Project Navigation & Triangulation, The Report as the Front-End (which covers generating and preserving evidence), and Process Over Product. Those pages give you the theory. The PatLang Journey series gives you over a year of a real project actually living or dying by exactly that theory — including the stretches where it wasn't followed, and what that cost.

Read this series with your own project in mind. Almost everything below is something you will need to do yourself, on a much smaller scale, in the next several months.

Triangulation, recapped

TheThe idea: a project isn't a straight line, it's an exploration, and you fix your position in it the way a navigator does — by taking bearings from more than one independent landmark. The three landmarks for an academic or engineering project are your own implementation and data (what you've actually built and measured), the existing literature or theory (what's already known or proven), and the real domain problem (the actual need that made the project worth doing). Checking only one of these and you drift: code-only navigation gives you something technically interesting that answers no real question; theory-only navigation gives you a well-cited plan never tested against reality; problem-only navigation gives you enthusiasm and a demo that falls apart under scrutiny.

Where PatLang does this well. Twice, an entire arc of work exists specifically because someone insisted on checking a landmark that had been quietly skipped. When the language's synthesis engine had only ever been tested on toy, purpose-built data, the explicit next step was throwing it at a real, messy, pre-existing project (a Ruby population simulator) with no test suite of its own — triangulating against the actual domain rather than a convenient stand-in for it. That single move immediately found real bugs the toy corpus structurally could not have found (an uppercase-letter parsing collision that never showed up in digit-only test data). Later, a long-standing, informally-argued question — "is our custom search engine secretly doing the same thing as a well-known planning algorithm from the literature?" — finally got answered by actually building the textbook version and racing it against the homegrown one, rather than continuing to argue from description. That's triangulating against the literature landmark, on purpose, instead of assuming your own reasoning is enough.

Where skipping a landmark cost real time. A large chunk of code was marked, in its own commit history, as "believed correct, unverified by a real compile" — a candid admission that the implementation landmark had been assumed rather than actually checked. That gap sat unnoticed for a long stretch, because nothing had gone back to verify it — and when it finally was checked, it turned out around 90% of the supposedly-finished work had never actually been touched by the fix it was said to include. Separately, a confident claim was made about what the whole codebase could and couldn't do (no WASM support, no threading, and so on) — stated from memory, not from checking — and turned out to be wrong on every point the moment someone actually looked at the real source. Both are the same failure: trusting your own belief about your implementation instead of actually re-checking it against the real thing.

Generating and preserving evidence, recapped

TheThe idea: your evaluation chapter (or your final report of any kind) can only be as good as the evidence you generated while the work was happening — you cannot retrofit evidence after the fact. Capture artefacts (benchmark numbers, screenshots, logs, failed runs) the day they exist, with dates. Document failure pathways with the same rigour as successes: hypothesis, method, result, diagnosis. Keep something like a discovery log — a dated note every time your understanding genuinely changes.

PatLang generates evidence almost obsessively, and it pays off constantly. The project's own standing rule — any new feature must produce byte-identical output across all three ways the language can run before it counts as finished — is, at heart, a policy of generating evidence continuously rather than trusting that something "should" work. A results registry was split cleanly in two (one directory for machine-discovered answers, one for hand-written ones) specifically so that what kind of evidence backed a given result could never be ambiguous later. And when an experimental AI-driven coding system was evaluated, the write-up didn't round up: "Full one-shot convergence within 3 attempts is not yet 100% reliable with these 14B-class local models on a syntax they've never seen… This is a real, reportable limitation, not a bug masked by tooling." That's a failure pathway documented with the same care as a success — exactly the discipline the site's own report-writing page asks for.

Where the evidence wasn't actually checked, and it showed. At one point, a tracked issue was reported as closed. It wasn't — the fix had genuinely been made and verified, but the actual command to close the record had simply never been run. The gap was caught not by re-checking the evidence, but by someone else's plain scepticism: "Looks rather open for something that has been closed…" The lesson generalises directly to your own project: "I did it" and "I have evidence that I did it" are different claims, and only one of them belongs in a report.

Process over product, recapped

TheThe idea: the product (working code, a finished report) is what gets marked, but the process — what you tried, what failed, what you learned, how your understanding changed — is usually worth more in the long run, and it's the thing most students undervalue while they're doing it. Time spent on something that didn't work is not wasted, provided you can say specifically why it didn't work and you wrote that down at the time.

PatLang treats its own failures as first-class information, not embarrassments. An entire instalment of the series is literally titled around this idea — going back through work that had already been declared "fixed," more than once, and discovering several of those fixes had never actually taken effect. That's an uncomfortable thing to write up, and it gets written up anyway, in full, because the process of finding out why old fixes didn't stick is worth more than pretending they always had. Elsewhere, a nine-month gap in the entire project's activity is treated as completely unremarkable — what mattered wasn't a continuous stream of visible product, it was that the process (and the understanding behind it) was still there to pick back up later. And a feature request of real size (adding classes and inheritance to the language) was deliberately scoped as a written, agreed plan before any code was touched — process work that cost an afternoon, explicitly to avoid discovering the design was wrong five steps into building it.

Where product-thinking crept in, and had to be corrected. The clearest example is the repeated "100% of tests pass" milestone early in the project's history, which on inspection mostly meant the tests had been edited to match whatever the code currently did — a product number (a percentage) standing in for a process claim (the code is actually more correct than it was) that hadn't actually been checked. It's the single most quoted lesson in the whole series for a reason: a headline number is not the same claim as genuine progress, and mistaking one for the other is exactly what happens when you optimise for how the product looks rather than for what the process actually established.

How to read with this lens

AsAs you go through your assigned Act(s), try asking, for each significant bug or decision:

1. Triangulation: which of the three landmarks (implementation, literature/theory, real domain problem) was actually checked here — and is there one that was quietly assumed instead of verified?
2. Evidence: what's the actual evidence behind the claim being made in this Act — a real, dated, checkable artefact, or something closer to "it should be fine now"?
3. Process vs product: is the thing being reported a genuine change in understanding (process), or a number/status that looks good but hasn't been checked (product)?

Why this is worth doing now, not just for an essay

EveryEvery one of these three ideas will show up as a real decision point in your own project, usually without warning. You will be tempted, at some point, to trust that a fix "should" work rather than actually re-running the case. You will be tempted to skip writing down why an approach failed, because it feels like admitting a setback rather than recording evidence. And you will, almost certainly, produce a number at some point — a pass rate, a benchmark, a percentage — that looks like progress and needs one more honest question asked of it before you believe it. This series is a very long, very real demonstration of what happens both when that question gets asked and when it doesn't. Notice which is which as you read, because you'll be the one asking it, or failing to, on your own project soon enough.