K-Blades vs. Self-Organizing Maps: How the Live Concept Atlas Works Now

For new readers: the map you get from pressing M anywhere on this site — the "Cartographic Topological Map" with hex-shaped territories, mountain/forest/swamp glyphs, and dashed roads between articles — used to be generated by a Self-Organizing Map (SOM), a kind of neural grid. It's now generated by a different technique entirely: k-blade geometric algebra, described across three research papers linked below. This page explains what changed, why, and what it means for how the map looks and behaves.

What Changed, in One Sentence

The atlas generator went from training a 40×40 grid of neurons for up to 800 seconds every time the site's content changed, to computing a pairwise similarity structure directly from page embeddings in about 230 milliseconds — and the map itself went from a fixed hex grid to continuous, force-directed node positions with genuine convex-hull territories instead of grid-cell colouring.

What the SOM Version Did

The retired som_atlas.py pipeline trained a 40×40 Pure-Python Self-Organizing Map (1,600 neurons) over broken-stick-selected TF-IDF vectors for 30 epochs. Each page was assigned to its Best Matching Unit (BMU) — the trained neuron closest to that page's vector — giving it an integer grid coordinate. A U-Matrix (the average distance between each neuron and its immediate grid neighbours) supplied an elevation value per cell, which drove mountain/forest/swamp placement and a Dijkstra pathfind for roads that preferred low-elevation "valleys." Topic territories were a discrete Voronoi partition: every grid cell coloured by whichever page-node was nearest to it, with visible border lines wherever two differently-coloured cells touched.

This worked, and had real strengths: a stable, dense coordinate grid that every rendering feature (hex tiles, contended borders, terrain, road pathfinding) could share a single contract against. Its cost, though, was structural rather than accidental — see below.

What the K-Blade Version Does Instead

The live kblade_atlas.py pipeline represents each page as a unit embedding vector (nomic-embed-text, 768-dim, cached), and computes pairwise similarity between every pair of pages using a k-blade generalized sine — the product of the sines of the principal angles between two subspaces, which reduces to the classic bivector norm for single vectors. Pages are grouped into clusters by modularity-maximizing graph communities over a sparsified top-K similarity graph, then laid out with a force-directed spring layout (seeded from an MDS projection of the k-blade distances) so that pages in the same cluster actually end up spatially close together, not just similarity-adjacent. Cluster territories are rendered as real convex hulls around each cluster's member coordinates — not grid-cell colouring. Terrain (mountain/forest/swamp) comes from a sparse jittered point sample tested against those hulls: a sample point outside every hull is a mountain (a genuine conceptual gap), inside exactly one hull is forest (settled, single-topic ground), and inside two or more hulls is swamp — a direct, visual rendering of the overlapping-membership finding below. Roads are a minimum spanning tree over physical canvas distance, restricted to edges that are also real similarity-graph edges, plus one bridge edge between every pair of adjacent clusters.

Why Switch: What the Papers Found

Three research papers (linked below, in Research Papers) document the investigation that led here, and are worth reading in full for the actual numbers and the failure modes along the way — summarized:

Geometric algebra needed the right algorithm, not just the right metric. An initial attempt paired k-blade similarity with the conventional greedy agglomerative merge and got a strictly worse hierarchy than plain cosine similarity — direct inspection showed a genuine chaining pathology, not a benchmarking artifact. Swapping only the clustering algorithm (to modularity communities, or a rate-of-change growth procedure), while holding the same k-blade similarity values fixed, recovered performance matching the best cosine baseline. The lesson that shaped the production engine: representation and algorithm are separable design choices, and kblade_atlas.py uses graph-community clustering specifically because that is the combination the research validated.
Pages are legitimately multi-topic, and it's measurable, not just anecdotal. Between 34% and 46% of pages, depending on method, genuinely qualify for membership in more than one cluster under a fixed affinity-margin rule. The live atlas surfaces this directly as swamp terrain wherever cluster hulls overlap, rather than forcing every page into exactly one Voronoi cell the way the SOM version did.
The speed difference is not a rounding error. A cold SOM training run (cache bypassed) took 802.1 seconds on the site's 160-page corpus, 99.4% of it in the 30-epoch neuron training loop itself; the equivalent k-blade similarity-plus-clustering pipeline took 230 milliseconds — roughly 3,485× faster. The mechanism is structural: the SOM's cost is grid-size×epochs (fixed, independent of corpus size), while the k-blade pipeline's cost is corpus-size-scaling pairwise linear algebra that stays cheap at this corpus's scale. This is what makes rebuilding the atlas on every ordinary site build practical, rather than something to cache aggressively and dread invalidating.

The efficiency paper is explicit that this was not, at the time, a fully like-for-like comparison — the SOM's U-Matrix and fixed grid gave rendering features (terrain elevation, road pathfinding) that the k-blade pipeline, as it stood then, did not yet replace. Building the convex-hull territories, sparse terrain sampling, and MST road network described above is that replacement work, carried out directly in kblade_atlas.py.

Bridge Text: How the "Wayfarer Reading Guidance" Prose Actually Gets Written

Every road on the map, and every pair of pages you can route between, gets a short contextual paragraph explaining the connection — the text that appears when you hover a road or click through from one article to a related one. Earlier revisions of this page described that text as deterministically templated, built once at ordinary site-build time with no language model involved. That's no longer how it works. The live mechanism, in tools/site/concept_experiments/inverter_site_bridge_build.py, is a real local LLM (qwen2.5, run offline via llama.cpp) writing genuine prose — but only for a carefully scoped subset of pairs, with everything else assembled live in the browser.

Only literal graph edges get an LLM call. On the current 165-page corpus that's 975 edges — the same top-K similarity-graph edges used for the roads themselves — out of 13,530 possible unordered page pairs. Each edge gets its own two-paragraph-ish piece of "scaffolding guidance" (120–180 words), generated from a short cached summary of each page rather than the raw page text, and framed explicitly as a teacher's guidance to a student moving from one page to the other: how closely related the topics really are, the actual conceptual bridge between them, and two or three concrete things to watch for making that jump. Both directions (A→B and B→A) are generated and stored separately, not derived from one another.
Every other pair is routed and stitched live, in your browser, at request time. The build publishes a small connectivity-guaranteed routing graph (assets/bridge-graph.json: just node ids and weighted edges, no text) alongside the per-edge bridge files. When you hover a node that isn't graph-adjacent to wherever you started, site.js runs a real Dijkstra shortest-path search over that graph, then fetches and concatenates the already-generated, already-correctly-directed text for each hop along the path, with a "Step N: A → B" header between hops. Nothing beyond the 975 atomic edges is ever pre-written or stored.
That live-routing design replaced an earlier one with a real, structural bug. The first version of this pipeline precomputed and stored a combined file for every multi-hop pair, stitched together at build time. It had two problems: a stitched file baked in only one direction, so a reader arriving at the same pair from the opposite direction saw guidance that read backwards; and because it stored a file per pair rather than per edge, the artifact set grew without bound as pages were added. Moving the routing itself to request time — walking the graph fresh every time, always in the actual direction of travel, from atomic edge text that's already correct in both directions — removed both problems at their root rather than patching around either.

This generation happens offline, ahead of an ordinary site build, not during it: build.py runs a cheap, no-LLM resume-check on every build and, if any edges are missing coverage — typically just the handful touching a page that was just added — automatically generates only those before publishing, so an ordinary content edit is never blocked on a multi-minute LLM run. A separate, later cleanup pass mechanically fixes wording artifacts (literal "Page A"/"Page B" placeholders, repetitive stock openers) that showed up in the raw prompt output before that pass existed, and is re-applied automatically to anything freshly generated.

One loose end, for anyone reading the generator code directly: kblade_atlas.py still computes the older deterministic-template bridge text this page used to describe (som_atlas.generate_pair_bridge, keyed by cluster labels and a hashed template choice) and still writes it into site-atlas.json as a contextual_bridges field. It's no longer read anywhere — site.js has no reference to that field at all. It's inert, superseded data, not a second live system running alongside the one described above.