K-Blades vs. Self-Organizing Maps: How the Live Concept Atlas Works Now

For new readers: the map you get from pressing M anywhere on this site — the "Cartographic Topological Map" with hex-shaped territories, mountain/forest/swamp glyphs, and dashed roads between articles — used to be generated by a Self-Organizing Map (SOM), a kind of neural grid. It's now generated by a different technique entirely: k-blade geometric algebra, described across three research papers linked below. This page explains what changed, why, and what it means for how the map looks and behaves.

What Changed, in One Sentence

The atlas generator went from training a 40×40 grid of neurons for up to 800 seconds every time the site's content changed, to computing a pairwise similarity structure directly from page embeddings in about 230 milliseconds — and the map itself went from a fixed hex grid to continuous, force-directed node positions with genuine convex-hull territories instead of grid-cell colouring.

What the SOM Version Did

The retired som_atlas.py pipeline trained a 40×40 Pure-Python Self-Organizing Map (1,600 neurons) over broken-stick-selected TF-IDF vectors for 30 epochs. Each page was assigned to its Best Matching Unit (BMU) — the trained neuron closest to that page's vector — giving it an integer grid coordinate. A U-Matrix (the average distance between each neuron and its immediate grid neighbours) supplied an elevation value per cell, which drove mountain/forest/swamp placement and a Dijkstra pathfind for roads that preferred low-elevation "valleys." Topic territories were a discrete Voronoi partition: every grid cell coloured by whichever page-node was nearest to it, with visible border lines wherever two differently-coloured cells touched.

This worked, and had real strengths: a stable, dense coordinate grid that every rendering feature (hex tiles, contended borders, terrain, road pathfinding) could share a single contract against. Its cost, though, was structural rather than accidental — see below.

What the K-Blade Version Does Instead

The live kblade_atlas.py pipeline represents each page as a unit embedding vector (nomic-embed-text, 768-dim, cached), and computes pairwise similarity between every pair of pages using a k-blade generalized sine — the product of the sines of the principal angles between two subspaces, which reduces to the classic bivector norm for single vectors. Pages are grouped into clusters by modularity-maximizing graph communities over a sparsified top-K similarity graph, then laid out with a force-directed spring layout (seeded from an MDS projection of the k-blade distances) so that pages in the same cluster actually end up spatially close together, not just similarity-adjacent. Cluster territories are rendered as real convex hulls around each cluster's member coordinates — not grid-cell colouring. Terrain (mountain/forest/swamp) comes from a sparse jittered point sample tested against those hulls: a sample point outside every hull is a mountain (a genuine conceptual gap), inside exactly one hull is forest (settled, single-topic ground), and inside two or more hulls is swamp — a direct, visual rendering of the overlapping-membership finding below. Roads are a minimum spanning tree over physical canvas distance, restricted to edges that are also real similarity-graph edges, plus one bridge edge between every pair of adjacent clusters.

Why Switch: What the Papers Found

Three research papers (linked below, in Research Papers) document the investigation that led here, and are worth reading in full for the actual numbers and the failure modes along the way — summarized:

Geometric algebra needed the right algorithm, not just the right metric. An initial attempt paired k-blade similarity with the conventional greedy agglomerative merge and got a strictly worse hierarchy than plain cosine similarity — direct inspection showed a genuine chaining pathology, not a benchmarking artifact. Swapping only the clustering algorithm (to modularity communities, or a rate-of-change growth procedure), while holding the same k-blade similarity values fixed, recovered performance matching the best cosine baseline. The lesson that shaped the production engine: representation and algorithm are separable design choices, and kblade_atlas.py uses graph-community clustering specifically because that is the combination the research validated.
Pages are legitimately multi-topic, and it's measurable, not just anecdotal. Between 34% and 46% of pages, depending on method, genuinely qualify for membership in more than one cluster under a fixed affinity-margin rule. The live atlas surfaces this directly as swamp terrain wherever cluster hulls overlap, rather than forcing every page into exactly one Voronoi cell the way the SOM version did.
The speed difference is not a rounding error. A cold SOM training run (cache bypassed) took 802.1 seconds on the site's 160-page corpus, 99.4% of it in the 30-epoch neuron training loop itself; the equivalent k-blade similarity-plus-clustering pipeline took 230 milliseconds — roughly 3,485× faster. The mechanism is structural: the SOM's cost is grid-size×epochs (fixed, independent of corpus size), while the k-blade pipeline's cost is corpus-size-scaling pairwise linear algebra that stays cheap at this corpus's scale. This is what makes rebuilding the atlas on every ordinary site build practical, rather than something to cache aggressively and dread invalidating.

The efficiency paper is explicit that this was not, at the time, a fully like-for-like comparison — the SOM's U-Matrix and fixed grid gave rendering features (terrain elevation, road pathfinding) that the k-blade pipeline, as it stood then, did not yet replace. Building the convex-hull territories, sparse terrain sampling, and MST road network described above is that replacement work, carried out directly in kblade_atlas.py.

Bridge Text: How the "Wayfarer Reading Guidance" Prose Gets Written

Every road on the map, and every pair of pages in the same cluster, gets a short contextual paragraph explaining the connection — the text that appears when you hover a road or click through from one article to a related one. Writing one of these by hand for every pair was never realistic: on the current 154-page corpus, the pipeline generates 4,520 of them. Instead, som_atlas.generate_pair_bridge (shared by both the retired SOM path and the live k-blade path) builds each paragraph from three real, per-pair signals rather than a fixed template:

Similarity-graded connective language. Every pair carries a real 0–1 strength — the same k-blade generalized-sine similarity used for clustering, for pairs the k-blade pipeline generates; a concept-overlap-ratio proxy for the retired SOM path, which never stored a per-pair score. That strength selects one of three phrasing registers ("closely extends" / "builds on" / "loosely connects to"), so a strongly related pair and a weakly related one don't read in identical language.
A small, deterministically-chosen template pool. Each of the three structural cases (a same-cluster road, a same-cluster non-road pairing, a cross-cluster pairing) has 3–4 sentence-structure variants. Which variant a given pair gets is chosen by hashing the pair's own page ids (md5(from_id::to_id)) — reproducible across rebuilds (the same pair always reads the same way) while still varying prose across thousands of pairs, without hand-authoring a blurb for each one.
Real cluster identity as the throughline. Rather than falling back to a generic phrase like "specialized domain methodologies" when two pages don't share explicit keywords, the live pipeline passes each page's actual k-blade cluster label — itself derived from the real title-word frequency across that cluster's member pages (e.g. "PROGRAMMING & NAVIGATION", "INTERVAL & TESTING"), not a hand-picked category name. About a quarter of all generated bridges use this cluster-aware framing directly, e.g. "You're leaving the PROGRAMMING & NAVIGATION country for INTERVAL & TESTING territory..." — language that reflects the site's actual discovered structure rather than a fixed, hand-maintained category list.

None of this required a language model at request time: it's deterministic template assembly over data the atlas generator already computed for clustering and layout, run once at build time.