Algorithm, Not Metric: Why Pairwise and K-Blade Geometric Algebra Appeared to Fail at Concept Hierarchy Construction, and What Fixed It

Authors: Pat Parslow (Draft)

Experiments were run interactively with AI-assisted tooling (code generation, execution, and result inspection); all reported numbers are from real runs against the site's own 160-page corpus, not simulated or illustrative. Code referenced throughout is in tools/site/concept_experiments/.

Abstract

WeWe investigate whether geometric algebra (GA) — specifically, representing document embeddings as blades and clusters as oriented subspaces — offers a genuine advantage over plain cosine similarity for building a hierarchical concept model from a small (160-page) real-world corpus. An initial series of experiments (A–C) using pairwise bivectors and rank-r subspace blades, combined with greedy bottom-up agglomerative merging, produced a uniformly negative result: GA-derived hierarchies were substantially worse than a plain cosine baseline on every quality metric tested, and direct inspection of the resulting trees confirmed a genuine chaining pathology rather than a measurement artifact. A follow-up series (D–E) tested whether this failure was attributable to the merge algorithm rather than the similarity metric, by pairing the same GA-derived similarities with two different global clustering procedures: modularity-maximizing graph communities, and a rate-of-change densest-subgraph growth algorithm. Both interventions substantially reversed the negative result. Under the best-tuned growth-clustering configuration, the GA-derived (k-blade, rank≤3) hierarchy reached an Adjusted Rand Index (ARI) of 0.634 against a folder-based ground truth, matching the best agglomerative cosine baseline (0.638) and clearly outperforming a mean-vector bivector variant (0.093–0.109) tested earlier with the same greedy algorithm. We conclude that the earlier negative verdict on GA for this task was largely an artifact of algorithm choice, not a property of the geometric representation itself, and we document two independent, reproducible fixes (average- rather than total-weight growth criteria; z-scored rather than raw similarity) that were both necessary before the effect could be observed at all.

1. Introduction

GeometricGeometric algebra represents oriented k-dimensional subspaces (k-blades) as first-class algebraic objects, generalizing the notion of a single direction vector to genuine regions or "volumes" in a vector space. This is an appealing framework for concept modelling: individual documents are naturally single vectors, but a coherent topic or cluster of documents plausibly occupies a genuine subspace, not just a mean direction. We set out to test this idea concretely, against a real corpus, rather than treat it as a purely theoretical proposal.

The corpus is 160 pages from a personal/portfolio and teaching website (parslow.net), spanning topics from programming-language internals to ethics, security, and geometric algebra itself. Each page is embedded with nomic-embed-text (768-dim), and the site's own directory structure provides a convenient — though, as Section 6 discusses, imperfect — ground-truth topic labelling for quantitative evaluation.

The investigation proceeded in two clearly separated phases. Phase 1 (Sections 2–4, Tests A–C) evaluated GA representations of increasing richness — from a closed-form pairwise bivector norm up to genuine rank-r subspace blades per cluster — using the conventional algorithm for building a hierarchy from a pairwise or subspace distance: greedy bottom-up agglomerative merging. Every configuration in Phase 1 underperformed a plain cosine baseline, in several cases severely. Phase 2 (Section 5, Tests D–E) asked whether this was a property of the metric or the algorithm, and found decisively that it was the algorithm: the same underlying similarity values, paired with two different global (not greedy-local) clustering procedures, produced results competitive with the cosine baseline.

2. Background

2.1 Bivectors and the Lagrange identity

ForFor two vectors a, b, the geometric product decomposes as

\[ ab = a \cdot b + a \wedge b \]

where \(a \cdot b\) is the familiar scalar (cosine-related) inner product and \(a \wedge b\) is the bivector (outer/wedge) part. For unit vectors, the Lagrange identity gives an exact closed form for the bivector's norm without constructing any general multivector algebra:

\[ \lVert a \wedge b \rVert^2 = 1 - (a \cdot b)^2 \]

i.e. \(\lVert a \wedge b \rVert = \sin\theta\) where \(a \cdot b = \cos\theta\). This is a plain \(O(n^2)\) NumPy computation at any dimension n, with no basis-blade enumeration and no risk of the exponential (\(2^n\)) cost that general-purpose GA libraries (e.g. kingdon) incur once n exceeds roughly 20 — measured directly in this work at 17s and >11GB RAM for a single Algebra(22) construction, before being terminated.

2.2 From bivectors to k-blades: generalized principal angles

The two-vector wedge generalizes to a genuine k-blade by representing a set of vectors (e.g. all members of a cluster) by an orthonormal basis \(A \in \mathbb{R}^{d \times r}\) of its r dominant directions (the top-r right singular vectors of the member matrix). Given two such bases \(A\) (rank \(r_A\)) and \(B\) (rank \(r_B\)), the singular values of \(A^\top B\) are the cosines of the principal angles between the two subspaces; the product of the corresponding sines is the natural generalization of the two-vector bivector norm to genuine k-blades:

\[ \sigma(A,B) = \prod_{i=1}^{\min(r_A,r_B)} \sqrt{1 - \cos^2\theta_i}, \qquad \cos\theta_i = \text{singular values of } A^\top B \]

This reduces exactly to the Lagrange-identity bivector norm when \(r_A = r_B = 1\), is computed via a small \((r \times r)\)-scale SVD (cost \(O(r^2 d)\), never \(O(2^d)\)), and requires no dimensionality reduction of the underlying embeddings — the full 768-dim vectors are used directly. This is the "sparse where appropriate, dense where it matters" design used throughout: no blanket PCA is applied up front (which would throw away information before it is needed), but each cluster's own basis is a small, cheap, per-cluster computation.

3. Corpus, Ground Truth, and Metrics

160 pages, embedded with nomic-embed-text (chosen over mxbai-embed-large after a per-signal AUC comparison: 0.8924 vs 0.7689 on a same-folder/different-folder pair-discrimination benchmark, with mxbai also roughly 3x slower — see Appendix). Ground truth is the site's own directory structure (17 folders). Section 6 presents direct evidence that this ground truth is imperfect — several folder placements reflect an editorial/organisational decision (e.g. an article about a technique being filed under "learning and teaching" advice rather than under the technique's own topic area) rather than a single-topic classification, and pages are frequently and legitimately multi-topic.

Quality is reported via Adjusted Rand Index (ARI) and Normalized Mutual Information (NMI) against this folder ground truth, and cophenetic correlation (agreement between a dendrogram's implied distances and the original distance matrix) where a hierarchy, rather than a flat partition, is being evaluated.

4. Phase 1: Pairwise and Subspace Blades with Greedy Agglomerative Merging

4.1 Test A — pairwise bivectors carry no information beyond cosine

ComputingComputing the exact bivector norm (Section 2.1) at the full, unreduced 768 dimensions and using it to rank same-folder vs. different-folder page pairs gives AUC = 0.8924 — numerically identical to raw cosine similarity's AUC on the same pairs (also 0.8924), and identical again to a naive combination of the two signals. This is explained, not merely observed: because \(\sin\theta\) is a monotonic function of \(\cos\theta\) whenever the sample's pairwise cosines share a consistent sign (true throughout this corpus), the bivector norm is a strictly rank-equivalent relabelling of cosine similarity for a two-vector wedge. Increasing the retained embedding dimensionality (the original motivation for this test) therefore cannot help — a two-vector wedge structurally cannot carry information a cosine similarity does not already carry.

4.2 Test B — mean-vector bivector merge is worse than cosine

Building an agglomerative hierarchy by repeatedly merging whichever two clusters have the smallest bivector norm between their mean direction vectors (the natural direct generalization of Test A to a hierarchy) performed markedly worse than average-linkage on cosine distance:

	cophenetic corr.	ARI (k=17)	NMI (k=17)	silhouette
baseline (cosine avg-linkage)	0.793	0.638	0.718	+0.139
mean-vector bivector merge	0.578	0.109	0.303	−0.013

4.3 Test C — genuine rank-r k-blades do not fix it, at any rank

Replacing the mean-vector representation with a genuine rank-r orthonormal-basis blade per cluster (Section 2.2), merged by the generalized-sine criterion, produced a smaller but still substantial gap, and — critically — the gap did not close as rank increased:

rank	build time (s)	cophenetic corr.	ARI (k=17)	NMI (k=17)
1	11.67	0.5815	0.1088	0.3028
2	13.47	0.6690	0.0919	0.2675
3	13.14	0.6467	0.0928	0.2741
5	13.47	0.6497	0.0768	0.2535
8	13.40	0.6287	0.0884	0.2612
12	14.82	0.6372	0.0403	0.1556
20	16.03	0.6374	0.0729	0.2441
30	15.52	0.6184	0.0663	0.2323

Cophenetic correlation peaks around rank 2–3 and then plateaus or mildly declines; ARI/NMI against the folder ground truth stay uniformly low across the entire sweep, with no rank-dependent trend at all. "Give the blade more of its own data" (the premise motivating the rank sweep) is therefore not the missing ingredient.

4.4 Direct inspection: a genuine chaining pathology, not a bad cut height

Two direct checks confirmed that the poor quantitative scores reflected a real structural problem in the tree, not merely an unlucky choice of where to cut it for the k=17 comparison. First, the top of the k-blade dendrogram, examined directly, merges exactly one leaf at a time into an ever-growing supercluster (a 159-member cluster gains one leaf to become 158…157…156, and so on) — the textbook signature of single-linkage-style chaining. Second, cutting the tree at its own single largest merge-distance gap (0.149→0.331, roughly four times any neighbouring gap) — rather than at an arbitrary k — produced 154 clusters out of 160 pages: almost every page as its own singleton, with only a handful of 2–3-page clusters (largely literal duplicate pages) below the gap. There is no useful intermediate structure hiding beneath a bad cut point; the merge criterion itself has essentially bimodal discriminative power — adequate for near-duplicates, uninformative for everything else — when paired with greedy nearest-pair merging.

5. Phase 2: Isolating Algorithm from Metric

GreedyGreedy bottom-up agglomerative merging repeatedly commits to "merge the single closest remaining pair," with no notion of overall graph structure — exactly the mechanism that produces chains. Phase 2 tests the same underlying k-blade (rank≤3) similarity values under two structurally different, non-greedy-pairwise algorithms.

5.1 Test D — modularity-maximizing graph communities

Each page was connected to its top-K most similar neighbours in a weighted graph (sparsified rather than dense — a fully connected graph has no genuine community structure to discover), and partitioned using greedy modularity maximization (Clauset–Newman–Moore, via networkx). Modularity maximization is itself a global objective — comparing a candidate community's internal edge weight against a chance expectation given node degree — rather than a literal heaviest-subgraph search, but it is not vulnerable to the same one-nearest-neighbour chaining failure as agglomerative merging.

	#communities	ARI	NMI
k-blade (rank≤3), K=8	8	0.3629	0.5538
cosine, K=8	5	0.3344	0.5415

A sweep of K ∈ {4,6,8,10,15,20,30} confirmed this was not a fortunate choice of K: k-blade tracks cosine closely across the whole range, ahead at some values (e.g. K=8) and behind at others (e.g. K=15: cosine 0.514 vs. k-blade 0.368), but never collapsing to Test C's near-zero ARI.

5.2 Test E — rate-of-change densest-subgraph growth

A second, more direct answer to "when is a cluster complete" avoids any fixed hyperparameter (k or top-K) entirely: seed a cluster with the most globally central unclaimed page, then repeatedly add whichever unclaimed page has the highest similarity to the current cluster, tracking the marginal gain at each step; stop growing when that marginal gain drops sharply relative to the previous step (an elbow), rather than at a predetermined size. This is a greedy densest-subgraph-style growth procedure, in contrast to Test D's global modularity objective.

Two implementation details were necessary before this procedure produced any structure at all, and are reported here because both failures are non-obvious and easy to reproduce by omission:

• Total vs. average weight. Scoring candidates by their total similarity to the current cluster inflates the score for any large cluster regardless of fit quality — even a weak match accumulates a large sum once the cluster has many members — so no elbow ever appears and the procedure degenerates to one all-consuming cluster at every threshold tested (0.3–0.9). Scoring by average similarity (the standard densest-subgraph density measure) is size-invariant and restores a real, sharply-varying signal.
• Raw vs. z-scored similarity. Raw cosine similarity for this embedding model occupies a narrow band (~0.70–0.86 across nearly all pairs in this corpus — the same anisotropy "floor" documented earlier in this project's broader embedding-ensemble work) that decays too gently for any elbow to be detectable. Z-scoring each similarity matrix against its own corpus-wide mean and standard deviation before growth restores the necessary contrast.

With both fixes applied, sweeping the elbow-sensitivity ratio (the threshold below which a gain drop counts as an elbow):

ratio	k-blade #clusters	k-blade ARI	k-blade NMI	cosine #clusters	cosine ARI	cosine NMI
0.9	70	0.0494	0.5967	41	0.6325	0.6925
0.8	38	0.6338	0.6958	35	0.5707	0.6625
0.7	28	0.4753	0.6417	20	0.4493	0.5999
0.6	17	0.4691	0.6056	16	0.4510	0.5959
0.5	17	0.4691	0.6056	15	0.4288	0.5604
0.4	17	0.4560	0.5615	11	0.4243	0.5371
0.3	15	0.4490	0.5582	11	0.4243	0.5371

Two honest observations follow from this table, and both matter. First, at their respective best operating points, k-blade (ratio 0.8: ARI 0.634) and cosine (ratio 0.9: ARI 0.633) are essentially tied with each other, and both are essentially tied with the best agglomerative-cosine baseline from Phase 1 (0.638) — this is the central positive result of Phase 2. Second, the two metrics do not share a best operating point (0.8 vs. 0.9), and each is markedly worse at the other's optimum (k-blade collapses to ARI 0.049 at ratio 0.9; cosine drops to 0.571 at ratio 0.8) — so this is a claim of parity under correct tuning, not of one representation dominating the other, and the elbow-ratio is a real, sensitive hyperparameter for this procedure rather than a robust default.

Manual inspection of the k-blade clustering at its best setting found genuinely coherent, non-folder-aligned groupings, including a 4-page cluster grouping a "triangulation" page (filed, by folder, under general project-advice) together with three explicitly spatial-cartography pages — recovering, from an entirely different algorithm, a specific cross-folder relationship anticipated on inspection grounds in Section 6.

6. Is the Ground Truth Trustworthy?

AA direct, manual comparison of cluster membership against folder membership (rather than only the summary ARI/NMI statistics) surfaced several cases where the folder structure, not the discovered hierarchy, is the less reliable signal. A page titled "programming," filed under a folder about games, is placed by the cosine hierarchy together with object-oriented-programming fundamentals pages — a more topically accurate placement. A "triangulation" page, filed under general project advice, is placed by both the cosine baseline and, independently, the Test E k-blade growth-clustering, together with the site's spatial-cartography pages, reflecting that its content concerns the same underlying spatial technique even though its folder reflects a pedagogical framing decision. A cluster spanning three different AI/ethics/education folders (chatbot healthcare ethics, accessibility and learning, several "AI in higher education" pages) is grouped coherently by the cosine hierarchy despite being split across folders by the site's own navigation structure.

This matters for how the entire quantitative comparison in this paper should be read: agreement with the folder ground truth is a reasonable, cheap proxy, but disagreement is not automatically evidence of a worse hierarchy — some of the "errors" found by ARI/NMI are, on inspection, the discovered hierarchy correctly reflecting that pages are legitimately multi-topic, which the single-folder-per-page ground truth cannot represent. The one place this caveat does not rescue a result in this paper is Test C's degenerate k-blade-agglomerative tree (Section 4.4): direct inspection there showed a real structural failure (near-total chaining, no usable intermediate clusters at any cut height), not a defensible alternative structure being penalized by an imperfect ground truth.

7. Discussion

TheThe overall trajectory of this investigation is itself the main finding. A geometric representation (pairwise bivectors, then genuine rank-r k-blades) was tested, found uniformly wanting against a strong baseline, and diagnosed by direct inspection as suffering a genuine structural pathology (chaining) rather than a benchmarking artifact — a legitimate basis, at that point, for concluding the representation itself was the problem. That conclusion turned out to be wrong, or at least incomplete: swapping only the clustering algorithm, while holding the k-blade similarity values fixed, recovered performance competitive with cosine under two independently-implemented, structurally different global algorithms (modularity communities; rate-of-change densest-subgraph growth). The representation and the algorithm used to consume it are separable design choices, and an evaluation that only ever tests one algorithm per representation risks attributing an algorithmic weakness to the representation.

A second, more specific observation: greedy agglomerative merging's chaining failure mode is not unique to GA-derived metrics in principle, but this corpus's k-blade similarity happened to expose it far more severely than cosine did (Test B/C's ARI of ~0.09–0.11 vs. cosine's 0.638 under the identical algorithm). One plausible explanation, consistent with Test A's finding that generalized-sine metrics saturate differently near \(\theta \approx 0\) than cosine does, is that the k-blade metric's discrimination is concentrated in a different, narrower distance range than cosine's, making it more sensitive to exactly the kind of "always take the single nearest neighbour" myopia that greedy agglomerative merging exhibits. This is offered as a hypothesis, not a proven mechanism; distinguishing it from other explanations was out of scope for this investigation.

Both fixes required to make Test E's growth-clustering procedure work at all — average- rather than total-weight scoring, and z-scored rather than raw similarity — are, in retrospect, generic prerequisites for any rate-of-change/elbow-based stopping rule on this kind of data, not specific to GA. We record them here explicitly because both failure modes (a monotonically-growing score that never elbows; a too-flat similarity distribution that never elbows) produce the same visible symptom — one giant undifferentiated cluster — and are easy to misattribute to "the elbow idea doesn't work" rather than to a fixable measurement choice.

8. Limitations

• The corpus is small (160 pages) and drawn from a single site with a particular authorial voice; generalization to larger, more heterogeneous corpora is untested.
• The folder-based ground truth, while shown to be imperfect (Section 6), was still used for all quantitative comparisons; no alternative, more granular ground truth (e.g. human-annotated multi-label topics) was constructed.
• Test E's elbow-ratio is a sensitive hyperparameter with different optima for different similarity metrics (Section 5.2); no principled, data-driven method for selecting it per-metric was developed here.
• Only one embedding model (nomic-embed-text) was used for the Phase 2 experiments; whether the algorithm-vs-metric conclusion holds for other embedding spaces is untested.
• "Genuine k-blade" here means a subspace basis of a cluster's dominant directions, not a wedge of the cluster's actual member vectors (which becomes degenerate once cluster size exceeds the ambient dimensionality's practical rank budget for a small corpus like this) — a scoping simplification noted at the time and not revisited.

9. Conclusion

PairwisePairwise bivectors carry no ranking information beyond cosine similarity for unit vectors (an exact mathematical result, not an implementation limitation), and genuine rank-r k-blades, evaluated via the conventional greedy agglomerative merge algorithm, produced hierarchies that were not merely unhelpful but structurally degenerate. Both findings would, on their own, support abandoning the geometric-algebra approach to this task. However, holding the same k-blade similarity values fixed and substituting two different global clustering algorithms for greedy agglomerative merging recovered performance matching the best cosine baseline found anywhere in this investigation. The correct conclusion is therefore not "geometric algebra does not help here," but "greedy nearest-pair agglomeration was an unusually poor algorithmic match for this particular metric, on this corpus" — a conclusion only reachable by treating algorithm and metric as separately falsifiable, and by directly inspecting failed results rather than accepting a summary statistic at face value.

Appendix: Reproducibility

AllAll code referenced is in tools/site/concept_experiments/: ga_bivector_exact.py (Test A), ga_native_hierarchy.py (Test B), ga_kblade_hierarchy.py / ga_kblade_rank_sweep.py (Test C), ga_kblade_graph_communities.py / ga_kblade_topk_sweep.py (Test D), ga_kblade_growth_clustering.py (Test E), dump_hierarchy_comparison.py / inspect_kblade_tree_structure.py / kblade_natural_cut.py (Section 4.4/6 manual inspection). All figures in this paper are read directly from these scripts' stdout on the cached embedding data (ensemble_cache.npz) for the 160-page corpus described in Section 3; no numbers are estimated or illustrative.