ConceptGA: Geometric Algebra for Concept Representation and Reasoning
Abstract
We present ConceptGA, a novel framework for representing and manipulating concepts using k-blades from Geometric Algebra instead of traditional vector embeddings. Unlike standard embeddings where concepts are points on a hypersphere, ConceptGA represents concepts as directed volumes (k-blades) where grade corresponds to abstraction level and orientation encodes asymmetric relations.
1. Introduction
Standard embeddings map concepts to vectors on a hypersphere — similarity is cosine angle. This is the representation behind word2vec-style models, where relationships between concepts show up as vector offsets: the well-known result that vec(king) - vec(man) + vec(woman) lands near vec(queen) demonstrated that a single linear operation, vector addition, could capture at least some relational structure in an embedding space[1]. That result is also the ceiling of what a flat vector space can express: every relation, whether IS-A, PART-OF, or CAUSES, has to be encoded as the same kind of offset, with no way to distinguish the type of relation from its magnitude, and no native operation for asymmetric or compositional structure. This has limitations:
| Limitation | Vector Embeddings | ConceptGA (k-blades) |
|---|---|---|
| Compositionality | Linear superposition only | Geometric product encodes relations (meet/join) |
| Hierarchy | Flat | Grade = conceptual abstraction level |
| Directionality | Single orientation | Edge directions = asymmetric relations (IS-A, PART-OF, CAUSES) |
| Negation | -v = opposite point | Dual = orthogonal complement = "not this concept" |
| Intersection | Not defined | Meet (\(\wedge\)) = shared subspace = common substructure |
2. Algebraic Foundations
ConceptGA is built directly on Geometric Algebra (GA), the coordinate-free algebra of subspaces developed from Hermann Grassmann's exterior algebra and William Kingdon Clifford's algebra, and given its modern, unified treatment by David Hestenes[2]. GA extends ordinary vector algebra with a single associative geometric product that simultaneously carries both the metric (inner-product) and the orientation/incidence (outer-product) content of two vectors. Where standard linear algebra needs separate machinery for dot products, cross products, determinants, and rotations, GA expresses all of these as instances of one product acting on elements of different grade. This is what makes it a natural fit for concept representation: a single algebraic operation can carry similarity (grade-0 part), relation structure (grade-2 and higher parts), and orientation (sign) at once, rather than requiring bespoke similarity, relation, and negation operators bolted onto a flat vector space.
2.1 Geometric Algebra G(V, Q)
Let V be an n-dimensional vector space. The geometric product of vectors a, b:
\[ ab = a \cdot b + a \wedge b \]
where a·b is the symmetric inner product and a∧b is the antisymmetric outer product. The standard modern reference for this algebra and its computational treatment — including the blade, meet, join, and dual operations used throughout this specification — is Dorst, Fontijne & Mann's Geometric Algebra for Computer Science[3].
2.2 k-Blades as Concepts
A k-blade is the outer product of k linearly independent vectors:
\[ B_k = v_1 \wedge v_2 \wedge \dots \wedge v_k \]
- Grade k = abstraction level (1 = atomic, higher = composite)
- Magnitude = salience/centrality of concept
- Basis indices = defining features
- Orientation = asymmetric relations (IS-A, PART-OF, CAUSES)
3. Core Operations
The meet and join operators below are the same operators Dorst, Fontijne & Mann define for combining subspaces in conformal geometric algebra[3], where meet computes the common subspace of two blades and join computes their span. ConceptGA reuses them unchanged, reinterpreting "subspace" as "shared conceptual substructure": two concept blades that share a sub-blade under meet share that substructure as a component of their meaning, exactly as two geometric objects that share a meet share a common point, line, or plane.
3.1 Similarity
\[ \operatorname{sim}(A, B) = \langle A \widetilde{B} \rangle_0 \quad \text{(scalar part of geometric product)} \]
3.2 Meet (Intersection)
\[ A \vee B = \langle A^{*} \cdot B \rangle \quad \text{(dual of } A \text{ contracted with } B\text{)} \]
3.3 Join (Union)
\[ A \wedge B \quad \text{(outer product)} \]
3.4 Dual (Negation)
\[ \neg A = A^{*} = A I^{-1} \quad \text{(dual w.r.t. pseudoscalar)} \]
4. Reasoning Operations
4.1 Analogy
\[ \begin{aligned} R_{A \to B} = B A^{-1} \quad \text{(versor transformation)} \\[2pt] D = R_{A \to B}\, C = B A^{-1} C \end{aligned} \]
4.2 Abstraction/Specification
\[ \begin{aligned} \operatorname{abstract}_j(A) = \langle A \rangle_j \quad \text{(drop features)} \\[2pt] \operatorname{specify}(A, F) = A \wedge F \quad \text{(add features)} \end{aligned} \]
5. CKB Integration
Each CKB belief becomes a concept blade:
"Vaccines reduce severe disease"
→ e_vaccine ∧ e_reduce ∧ e_severe ∧ e_disease ∧ r_CAUSES
Surprise as prediction error:
\[ \operatorname{surprise} = 1 - \operatorname{sim}_w(P, O) \]
Counterfactuals:
\[ \mathrm{CF}_{\neg F}(B) = B \wedge \neg F \]
6. Evaluation
| Metric | Target |
|---|---|
| Belief similarity nDCG@10 | > Vector baseline |
| Contradiction detection F1 | > Textual entailment |
| Analogy completion accuracy | > Word2vec/GPT |
| Surprise prediction AUC | > Heuristic |
The analogy-completion baseline is the vector-offset method from Mikolov, Yih & Zweig[1]: ConceptGA's versor-transformation analogy operator (§4.1) is a direct generalisation of that vector-offset arithmetic — instead of a single subtraction/addition applied to points, it composes an actual transformation (a versor) derived from one pair of blades and applies it to a third, which should degrade less as relations become asymmetric or as the objects being related rise in grade above simple atomic vectors.
7. Conclusion
ConceptGA provides a mathematically rigorous, compositional framework for concept representation that naturally supports hierarchy, relations, negation, and reasoning operations — all grounded in the single algebra that also powers spatial reasoning (SGA engine). The central bet of this specification is that the limitations of flat vector embeddings identified in §1 are not incidental but structural: no amount of extra dimensions or training data gives a point on a hypersphere a native notion of grade, orientation, or subspace intersection, because those concepts simply do not exist in that representation. Geometric algebra was developed, independently of any machine-learning motivation, specifically to unify such structure under one algebraic product[2]; ConceptGA's contribution is to test whether that unification, already established for physics and computer graphics, carries over to concept representation and reasoning. The evaluation criteria in §6 are the intended empirical test of that bet against the vector-embedding baselines it aims to surpass.
References
- T. Mikolov, W. Yih and G. Zweig, "Linguistic Regularities in Continuous Space Word Representations," Proceedings of NAACL-HLT 2013, pp. 746–751. https://aclanthology.org/N13-1090/
- D. Hestenes, New Foundations for Classical Mechanics, 2nd ed., Kluwer Academic Publishers, 2003. https://link.springer.com/book/10.1007/0-306-47122-1
- L. Dorst, D. Fontijne and S. Mann, Geometric Algebra for Computer Science: An Object-Oriented Approach to Geometry, Revised Edition, Morgan Kaufmann, 2009. https://www.geometricalgebra.org/