"The atoms are the phonemes of thought. Concepts are words."
We discovered that flipping the direction of the DTO transformation dramatically improves fidelity:
| Approach | Direction | Fidelity |
|---|---|---|
| Old | 10KD → compress → 1024D → expand → 10KD | 62.5% |
| New | 1024D → expand → 10KD → compute → compress | 95-99% |
The key insight: 1024D qualia should be the source of truth, not a projection of 10KD.
When concepts are defined as SPARSE combinations of atoms (~10-50 active out of 1024), round-trip fidelity approaches near-perfect levels.
Random projection (Johnson-Lindenstrauss) preserves relative distances but not exact values.
Each 1024D output dimension = weighted sum of ALL 10000 input dimensions
Information is smeared across all dimensions. When we invert:
- ~50% of bits are random (no signal)
- ~12.5% signal survives
- Total fidelity ≈ 62.5%
10KD → 1024D is a 10:1 compression.
Best case (perfect separation):
- 1024 independent "atoms"
- Each atom encodes 10 bits of detail
- Round-trip: preserve majority of each block
Worst case (random mixing):
- All 10000 dims contribute to all 1024 outputs
- Inversion is ill-conditioned
- Fidelity collapses to random + signal ≈ 62.5%
Instead of treating 10KD as "real" and 1024D as "projection", flip it:
1024D is "real" (the clean atom space)
10KD is "expansion" (the computation workspace)
┌─────────────────────────────────────────────────────────────────┐
│ 1024D QUALIA SPACE │
│ (Source of Truth) │
│ │
│ Concepts defined as SPARSE atom combinations: │
│ cat = [0.1, 0, -0.5, 0, 0, 0.8, 0, ...] (~20 active) │
│ on = [0, 0.3, 0, 0, 0.2, 0, 0, ...] (~10 active) │
│ mat = [0, 0, 0.4, 0, 0, 0, -0.3, ...] (~15 active) │
│ │
│ Operations: search, blend, communicate, learn │
└───────────────────────────────┬─────────────────────────────────┘
│
EXPAND (deterministic)
expansion @ qualia
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ 10KD RESONANCE SPACE │
│ (Computation Workspace) │
│ │
│ Binary vectors for exact computation: │
│ cat_10k = expand(cat) → 1250 bytes packed │
│ on_10k = expand(on) → 1250 bytes packed │
│ mat_10k = expand(mat) → 1250 bytes packed │
│ │
│ Operations: XOR bind, Clean Room, Triple Rub, NARS │
└───────────────────────────────┬─────────────────────────────────┘
│
COMPRESS (recover)
pinv(expansion) @ bipolar
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ RESULT IN 1024D │
│ │
│ triple = compress(cat_10k ⊕ on_10k ⊕ mat_10k) │
│ (~40 active atoms, ~95% fidelity) │
└─────────────────────────────────────────────────────────────────┘
Sparse vectors have low interference.
When we expand a sparse 1024D vector to 10KD:
- Each active atom contributes a unique "signature" to the 10KD space
- Few active atoms → signatures don't overlap much
- Compression can identify which atoms were active
Fidelity vs Sparsity (empirically measured):
| Active Atoms (k) | Round-Trip Fidelity |
|---|---|
| 10 | 99.98% |
| 25 | 99.32% |
| 50 | 98.70% |
| 100 | 97.81% |
| 200 | 96.16% |
Approximate formula:
fidelity ≈ 1 - 0.01 * k (for k << 1024)
We only need ~10^6 concepts (human vocabulary size).
If each concept uses ~50 atoms from 1024:
C(1024, 50) ≈ 10^93 possible concepts
This is astronomically more than we need. The atom space is not a bottleneck.
| Level | Qualia | Language |
|---|---|---|
| Atoms | 1024 basis vectors | ~40 phonemes |
| Concepts | Sparse combinations | Words |
| Composites | TRIPLE bindings | Sentences |
Just as words are sparse combinations of phonemes, concepts are sparse combinations of atoms.
class CleanQualiaDTO:
"""
Bidirectional 1024D ↔ 10KD with high fidelity for sparse vectors.
1024D is the SOURCE OF TRUTH.
10KD is the COMPUTATION WORKSPACE.
"""
def __init__(self, seed=42):
rng = np.random.RandomState(seed)
# Expansion matrix: (10000, 1024)
# Each column is an atom's "signature" in 10KD
self.expansion = rng.randn(10000, 1024).astype(np.float32)
# Normalize columns
norms = np.linalg.norm(self.expansion, axis=0, keepdims=True)
self.expansion /= norms
# Compression: pseudo-inverse
self.compression = np.linalg.pinv(self.expansion)
def expand(self, qualia):
"""1024D sparse → 10KD binary (deterministic)"""
expanded = self.expansion @ qualia
bits = (expanded > 0).astype(np.uint8)
return np.packbits(bits)
def compress(self, resonance):
"""10KD binary → 1024D sparse (high fidelity for sparse)"""
bits = np.unpackbits(resonance)[:10000].astype(np.float32)
bipolar = bits * 2 - 1
return self.compression @ bipolar
def round_trip_fidelity(self, qualia):
"""Measure fidelity for this specific qualia."""
resonance = self.expand(qualia)
recovered = self.compress(resonance)
return cosine_similarity(qualia, recovered)| Operation | Time (1000 vectors) | Throughput |
|---|---|---|
| Batch expand | ~155ms | 6.5K/sec |
| Batch compress | ~89ms | 11K/sec |
| Cosine search | ~4ms | - |
# Base concepts: very sparse (~10-20 atoms)
cat = np.zeros(1024)
cat[[42, 107, 256, 512, 789, 823, 901, 945, 999, 1001]] = np.random.randn(10)
dog = np.zeros(1024)
dog[[42, 107, 256, 333, 444, 555, 666, 777, 888, 1010]] = np.random.randn(10)
# Note: cat and dog share some atoms (42, 107, 256) - they're both animals!
on = np.zeros(1024)
on[[10, 20, 30, 40, 50]] = np.random.randn(5) # Relations are sparserdto = CleanQualiaDTO()
# Expand to workspace
cat_10k = dto.expand(cat)
on_10k = dto.expand(on)
mat_10k = dto.expand(mat)
# Bind using XOR (exact)
ROLE_S = random_binary_10k()
ROLE_R = random_binary_10k()
ROLE_O = random_binary_10k()
triple_10k = xor(xor(xor(cat_10k, ROLE_S), xor(on_10k, ROLE_R)), xor(mat_10k, ROLE_O))
# Compress back to 1024D
triple_1024 = dto.compress(triple_10k)
# ~40 active atoms, ~95% fidelity# Store in database
db.store(
id="cat_on_mat",
qualia=triple_1024, # For search, blend, communicate
resonance=triple_10k, # For further binding
sparsity=count_active(triple_1024),
fidelity=dto.round_trip_fidelity(triple_1024)
)The 10KD space is overcomplete for 1024 atoms. Each atom carves out a hyperplane in 10KD. Sparse combinations select intersection of hyperplanes. Compression identifies which hyperplanes were selected.
Dense vectors in 1024D → overlapping hyperplanes → hard to separate. Sparse vectors in 1024D → distinct hyperplane intersections → easy to separate.
This is why sparsity directly correlates with fidelity.
XOR binding in 10KD creates new hyperplane intersections. These are still separable if inputs were sparse. Fidelity degrades gracefully: k1 + k2 + k3 active atoms → fidelity(k1+k2+k3).
The Clean Room operation is essentially:
- Compress noisy 10KD → 1024D
- Find nearest KNOWN sparse concept
- Expand back to 10KD
This works because known concepts are sparse attractors in 1024D.
Per-dimension confidence = how strongly each atom is activated. High activation → confident about this "phoneme". Low activation → uncertain.
Sparse high-confidence = clean concept. Dense low-confidence = noisy/uncertain.
Visual cortex represents images as sparse combinations of basis functions. Our atoms play the same role for concepts.
Sparse signals can be recovered from fewer measurements than Nyquist. Our 1024D → 10KD → 1024D is a form of compressed sensing.
High-dimensional binary vectors for symbolic computation. We add the 1024D sparse layer for clean definitions.
Random projection preserves similarity. Our expansion matrix is a form of LSH, optimized for sparse inputs.
╔══════════════════════════════════════════════════════════════════════╗
║ DRAGONFLY-VSA ARCHITECTURE ║
╠══════════════════════════════════════════════════════════════════════╣
║ ║
║ ┌────────────────────────────────────────────────────────────────┐ ║
║ │ 1024D QUALIA SPACE │ ║
║ │ (Source of Truth) │ ║
║ │ │ ║
║ │ • Sparse atom combinations (~10-50 active) │ ║
║ │ • Clean concept definitions │ ║
║ │ • Vector DB storage (Upstash, Pinecone) │ ║
║ │ • Cross-model communication (GPT, Grok, Gemini) │ ║
║ │ • Smooth interpolation and blending │ ║
║ │ • Gradient-based learning │ ║
║ │ • 95-99% round-trip fidelity │ ║
║ └──────────────────────────┬─────────────────────────────────────┘ ║
║ │ ║
║ EXPAND │ COMPRESS ║
║ (lossless) (high fidelity) ║
║ │ ║
║ ┌──────────────────────────▼─────────────────────────────────────┐ ║
║ │ 10KD RESONANCE SPACE │ ║
║ │ (Computation Workspace) │ ║
║ │ │ ║
║ │ • Binary vectors (1250 bytes packed) │ ║
║ │ • XOR binding (exact, self-inverse) │ ║
║ │ • Clean Room filtering (attractor convergence) │ ║
║ │ • Triple Rub consensus (hallucination detection) │ ║
║ │ • NARS inference (PRODUCT, IMAGE, TRIPLE) │ ║
║ │ • AVX-512 accelerated (near-GPU speed) │ ║
║ │ • Deterministic, exact operations │ ║
║ └────────────────────────────────────────────────────────────────┘ ║
║ ║
║ Key Insight: Sparsity enables high-fidelity signal separation. ║
║ Atoms are phonemes. Concepts are words. Bindings are sentences. ║
║ ║
╚══════════════════════════════════════════════════════════════════════╝
Train the expansion matrix on real data to maximize:
- Separation between known concepts
- Sparsity of representations
- Round-trip fidelity
- Level 1: 256 "super-atoms" (very coarse features)
- Level 2: 1024 atoms (current level)
- Level 3: 4096 "sub-atoms" (fine details)
Multi-resolution representation for different fidelity needs.
Adapt sparsity based on concept complexity:
- Simple concepts: k=10 (99.9% fidelity)
- Complex concepts: k=100 (97% fidelity)
- Relations: k=5-20 (very sparse)
Atoms can shift over time as the system learns. New experiences refine atom signatures. The expansion matrix becomes a learned feature extractor.
The signal separation problem is solved by flipping the direction:
- Define concepts as sparse 1024D vectors (the atoms)
- Expand to 10KD for computation (deterministic)
- Compute using exact XOR operations (Clean Room, Triple Rub)
- Compress back to 1024D (high fidelity for sparse)
- Store both representations (1024D for search, 10KD for binding)
Sparsity is the key. With ~50 active atoms, we achieve 98%+ fidelity. The atoms are the phonemes of thought. Concepts are words. The 10KD workspace is where thinking happens. The 1024D qualia space is where meaning lives.
Created: 2026-01-24 Part of: Dragonfly-VSA v0.7.3 Repository: https://github.com/AdaWorldAPI/dragonfly-vsa