# Alternative Paradigm Research Inventories

**Created**: 2025-03-25
**Purpose**: Comprehensive research inventories for alternative AI/ML paradigms not currently covered by ARES

---

## Overview

This project contains structured research inventories for six alternative AI/ML paradigms that represent fundamental departures from current mainstream approaches (Transformers, backpropagation, pre-trained embeddings):

1. **Hyperdimensional Computing (HDC)**
2. **Vector Symbolic Architectures (VSA)**
3. **Sparse Distributed Representations (SDR)**
4. **Alternatives to Backpropagation**
5. **Compositional Generalization**
6. **Localist Representations**

---

## Why These Paradigms Matter

### Current ARES Coverage Gap

| Paradigm | Current ARES Coverage | Strategic Value |
|----------|----------------------|-----------------|
| HDC | None | High - 100-1000x parameter reduction possible |
| VSA | None | High - Guaranteed compositional generalization |
| SDR | Indirect (sparsity for efficiency only) | High - Brain-plausible semantic learning |
| Alt. Backprop | None | Critical - Training without massive compute |
| Comp. Generalization | None | Critical - Human-like reasoning |
| Localist | None | Medium - Alternative to distributed representations |

### Potential Impact on ARES Goals

- **Memory Efficiency**: HDC/VSA operate on high-dimensional vectors with simple operations
- **Reasoning**: Compositional generalization via algebraic operations
- **Developer Productivity**: No massive pre-training required; train from scratch
- **Consumer Hardware**: Can train meaningful models on 8GB GPUs or CPUs

---

## Directory Structure

```
alternative-paradigm-research-inventories/
├── README.md (this file)
├── MANIFEST.md (knowledge manifest)
├── topics/               # Detailed research briefs for each paradigm
│   ├── 01_hyperdimensional_computing.md
│   ├── 02_vector_symbolic_architectures.md
│   ├── 03_sparse_distributed_representations.md
│   ├── 04_alternatives_to_backpropagation.md
│   ├── 05_compositional_generalization.md
│   └── 06_localist_representations.md
├── papers/              # Paper summaries and key findings
│   ├── arxiv_seeds_2023_2026.txt  # Backfill seed list
│   └── categorized_paper_summaries.md
└── knowledge/           # Structured knowledge entries for ARES ingestion
    ├── topics_mechanisms_benefits.json
    └── research_questions_gaps.json
```

---

## Usage

### For ARES Research Planning

1. **Review topic briefs** in `topics/` to understand each paradigm
2. **Check backfill seeds** in `papers/arxiv_seeds_2023_2026.txt` for candidate papers
3. **Ingest structured knowledge** from `knowledge/` into ARES knowledge base
4. **Generate experiments** based on research questions and gaps

### For Backfill Cycle Targeting

The `papers/arxiv_seeds_2023_2026.txt` file contains arXiv IDs for high-priority papers in each paradigm. These can be:
- Added to manual backfill queue
- Used to seed automated backfill discovery
- Prioritized for human review

---

## Next Steps

1. **Immediate**: Ingest structured knowledge into ARES
2. **Short-term**: Run backfill on seed papers (2023-2026)
3. **Medium-term**: Design self-experiments testing key hypotheses
4. **Long-term**: Build prototype implementations combining best techniques

---

## Key Hypotheses to Test

### H1: HDC-Based Language Models
**Claim**: Language models using 10,000-dimensional hypervectors with binding operations can achieve comparable performance to transformers with 100-1000x fewer parameters.

**Test**: Implement HDC LM on small corpus (WikiText-2), compare perplexity vs transformer baseline.

### H2: VSA for Compositional Generalization
**Claim**: Vector symbolic architectures naturally support compositional generalization without catastrophic forgetting.

**Test**: Design a systematic generalization task (e.g., SCAN-like), compare VSA vs transformer performance on held-out compositions.

### H3: SDR Learning Without Backprop
**Claim**: Sparse distributed representations can be learned using local Hebbian-style rules instead of global backpropagation.

**Test**: Implement SDR-based classifier trained with Hebbian learning, compare accuracy vs backprop-trained baseline.

---

## References

- Kanerva (2009) - Hyperdimensional Computing: An Introduction to Computing with Distributed Vectors
- Gayler (2003) - Vector Symbolic Architectures Answer Jackendoff's Challenges for Cognitive Neuroscience
- Plate (1995) - Holographic Reduced Representations
- Rumelhart et al. (1986) - Localist vs Distributed Representations
- Marcus (2001) - The Algebraic Mind: Integrating Connectionism and Cognitive Science

---

**Status**: Research Inventory Created  
**Last Updated**: 2025-03-25  
**ARES Integration**: Pending knowledge ingestion