Entropy-Gated State Speculative Decoding
High entropy tokens carry more information and require higher state fidelity. Low entropy tokens (tokens, stop words) can be processed with 4-bit states. This dynamic switching will reduce average memory bandwidth by 25%.
ID: entropy-gated-state-speculative-decoding
Folder: inventions/entropy-gated-state-speculative-decoding
Created: 2026-03-09 07:02:34
Updated: 2026-03-10 06:26:36
Files: 11
Source: student_autonomy
README.md
ARES's plain-English description of what this invention does and how to run it.
# Entropy-Gated State Speculative Decoding
This project demonstrates a novel optimization for State Space Models (SSMs) where the precision of the recurrent state is dynamically adjusted based on the information content (entropy) of the current token.
## Concept
- **Hypothesis**: High entropy tokens (unpredictable) require high state fidelity. Low entropy tokens (predictable stop words) can be processed with low-precision states (e.g., 4-bit).
- **Benefit**: Reduces average memory bandwidth usage during inference without significant degradation in model coherence.
## Installation
No external dependencies are required. This project uses only the Python standard library.
```bash
# Ensure you have Python 3.8+
python run_demo.py
```
## Usage
Run the demonstration to see the dynamic precision switching in action:
```bash
python run_demo.py
```
The demo simulates an SSM processing a sequence of tokens with varying entropy levels and reports the effective bandwidth savings.
| Path | Bytes |
|---|
| DESIGN_BRIEF.md |
1030 |
| entropy_gated_state_speculative_decoding/__init__.py |
336 |
| entropy_gated_state_speculative_decoding/__pycache__/__init__.cpython-311.pyc |
596 |
| entropy_gated_state_speculative_decoding/__pycache__/__init__.cpython-313.pyc |
564 |
| entropy_gated_state_speculative_decoding/__pycache__/ssm_core.cpython-311.pyc |
7132 |
| entropy_gated_state_speculative_decoding/__pycache__/ssm_core.cpython-313.pyc |
6184 |
| entropy_gated_state_speculative_decoding/ssm_core.py |
5286 |
| invention.json |
1450 |
| pyproject.toml |
295 |
| README.md |
1019 |
| run_demo.py |
3042 |
Manifest
Structured metadata ARES recorded when it created this project.
{
"id": "entropy-gated-state-speculative-decoding",
"title": "Entropy-Gated State Speculative Decoding",
"summary": "High entropy tokens carry more information and require higher state fidelity. Low entropy tokens (tokens, stop words) can be processed with 4-bit states. This dynamic switching will reduce average memory bandwidth by 25%.",
"source": "student_autonomy",
"kind": "invention",
"path": "inventions/entropy-gated-state-speculative-decoding",
"created_at": "2026-03-09 07:02:34",
"updated_at": "2026-03-10 06:26:36",
"project_status": "built",
"project_entrypoint": "run_demo.py",
"smoke_test_status": "passed",
"smoke_test_output": "============================================================ Entropy-Gated State Speculative Decoding Demo ============================================================ [Config] Initializing Token Sequence (Length=256)... [Config] Initializing Entropy-Gated SSM... [Run] Processing sequence... Step 000 | Entropy: 2.0000 | Mode: HIGH_PRECISION (16-bit) Step 001 | Entropy: 2.0000 | Mode: HIGH_PRECISION (16-bit) Step 002 | Entropy: 0.3625 | Mode: LOW_PRECISION (4-bit) Step 003 | Entropy: 0.3625 | Mod",
"generated_files": 5,
"project_generated_at": "2026-03-09 07:03:09",
"source_hypothesis_id": "hyp-entropy-gated-state-speculative-decoding",
"source_exp_path": "experiments\\exp_self.20260309070151.002_20260309_070222"
}