# Cross-Layer State Distillation (CLSD)

## Overview

Cross-Layer State Distillation (CLSD) is a model compression and optimization technique where the internal state (activations) of a "teacher" layer is used as a target for a "student" layer. 

Unlike traditional Knowledge Distillation (which aligns final outputs), CLSD aligns intermediate feature maps. This forces the student layer to learn richer, more abstract representations earlier in the network, effectively "short-circuiting" the depth required to solve a problem.

## Project Scope

This package provides a minimal, dependency-free implementation of CLSD using pure Python.

**Features:**
*   **Zero Dependencies:** Uses only Python standard library (`random`, `math`).
*   **Custom Neural Engine:** Includes a tiny matrix/tensor library and autograd logic for the distillation task.
*   **CLSD Trainer:** Logic to minimize the Mean Squared Error (MSE) between a source layer and a target layer.

## Limitations

*   **Simulation Only:** This implementation is intended for educational and algorithmic demonstration. It uses synthetic random data and small network dimensions.
*   **Performance:** Pure Python matrix operations are significantly slower than C-accelerated libraries like NumPy or PyTorch. Do not use for production training.
*   **Scope:** The backward pass is optimized specifically for the distillation loss (MSE between layers), not a general computational graph.

## Installation

No installation required. Ensure you have Python 3.7+.

## Usage

Run the smoke test to see CLSD in action:

```bash
python run_demo.py
```

### Code Example

```python
import random
from cross_layer_state_distillation_clsd import Layer, CLSDTrainer, Tensor

# 1. Define a simple architecture
input_size = 4
hidden_size = 4

# A 'shallow' layer we want to enhance
student_layer = Layer(input_size, hidden_size)

# A 'deep' layer with rich representations to mimic
teacher_layer = Layer(input_size, hidden_size)

# Initialize Trainer
trainer = CLSDTrainer(learning_rate=0.1)

# 2. Generate synthetic input
data = Tensor([[random.random() for _ in range(input_size)]])

# 3. Get States
teacher_state = teacher_layer.forward(data)
student_state = student_layer.forward(data)

print(f"Initial Distillation Loss: {trainer.compute_loss(student_state, teacher_state):.4f}")

# 4. Distill: Update student weights to match teacher's state
trainer.distill_step(student_layer, data, teacher_state)

# 5. Verify improvement
new_student_state = student_layer.forward(data)
print(f"Post-Distillation Loss: {trainer.compute_loss(new_student_state, teacher_state):.4f}")
```

<!-- ARES_AUTO_VERIFIED_SUMMARY:START -->
## Verified Project Notes

- Package import path: `cross_layer_state_distillation_clsd`
- Entrypoint: `run_demo.py`
- Delivery mode: `prototype`
- Release tier: `prototype`
- Verification status: `PASS`
- Clean-room release gates: `NOT_RUN`
- Public exports: `CLSDTrainer, Layer, Tensor`
- Python files detected: `run_demo.py, cross_layer_state_distillation_clsd/__init__.py, cross_layer_state_distillation_clsd/core.py`

## Verification Commands

- `PASS` `"/home/corbybender/ares/.venv-linux/bin/python" -m py_compile "run_demo.py"`
- `PASS` `"/home/corbybender/ares/.venv-linux/bin/python" -m compileall "cross_layer_state_distillation_clsd"`
- `PASS` `"/home/corbybender/ares/.venv-linux/bin/python" run_demo.py`

## Current Limits

- No additional consistency warnings were detected by the local audit.
<!-- ARES_AUTO_VERIFIED_SUMMARY:END -->