# Zero-Copy Memory-Mapped State Streaming
A lightweight, zero-copy IPC (Inter-Process Communication) mechanism for Python using standard library `mmap`. This tool allows separate processes to share state by mapping a common file into their virtual address space, eliminating the overhead of serialization or data duplication.
## Features
* **Zero-Copy Access**: Readers and writers access the same physical memory pages.
* **Lock-Free Reads**: The reader process can view the latest state without acquiring locks (optimized for polling).
* **Standard Library**: No external dependencies. Uses `mmap`, `struct`, and `os`.
* **Binary Protocol**: Uses `struct` packing for deterministic, low-latency data layout.
## Installation
No pip install required. Simply ensure you are on a Unix-like OS or Windows (mmap support varies by OS).
## Usage
### Basic Writer
```python
from zero_copy_memory_mapped_state_streaming import MappedState
import time
# Initialize a 1024-byte shared memory region
state = MappedState("demo.bin", size=1024, mode='writer')
for i in range(5):
# Write a payload (bytes) and a metadata header
state.write(sequence_id=i, timestamp=time.time(), payload=f"Data-{i}")
time.sleep(0.1)
state.close()
```
### Basic Reader
```python
from zero_copy_memory_mapped_state_streaming import MappedState
# Connect to the existing memory region
state = MappedState("demo.bin", size=1024, mode='reader')
while True:
data = state.read()
if data:
print(f"Read: {data}")
# In a real scenario, you might poll here
# or use a separate synchronization primitive
break
```
## Limitations
* **Concurrency**: This specific simplified implementation prioritizes read speed over strict write-locking. Writers should ideally coordinate if multiple exist, though the demo uses a single writer.
* **OS Support**: Relies on OS-specific `mmap` behavior.
## Running the Demo
Execute the smoke test to verify zero-copy functionality:
```bash
python run_demo.py
```
<!-- ARES_AUTO_VERIFIED_SUMMARY:START -->
## Verified Project Notes
- Package import path: `zero_copy_memory_mapped_state_streaming`
- Entrypoint: `run_demo.py`
- Delivery mode: `prototype`
- Release tier: `prototype`
- Verification status: `PASS`
- Clean-room release gates: `NOT_RUN`
- Public exports: `MappedState, StateStruct`
- Python files detected: `run_demo.py, zero_copy_memory_mapped_state_streaming/__init__.py, zero_copy_memory_mapped_state_streaming/core.py`
## Verification Commands
- `PASS` `"/home/corbybender/ares/.venv-linux/bin/python" -m py_compile "run_demo.py"`
- `PASS` `"/home/corbybender/ares/.venv-linux/bin/python" -m compileall "zero_copy_memory_mapped_state_streaming"`
- `PASS` `"/home/corbybender/ares/.venv-linux/bin/python" run_demo.py`
## Current Limits
- README mentions streaming, but no streaming interface was detected in the local code.
<!-- ARES_AUTO_VERIFIED_SUMMARY:END -->
| Path | Bytes |
|---|---|
| __pycache__/run_demo.cpython-314.pyc | 4321 |
| DESIGN_BRIEF.md | 803 |
| invention.json | 3136 |
| pyproject.toml | 346 |
| README.md | 2946 |
| run_demo.py | 3178 |
| zero_copy_memory_mapped_state_streaming/__init__.py | 218 |
| zero_copy_memory_mapped_state_streaming/__pycache__/__init__.cpython-314.pyc | 474 |
| zero_copy_memory_mapped_state_streaming/__pycache__/core.cpython-314.pyc | 6829 |
| zero_copy_memory_mapped_state_streaming/core.py | 3768 |
{
"id": "zero-copy-memory-mapped-state-streaming",
"title": "Zero-Copy Memory-Mapped State Streaming",
"summary": "Zero-Copy Memory-Mapped State Streaming -> Success (score=6.33). Promote this line toward an invention brief.",
"source": "student_autonomy",
"kind": "invention",
"path": "inventions/zero-copy-memory-mapped-state-streaming",
"delivery_mode": "prototype",
"release_tier": "prototype",
"release_verification_status": "not_run",
"created_at": "2026-03-29 07:52:30",
"updated_at": "2026-03-29 08:01:21",
"project_entrypoint": "run_demo.py",
"smoke_test_status": "passed",
"smoke_test_output": "Starting Zero-Copy Memory-Mapped State Streaming Demo... [Writer] Process started. [Writer] Wrote sequence 0 [Reader] Process started. [Reader] Detected update: Seq=0, Time=1774788879.1354, Payload=ARES_STATE_001 [Writer] Wrote sequence 1 [Reader] Detected update: Seq=1, Time=1774788879.3445, Payload=ARES_STATE_001 [Writer] Wrote sequence 2 [Reader] Detected update: Seq=2, Time=1774788879.5499, Payload=ARES_STATE_001 [Writer] Wrote sequence 3 [Reader] Detected update: Seq=3, Time=1774788879.7561",
"generated_files": 5,
"project_generated_at": "2026-03-29 07:54:40",
"source_exp_path": "experiments\\exp_self.20260308173513.009_20260308_173548",
"verification_status": "passed",
"verification_results": [
{
"command": "\"/home/corbybender/ares/.venv-linux/bin/python\" -m py_compile \"run_demo.py\"",
"passed": true,
"returncode": 0,
"timed_out": false,
"stdout_excerpt": "",
"stderr_excerpt": ""
},
{
"command": "\"/home/corbybender/ares/.venv-linux/bin/python\" -m compileall \"zero_copy_memory_mapped_state_streaming\"",
"passed": true,
"returncode": 0,
"timed_out": false,
"stdout_excerpt": "Listing 'zero_copy_memory_mapped_state_streaming'...",
"stderr_excerpt": ""
},
{
"command": "\"/home/corbybender/ares/.venv-linux/bin/python\" run_demo.py",
"passed": true,
"returncode": 0,
"timed_out": false,
"stdout_excerpt": "Starting Zero-Copy Memory-Mapped State Streaming Demo...\n[Writer] Process started.\n[Writer] Wrote sequence 0\n[Reader] Process started.\n[Reader] Detected update: Seq=0, Time=1774788881.3205, Payload=ARES_STATE_001\n[Writer] Wrote sequence 1\n[Reader] Detected update: Seq=1, Time=1774788881.5259, Payload=ARES_STATE_001\n[Writer] Wrote sequence 2\n[Reader] Detected update: Seq=2, Time=1774788881.7313, Payload=ARES_STATE_001\n[Writer] Wrote sequence 3\n[Reader] Detected update: Seq=3, Time=1774788881.9366, Payload=ARES_STATE_001\n[Writer] Wrote sequence 4\n[Reader] Detected update: Seq=4, Time=1774788882.1424, Payload=ARES_STATE_001\n[Reader] Finished.\n[Writer] Finished.\n\n--- Verification ---\nProcesses completed successfully.\nShared memory file verified.\nINVENTION_SMOKE_TEST: PASS",
"stderr_excerpt": ""
}
],
"project_status": "built"
}