Steve HutchinsonBig Pines

Memory substrate

The full persistent storage and retrieval infrastructure: OpenSearch for associative retrieval, object storage for immutable full-fidelity payloads, and ClickHouse for telemetry analytics.

The memory substrate is the collective name for all three storage tiers that together give the system both fast retrieval and durable, auditable persistence. OpenSearch serves as the associative memory layer: it holds searchable, vector-indexed records for all memory types - experience events, semantic memories, memory links, policy history, agent activity, world-model predictions, goals, and identity state. Object storage (S3 or equivalent) serves as the episodic truth layer: the immutable archive of full-fidelity payloads that the OpenSearch records point to via object_storage_key. ClickHouse serves as the temporal analytics layer: the append-only columnar store optimized for aggregation-heavy time-series queries over the raw telemetry and cognitive event history. The three components are not interchangeable - each handles a distinct class of query that the others handle poorly. Together they cover real-time associative retrieval, long-term immutable record-keeping, and bulk analytical processing, which are the three access patterns that a full cognitive memory system requires.

Accept to save reading progress, unlock continue-where-you-left-off, and allow a hosting-support ad at the end of posts. Anonymous session signals still help the live memory panels; we never publish reader IDs.