Object storage
A cloud-native blob storage service (S3 or equivalent) used as the episodic truth layer for full-fidelity experience payloads and as the backing store for diskless Kafka topics. Optimized for high durability and low cost at the expense of retrieval latency compared to local disk or in-memory stores.
Object storage serves two distinct roles in the Cognitive Substrate. First, it is the episodic truth layer: the immutable archive of full-fidelity experience event payloads and raw reasoning traces. OpenSearch stores indexed metadata with a pointer (object_storage_key) to the full payload here. This design keeps the search index lean while preserving complete records for audit, replay, and re-consolidation. Second, it is the backing store for diskless Kafka topics, where brokers route events to and from object storage rather than local disks. This makes brokers stateless and dramatically reduces the cost of long-retention topics at the expense of higher write latency. The cold tier of tiered storage also uses object storage for archived low-value memories that rarely need retrieval but must be preserved for compliance or historical analysis.