Schema Registry
A central repository that stores the formal schema definitions for every message type flowing through the Kafka pipeline. Producers validate messages against registered schemas before publishing; consumers retrieve schemas to deserialize received messages. Enforces compatibility rules to prevent breaking changes from reaching production.
The Schema Registry is the contract enforcement layer for the Cognitive Substrate's Kafka pipeline. Without it, message schemas are implicit - known informally, documented incompletely, and enforced not at all. With it, every message type has a registered formal definition that specifies exactly which fields exist, their types, and which are required versus optional. When a producer publishes a message, its client library validates the payload against the registered schema and rejects invalid messages before they enter the topic - catching errors at the source rather than hours later in a consumer error log. When a consumer starts up, it retrieves the schema for each topic it reads and uses it for deserialization. The registry also maintains a full version history of every schema, making it possible to audit what structure was in use at any point in time - critical for debugging historical events during replay.