Building the Weekly Memory Digest: Closing the Loop

Right now, the relationship between my writing on BigPines.net and the Cognitive Substrate is strictly one-way. Reader behavior - page views, scroll depth, code copies, article completion - gets turned into ExperienceEvents, embedded, and stored in OpenSearch. The memory system is learning about my content and my audience. But it has no channel to speak back.

That needs to change.

The Weekly Memory Digest

The idea is simple but powerful: every week, the system autonomously generates a concise, high-signal report about what it has observed from my writing and reader behavior, then delivers it to me.

How to Build It

Digest Trigger

A scheduled job - either via Temporal or a straightforward cron in a new worker - runs every Sunday. It queries OpenSearch for all ExperienceEvents from the past 7 days tagged with source:blog or source:reader_telemetry.

Insight Generation

Rather than using a language model to write the digest (which would compromise the extractive philosophy and lose traceability), the Digest Worker runs targeted operations against the same data the ingest worker already produces. Here is what that data looks like - the mapper that converts raw blog events into embedded experiences:

// packages/ingest-worker/src/mapper.ts

export function mapTelemetryToExperience(event: TelemetryEvent): ExperienceEvent {
  const context: EventContext = {
    sessionId: event.sessionId,
    agentId: 'ingest-worker',
    traceId: randomUUID(),
  }

  return {
    eventId: randomUUID(),
    timestamp: event.timestamp,
    type: 'environmental_observation',
    context,
    input: {
      text: buildSummary(event),
      // Embedding generated by the OpenSearch ingest pipeline at index time.
      embedding: [],
    },
    importanceScore: importanceScore(event),
    tags: buildTags(event),
  }
}

// Importance scoring reflects signal strength:
//   article_complete  = 0.85 + reading-time bonus (highest)
//   scroll_depth 90%  = 0.75
//   copy_code         = 0.65
//   search_query      = 0.70 if zero results (niche curiosity), 0.50 otherwise
//   focus_loss        = 0.10–0.30 (dwell-time weighted)
//   nav_click         = 0.05 (lowest)
function importanceScore(event: TelemetryEvent): number {
  switch (event.type) {
    case 'article_complete':
      return Math.min(0.85 + (event.payload.readingTimeMs / 600_000) * 0.15, 1.0)
    case 'scroll_depth':
      return { 90: 0.75, 75: 0.55, 50: 0.3, 25: 0.15 }[event.payload.depth]
    case 'search_query':
      return event.payload.resultCount === 0 ? 0.7 : 0.5
    // ...
  }
}

Each event also gets tagged with engagement category (engagement:deep, engagement:conversion, engagement:exit), article slug, and semantic topic tags. The Digest Worker queries these tags to find the week's signal without needing to parse freeform text.

• Aggregation queries against OpenSearch to surface frequency and co-occurrence patterns
• Analysis of the abstraction ladder for newly formed or updated patterns at the Concept and Principle levels
• Examination of trust score changes and any Memory Critique Events from the past week
• Statistical anomaly detection on reader behavior metrics - scroll depth drop-offs, high-completion articles, unexpected traffic patterns

Output Format

The worker generates a structured Markdown report with fixed sections:

• Top Insight - the single most interesting observation from the week
• Emerging Patterns - themes or ideas recurring with increasing frequency
• Interesting Tensions - places where recent writing appears to contradict older positions
• Knowledge Gaps - topics referenced repeatedly but never explored in depth
• Recommendations - concrete suggestions for upcoming writing based on observed patterns

Delivery

The completed digest posts to a dedicated Slack channel or saves to a private location.

An Example

Here is what a realistic digest might look like:

Cognitive Substrate - Weekly Memory Digest Week of May 11-17, 2026

Top Insight This Week You have written about memory systems three times this month, but all three pieces focus heavily on how memory works and almost none on why a human would actually want to use it. The system flagged this as a potential gap between technical depth and user value.

Emerging Patterns Strong recurring interest in "forgetting as intelligence." Increasing focus on self-correction and critique mechanisms. Noticeable shift from infrastructure talk toward agent cognition.

Interesting Tension A contradiction between the May 8 post praising extractive, auditable memory and the May 15 draft exploring agent critique of its own memories. One favors stability, the other favors evolution.

Knowledge Gap Repeated references to "agent improvement" but no concrete metrics or success criteria have been defined yet.

Recommendation Consider writing a piece that bridges the technical memory work with real user outcomes.

Why This Matters

This closes the loop. The memory system stops being a silent observer and becomes an active collaborator in my thinking and writing process. The digest does not tell me what to think - it tells me what the data has been quietly suggesting, consistently, across weeks of real behavior.

That is the kind of feedback that is easy to miss when you are head-down writing. The system has been watching. Now it can speak.