Three Decisions Behind a Working Context Layer

Token Discipline as a Design Constraint

Token budget is not an optimization I got to after the system worked. It is the constraint the structure was built around, and it is the reason the structure looks the way it does.

Python Everywhere, AI on Rails

This is the decision I argue about most, so here is the blunt version: do not let an agent auto-run your pipeline. Write Python that runs on a schedule, and let that Python call an AI API at the specific points where intelligence is actually required.

The AI is a subroutine. It is not the runtime.

The worked example

Meeting ingest is the pipeline I have run the longest, so it is the honest one to show. A scheduled Python script pulls new recordings. Python converts the raw export into clean markdown. Then, and only then, three AI passes produce a summary, a structured brief, and a one-card gloss, each driven by its own versioned prompt file. Routing items into the right project folder and rebuilding the index are pure Python again.

Seven stages, three of which cost tokens. The other four are deterministic, debuggable, and free.

Prompts are files, not string literals

Every AI call in the pipeline loads its instructions from a markdown file on disk. Not an f-string three functions deep in a processor. This sounds like a small thing and it is not: a prompt in a file gets versioned, diffed, reviewed, and reverted. A prompt buried in code drifts, and you find out six weeks later when the output shape quietly changed.

# Meeting Gloss Prompt — v1.0
# Flag definitions live in prompts/flags/meeting_flags.md
# Last updated: 2026-06-27

## What a Gloss is

A Gloss is a catalog card. It is NOT a summary of content. It does
not narrate what was discussed. It is descriptive of THE FILE, not
THE EVENTS within the file.

Example:
GOOD: "Internal 1:1 between two team members covering team
management, service initiative planning, and deal pipeline."

BAD: "A strategic discussion where the lead approved the service
initiative and aligned on team role changes."

The first describes the file. The second describes what happened.
Glosses are the first kind.

## Accuracy requirements

- Never add a last name if only a first name was used in the source
- Never assign a title, role, or company affiliation not stated
- Never name a person, company, or product that was not in the source
- When in doubt, omit rather than invent

Prompt caching pays for the structure

Because the system prompt and the shared flag definitions are identical across every call in a run, they go up marked as cacheable. The first call writes the cache at a small premium. Every call after that reads it at roughly a tenth of the input price. Three tier passes over a day’s worth of meetings means the fixed instructions get billed properly once and discounted for the rest of the run.

This only works because the prompt is stable and reused, which is a direct consequence of it being a file rather than something assembled per call.

Multi-client work without login juggling

Different work bills to different accounts. Rather than signing in and out, the client resolves an API key profile from the environment and the scheduled run passes whichever profile matches the context it is operating in. Adding an account is an environment variable, not a code change.

# Profiles let one pipeline bill several accounts without
# signing in and out. The scheduled run passes whichever
# profile matches the context it is operating in.

import os
import anthropic


def resolve_key(profile: str = "default") -> str | None:
    """Return the API key for a profile, falling back to the default."""
    if profile == "default":
        return os.getenv("ANTHROPIC_API_KEY")
    return (
        os.getenv(f"ANTHROPIC_API_KEY_{profile.upper()}")
        or os.getenv("ANTHROPIC_API_KEY")
    )


class AIClient:
    def __init__(self, model="claude-sonnet-4-6", key_profile="default"):
        api_key = resolve_key(key_profile)
        if not api_key:
            raise EnvironmentError(f"No API key for profile {key_profile!r}")
        self.model = model
        self._client = anthropic.Anthropic(api_key=api_key)

What a processor actually looks like

Load the prompt. Read the input tier. Call the API. Write the output tier. That is the entire shape, and it is the same shape for all three AI stages. The interesting logic, which file to process, whether it has already been done, where the result belongs, lives in Python on either side of the call.

from pathlib import Path

from shared.ai_client import AIClient
from shared.prompt_loader import load_prompt

STAGING = Path("<context-layer>/_staging/summarized")

system_prompt = load_prompt("summarize_meeting.md")
flags         = load_prompt("flags/meeting_flags.md")

client = AIClient(key_profile=args.key_profile)


def process_file(stem: str, path: Path) -> dict:
    cleaned = path.read_text(encoding="utf-8")

    result = client.generate(
        system_prompt=system_prompt,
        user_content=f"# Meeting To Process\n{cleaned}",
        cache_system=True,                 # stable across the whole run
        cacheable_prefix=f"# Flag Definitions\n{flags}\n",
    )

    (STAGING / f"{stem}.md").write_text(result["text"], encoding="utf-8")
    return {"cost_usd": result["cost_usd"], "tokens": result["input_tokens"]}