Hightower's AI Harness Engineering

Hightower's AI Harness Engineering

Loop Engineering + Graph Engineering: Building a Compounding Second Brain with OKF-based LLM Wiki

From Session Loops to Persistent Graphs: OKF, the Second Brain, and Compounding Agent Systems

Rick Hightower's avatar
Rick Hightower
Aug 15, 2026
∙ Paid
Title card reading From Session Memory to a Compounding System by Rick Hightower, over an illustration of dissolving session bubbles on one side and a stream of structured documents passing through a schema-enforced gate into a permanent repository vault

From Session Memory to a Compounding System

How an append-only event log, a Git-native knowledge graph, and bounded ContextPacks turn throwaway agent sessions into a system that gets smarter with every run

You and an agent solved a hard problem yesterday. The reasoning was sharp, and the trade-offs were clear. Today, a different agent reopens the exact decision you already settled, and every hour you spent is gone. You are not losing time to a weak model. You are losing it to an architecture that throws the answer away when the session ends.

Summary: This article shows how to stop paying for the same reasoning twice. It covers three foundations that decide whether knowledge survives a session. It covers an append-only event log that makes agent work visible, and a Git-native knowledge graph that holds both the why and the what. It then covers the ContextPack, which provides each new run with a bounded, ranked slice of that graph rather than a blank context. Every tool named here is open source, and the code is linked at the end.


The loss repeats every day in engineering organizations that adopted coding agents. Quality drops under machine volume. Teams argue the same decisions again. Knowledge stays tribal, locked in chat logs and in the heads of a few power users. Spending goes up, outcomes remain uneven, and the system gets weaker over time rather than stronger.

The root cause is architectural. Most teams treat agent work as private and temporary. The opposite is what you need. Every successful run and every failed run should make the next run better. A new teammate should clone a repository and get real value on day one, without first becoming a prompt expert.

Three distinctions that shape everything

Table of three distinctions, listing harness engineering against raw agent calls, write authority against letting the model write shared state, and memory against relying on one context window

Reading the table. Distinction names the choice under discussion, and each one is a decision you make once and live with. The wrong default states what teams do when nobody decides, which is the behavior you get for free. The working pattern states what to do instead, written as the shape of the fix rather than a product name.

Harness engineering is more than orchestration. A harness is the control plane. It holds the hooks, the evaluation surfaces, and the guardrails. Raw agent calls are fine while you explore. Production work needs a harness that can observe, constrain, and improve itself.

Write authority decides whether the system stays safe with many writers. A language model is excellent at judgment and extraction. It is dangerous as the final writer of shared state. Let the model propose, and let schema-enforced code perform the commit. The deterministic write boundary is what makes it safe for many agents and humans to share a single knowledge base.

Memory comes in three kinds, and teams collapse them at their cost. Procedural memory lives in AGENTS.md, CLAUDE.md, and skills, and it answers how the team works. Working memory is the current context window, and it is fragile. Long-term institutional memory belongs in the repository.

The fragility of working memory is measured, not folklore. The paper “Lost in the Middle: How Language Models Use Long Contexts”, by Nelson F. Liu, Kevin Lin, John Hewitt, and colleagues, showed that models recall the beginning and the end of a long context better than the middle. Vector stores and private per-user memory help an individual. A team needs something reviewable, attributable, and mergeable under Git.

Share

Making the work visible

The first practical layer is an append-only event log. Work items become events with stable identifiers. Current state comes from a deterministic fold over the log. Concurrent agents and separate worktrees write safely, because merges are union-based instead of conflict-prone.

Keep the organizational ticket system as the system of record for humans. For agents, the local event log becomes the source of truth. Generate status and roadmaps from the fold. The work becomes visible instead of staying trapped in private sessions.

Diagram of an append-only event log with stable event identifiers folding deterministically into current state, feeding a generated board and roadmap, with two agent worktrees writing into the log through union merges

The project second brain

A knowledge graph sits atop the event log. It is ordinary Markdown and YAML under version control. Different plugins capture different slices of reality.

Table of the three second-brain layers, showing PKC answering why, SAC answering what is running, and DEKC answering what the data does

Reading the table. Layer gives the three-letter name of the capture plugin, which is also the name of its repository. The question it answers is the one the layer exists to settle, and it is the fastest way to decide where a new fact belongs. What it holds lists the node types that layer stores, so you can see at a glance whether your fact has a home.

PKC, or Project Knowledge Capture, records the reasoning. It holds the meetings, decisions, experiments, assumptions, and open questions. A decision with its context attached does not need re-litigation.

SAC, or System Architecture Capture, records the running system. It holds services, ownership, dependencies, and blast radius. It answers the question of what breaks when you change this.

DEKC, or Data Engineering Knowledge Capture, records the data platform. It holds lineage, medallion layers, streams and jobs, the semantic layer, and the glossary.

The ContextPack

The interface between the graph and any agent is the ContextPack. A ContextPack is a bounded, ranked, typed-hop subgraph.

It works in three steps. First, it follows a limited number of edges out from a root concept. Second, it ranks the resulting nodes by relevance and impact. Third, it returns only what fits in the budget.

Compare it against the two common alternatives. A full dump does not fit inside context limits. Pure vector retrieval fits, but it returns a bag of similar fragments and drops causality, direction, and impact order. The ContextPack maintains the structure and stays within budget.

Diagram comparing three retrieval approaches: a full graph dump that overflows the context budget, vector similarity that returns unordered fragments, and a ContextPack that follows typed hops from a root concept, ranks by impact, and fits the budget

Closing the loop'


If you are a paid subscriber, thank you. Your support makes this work possible.

If you are a free subscriber and find these articles useful, please consider upgrading. A paid subscription is $80 per year or $8 per month.

Share Hightower's AI Harness Engineering

User's avatar

Continue reading this post for free, courtesy of Rick Hightower.

Or purchase a paid subscription.
© 2026 Rick Hightower · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture