Learning System Guide

How the system captures knowledge from sessions, stores it in a unified DB, materializes it as principles Claude can act on, and runs gyms to evaluate and improve components.

1. Why This Exists

Every Claude Code session starts with limited memory. Hard-won insights from yesterday's debugging session are gone.

The learning system closes this loop. It mines sessions for patterns, stores them as evidence-backed principles, and injects relevant ones into future sessions before the same mistakes are repeated.

2. Core Concepts

Four nouns you need to understand:

3. Walkthrough: Observation to Behavior Change

Tracing one real example through the entire system:

$ learn add "Gradio 6 silently ignores head= on gr.Blocks — must pass to launch()"
Classifying with gemini-2.5-flash...
  Type: pattern | Project: rivus | Tags: gradio, css, gotcha
  Linked to: No Silent Failures (strength: 0.8)
  Linked to: Prefer Context-Native Mechanisms (strength: 0.6)
  Instance ID: inst_a8f3c2
  Embedding generated ✓

$ learn show no-silent-failures
  Principle: No Silent Failures
  Status: active | Type: principle | File: dev.md
  Instances: 31 | Applications: 219
  Full text: "Never let a function silently ignore, drop, or swallow input.
  If something can't be processed, raise an error or log a warning..."

4. Self-Improvement Loops

Beyond capturing knowledge, the system actively improves itself through two mechanisms: auto-advise (learning what to do proactively) and gyms (testing and optimizing components).

Auto-Advise: Learning Proactive Behavior

Mines follow-up patterns from sessions — cases where the user's next request was a predictable action Claude should have done proactively.

The pipeline: SessionEnd hook → Opus extracts follow-up patterns → stored as principles in auto-advise.md → injected into future sessions like any other principle. Currently 8 auto-advise patterns active.

Gyms: Generate → Evaluate → Learn

Offline evaluation environments that test and optimize system sub-components using historical data. Each gym tries variants of a prompt or method, scores them with LLM judges, and picks the winner.

Gym findings feed back into the main learning loop as new instances or principle refinements. See Section 12 for detailed gym documentation.

5. Status: What Works & What Doesn't

An honest assessment of where the system stands, drawn from production stats, the presentation report ENHANCE markers, and learning/TODO.md.

What works well

In progress

Planned / known gaps

Measurement gaps (from report ENHANCE markers)

The presentation report flags 9 areas where the system lacks measurement:

1. No causal metrics — stats are descriptive (1,436 instances), not causal (error rates before/after)
2. No before/after session comparison — the killer metric ("sessions with principles had X% fewer errors") doesn't exist yet
3. Application tracking sparse — most principles have few tracked applications; automation needed
4. Sandbox results not aggregated — per-principle A/B data exists in sandbox_results.db but isn't summarized
5. No worked failure→repair example — the 8-candidate scoring pipeline needs a concrete trace
6. Gym results under-documented — Badge gym ran 5 variants but results aren't in the report
7. Verification levels unproven — L0-L1 claimed but evidence not shown
8. Prior work comparison too high-level — needs specific paper references (Voyager, Reflexion, LATS)
9. KB gym design choice rationale missing — 5-case test and 70% threshold need justification

6. Quick Start

Add knowledge

# LLM auto-classifies, links to existing principles
learn add "Gradio 6 silently ignores head= on gr.Blocks"

# Store directly, no LLM classification
learn add "Always use python not python3" --raw --type convention

# Add with project context
learn add "Redis TimeSeries keys use ts:{SYMBOL}:price:raw" --project moneygun

Search and browse

# Keyword search
learn find "gradio gotchas"

# Semantic search (uses embeddings — finds conceptual matches)
learn find -s "silent failures"

# List principles with full text
learn principles -v

# Show a specific principle + evidence chain
learn show no-silent-failures

Regenerate materialized files

# Preview changes without writing
python learning/schema/materialize.py --principles-only --dry-run

# Write all outputs
python learning/schema/materialize.py

Health check

learn health    # embedding coverage, orphans, staleness
learn stats     # summary statistics

7. How Data Flows

There are two main data paths: automatic (from sessions) and manual (from the CLI).

Automatic: Session → Principle

Runs automatically on every session end, as a fire-and-forget subprocess.

Manual: Observation → Classified Instance

Materialization → Claude's Context

8. Runtime Components

Two background workers handle the automation. Understanding them is key to debugging.

Why multiple LLMs?

9. CLI Reference

The learn command (~/.local/bin/learn → tools/bin/learn) is the primary interface. All commands operate on learning/data/learning.db.

10. Principle Files

materialize.py generates one .md file per domain from the DB. These live at ~/.claude/principles/ and are read-only views — edit via the CLI or DB, then re-materialize.

Top 10 Principles by Evidence

11. Session Review

Analyzes how work was done (efficiency, errors, patterns). Complements doctor/chronicle which analyzes what was done (topics, accomplishments). All tools read from the same ~/.claude/projects/ JSONL transcripts.

# Mine failures from last 30 days
python -m learning.session_review.failure_mining --clear --days 30

# Score with defaults (flash + grok_think, majority vote)
python -m learning.session_review.pair_judge_compare

# Analyze tool errors
python -m learning.session_review.tool_error_analysis --days 3

# Run principle A/B test in Docker
python -m learning.session_review.sandbox_replay \
  --prompt "Find all Gradio apps" --commit HEAD

12. Gyms (Detailed)

Offline evaluation environments that test and optimize system sub-components using historical data. Each gym runs a generate → evaluate → learn loop: try variants of a prompt/method, score them with LLM judges, pick the winner. Gym findings can become new learning instances or principle refinements, feeding back into the main loop. All gyms extend lib/gym/GymBase.

# Quick test (5 sessions)
python -m learning.gyms.badge.gym --max-sessions 5

# Full run with HTML report
python -m learning.gyms.badge.gym --max-sessions 20 --report

# Bulk URL comparison
python learning/gyms/fetchability/fetchability_tool.py \
  --url-file /tmp/urls.txt --parallelism 8

Auto-Advise Pipeline (detailed)

How it works:

1. SessionEnd → learning_worker.py runs Stage 3: extract_follow_ups(condensed)
2. Opus identifies 0-3 follow-up patterns from the condensed transcript
3. Patterns stored as Principle entries directly in learning.db (with dedup by auto-advise/{slug} ID)
4. materialize.py generates ~/.claude/principles/auto-advise.md
5. principles_worker.py (unchanged) matches auto-advise principles like any other principle

Why direct DB write? The learn add --type principle path always triggers LLM re-classification, which is wasteful since Opus already classified the pattern. store_follow_ups() writes Principle entries via LearningStore.add_principle() directly.

13. Skills

Three Claude Code skills cover the learning loop:

14. Directory Map

15. DB Schema

Key relationships

Instance sources: 1,324 from session_reflection (automatic), 112 from manual (CLI).

16. Design Decisions

DB-native principles

learning.db is the single source of truth. The ~/.claude/principles/*.md files are materialized views — generated by materialize.py. Never edit .md files directly; use the CLI or DB, then re-materialize.

LLM classification on add

learn add uses Gemini Flash (~$0.0003/call) to classify observations and link them to existing principles. Flash agrees with frontier models 90%+ on structured classification. Use --raw to skip when classification is predetermined.

Dedup by abstraction

Before proposing a new principle, check if an existing one covers the case at a higher abstraction level. "Order tabs by user frequency" is an instance of "Importance Ordering & Attention Budget" — link the instance, don't create a near-duplicate principle.

Auto-advise: direct DB write

Follow-up patterns bypass learn add --type principle (which re-classifies via LLM) and write Principle entries directly. Opus already classified them during extraction — re-classification would be wasteful.

Session skipping heuristics

learning_worker.py skips sessions with <3 user messages or <2 minutes duration. Also checks whether a manual wrapup already extracted learnings for the same session.

17. Troubleshooting