How Pinky Brain works

End-to-end explanation: what it does, who creates the documents, how they are indexed and searched, and how it connects to a project / to Claude Code.

1. The idea in one sentence

The .md files with frontmatter are the source of truth of the knowledge; pinky_brain indexes them (SQLite + hybrid search) and retrieves them when an agent or a human needs them. The index is derived and disposable — it is rebuilt from the .md files.

2. Two parts worth not confusing

PartWhat it isWho manages it
The knowledgethe .md files (gotchas, patterns, decisions, diary)created by the agent or the human (see §3)
The enginethe Rust code that indexes and searches (CLI + MCP + hooks)pinky_brain
pinky does not invent documents. The engine only stores, indexes and searches. Deciding what and when to save is agent behavior, guided by a rule.

3. Who creates the documents? (the key question)

Three sources, none of them "binary magic":

  1. The agent, following the rule rules/use-pinky-brain.md. That rule is the "skill" that tells the agent SEARCH before acting, SAVE after discovering a gotcha/pattern/decision, leave breadcrumbs, write diary. When the agent meets one of those conditions, it creates a .md with frontmatter in documentation/ (of the project) or in ~/.pinky/brain/ (global).
  2. The stop hook (from pinky-hooks): when a session/task ends, it **adds a diary entry** automatically in documentation/diary/YYYY-MM-DD.md.
  3. You, by hand: any .md with the correct frontmatter counts.

Without the rule (#1), pinky still works as a search engine, but the knowledge is not captured on its own: it is the rule that turns "I discovered something" into "I saved it".

4. The knowledge life cycle

   CREATE               INDEX                SEARCH                 MAINTAIN
 (new .md)    ──▶  pinky reindex   ──▶  brain_search / pinky   ──▶  dedup · stale
 agent/hook/         (parse + chunk        search (BM25+vector       · rollup
 human               + embed → SQLite)     + RRF [+ rerank])         · telemetry
  • Create: a .md is born (rule, hook or by hand).
  • Index: pinky reindex <folder> parses the frontmatter, splits the body into chunks, embeds them (local model multilingual-e5-small) and stores them in the SQLite index (FTS5 + sqlite-vec). It is incremental: it only re-indexes what changed.
  • Search: hybrid search — full-text (BM25, with the title and tags weighing more than the body) + vector (semantic), fused with Reciprocal Rank Fusion, optionally with lexical rerank (--rerank). Snippets are centered on the match (not the start of the document).
  • Maintain: dedup (near-duplicates), stale (old entries to re-verify), rollup (diary summary), telemetry (what is used / what never is), eval (retrieval quality).

5. The three access points to the engine

The same index is used in three ways:

  1. CLI pinky — for you / scripts: reindex, search, dedup, stale, rollup, backlinks, eval, telemetry, stats, doctor.
  2. MCP pinky-mcp — for the agent: it exposes the brain_search tool (and brain_stats). The agent calls it to query the knowledge without grepping. It is registered in Claude Code (make mcp-register).
  3. Hooks pinky-hooks — they automate usage from Claude Code:
    • SessionStart: shows brain stats on startup.
    • PreToolUse(Read): injects relevant knowledge before reading a file.
    • PreToolUse(Write/Edit): warns about related gotchas before writing.
    • Stop: writes the diary entry.

The hooks inject context proactively — they don't require the agent to call brain_search. That's useful in a single long interactive session. In headless / orchestration runs where each claude -p invocation is a brand-new session (each pipeline stage = its own session), the per-session SessionStart and per-Read PreToolUse injections multiply thousands of additive tokens without substituting what the agent was going to read anyway. Set PINKY_HEADLESS=1 to silence the push injections: the brain_search MCP tool keeps working (pull model — exactly what it was designed for). Other knobs: PINKY_HOOK_HITS (default 3) controls how many hits each hook injects, PINKY_HOOK_SNIPPET (default 240) the snippet length. A per-session dedup also avoids re-injecting an entry_id already served to the same session.

6. Format of a document

Every indexable .md carries YAML frontmatter. Example of a gotcha:

---
type: gotcha            # gotcha | pattern | decision | diary | guide | note
project: demo
created: 2026-06-30
last_verified: 2026-06-30
tags: [postgres, pool]
---
# Gotcha title

Body in markdown. The first `# ...` line is used as the title if there's no `title`
in the frontmatter.

type, tags, project, created, last_verified become indexed metadata (for filtering and for the age-based decay).

Optional supersedes marks the entries this one replaces — one path, or a list when a note consolidates several older ones:

supersedes: decisions/auth-v1.md
# …or
supersedes: [decisions/auth-v1.md, decisions/sessions.md]

The list fields (tags, supersedes) also accept a bare scalar (tags: postgres). A field with an unusable value (say type: [a, b]) falls back to its default on its own: the rest of the frontmatter is still indexed, and pinky lint names the offending field.

7. Two levels of knowledge

  • Project-level: lives in <project>/documentation/, versioned with the code.
  • Global (across projects): ~/.pinky/brain/ — a separate git repo, syncable between your machines with git pull/push.

The index (brain.db) is never versioned; it is rebuilt with reindex.

8. How to connect it to a project + to Claude Code

# 1. Install the binaries (pinky, pinky-mcp, pinky-hooks)
make up                                   # in the pinky_brain repo

# 2. Index the project's knowledge
pinky reindex ./documentation --project myproject

# 3. Give the tool to the agent (registers the MCP in Claude Code)
make mcp-register                         # → claude mcp add pinky -- pinky-mcp

# 4. Have the agent capture knowledge: add the rule to your CLAUDE.md
#    or load it as a rule/skill — see rules/use-pinky-brain.md

# 5. (optional) Configure the Claude Code hooks pointing to `pinky-hooks`
#    SessionStart / PreToolUse(Read|Write) / Stop

With that: the agent searches before acting (brain_search), saves what it discovers (rule), and leaves diary when finishing (hook). The knowledge grows on its own as you work.

9. Summary

  • pinky_brain = knowledge engine (store + index + search), not an author.
  • The documents are created by the agent (rule), the stop hook, or you.
  • A single index, three accesses: CLI, MCP (for the agent), hooks (Claude Code).
  • Hybrid search, maintenance (dedup/stale/rollup/telemetry/eval), 2 levels (project/global). See PLAN.md for the complete architecture.