Vault Architecture: Design Assessment and Gaps
Sources: Session notes, 2026-04-03; 2026-04-04; 2026-05-12 Raw: ECC Shorthand Guide; Vault Structure Assessment; Vault Linter Design; claude-obsidian vs bot_vault; Karpathy LLM Wiki Analysis Updated: 2026-08-12
Overview
The bot_vault’s three-layer architecture (Clippings → ai/ analysis → research/ synthesis) is structurally correct relative to the Karpathy LLM Knowledge Base pattern. The gaps are not structural but workflow-level: the index does not auto-update, naming conventions diverged, and the robot-to-human communication channel is underdeveloped relative to the external signal ingestion pipeline.
Architecture Mapping
| Karpathy Layer | bot_vault Equivalent | Status |
|---|---|---|
raw/ | Clippings/ | Correct |
wiki/ (per-article analysis) | ai/ | Correct, but naming problem |
wiki/ (Q&A outputs) | notes/ | Correct |
wiki/ (concept synthesis) | research/ | Correct but isolated, manual |
| LLM auto-maintained index | interest-map.md | Manual maintenance — key gap |
Known Structural Gaps
High Priority: ai/ Naming Dual-Track Problem
ai/ contains 54 title-named legacy files (e.g., Clippings-How to Train Your Robots Demonstration Modality.md) alongside 432 slug-named files from a later convention change. The early /analyze-vault skill used title naming; the current version uses slugs. The 54 unrenamed files are the root cause of 37 broken wikilinks in interest-map.md and research/ cross-references.
Fix: one-time rename of the 54 title-named files to slug format, with simultaneous wikilink updates.
High Priority: interest-map.md Does Not Auto-Update
After /analyze-vault runs, interest-map.md remains static. 100+ new ai/ analyses are invisible to the index. This breaks the compounding loop: new knowledge is filed but never becomes discoverable through the primary navigation structure.
Fix: add an “update interest-map” step to the end of /analyze-vault SKILL.md.
Medium Priority: No Concept Article Layer
Each ai/ file corresponds to one source; there is no cross-article concept synthesis layer. The Karpathy model distinguishes per-article analysis from concept articles written by the LLM across multiple sources. The research/ directory partially fills this role but is populated manually and inconsistently.
Fix: the Layer 2 semantic linter’s connection discovery is the automated entry point for identifying concept article candidates.
Vault Linter Design (Two-Layer)
Layer 1 — Python Structural Checks (fast, deterministic)
Implemented in scripts/lint-vault.py, no LLM required:
- Frontmatter completeness: Clippings missing title/source/analyzed/source_hash; notes/ missing date/tags
- Tag taxonomy enforcement: compare against vault-codemap whitelist, find typos or stale tags
- Broken wikilinks: scan all
[[...]]links, find references to nonexistent files - Analyzed backlog:
analyzed: falseitems older than N days - Duplicate detection: use
source_hashto find repeated clips - Orphan detection: notes/ and research/ files with no inbound wikilinks
Layer 2 — LLM Semantic Checks (slower, judgment-based)
Integrated as /lint-vault skill:
- Interest alignment: find low-relevance articles relative to interest-map; find emerging topic clusters not yet reflected in the map
- Connection discovery: find concepts mentioned across multiple Clippings but never synthesized → candidate for a new concept article
- Staleness check: compare new ai/ article tags against interest-map.md categories
- Missing description imputation: fill empty description fields with a one-sentence summary
Output format: results written to inbox/lint-YYYY-MM-DD.md with three sections: red (structural fixes), yellow (knowledge health), green (semantic discoveries).
Implementation order: Layer 1 script → /lint-vault skill wrapper with --semantic flag → weekly cron + Discord notify.
The Compounding Loop and Agent Architecture
Karpathy’s key claim: every Q&A session output should be “filed back into the wiki” to close the compounding loop. Without this feedback, the knowledge base is a write-only accumulator — inputs arrive but the output of each query does not feed the next.
bot_vault’s missing loop element: /analyze-vault processes Clippings → writes ai/ notes, but the loop stops there. Q&A answers, research syntheses, and conversation outputs are not systematically filed back. The notes/ directory partially fills this role (ad-hoc Q&A saves via /save-note), but there is no structural guarantee.
Minimum viable compounding loop: two commands close the gap with minimal engineering:
/qa-vault— query → search vault → synthesize answer → save to notes//save-note— capture conversation insight → save to notes/
Both exist in the current vault SKILL.md. The remaining gap is that /analyze-vault does not automatically call either; the feedback is human-triggered, not automatic.
JUMPERZ agent architecture insight (from analysis of the Karpathy pattern): agents operating on large knowledge bases do not need unlimited context windows — they need good file organization and a readable index. An agent that can read interest-map.md and navigate to specific articles knows where to look without loading everything. The bot_vault structure is already compatible with this model; interest-map.md auto-update is the missing piece that enables agent navigation without full-context loading.
claude-obsidian vs bot_vault Comparison
claude-obsidian (4.6k stars) offers complementary design patterns:
Hot Cache (most valuable to port): claude-obsidian maintains wiki/hot.md (~500 words) as a recent-context cache. bot_vault cold-starts from vault-codemap.md on every skill execution, consuming redundant tokens. A hot.md equivalent would reduce per-skill context overhead.
Data flow directions are complementary: bot_vault is strong on external signal ingestion (Discord, arXiv, Linkding → vault); claude-obsidian is strong on internal self-maintenance (linting cadence, multi-round autoresearch, canvas auto-generation).
Immutable source layer: claude-obsidian uses .raw/ vs wiki/ strict separation. bot_vault’s Clippings/ plays the raw role but the processed layer (ai/) is not formally defined as such in the schema.
/autoresearch multi-round loop: claude-obsidian runs 3 rounds of search → fetch → synthesize → file before writing a final synthesis. bot_vault’s /research-vault is single-pass with no gap-filling iteration.
Memory architecture: bot_vault uses dual-layer memory (claude-mem MCP semantic search + Obsidian vault); claude-obsidian relies only on the vault itself. bot_vault has stronger cross-session query capability.
cs.HC Filtering Issue
Subscribing to cs.HC for HRI coverage introduces noise: the category includes dark patterns, educational technology, and visualization research. The per-category quota forces retention of low-relevance papers.
Fix: apply a separate keyword gate for cs.HC entries (robot, hri, haptic, etc.) rather than relying on quota-based inclusion.
Open Questions
- After fixing the ai/ naming dual-track problem, how many of the 37 broken links are automatically resolved vs. requiring manual disambiguation?
- The
/autoresearchmulti-round gap-filling loop in claude-obsidian: would adding it to/research-vaultincrease research quality enough to justify the additional token cost? - Hot Cache maintenance: what triggers a hot.md refresh — every skill run, every n runs, or explicit user request?