Origin: Andrej Karpathy’s “LLM Wiki” Vision
In April 2024, renowned AI researcher Andrej Karpathy published a Gist titled “LLM Wiki,” proposing an ideal form for a knowledge base. The core idea is that a Large Language Model (LLM) should not just be a tool for retrieving information from vast training data, but an intelligent agent capable of actively reading, linking, organizing, and maintaining a personal knowledge base. In this vision, the user inputs fragmented materials, and the model is responsible for integrating them into a structured knowledge network. When answering questions, the answers are drawn from this curated network rather than the model’s inherent memory, ensuring accuracy and traceability. This concept has pointed the way for a new generation of knowledge management tools.
Project Core: The Automated Knowledge Engine of Claude-Obsidian
Claude-Obsidian is a practical implementation of Karpathy’s “LLM Wiki” concept, an open-source project that has garnered over 7,200 stars on GitHub. It deeply integrates Anthropic’s Claude model with the popular local Markdown note-taking tool, Obsidian, to create an automated “knowledge engine.”
The core of its workflow is to achieve what the author calls “compounding knowledge.” Users can place any form of material—such as web pages, PDFs, code snippets, chat logs, or even YouTube video notes—into a designated directory, and Claude-Obsidian will use the Claude model to automatically perform the following tasks:
- Information Extraction and Organization: Automatically reads the source material, extracts key entities (like people, organizations, projects) and concepts (like theories, patterns, methods), and creates separate Markdown note pages for them.
- Knowledge Graph Construction: While creating notes, it automatically establishes bidirectional cross-reference links between pages, weaving isolated knowledge points into an interconnected knowledge graph.
- Source Archiving: Generates a source page for each piece of original material, ensuring that all knowledge is verifiable.
Through this automated process, the addition of each new piece of information enhances the depth and breadth of the existing knowledge network, allowing the value of the knowledge base to grow over time.
Key Features: Self-Maintenance and Data Sovereignty
Claude-Obsidian is not just an information processor, but a system with self-management capabilities. It offers a range of advanced features to ensure the long-term health and usability of the knowledge base:
- Contradiction Detection: The system can identify conflicting arguments within the knowledge base, flag them, and attach their respective source links, making it easier for users to discern and correct them.
- Conversational Memory: Utilizes a file named
hot.md to record the context of recent conversations. This prevents the model from needing to re-process background information in continuous interactions, improving efficiency and token economy.
- Health Checks: Includes 8 types of built-in health check (Linting) rules. With a single click, it can scan and report issues like “orphan notes” (pages with no links), dead links, outdated claims, and missing citations, helping users easily maintain a clean and efficient knowledge base.
- Complete Data Sovereignty: The entire system is based on locally stored plain-text Markdown files, without relying on any cloud databases or proprietary formats. This means users have full control over their data, eliminating subscription fees and avoiding the privacy risks associated with cloud services.
Practical Guide: Installation and Core Workflow
Users can deploy Claude-Obsidian in two main ways:
- Directly Clone the Repository: Download the project using the
git clone command, then execute the bash bin/setup-vault.sh script. This script will automatically configure Obsidian’s views, styles, and other settings for an out-of-the-box experience.
- Install as a Claude Code Plugin: In the Claude Code environment, add and install
AgriciDaniel/claude-obsidian through its plugin marketplace.
After installation, open the corresponding folder in Obsidian as a Vault. The system will guide you to trust the author and enable a series of recommended plugins, including Calendar, Thino, and Excalidraw, to enhance the note-taking experience. Next, start Claude Code in the project directory and enter the /wiki command. The system will then walk you through initializing the knowledge base, creating core files like index.md and hot.md.
The daily core workflow includes:
- Knowledge Ingestion: Drag and drop files into the
.raw/ directory, then instruct Claude to process them.
- Intelligent Q&A: Ask questions in natural language. The model will retrieve information following the priority order of
hot.md -> index.md -> specific pages, and provide answers with citations.
- Maintenance and Visualization: Periodically use the
lint command for health checks, or use the /canvas command to visually organize notes, images, PDFs, and other elements on a canvas compliant with the JSON Canvas 1.0 specification.
