What I Learned Building AI Agents on Top of the Obsidian CLI

I use Obsidian extensively as my personal knowledge management system. It tracks everything: daily notes, tasks, projects, meetings, people, and more. Over the past year I’ve been building AI agent workflows that operate on my vault – using tools like Claude Code and pi (an open-source coding agent) to automatically gather context, update notes, and maintain my system.

When Obsidian released their CLI in version 1.12, I was excited. Here was an official way to interact with the vault programmatically: read files, get backlinks, rename notes with automatic link updates. I immediately started building my agent tooling on top of it.

It worked great – until it didn’t. This post is about what I learned along the way.

The Setup

My vault has over 700 person files, hundreds of daily notes, tasks, projects, and more. I built a set of tools that my AI agents use to gather context about any wikilink in the vault:

  1. Get the note itself – read the full markdown content
  2. Find content references – find every place a wikilink is mentioned in other notes, with intelligent context extraction (preserving bullet hierarchies and paragraph boundaries)
  3. Find frontmatter references – find every note that references the wikilink in its YAML frontmatter (useful for structured relationships like meeting attendees or task assignments)

These tools power skills like “update a person note,” which gathers context from across the vault and populates a person file with relationship details, professional info, and personal notes. The whole thing is driven by the AI agent, which reads the skill instructions, calls the tools, analyzes the results, and writes the updated file.

The Obsidian CLI: What It Does Well

The Obsidian CLI is genuinely valuable for certain things.

Renaming notes is the killer feature. When you rename a note through the CLI, Obsidian automatically updates every wikilink across the entire vault. You simply cannot replicate this safely with grep and sed. The link index knows about aliases, case normalization, and ambiguous matches. This is a write operation that fundamentally requires Obsidian’s understanding of the vault’s link graph.

Backlink discovery is also useful. The CLI can tell you which files link to a given note, using Obsidian’s resolved link index. This handles edge cases that a simple text search might miss.

Single file reads are convenient when you don’t know the exact file path. Obsidian resolves the shortest unambiguous name, so you can ask for “John Smith” without needing to know it lives at People/John Smith.md.

Where It Breaks Down

The problem showed up when I tried to scale things up. I wanted to batch-update all 700+ empty person files in my vault — running the AI agent on each one sequentially. The agent would gather context, analyze it, and write the updated file. Simple enough.

Except each person update requires gathering context, which means:

  1. One CLI call to get backlinks (which files mention this person?)
  2. One CLI call per backlinked file to read its content

Each CLI call takes roughly one second. For someone mentioned in 561 files across my vault, that means find-refs alone would take over nine minutes. Even for a lightly-referenced person with 10 backlinks, that’s 10 seconds of I/O before the AI agent can even start thinking.

When I tried to run this at batch scale, the Obsidian CLI calls would sometimes hang entirely. I’d kick off an update of 5 people and the script would stall on the second or third person, waiting for CLI responses that never came.

The root cause is architectural: the Obsidian CLI is a one-call-at-a-time interface. Every invocation is a separate subprocess that communicates with the running Obsidian instance. There’s no batching, no streaming, no way to say “read me these 50 files in one shot.” And since the CLI requires Obsidian to be running, you’re limited by however fast Obsidian can service these requests.

The Fix: Filesystem for Reads, CLI for Writes

The solution was straightforward once I stopped trying to force everything through the CLI. I added a filesystem backend to my context-gathering tools that bypasses the CLI entirely for read operations:

  • Backlink discovery: Instead of obsidian-cli backlinks, I use ripgrep to scan the entire vault for [[Person Name]]. Ripgrep can search every markdown file in the vault in about 50 milliseconds.
  • File reads: Instead of obsidian-cli read, I just read the file directly from disk. This is essentially instant.

The results were dramatic. Gathering context for a person went from 10-60+ seconds (depending on reference count) down to about 0.2-0.4 seconds. The batch update script that was hanging after one person now processes five people in under a minute.

I kept the CLI available as a backend for interactive use where the convenience of name resolution is nice, and for write operations like rename-note where the CLI’s link-updating capability is difficult to replace.

Lessons for AI Agent Tooling

Building this system taught me a few things that I think generalize beyond Obsidian:

The control plane / data plane distinction matters. The Obsidian CLI is a great control plane – it knows things about your vault’s structure that the filesystem doesn’t, and it can make coordinated changes. But it’s a poor data plane. When an AI agent needs to read 50 files to build context, you want the fastest possible path to those bytes. That’s the filesystem.

AI agents have different performance requirements than humans. A one-second response time is usually not a huge deal for a human typing commands. But AI agent workflows involve tight loops of tool calls — gather context, analyze, gather more context, write output. Each second of latency compounds. A tool that feels fast to a human can be the bottleneck that makes an agent workflow impractical.

Design tools for fan-out. The most common pattern in my agent workflows is: “find all the things related to X, then read all of them.” This is inherently a fan-out operation. Tools that can only handle one item at a time will always be the bottleneck. If I were designing the Obsidian CLI for agent use cases, I’d add batch operations: read multiple files in one call, return backlinks with content included.

Start with the simplest thing that works, then optimize the bottleneck. My original tools used the CLI for everything, and they worked fine for single-note updates. It was only when I tried to batch-process hundreds of notes that the performance issue became obvious. The right time to add the filesystem backend was exactly when I did – when I had a concrete bottleneck, not before.

What Would Make the Obsidian CLI Better for Agents?

The Obsidian CLI is still in active development, and there are a few additions that would make it much more agent-friendly:

  • Batch read: obsidian-cli read file=A file=B file=C returning all contents in one response
  • Backlinks with content: obsidian-cli backlinks file=X --include-content so you get everything in a single round-trip instead of N+1 calls
  • A persistent connection mode: A socket or pipe interface that keeps a connection open, avoiding subprocess overhead per call

These would eliminate the need for the filesystem workaround in most cases. Until then, the hybrid approach works well – use CLI for writes and single lookups, filesystem for bulk reads.

Conclusion

If you’re building AI agent workflows on top of Obsidian, my advice is: use the CLI for what it’s uniquely good at (write operations, link-aware mutations, single convenient lookups), and use direct filesystem access for bulk context gathering. The CLI is a powerful tool, but AI agents are demanding users with very different performance needs than humans. Designing your tooling around that reality will save you a lot of frustration.

Notes on “Why Take Notes” by Mark Nagelberg

  • Author:: [[Mark Nagelberg]]
  • Source:: link
  • Reading Status:: [[complete]]
  • Review Status:: [[complete]]
  • Anki Tag:: nagelberg_why_take_notes
  • Anki Deck Link:: link
  • Blog Notes URL:: link
  • Tags:: #[[Spaced Repetition Newsletter]] #[[Blog Posts]] #[[PKM]] #[[note-taking]] #[[triple-pass system]] #retrieval #elaboration #[[knowledge management]] #[[articles]]
  • Notes

    • Two pillars of the "Triple-Pass System":
      • Note-taking
      • Spaced repetition
    • Why note-taking is necessary:
      • Preparing for [[Spaced Repetition]]
        • You don’t want to add directly to spaced repetition on first read – you’ll add too much unnecessary information or miss important context.
      • [[retrieval]] and [[elaboration]] practice
        • Reviewing, consolidating, and connecting your notes involves both retrieval and elaboration, which are beneficial for learning.
      • Computer-aided information [[retrieval]] and [[idea generation]]
        • Your notes can store more information and detail.
        • Digital search tools make it easy to look up information in your notes as well as find unexpected connections and insights you wouldn’t get from memory alone.

Why Take Notes?

Photo by Kelly Sikkema on Unsplash

You can access an Anki deck and Roam Research notes on this article here.

My personal knowledge management system (the “Triple-Pass System”) has two key pillars:

  • Note-taking (AKA “second brain”) to store consolidated information on the content that I read
  • Spaced repetition to retain the most important information from my notes forever, at minimal cost

You might wonder: why is note-taking necessary at all when you have the powerful mental prosthetic of spaced repetition? Spaced repetition makes memory a choice. Why put in the extra effort to take notes when you can just add stuff to your spaced repetition system and be done with it forever?

There are a few reasons why note-taking deserves a place in my personal knowledge management system:

  • Preparing for spaced repetition
  • Retrieval and elaboration practice
  • Computer-aided information retrieval and idea generation

Preparing for Spaced Repetition

It’s usually not a good idea to add information directly to your spaced repetition system while you are reading content for the first time. You are much more likely to add unnecessary information or miss the crux because you lack context from reading the whole piece.

Instead, using your notes as an initial place to store information gives you time to let the information “stew” before you add it to spaced repetition. This helps you faithfully follow rule number 1 of formulating knowledge: do not learn if you do not understand.

Taking notes first has the added benefit of flexibility over when you add material to spaced repetition. You can add to spaced repetition right away, or you can do it later when you have more time. In contrast, going straight to spaced repetition requires you to either add material right away or re-read the entire source document later, essentially starting from scratch.

Retrieval and Elaboration Practice

The research literature on efficient learning tells us that retrieval and elaboration (i.e. recalling things you have learned and re-interpreting them) is extremely beneficial.

This is exactly what you do when note-taking:

  • Review the material (which usually requires some recall)
  • Consolidate it into a form that’s easily consumable (elaboration)
  • Make connections with existing knowledge (recall and elaboration)
  • Intersperse ideas and alternate interpretations throughout your notes (elaboration)

Computer-Aided Information Retrieval and Idea Generation

If you think of your brain as a computer, your spaced repetition system is like your RAM: quickly accessible information. This is ideal for knowledge that you will benefit from being able to retrieve quickly.

In contrast, your note-taking system is like a hard drive: it’s slower to access (since you have to open your note-taking app), but it is capable of storing much larger quantities of information. This is great for looking up details that were not practical to commit to spaced-repetition.

Most digital note-taking tools also have sophisticated search functions that allow you to efficiently look up what you need. These features not only help with information retrieval, but also idea generation by helping you make unexpected connections with other knowledge. Roam Research is particularly good in this area, using a graph-based data model that lets you explore connections between your notes.

The Triple-Pass Method to Remember What you Learn, Forever

You read books. You watch videos. You listen to podcasts. There is a firehose of high-quality information available at your fingertips.

But how much do you actually get out of your media consumption? How much do you remember? For a long time, my answer was “not a whole lot”. I was often shocked at how few key points I could recall from material I had read only days or weeks earlier.

To me, this is unacceptable. Why bother doing so much reading if I’m just going to forget it all? Sure, if reading for entertainment, then no big deal. But most of the material I spend my time on is highly relevant to my professional and personal life. I want this knowledge to compound.

I’ve spent years experimenting, testing, and tweaking systems to get more value out of the media I consume and remember key points forever. I’ve eventually landed on a solution that I want to share with you: the “triple-pass” system. Using this system, I’m confident I’m getting everything I can out of my reading, and not a moment is wasted.

High-level Overview of the Triple-Pass System

At the highest level, the triple-pass system consists of the following stages:

  • First Pass: Active Media Consumption. Take notes and highlight important points from the media you consume (e.g. books, articles, videos).
  • Second Pass: Consolidate and Summarize Notes. Export notes and highlights from the first pass to a central note-taking system, then review, refine, and consolidate. This pass produces what Tiago Forte calls your “second brain“: an external system storing your knowledge in a format that can be easily searched and retrieved for use.
  • Third Pass: Commit to Long-Term Memory using Spaced Repetition. Add the most important parts of your notes from the second pass to a spaced repetition system. This stores the information efficiently in your long-term memory so you can access it in-the-moment when needed.

I go into each of these three stages in more detail below, including the technologies I use at each stage.

First Pass: Active Media Consumption

At this stage, you consume the media that you want to absorb. The key feature about this stage is that it is active media consumption, i.e. highlighting and taking notes as you read.

Here are some tips for getting the most out of the first pass:

  • Err on the side of more highlighting rather than less. It’s good to have a lot of context in your highlights. You can always eliminate things that are redundant or unnecessary in the second pass.
  • Highlight chapter titles. Titles provide useful context for organizing your notes and understanding the broader picture for an excerpt.
  • Prioritize the new, useful, and the interesting. There is no need for anything useless and uninteresting to exist in your knowledge base. Also, avoid including things you already know well.
  • Look for scaffolding. Keep an eye out for material that provides a “platform” to tackle more advanced concepts. For example, I always highlight definitions of important terms I’m unfamiliar with. I also like information about people, places, or things, since I can use these as context to learn related topics faster.

Technology I use for the First Pass

The tools I use at this stage vary depending on the type of media I’m consuming. There are two main criteria: i) the tool must allow me to record highlights and notes digitally, and ii) the tool must allow me to export materials to my note taking system (Roam) with little effort.

  • Books: Usually I purchase books on Kindle, which has excellent highlighting. I then use Readwise to automatically sync highlights to Evernote, and can copy and paste from Evernote to Roam (apparently Readwise now has a direct export to Roam, so it looks like I can cut out the Evernote middleman). For physical books, I use the Readwise mobile application, which has an OCR feature that lets you take a picture of the page you are reading and highlight it.
  • Articles / Blogs: I try to read all articles and blog posts on Instapaper, which allows me to highlight and take notes on articles. Again, Readwise allows me to import these notes into Evernote, which I copy / paste into Roam.
  • PDFs: At the moment I don’t have a good solution for PDFs. Since I can’t highlight them easily, I’ll often just take notes directly to Roam as I read, with the PDF open in one window and Roam open in another.
  • Videos: Typically I’ll sit at my computer when watching videos. So, I keep Roam open while watching the video and take notes, with timestamps. Here’s an example of video notes I put together for a Jeff Bezos Lecture on innovation.
  • Podcasts: I do listen to some podcasts (especially Conversations with Tyler), although I haven’t found a great tool for taking podcast notes. I usually listen to podcasts when I’m on the go, so note taking in the browser is typically not possible. Usually, if there’s a podcast that I listen to that’s really good, I’ll just revisit it and take notes while I listen at my computer desk.

Second Pass: Consolidate and Summarize Notes

In this pass, edit the highlights and notes that you’ve exported to your note-taking system. Activities here can include:

  • Deleting irrelevant or redundant notes
  • Summarizing excerpts into key points, while keeping some direct quotes from the material that is notable or quote-worthy
  • Reformulating material into your own words
  • Bold-facing the most important points so your notes are “glanceable”
  • Creating commentary on the material, expanding on points that you liked, critiquing points you disagreed with, filling in missing arguments, creating connections to other material in your knowledge base, or creating a high-level “book-review” style summary
  • Marking particularly important material for long-term memory

A key advantage of the second pass is it produces a valuable digital asset that you can draw on the rest of your life: you now have a searchable, quick-to-read summary of what you have read. The knowledge is now part of your “second brain” for easy access and connection to your existing knowledge.

Yet another advantage of this second pass is that the act of editing helps you absorb the material. This is because it requires elaboration and recall, which are both well-known to foster learning and memory.

Technology I use for the Second Pass

My note taking tool of choice is Roam. It is a fantastic piece of software, although it’s difficult to explain its value in words (you really just have to try it). I recommend looking into it if you are not already heavily invested in an existing note taking app. I find it allows me to easily make connections between knowledge, and its incredible functionality has led me to ditch Evernote, Asana, and 90% of Google Drive.

Third Pass: Commit to Long-Term Memory Using Spaced Repetition

In this third and final pass, add material material flagged “long-term memory” in the second pass to a spaced repetition system.

Spaced repetition is reviewing material at increasing intervals of time, allowing you to remember material with minimal effort. I won’t go into more detail here, but I highly recommend this overview by Gwern Branwen. You can also subscribe to my Spaced Repetition Newsletter.

The advantage of this third pass is access to your knowledge in the moment, when it matters, without any external note taking tools. This is useful for the many situations where it’s not feasible to consult your notes. For example: job interviews, meetings, or creative work where speed of thought is important and you need lots of in-memory scaffolding to make progress.

One nice feature of this third pass is you’ve thoroughly vetted the material in the first two passes. This means you’ll be less likely to add things you don’t need to your spaced repetition system, and you’ll only add information you understand (see rule 1 of flashcard knowledge construction: do not learn what you do not understand).

Technology I use for the Third Pass

Personally, I use Anki, which is probably the most popular spaced repetition software tool today. Some examples of other options include Mnemosyne, SuperMemo or even paper flashcards if you’re a technophobe.

Examples of the Triple-Pass System in Action

To get a feel for what the end result of this system looks like, here are a couple of examples:

This Seems like a lot of Work…

It’s true that using this system will almost certainly mean you’ll take more time to consume media. Compared to just passively reading a book, the highlighting, summarizing, and spaced repetition all add time.

But before you dismiss it, ask yourself a couple questions.

First: are you a genius that effortlessly absorbs the materials you consume? I don’t mean this sarcastically. People like this exist, like Tyler Cowen. If yes, there’s no real benefit to you from this system. It would just slow you down.

Second: why are you consuming the media in the first place? Is it something you really want to remember? If the answer is yes, then I believe using a system like this is a no-brainer.

Yes, it takes some extra time. But if you are consuming high-quality material relevant to your life, the benefits of that extra 10-30% of effort is well worth it.

For access to my shared Anki deck and Roam Research notes knowledge base as well as regular updates on tips and ideas about spaced repetition and improving your learning productivity, join “Download Mark’s Brain”.