Product Requirements

What indx is, who it's for, the goals and non-goals that bound v1.

Status: Draft v0.1 · 2026-05-03 Owner: indx core team One-liner: A self-hostable, AI-native knowledge vault that is Obsidian-compatible on disk and agent-driven by design.

1. Vision

Your second brain, designed to be operated by AI agents and humans equally.

A vault is a folder of plain markdown. indx keeps that contract — same files, same [[wikilinks]], same .canvas, same .obsidian/ — and adds three first-class machine surfaces (CLI, HTTP API, MCP) plus a thin humane web UI.

The product principle: every operation that a human can do in the UI must be doable by an agent in one tool call, deterministically, without screen-scraping.

2. Background & motivation

Obsidian is excellent for humans but built around a desktop GUI and a sandboxed plugin model. Driving it from an agent is awkward (URL handlers, brittle filesystem patches, plugin gymnastics).
Existing “AI-on-notes” stacks are usually wrappers that read a vault but cannot safely write to one — and lock users into a proprietary store.
The agent ecosystem (MCP, OpenAI tool calling, Claude Skills) is converging on tool-shaped APIs. The vault should expose itself in that shape.
Self-hosting is non-negotiable for users with private notes. Single-binary / single-container deployment is the expected experience.

3. Users

Persona	Description	Why they care
Agent operator (primary)	A power user running Claude Code, Cursor, an autonomous agent, or a custom workflow against their notes.	Needs deterministic tools, structured outputs, idempotent writes.
Self-hoster (secondary)	Privacy-focused user who already runs their own services.	Wants single-container deployment, low resource footprint, no telemetry.
Existing Obsidian user (secondary)	Has years of `.md` files in a vault folder, possibly synced via iCloud/Git/Syncthing.	Must not be forced to migrate or convert formats.
Library author (tertiary)	Wants to script against vaults from Python, Go, Rust.	Needs a stable HTTP API and OpenAPI spec.

The “agent” is treated as a real user with real needs, not as a backend integration.

4. Goals

Obsidian on-disk compatibility. Open an existing vault directory unchanged and write back changes that Obsidian can re-open without corruption.
Three equally-supported surfaces. CLI, HTTP API, and MCP all expose the same capability set; the web UI is built on top of the API.
Agent-grade ergonomics. Determinism, idempotency, structured errors, schema introspection, atomic edits, ETags, dry-run.
Single-container deployment. One docker run with a vault volume mount; no external Postgres/Redis/etc. required.
Lightweight. <100 MB image, <256 MB RAM at idle, cold start <2 s, p95 read <50 ms, p95 search <200 ms on 10 k notes.
Bring-your-own AI. Embeddings and chat models are pluggable (Vercel AI Gateway, OpenAI-compatible endpoints, local Ollama). The product works fully without any AI key — semantic search becomes lexical-only.
Open spec, open source. MIT-licensed; the API is OpenAPI 3.1; the on-disk format is Obsidian’s existing format plus a small .indx/ index folder that is recreatable.

5. Non-goals (v1)

Plugin ecosystem parity with Obsidian.
Native mobile apps. (Web UI must be responsive; that’s the bar.)
Real-time multi-user collaborative editing.
End-to-end encryption of vault contents at rest. (Trust the host filesystem; document the threat model.)
Built-in chat assistant in the UI. (Ship the surfaces; let agents bring themselves.)
Built-in vault sync between machines. (Use Git / Syncthing / iCloud — the vault is just a folder.)

6. Key user journeys

J1. Agent reads and edits a note

An agent calls note_read with a path, gets the markdown body + parsed frontmatter + structured outline, calls note_patch to insert a new section under a specific heading, and gets back the new ETag. No diff-then-write race conditions; no GUI involved.

J2. Agent searches across the vault

An agent calls vault_search with { q, mode: "hybrid", limit: 20 }. Receives ranked results with snippets, paths, scores, and link counts. Drills in by note_read on selected paths.

J3. Agent maintains the link graph

An agent calls link_backlinks for a note, sees orphans, calls note_patch to add wikilinks. Calls vault_status and confirms orphan_count decreased.

J4. Human inspects what an agent did

The user opens the web UI, sees a recent-changes panel with who: agent annotations, scrubs through diffs, and rolls back if needed. (v1 ships the panel; rollback is v1.1.)

J5. Self-hosting setup

User runs one docker run, points it at an existing Obsidian vault, gets a token, and is editing via UI/CLI/API/MCP in under 60 seconds.

J6. Existing vault zero-touch

User points indx at a 5-year-old Obsidian vault. Indx reads .obsidian/ settings (read-only), builds its index in .indx/, and never writes to .obsidian/. User can quit indx, open Obsidian on the same folder, and see no surprises.

7. Functional scope (MVP)

Area	In scope (v1)	Out of scope (v1)
Notes	CRUD, move, frontmatter, body, headings, blocks, wikilinks, embeds, callouts, tags, math, mermaid	Inline plugins, custom CSS
Canvas	Read + write `.canvas` (JSON Canvas 1.0 spec)	Visual canvas editor (read-only viewer in v1, edit by patch)
Bases	Read + query `.base` files	Visual builder UI
Search	Lexical (SQLite FTS5), tag, frontmatter filters, hybrid (lexical + vector)	Cross-vault search
Graph	Backlinks, forward links, orphan detection, tag co-occurrence	Force-directed visual graph (basic version only)
Daily notes	Date-templated note creation	Calendar plugin parity
Watch	Detect external file changes (chokidar), reindex incrementally, broadcast over SSE	Real-time collaborative cursors
Auth	Static bearer tokens with optional scopes	OIDC, per-user vaults, RBAC
Multi-vault	One vault per running container	Vault-switcher UI

8. Quality bar (success metrics)

Metric	Target
Docker image size (compressed)	< 100 MB
Idle RAM	< 256 MB
Cold start to ready	< 2 s
p50 / p95 note read	5 / 50 ms
p50 / p95 search (10 k notes)	30 / 200 ms
Round-trip Obsidian compatibility	100% — sample 1000-note vault open/edit/save in indx must produce byte-identical files in Obsidian when no edit is intended; semantic-identical when edits occur
Agent task success rate (internal eval)	≥95% on 50-task benchmark covering CRUD, search, link-graph, canvas

9. Constraints & principles

The vault is the truth. Indx never owns user data; it indexes a directory. Deleting .indx/ must be safe and recoverable by reindex.
Idempotency-by-default. Every write tool accepts an Idempotency-Key; every patch is content-addressed (ETag).
No hidden state. All writes go through the same code path whether they originate in UI, API, CLI, or MCP.
No telemetry. Zero outbound calls unless an AI provider is configured.
Boring tech. SQLite, Node 24 LTS, Next.js, plain markdown. No exotic services in the container.
Schema-first. Zod schemas → TypeScript types → JSON Schema → OpenAPI 3.1 → MCP tool schemas. One source of truth.

10. Risks

Risk	Mitigation
Obsidian-format edge cases (block IDs, exotic embeds) corrupt files on round-trip	Property-based tests against a corpus of real-world vaults; AST-preserving editor
Agent writes a malformed wikilink and breaks the graph	Schema validation on every write; auto-fix common mistakes; emit warnings on parse anomalies
Index drift after external edits	File watcher + content hashing; `vault_reindex` is always available and cheap
Single-container constraint conflicts with embeddings provider	Embeddings provider is remote (Vercel AI Gateway / OpenAI-compatible) and optional; vector index can be SQLite-based locally
Token-based auth too weak for multi-user setups	v1 ships single-tenant; multi-user OIDC is a v2 line item, documented up front

11. Open questions

Q1. Should .indx/ live inside the vault (visible to Git) or in a separate path? Lean: inside vault, with a recommended .gitignore line.
Q2. Default embeddings model? Lean: none — semantic search is opt-in.
Q3. Should the CLI talk to a running indx server, or hit the vault directly? Lean: both — indx --local mode for offline scripts; default mode hits the API for consistency.
Q4. How do we handle Obsidian plugin-specific frontmatter we don’t understand? Lean: round-trip preserve, surface as unknown_keys in API responses.

12. Release plan

v0.1 (alpha): CLI + API + MCP for notes (CRUD, patch, search, links, tags). Web UI: file tree, editor, search, recent activity. Single Docker image. Bearer auth.
v0.2 (beta): Canvas read+write, Bases query, SSE event stream, OpenAPI client codegen, vector search, AI runtime skeleton (see AI.md) — ai_status, ai_toc (single-note offline mode), opt-in chat ops behind provider config.
v1.0: Production-ready bar on metrics in §8, agent-task benchmark passing, docs site, hosted demo, full AI runtime (summarize / ask / toc MOC mode / relate) with citation verification, cache, daily cost ceiling.
v1.1+: Diff/rollback UI, multi-tenant OIDC, plugin compatibility shim for a curated subset of Obsidian plugins.