Implementation Plan

Repository layout, per-package policy, surface adapters, cross-cutting policies.

Status: Draft v0.1 · 2026-05-03 Companion docs: PRD.md, SPEC.md, API.md, CLI.md, MCP.md, AI.md

This document is the bridge between the specs and the codebase. It defines:

The full repository file tree.
The internal API of every package and the route surface of the web app.
A per-file implementation policy — what each file is responsible for, what it must not do, and how it interacts with the rest of the system.

The document is opinionated. Where a choice could go several ways, the answer here is the one we ship.

0. Architectural ground rules (recap)

These are load-bearing across every file in the repo. If a file would violate one, the file is wrong, not the rule.

One source of truth for shapes. Every payload is a Zod schema in @indx/shared. TypeScript types, JSON Schema (for MCP), and OpenAPI 3.1 (for the HTTP API) are derived — never hand-written.
One engine, three surfaces. @indx/core exposes one capability per operation. apps/web route handlers, packages/cli commands, and packages/mcp tools are thin adapters that translate transport in/out and call core. No business logic in the adapters.
The vault is the truth. Anything in .indx/ is reproducible from the vault. Nothing in .obsidian/ is ever written by indx.
Atomic, content-addressed writes. Every write goes through core.vault.writeNote (or a .canvas / .base peer). ETag = first 16 hex of xxhash64(bytes). If-Match is honored everywhere, including the CLI in local mode.
No hidden actors. Every write carries an Actor context (ui/api/cli/mcp/fs). Events log it. Tests assert it.
Errors are typed. Core throws IndxError subclasses with stable codes. Adapters map to HTTP status / CLI exit code / MCP isError. No throw new Error("...") outside of panic paths.
No telemetry, no outbound calls. The only outbound call indx ever makes is to a configured embeddings provider, and only for embedding requests.
Boring tech. SQLite, Node 24, Next.js 16, Zod, remark, chokidar. New dependencies require an entry in docs/DECISIONS.md (when that file exists) and a “why not the existing stack” note.

1. Repository layout

indx-app/
├── apps/
│   └── web/                              # Next.js 16 app — UI + REST + SSE + MCP HTTP
│       ├── next.config.mjs
│       ├── postcss.config.mjs
│       ├── tailwind.config.ts
│       ├── tsconfig.json
│       ├── package.json
│       ├── public/                       # static assets only (no business logic)
│       └── src/
│           ├── app/
│           │   ├── layout.tsx            # root RSC layout
│           │   ├── globals.css
│           │   ├── page.tsx              # landing surface index (current scaffold)
│           │   ├── (ui)/                 # human web UI route group
│           │   │   ├── layout.tsx        # app chrome (sidebar, command bar)
│           │   │   ├── page.tsx          # vault home (recent + pinned)
│           │   │   ├── notes/
│           │   │   │   └── [...path]/page.tsx     # editor + preview
│           │   │   ├── search/page.tsx
│           │   │   ├── graph/page.tsx
│           │   │   ├── tags/page.tsx
│           │   │   ├── canvas/[...path]/page.tsx
│           │   │   ├── bases/[...path]/page.tsx
│           │   │   └── activity/page.tsx
│           │   ├── api/                  # parallel to (ui), but pure routes
│           │   │   ├── v1/
│           │   │   │   ├── health/route.ts
│           │   │   │   ├── notes/
│           │   │   │   │   ├── route.ts                    # GET (list)
│           │   │   │   │   ├── [...path]/
│           │   │   │   │   │   ├── route.ts                # GET / PUT / PATCH / DELETE
│           │   │   │   │   │   └── move/route.ts           # POST
│           │   │   ├── search/route.ts
│           │   │   ├── links/
│           │   │   │   ├── unresolved/route.ts
│           │   │   │   ├── orphans/route.ts
│           │   │   │   └── [...path]/
│           │   │   │       ├── backlinks/route.ts
│           │   │   │       └── forward/route.ts
│           │   │   ├── tags/
│           │   │   │   ├── route.ts                        # GET (all)
│           │   │   │   ├── rename/route.ts                 # POST
│           │   │   │   └── [tag]/notes/route.ts            # GET
│           │   │   ├── canvas/[...path]/route.ts           # GET / PUT / PATCH
│           │   │   ├── bases/
│           │   │   │   └── [...path]/
│           │   │   │       ├── route.ts                    # GET / PUT
│           │   │   │       └── query/route.ts              # GET (paginated or NDJSON)
│           │   │   ├── vault/
│           │   │   │   ├── status/route.ts
│           │   │   │   ├── reindex/route.ts
│           │   │   │   └── config/route.ts
│           │   │   ├── ai/
│           │   │   │   ├── status/route.ts                  # GET (cheap, no upstream call)
│           │   │   │   ├── summarize/route.ts               # POST, SSE on Accept: text/event-stream
│           │   │   │   ├── ask/route.ts                     # POST, streaming
│           │   │   │   ├── toc/route.ts                     # POST (note | moc; write goes through notes pipeline)
│           │   │   │   ├── relate/route.ts                  # POST
│           │   │   │   ├── tag/route.ts                     # POST (suggest / apply via notes.patch)
│           │   │   │   ├── metadata/route.ts                # POST (suggest / apply via notes.patch)
│           │   │   │   └── extract/route.ts                 # POST (suggest / apply via notes.patch)
│           │   │   └── events/route.ts                     # SSE
│           │   ├── openapi.json/route.ts                   # served live
│           │   └── mcp/route.ts                            # MCP HTTP transport
│           ├── components/                # client + server components for the UI
│           │   ├── editor/CodeMirror.tsx
│           │   ├── editor/PatchPalette.tsx
│           │   ├── tree/FileTree.tsx
│           │   ├── search/SearchBar.tsx
│           │   ├── search/Results.tsx
│           │   ├── graph/GraphView.tsx
│           │   ├── activity/ActivityFeed.tsx
│           │   └── chrome/AppShell.tsx
│           ├── lib/
│           │   ├── server/
│           │   │   ├── core.ts            # singleton VaultHandle accessor
│           │   │   ├── runtime.ts         # boot vault on first request, cache
│           │   │   ├── auth.ts            # bearer + scope check
│           │   │   ├── actor.ts           # build Actor from request
│           │   │   ├── envelope.ts        # ok/err response builders
│           │   │   ├── etag.ts            # If-Match / If-None-Match helpers
│           │   │   ├── idempotency.ts     # write-replay cache
│           │   │   ├── rate-limit.ts      # per-token sliding window
│           │   │   ├── ndjson.ts          # streamed NDJSON helpers
│           │   │   ├── sse.ts             # SSE writer
│           │   │   └── path.ts            # decode + validate path params
│           │   └── client/
│           │       ├── api.ts             # browser API client (typed via shared)
│           │       └── sse.ts             # SSE consumer
│           └── middleware.ts              # auth + actor injection on /v1, /mcp
├── packages/
│   ├── shared/                           # Zod schemas, types, generators
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   └── src/
│   │       ├── index.ts
│   │       ├── schemas/
│   │       │   ├── note.ts                # already scaffolded
│   │       │   ├── patch.ts               # already scaffolded
│   │       │   ├── search.ts              # already scaffolded
│   │       │   ├── event.ts               # already scaffolded
│   │       │   ├── error.ts               # already scaffolded
│   │       │   ├── canvas.ts              # JSON Canvas 1.0 types + canvas patches
│   │       │   ├── base.ts                # .base file schema + query result rows
│   │       │   ├── link.ts                # links / backlinks / orphans / unresolved
│   │       │   ├── tag.ts                 # tag list / rename
│   │       │   ├── vault.ts               # status / config / reindex
│   │       │   ├── ai.ts                  # AiScope, SummarizeInput/Output, AskInput/Output, TocInput/Output, RelateInput/Output, AiUsage, AiWarning, Citation
│   │       │   └── http.ts                # request envelopes + headers
│   │       └── openapi/
│   │           ├── builder.ts             # zod → JSON Schema → OpenAPI 3.1
│   │           └── routes.ts              # route × schema registry
│   ├── core/                             # vault engine
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   └── src/
│   │       ├── index.ts                   # public exports
│   │       ├── config.ts                  # CoreConfig type, env loader
│   │       ├── errors.ts                  # IndxError subclasses
│   │       ├── actor.ts                   # Actor type + AsyncLocalStorage
│   │       ├── paths.ts                   # vault-relative path normalization
│   │       ├── hash.ts                    # xxhash64 wrapper, etag()
│   │       ├── vault/
│   │       │   ├── handle.ts              # VaultHandle (open/close, lifecycle)
│   │       │   ├── fs.ts                  # safe read/write/rename in vault root
│   │       │   ├── walk.ts                # full vault scan
│   │       │   ├── watch.ts               # chokidar wrapper
│   │       │   ├── note.ts                # readNote / writeNote / patchNote / move / delete
│   │       │   ├── canvas.ts              # readCanvas / writeCanvas / patchCanvas
│   │       │   ├── base.ts                # readBase / writeBase / queryBase
│   │       │   └── attachment.ts          # readAttachment (write deferred)
│   │       ├── md/
│   │       │   ├── pipeline.ts            # unified processor factory
│   │       │   ├── parse.ts               # parseMarkdown(buf) → MdAst
│   │       │   ├── serialize.ts           # serializeMarkdown(MdAst) → string
│   │       │   ├── frontmatter.ts         # YAML get/set/round-trip
│   │       │   ├── outline.ts             # heading extraction
│   │       │   ├── links.ts               # wikilink + embed extraction
│   │       │   ├── tags.ts                # inline + frontmatter tags
│   │       │   ├── blocks.ts              # ^block-id extraction
│   │       │   ├── plain.ts               # body-text projection for FTS
│   │       │   └── patch/
│   │       │       ├── apply.ts           # dispatch on Patch.op
│   │       │       ├── frontmatter.ts     # set/delete frontmatter ops
│   │       │       ├── body.ts            # append/prepend body
│   │       │       ├── heading.ts         # insert/replace/rename around headings
│   │       │       └── block.ts           # replace_block
│   │       ├── canvas/
│   │       │   ├── parse.ts
│   │       │   ├── serialize.ts
│   │       │   └── patch.ts               # add_node / update_node / add_edge / move_node ...
│   │       ├── base/
│   │       │   ├── parse.ts               # YAML → Base schema
│   │       │   ├── serialize.ts
│   │       │   └── query.ts               # filter + formula evaluator over the index
│   │       ├── index/
│   │       │   ├── db.ts                  # better-sqlite3 open + migrations
│   │       │   ├── schema.sql             # CREATE TABLE statements (SPEC §5.1)
│   │       │   ├── migrations/
│   │       │   │   └── 0001_init.sql
│   │       │   ├── notes.ts               # upsert/delete row, hash-diff
│   │       │   ├── links.ts               # link rows
│   │       │   ├── tags.ts                # tag rows
│   │       │   ├── blocks.ts              # block rows
│   │       │   ├── fts.ts                 # FTS5 wrappers
│   │       │   ├── vss.ts                 # optional sqlite-vss
│   │       │   └── reindex.ts             # full + incremental drivers
│   │       ├── search/
│   │       │   ├── lexical.ts
│   │       │   ├── semantic.ts
│   │       │   ├── hybrid.ts              # reciprocal rank fusion
│   │       │   ├── filters.ts             # path_glob, tag, frontmatter, links
│   │       │   ├── snippet.ts             # snippet rendering
│   │       │   └── cursor.ts              # opaque cursor codec
│   │       ├── links/
│   │       │   ├── resolve.ts             # raw [[X]] → vault path
│   │       │   ├── backlinks.ts
│   │       │   ├── forward.ts
│   │       │   ├── orphans.ts
│   │       │   └── unresolved.ts
│   │       ├── embeddings/
│   │       │   ├── provider.ts            # interface + factory
│   │       │   ├── vercel-ai-gateway.ts
│   │       │   ├── openai.ts
│   │       │   └── ollama.ts
│   │       ├── ai/
│   │       │   ├── index.ts               # AiApi (status/summarize/ask/toc/relate)
│   │       │   ├── chat/
│   │       │   │   ├── provider.ts        # ChatProvider interface + factory
│   │       │   │   ├── vercel-ai-gateway.ts
│   │       │   │   ├── openai.ts
│   │       │   │   └── ollama.ts
│   │       │   ├── prompt/                # versioned prompt templates per op
│   │       │   │   ├── summarize.ts
│   │       │   │   ├── ask.ts
│   │       │   │   ├── toc.ts
│   │       │   │   ├── relate.ts
│   │       │   │   ├── tag.ts
│   │       │   │   ├── metadata.ts
│   │       │   │   └── extract.ts
│   │       │   ├── scope.ts               # AiScope resolver (paths/glob/tag/query/note+links)
│   │       │   ├── retrieve.ts            # RAG retriever (delegates to search/*)
│   │       │   ├── chunk.ts               # token-budgeted chunker; map-reduce planner
│   │       │   ├── citations.ts           # post-hoc citation verification
│   │       │   ├── stream.ts              # SSE/NDJSON/MCP-progress shim
│   │       │   ├── cache.ts               # .indx/ai-cache.db (sqlite, content-addressed)
│   │       │   ├── cost.ts                # per-call cost estimator + daily ceiling
│   │       │   ├── apply.ts               # shared per-note apply driver (path-sorted, if_match, atomic per note)
│   │       │   ├── normalize.ts           # tag/frontmatter normalization (Obsidian list shape, ISO dates, …)
│   │       │   ├── validate.ts            # field/schema validators (regex, enum, min/max, JSON Schema)
│   │       │   └── ops/
│   │       │       ├── summarize.ts
│   │       │       ├── ask.ts
│   │       │       ├── toc.ts             # note-mode (no LLM) + moc-mode + write delegation
│   │       │       ├── relate.ts
│   │       │       ├── tag.ts             # vocabulary biasing + suggest/apply
│   │       │       ├── metadata.ts        # typed fields → constrained generation → validate → apply
│   │       │       └── extract.ts         # JSON Schema input → constrained generation → validate → apply
│   │       ├── events/
│   │       │   ├── bus.ts                 # in-process pub/sub (EventEmitter-ish)
│   │       │   ├── log.ts                 # rolling NDJSON to .indx/events.log
│   │       │   └── replay.ts              # Last-Event-ID replay
│   │       ├── auth/
│   │       │   ├── tokens.ts              # parse INDX_TOKEN[S], scope evaluator
│   │       │   └── scope.ts               # scope match (incl. path:<glob>)
│   │       ├── ratelimit/
│   │       │   └── sliding.ts             # in-memory sliding window
│   │       ├── idempotency/
│   │       │   └── cache.ts               # bounded LRU keyed by (token,method,path,key)
│   │       └── observability/
│   │           ├── log.ts                 # pino factory
│   │           └── otel.ts                # optional OTel hooks
│   ├── cli/                              # `indx` binary
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   ├── bin/indx.mjs                   # already scaffolded
│   │   └── src/
│   │       ├── index.ts                   # main(argv); already scaffolded
│   │       ├── parse.ts                   # arg parser (no `commander` dep)
│   │       ├── globals.ts                 # global flags + env precedence
│   │       ├── output.ts                  # ok / fail / ndjson writers
│   │       ├── exit.ts                    # exit code map
│   │       ├── transport/
│   │       │   ├── index.ts               # Transport interface
│   │       │   ├── local.ts               # in-process @indx/core
│   │       │   └── remote.ts              # fetch against /v1
│   │       └── commands/
│   │           ├── note.ts
│   │           ├── link.ts
│   │           ├── tag.ts
│   │           ├── search.ts
│   │           ├── canvas.ts
│   │           ├── base.ts
│   │           ├── vault.ts
│   │           ├── ai.ts                   # status / summarize / ask / toc / relate (+ --stream)
│   │           ├── serve.ts
│   │           ├── mcp.ts
│   │           └── schema.ts
│   └── mcp/                              # MCP server, shared by web /mcp and CLI stdio
│       ├── package.json
│       ├── tsconfig.json
│       └── src/
│           ├── index.ts                   # createMcpServer(deps)
│           ├── identity.ts                # name/version/instructions
│           ├── registry.ts                # tool/resource/prompt registry
│           ├── tools/
│           │   ├── index.ts               # registers all tools, gates by scope + AI availability
│           │   ├── note.ts                # note_read, note_write, note_patch, ...
│           │   ├── search.ts
│           │   ├── links.ts
│           │   ├── tags.ts
│           │   ├── canvas.ts
│           │   ├── base.ts
│           │   ├── ai.ts                  # ai_status, ai_summarize, ai_ask, ai_toc, ai_relate
│           │   └── vault.ts
│           ├── resources/
│           │   ├── index.ts               # registers vault://, vault://search?, ai:// (when enabled)
│           │   ├── note.ts
│           │   ├── search.ts
│           │   ├── graph.ts
│           │   ├── tags.ts
│           │   ├── canvas.ts
│           │   ├── base.ts
│           │   └── ai.ts                  # ai://summary/<scope>, ai://moc/<scope>
│           └── prompts/
│               ├── index.ts
│               ├── vault_summary.ts
│               ├── daily_note.ts
│               ├── link_orphans.ts
│               ├── cleanup_unresolved_links.ts
│               └── weekly_review.ts
├── tests/                                # cross-package integration + e2e
│   ├── corpus/                           # sample vaults for round-trip tests
│   ├── round-trip.test.ts
│   ├── api.test.ts
│   ├── cli.test.ts
│   ├── mcp.test.ts
│   └── docker.test.ts                    # boots image, hits /v1/health
├── scripts/
│   ├── gen-openapi.ts                    # writes openapi.json snapshot for CI
│   ├── gen-mcp-schemas.ts
│   └── corpus-fetch.ts                   # downloads test vaults
├── vault/                                # placeholder; gitignored except .keep
├── docs/                                 # PRD, SPEC, API, CLI, MCP, IMPLEMENTATION (this file)
├── Dockerfile
├── docker-compose.yml
├── package.json
├── pnpm-workspace.yaml
├── tsconfig.base.json
├── .env.example
└── README.md

Notes on layout:

The web app’s UI lives under a route group (ui)/ so its layout doesn’t leak into /v1, /mcp, /openapi.json. Those three are pure JSON/SSE and must not inherit any HTML chrome.
packages/mcp is its own package so the same registry serves both the Next.js HTTP transport (apps/web/src/app/mcp/route.ts) and the CLI stdio mode (packages/cli/src/commands/mcp.ts). No fork.
tests/ is at repo root because every package’s tests need a real vault directory and a built apps/web server. Per-package unit tests still live under each package’s src/__tests__/.

2. `@indx/shared` — schemas and generators

2.1 What this package owns

Every payload shape that crosses a process or surface boundary (HTTP request/response, MCP tool input/output, CLI JSON output, event log).
The OpenAPI 3.1 builder.
Nothing else. No I/O, no Node-only imports — this package must be safe to bundle into the browser too.

2.2 File-by-file policy

src/index.ts — pure barrel. Re-export schemas grouped by domain. Never add logic here.

src/schemas/note.ts (scaffolded) — VaultPath, NoteKind, OutlineEntry, Note. Add: NoteRef (path + etag only), NoteListItem (lightweight projection), NoteCreateInput, NoteReplaceInput, NoteListQuery.

src/schemas/patch.ts (scaffolded) — Patch discriminated union, PatchRequest. Add PatchResult = Note (read-after-write). The patch grammar must match SPEC §6.2 byte-for-byte.

src/schemas/search.ts (scaffolded) — SearchMode, SearchRequest, SearchHit, SearchResponse. Add Cursor brand type — never expose its inner shape.

src/schemas/event.ts (scaffolded) — Actor, VaultEvent discriminated union. Add EventEnvelope = { id: string; event: VaultEvent } for SSE wrappers.

src/schemas/error.ts (scaffolded) — ErrorCode, ApiError, Envelope. Add helpers ok(data) / err(error) schema-typed builders for the web app to import.

src/schemas/canvas.ts — JSON Canvas 1.0 schema (CanvasNode = TextNode | FileNode | LinkNode | GroupNode, CanvasEdge, Canvas = { nodes, edges }). Plus CanvasPatch discriminated union (add_node, update_node, remove_node, add_edge, remove_edge, move_node). Unknown fields preserved via .passthrough() per SPEC §4.8.

src/schemas/base.ts — .base YAML schema: BaseFile = { filters, formulas, properties, views }. BaseRow = Record<string, unknown>. BaseQueryRequest = { limit?, cursor? }.

src/schemas/link.ts — LinkRow, BacklinksResponse, ForwardLinksResponse, OrphansResponse, UnresolvedResponse.

src/schemas/tag.ts — TagCount, TagListResponse, TagRenameRequest, TagRenameResult.

src/schemas/vault.ts — VaultStatus, VaultConfig, ReindexRequest, ReindexJob.

src/schemas/http.ts — IdempotencyKey brand, header schemas, generic Paginated<T> helper.

src/openapi/builder.ts — wraps a Zod → JSON Schema converter (zod-to-json-schema) and returns an OpenAPIObject. Uses $ref for the canonical schema set; never inlines duplicates. Pure function: buildOpenApi(routes: RouteDef[]): OpenApiDocument.

src/openapi/routes.ts — registry: [{ method, path, summary, request: { params, query, headers, body }, responses, scopes }]. Every API route file in apps/web registers itself here at module-load time or (preferred for tree-shaking) the registry is the single literal that the web app’s route handlers also import.

Policy for new schemas:

Define the Zod schema here first.
Export the inferred TS type (export type X = z.infer<typeof X>).
Add a route entry in openapi/routes.ts if the schema is HTTP-bound.
Add the MCP tool schema in packages/mcp referencing the same Zod schema (don’t re-declare).

3. `@indx/core` — vault engine

3.1 What this package owns

The actual filesystem, parsing, indexing, search, link graph, events.
Every read and write the system performs.
Nothing surface-shaped: no HTTP, no MCP transport, no argv parsing.

3.2 Public surface (`src/index.ts`)

export { openVault, type VaultHandle } from "./vault/handle.js";
export type { CoreConfig } from "./config.js";
export { type Actor } from "./actor.js";
export {
  IndxError, NotFound, EtagMismatch, AlreadyExists, ParseFailed,
  ReadOnly, Forbidden, Reindexing, ValidationFailed,
  AiUnavailable, AiProviderError, AiQuotaExceeded, AiGroundingFailed,
  AiInputTooLarge, AiSchemaInvalid, AiApplyConflict
} from "./errors.js";

VaultHandle is the only object an adapter ever holds. Its shape:

interface VaultHandle {
  readonly config: CoreConfig;
  readonly notes:   NotesApi;       // read / write / patch / delete / move / list / outline / exists
  readonly canvas:  CanvasApi;      // read / write / patch
  readonly bases:   BasesApi;       // read / write / query
  readonly search:  SearchApi;      // search
  readonly links:   LinksApi;       // backlinks / forward / orphans / unresolved
  readonly tags:    TagsApi;        // list / notes / rename
  readonly vault:   VaultAdminApi;  // status / reindex / config get/set
  readonly ai:      AiApi | null;   // status / summarize / ask / toc / relate; null when no provider
  readonly events:  EventBus;       // subscribe / replay / publish (internal)
  close(): Promise<void>;
}

All methods are async, all accept an actor: Actor argument as the first argument. No method silently uses an ambient actor.

Each method’s input/output is typed with a schema from @indx/shared. For example:

notes.write(actor: Actor, input: { path: string; body: string; frontmatter?: unknown; ifMatch?: string; ifNotExists?: boolean }): Promise<Note>
notes.patch(actor: Actor, input: { path: string; ops: Patch[]; ifMatch?: string }): Promise<Note>

3.3 File-by-file policy

Lifecycle and config

src/config.ts — CoreConfig type (scaffolded). Add loadConfigFromEnv(env: NodeJS.ProcessEnv): CoreConfig and mergeWithVaultConfig(c: CoreConfig, vaultConfig?: VaultConfig): CoreConfig. Env wins for safety knobs (INDX_READONLY); vault config wins for vault-flavor knobs (markdown.list_marker).

src/vault/handle.ts — openVault(config):

Resolve vaultPath to an absolute, real path (fs.realpath); reject if not a directory.
Ensure <vault>/.indx/ exists; create if missing (unless readOnly).
Open the SQLite db; run migrations.
Read .obsidian/app.json if present; fold relevant fields into the merged config (read-only).
Read .indx/config.json; merge.
Kick a non-blocking full-walk reindex if the index is empty, else an incremental scan.
Start the chokidar watcher.
Return VaultHandle with the API objects wired up.

close() flushes pending events, stops the watcher, closes the db, drains the event log writer.

src/errors.ts — class hierarchy:

class IndxError extends Error {
  abstract readonly code: ErrorCode;     // matches @indx/shared
  abstract readonly status: number;      // suggested HTTP status
  details?: Record<string, unknown>;
}

with one subclass per ErrorCode. Adapters import these directly to map.

src/actor.ts — Actor type, an AsyncLocalStorage<Actor> for emergency lookup (only used by event log writer when an explicit actor arg can’t be threaded through). Helper withActor(a, fn) for adapters.

src/paths.ts — normalizeVaultPath(input: string): string — strip leading /, reject .., collapse duplicate slashes, NFC-normalize. resolveAbs(vaultRoot, p) — joins safely, throws Forbidden on escape. caseInsensitiveFallback(vaultRoot, p) — used only when case_insensitive_match is configured.

src/hash.ts — xxhash64(buf: Buffer): bigint, etag(buf: Buffer): string (first 16 hex chars). Implementations may use @node-rs/xxhash or a pure JS fallback; the contract is just stability.

Vault FS layer

src/vault/fs.ts — only file in the engine that calls node:fs directly outside of the watcher. Exposes:

readBytes(rel) / writeBytesAtomic(rel, buf, { ifMatch? }) / rename(from, to) / unlink(rel) / stat(rel).
writeBytesAtomic writes to <rel>.indx-tmp-<rand> then rename()s; refuses to leave temp files on error.
All paths are normalized first; refusal to operate outside vaultRoot is enforced here, not by callers.

src/vault/walk.ts — walkVault(root): AsyncIterable<{ rel, stat, buf? }>. Lazily yields; respects ignore config; never reads .obsidian/ content beyond what config-loading needs.

src/vault/watch.ts — chokidar wrapper with 200 ms debounce per path. Emits internal events {kind: 'add'|'change'|'unlink'|'rename', rel, mtime_ms} consumed by index/reindex.ts and events/bus.ts (with actor: { kind: "fs" }).

src/vault/note.ts — the orchestration layer. read(path) joins fs + index + md to return a hydrated Note. write(path, body, frontmatter, opts) is the canonical write path:

1. validateInputs()                       (Zod, paths.ts)
2. parseMarkdown(combined)                (md/parse.ts) — must succeed, else ParseFailed
3. if ifMatch: compare against current hash (index/notes.ts → hash)
4. fs.writeBytesAtomic(...)
5. index.upsertNote(rel, parsed)          (synchronous, before return)
6. events.publish({ type: "note.created"|"note.updated", actor, ... })
7. return read(path)                      (read-after-write)

Same shape for patch (delegates the AST mutation to md/patch/apply.ts), delete, move. move(from, to, { update_links }) does:

read source.
compute new path (validate, refuse overwrite unless Idempotency-Key flow says so).
atomically rename.
if update_links: rewrite incoming wikilinks via md/patch/apply.ts on each backlink source. This is one transactional sweep — emits one note.moved and one note.updated per rewrite.
update index, emit events.

src/vault/canvas.ts — peer of note.ts but for .canvas files. Patch ops live in canvas/patch.ts.

src/vault/base.ts — read/write/serialize the .base YAML; query delegates to base/query.ts.

src/vault/attachment.ts — read-only in v1: serve bytes with mime-sniffed Content-Type. Write deferred to v0.2 (matches PRD §7).

Markdown pipeline

src/md/pipeline.ts — exports a singleton unified processor with remark-parse, remark-frontmatter, remark-gfm, our wikilink/embed/callout/blockId plugins, and remark-stringify configured by the merged vault config. Plugins live as small files in md/:

md/links.ts — remark plugin that recognizes [[X]], [[X|Y]], [[X#H]], [[X#^id]], ![[X]]. Adds nodes of type wikilink / embed to the AST.
md/tags.ts — recognizes #tag outside of code spans/fences. Frontmatter tags: are read by frontmatter.ts, not here.
md/blocks.ts — recognizes ^block-id at end-of-block lines.
md/outline.ts — pure visitor over headings; returns OutlineEntry[].
md/plain.ts — strips formatting to a UTF-8 plain projection used for FTS body.
md/frontmatter.ts — getFrontmatter(ast) / setFrontmatter(ast, key, value) / deleteFrontmatter(ast, key). Round-trip: unknown keys are kept in original YAML order.

Round-trip rule: serialize(parse(buf)) must produce byte-identical output for the corpus in tests/corpus/. If a file fails this, the corresponding plugin is wrong, not the corpus.

src/md/patch/apply.ts — applyPatches(ast, ops): { ast, warnings[] }. Dispatches to op files. Heading matching is display-text exact match, not slug-fuzzy; a missing heading is not_found with the heading echoed in details.

src/md/patch/{frontmatter,body,heading,block}.ts — each implements one or two op types. They mutate the AST in place; the dispatcher re-serializes once at the end.

Canvas + base

src/canvas/parse.ts — parses JSON Canvas 1.0 with .passthrough() Zod; preserves unknown fields per spec.

src/canvas/patch.ts — applies CanvasPatch[] to the parsed canvas. Generates new node ids when omitted, rejects edges referencing missing nodes.

src/base/parse.ts — YAML → typed BaseFile.

src/base/query.ts — compiles the .base filters/formulas to SQL queries against the notes/tags/links tables. Formulas are evaluated in JS over the SQL result rows. Streams via async iterator so Accept: application/x-ndjson can flow rows without buffering.

Indexing layer

src/index/db.ts — opens better-sqlite3 with journal_mode = WAL, synchronous = NORMAL, foreign_keys = ON. Runs migrations. Exposes a typed Database wrapper with prepared-statement caches.

src/index/schema.sql — exact SQL from SPEC §5.1. Treat this file as a contract.

src/index/notes.ts — upsertNote(rel, parsed), deleteNote(rel), getNoteRow(rel). Each upsert:

Compares the new hash to the existing row’s hash; bails early if equal.
Writes notes, tags, links, blocks, notes_fts updates inside one transaction.

src/index/links.ts — link rows, with dst_path resolved via links/resolve.ts at index time. Re-resolution after rename happens in a separate sweep so we never block the write critical path.

src/index/fts.ts — searchFts(q, filters) with BM25; returns { path, score, matched_in[] }.

src/index/vss.ts — only loaded when INDX_VECTOR_STORE=sqlite-vss. Falls back gracefully if the extension isn’t available, logging once and disabling semantic mode for the process lifetime.

src/index/reindex.ts — full reindex (parallelized worker pool, but capped to keep RAM under §10.4 numbers) + incremental reindex driven by the watcher. Emits index.reindexed events.

Search

src/search/lexical.ts / semantic.ts / hybrid.ts — three implementations behind a common SearchProvider interface. hybrid.ts runs both, then reciprocal-rank-fuses the rankings (k=60). Filters from filters.ts are applied as SQL pre-filters whenever possible; only the residual filtering happens in JS.

src/search/cursor.ts — opaque base64url cursor: { algo: "bm25"|"vss"|"hybrid"; tieBreak: string; lastScore: number }. Bumping the algo version invalidates old cursors gracefully (returns next_cursor: null, no error).

Embeddings

src/embeddings/provider.ts — interface:

interface EmbeddingsProvider {
  readonly model: string;
  readonly dim: number;          // must match the vss table
  embed(texts: string[]): Promise<Float32Array[]>;
}

The factory returns null if INDX_EMBEDDINGS_PROVIDER is unset. Search code branches on null and falls back to lexical with warnings: ["embeddings_unavailable"].

src/embeddings/vercel-ai-gateway.ts — uses AI SDK v6’s embed/embedMany against the gateway. Token from INDX_AI_GATEWAY_KEY.

src/embeddings/openai.ts — uses INDX_OPENAI_BASE_URL so this also covers Ollama and any compatible endpoint.

src/embeddings/ollama.ts — convenience preset over the OpenAI-compatible client.

AI runtime

src/ai/* is the engine for the four built-in AI ops described in AI.md. It is purely additive — it does not touch any existing read/write path; instead it composes them.

src/ai/index.ts — exports AiApi: { status(actor), summarize(actor, input), ask(actor, input), toc(actor, input), relate(actor, input), tag(actor, input), metadata(actor, input), extract(actor, input) }. VaultHandle.ai exposes this when AI is enabled and is null otherwise — adapters branch on vault.ai === null to surface ai_unavailable.
src/ai/chat/provider.ts — ChatProvider interface mirroring EmbeddingsProvider: generate({ system, messages, json_schema?, max_output_tokens?, temperature?, seed?, signal? }) returns { text, structured?, usage, finish_reason }. The factory honors INDX_AI_PROVIDER and falls back to INDX_EMBEDDINGS_PROVIDER when unset; if neither is set the factory returns null.
src/ai/scope.ts — resolveScope(scope: AiScope, deps): { paths: { path, etag }[]; warnings: AiWarning[] }. Applies INDX_AI_ALLOW_GLOBS / INDX_AI_DENY_GLOBS and surfaces drops as globs_excluded. Pure function over the index — no chat provider needed.
src/ai/retrieve.ts — thin wrapper over search/{lexical,semantic,hybrid} for ai_ask. Re-uses the existing SearchProvider interface; returns top-k snippets keyed by { path, etag, line, score, matched_in[] }.
src/ai/chunk.ts — token-budgeted chunker + map-reduce planner. planSummary(scope, budget) returns a tree of { chunks: NoteChunk[], reduce: (chunkResults) => combine } so the orchestrator never holds the whole vault in RAM.
src/ai/citations.ts — verify(citations[], vault) → { kept[], dropped[] }. Reads each cited path via notes.read (cheap; FS hit only when not in cache), confirms the etag matches and the anchor/line resolves, drops with reason. The dispatcher fails the op (ai_grounding_failed) iff cite: true and kept is empty.
src/ai/cache.ts — content-addressed cache backed by a sibling SQLite db at .indx/ai-cache.db (own connection, journal_mode = WAL). Key derived per AI.md §8.1; value is the structured payload + ttl. Invalidation is implicit: cache lookup folds in the cited paths’ current etags, so a stale entry simply doesn’t match.
src/ai/cost.ts — per-call cost estimator (per-provider per-model rate table; updated as a small JSON shipped with the package) and the daily-spend ceiling (INDX_AI_DAILY_COST_USD). Over-cap → RateLimited with details { reason: "ai_quota_exceeded", reset_at }.
src/ai/stream.ts — surface-agnostic event emitter consumed by the API SSE writer, the CLI NDJSON writer, and the MCP progress notification helper. The runtime emits ai.partial / ai.citation / ai.usage deltas; the orchestrator finishes by calling complete(payload) which is what every adapter ultimately renders.
src/ai/ops/summarize.ts — orchestrates: resolve scope → chunk → per-chunk generate (with json_schema) → reduce → verify citations → emit. Honors style, length, include, language, audience.
src/ai/ops/ask.ts — retrieve top-k → assemble prompt → generate (streaming) → verify citations → return. The system prompt is locked-in templated text in prompt/ask.ts; freeform tone changes go through style, not by editing the system prompt at runtime.
src/ai/ops/toc.ts — mode: "note" walks the indexed outline (no chat call) unless include_descriptions: true; mode: "moc" clusters via group_by, asks the chat model for titles + (optional) per-item descriptions, returns toc_markdown + toc_tree. When write: { path } is set, delegates to vault.notes.write(actor, …) honoring if_match / if_not_exists — atomicity invariants are unchanged.
src/ai/ops/relate.ts — neighbor discovery via the existing vss (or lexical fallback), classification batched per source via chat.generate(json_schema=…), dedup + sort. With propose_links: true, returns draft PatchOp[] constructed against md/patch/heading.ts shapes — never applied.
src/ai/ops/tag.ts — loads the existing tag list (tag_list snapshot at scope-resolve time), composes the prompt with vocabulary controls, generates against a JSON Schema constrained to allowed tags, runs normalize.ts on each candidate (charset per SPEC §4.5, case-folding when matching existing tags), drops below-threshold suggestions, optionally drives apply.ts to write set_frontmatter/tags per path. Body #tag mentions are read but never rewritten.
src/ai/ops/metadata.ts — translates MetadataFieldSpec[] into a JSON Schema, generates per-note (or per-chunk for very large scopes), runs validate.ts on each value (pattern, enum, min, max, type coercion for date/datetime), drops invalids with metadata_invalid, and (under apply: true) drives apply.ts per path with the requested apply_mode. Special keys (tags/aliases/cssclasses) are routed through the same writer that notes.patch.set_frontmatter uses for FR-F-3 round-trip safety.
src/ai/ops/extract.ts — validates the caller’s schema upfront (AiSchemaInvalid on failure), generates with that schema, re-validates output records, and (under apply: true + destination: "frontmatter") writes via apply.ts to destination_key. destination: "json" is a pure read; no apply.ts call.
src/ai/apply.ts — shared driver for the apply path. Resolves scope, loads current etag per path, sorts paths, iterates writes via core.notes.patch(actor, …), honors per-path if_match, collects per-path outcomes, surfaces partial failures as warnings, and emits the roll-up ai.invocation with applied_paths. Per-note atomicity is inherited from notes.patch; cross-note atomicity is explicitly not attempted.
src/ai/normalize.ts — pure helpers: tag normalization, frontmatter list-shape preservation, date/datetime ISO 8601 coercion. Used by tag/metadata/extract apply paths and by their suggest paths so suggested values match what would be written.
src/ai/validate.ts — Zod-driven validators for MetadataFieldSpec plus a JSON Schema validator for ai_extract (Ajv 2020-12 strict mode). Validation runs both before generation (to derive the JSON Schema sent to the provider) and after (defense in depth — providers are not always faithful to the schema).
src/ai/prompt/*.ts — versioned system prompts per op. Versioning is a prompt_version constant baked into the cache key, so prompt changes are observable as cache invalidation rather than as silent drift.

The orchestrator never throws raw provider errors; upstream failures are wrapped as AiProviderError (502, code ai_provider_error). Timeouts honor INDX_AI_TIMEOUT_MS via AbortController.

Events, auth, rate-limit, idempotency, observability

src/events/bus.ts — typed pub/sub. subscribe(filter) returns an async iterator with backpressure (drops events with a dropped counter when the consumer falls more than 1024 events behind, exposed in SSE as a _meta event).

src/events/log.ts — appends every event as one JSON line to .indx/events.log. Rotates at 10 MB by renaming to events.log.1 (one rotation kept).

src/events/replay.ts — replayFrom(id): AsyncIterable<EventEnvelope> reads the rotated log files for SSE Last-Event-ID resume.

src/auth/tokens.ts — parses INDX_TOKEN (single string) or INDX_TOKENS_FILE (JSON [{token, scopes}]). Returns a TokenSet with constant-time match. No remote token store; no DB-backed tokens in v1.

src/auth/scope.ts — hasScope(scopes, required, path?): boolean. path:<glob> uses picomatch-like semantics (a tiny glob matcher we vendor; no new dep if avoidable).

src/ratelimit/sliding.ts — per-token sliding window counter in memory, default 100 rps / 1000 burst, configurable.

src/idempotency/cache.ts — bounded LRU keyed by (token, method, path, key), value { status, body, bodyHash }. 24h TTL. Hash collisions on the same key with a different body return idempotency_key_reused.

src/observability/log.ts — pino factory; all engine logs go through it. Adapters get child loggers with their own bindings ({ surface: "api" }).

src/observability/otel.ts — opt-in. Wraps the public APIs of VaultHandle with spans when enabled.

4. `apps/web` — Next.js app (UI + REST + SSE + MCP)

4.1 Boot and runtime

src/lib/server/runtime.ts owns the singleton vault. Pattern:

let bootPromise: Promise<VaultHandle> | null = null;
export function getVault(): Promise<VaultHandle> {
  if (!bootPromise) bootPromise = openVault(loadConfigFromEnv(process.env));
  return bootPromise;
}

All route handlers await getVault() at the start. The first request after process start waits for boot; subsequent requests are immediate. Reindex never blocks the event loop — openVault returns as soon as the index is queryable; full reindex runs in the background and emits a 503 reindexing only for queries that genuinely cannot answer until it completes.

Every API route declares export const runtime = "nodejs" and export const dynamic = "force-dynamic". We do not ship the engine to the edge runtime.

4.2 Middleware and request shape

src/middleware.ts runs only on /v1/* and /mcp:

Reads Authorization. Missing → 401 envelope.
Validates token via core.auth. Bad → 401. Computes scopes.
Reads/echos Idempotency-Key, If-Match, If-None-Match.
Builds Actor ({ kind: "api", id: tokenId } for /v1, { kind: "mcp", id: ... } for /mcp).
Stamps a request_id header (xxhash of bytes + time).
Forwards.

The web UI uses session cookies set by a sign-in route that the user hits with their bearer token once; cookies are HttpOnly; Secure; SameSite=Strict. UI requests carry the cookie; the cookie validates exactly like a bearer in the auth.ts helper.

4.3 Route handler shape

Every route under /v1 follows the same skeleton. Generic example:

import { NotesApi } from "@indx/shared";
import { getVault } from "@/lib/server/runtime";
import { ok, err, etagHeaders, parseIfMatch } from "@/lib/server";

export const runtime = "nodejs";
export const dynamic = "force-dynamic";

export async function GET(req: Request, ctx: { params: { path: string[] } }) {
  const path = decodePath(ctx.params.path);
  const include = parseInclude(req);
  const vault = await getVault();
  const actor = req.headers.get("x-indx-actor-context") /* set by middleware */;
  try {
    const note = await vault.notes.read(actor, { path, include });
    if (matchesIfNoneMatch(req, note.etag)) return new Response(null, { status: 304 });
    return ok(note, etagHeaders(note));
  } catch (e) {
    return err(e);              // typed mapping inside err()
  }
}

ok, err, etagHeaders, etc. live in lib/server/. They are the only places that build a Response object.

4.4 Per-route policy

Route	Method(s)	Policy notes
`/v1/health`	GET	Cheap; never depends on full reindex. Returns `{ status: "ok"\|"reindexing", vault, indexed_notes, last_reindex_at, uptime_s }`.
`/v1/notes/[...path]`	GET	`include` query param; `If-None-Match` → 304.
	PUT	`If-Match` honored; `Idempotency-Key` cached on success and on 4xx (so retries don’t re-write).
	PATCH	Body validated by `PatchRequest`; ops applied atomically.
	DELETE	204 + empty body; tombstone for 5 min, then 404; `If-Match` honored.
`/v1/notes/[...path]/move`	POST	`update_links` defaults to true; response includes `rewrites` count.
`/v1/notes`	GET	Paginated array by default; NDJSON when `Accept: application/x-ndjson` (see `lib/server/ndjson.ts`).
`/v1/search`	GET	If `mode=hybrid
`/v1/links/[...]/backlinks` etc.	GET	Cheap reads; cursor-paginated; `ETag` on the result set hash so polling clients can short-circuit.
`/v1/tags`	GET	Returns counts; cached in-memory invalidated by `note.updated`.
`/v1/tags/[tag]/notes`	GET	Glob-aware.
`/v1/tags/rename`	POST	Bulk; idempotent if `from` no longer exists.
`/v1/canvas/[...path]`	GET/PUT/PATCH	Same shape as notes but with `Canvas` payload.
`/v1/bases/[...path]`	GET/PUT	YAML behind JSON envelope.
`/v1/bases/[...path]/query`	GET	NDJSON or paginated array; cursor encodes the last row’s stable id.
`/v1/vault/status`	GET	Uses `vault.vault.status()`. Always returns 200.
`/v1/vault/reindex`	POST	Returns 202 with a `job_id`; subscribe via `/v1/events`.
`/v1/vault/config`	GET/PUT	Admin-scope only; PUT validates the new config and atomically swaps.
`/v1/events`	GET (SSE)	`text/event-stream`; supports `Last-Event-ID`; query filters `paths`, `kinds`. Uses Node `ReadableStream` + the event bus iterator.
`/openapi.json`	GET	`buildOpenApi(routes)` from `@indx/shared`. Reflects feature flags (omits embeddings ops if disabled).
`/mcp`	POST/GET	Hands the request to `packages/mcp` HTTP transport. The mcp registry is filtered by request scopes — write tools literally aren’t advertised to a `vault:read` token.

4.5 UI policy

The web UI is built on the same /v1 API via lib/client/api.ts. No server component fetches vault directly — RSCs may use getVault() for performance, but the data passes through the same Zod schemas as the API would. This keeps “the API is the canonical surface” (PRD §4 goal 2) honest.

UI components (src/components/) are dumb in the React-component sense:

editor/CodeMirror.tsx — client component, hosts a CodeMirror 6 instance with our markdown extensions; emits onPatch(ops) rather than raw text changes when the user uses the patch palette, falling back to whole-body update for free typing.
editor/PatchPalette.tsx — keyboard-first command surface that maps to the same patch grammar.
tree/FileTree.tsx — server-rendered tree from vault.notes.list({ path_glob: "**" }); client-side hydration only for expand/collapse state.
search/SearchBar.tsx + search/Results.tsx — search uses /v1/search directly for parity with agents.
graph/GraphView.tsx — basic force-directed graph; data from /v1/links/.... The “fancy” graph is post-v1 (PRD §4).
activity/ActivityFeed.tsx — subscribes to /v1/events via lib/client/sse.ts, renders actor.kind = "mcp" | "api" | "cli" | "ui" | "fs" chips.

The activity panel is the human visibility layer for J4 (“Human inspects what an agent did”): it must always show the actor and a diff link. Diff/rollback shipping in v1.1 — for v1, we link to a read-only “before/after” view based on the events log.

5. `packages/cli`

5.1 Structure

bin/indx.mjs (already scaffolded) loads dist/index.js and calls main().

5.2 File policy

src/index.ts (scaffolded) — replace the stub main with: parse global flags via parse.ts, dispatch to commands/<noun>.ts, render output via output.ts, exit via exit.ts. Never throw to top level — every error becomes a structured fail().

src/parse.ts — argv → { noun, verb, args, flags }. Hand-rolled (no commander/yargs dep; the surface is small enough). Supports --flag value, --flag=value, --bool, and --. Unknown flags → exit 2 with code: bad_flag.

src/globals.ts — resolves --vault / --remote / --token against env. Decides mode = "local" | "remote" per CLI §2.1: --remote wins; else INDX_URL ⇒ remote; else local.

src/output.ts — ok(data) writes one JSON line and exits 0. fail(code, error) writes { ok: false, error } and exits with the matching code. ndjson(iterable) writes one object per line, flushing.

src/exit.ts — exit code map. Single source of truth.

src/transport/index.ts — Transport interface mirroring VaultHandle (one method per CLI verb). Both implementations conform.

src/transport/local.ts — wraps openVault(loadConfigFromEnv(...)) and proxies. Closes the vault on exit. Used when mode === "local".

src/transport/remote.ts — fetch against ${INDX_URL}/v1. Adds Authorization, Idempotency-Key (defaulted to a random uuid per write), If-Match. Maps HTTP error envelope → CLI error.

Validates flags via Zod.
Calls transport.notes.<op>(...).
Renders the result through output.ok (or NDJSON for list --ndjson).

create --if-not-exists is implemented as a single transport call (writeNote({ ifNotExists: true })), not as a check-then-write — atomicity matters for agents.

src/commands/link.ts, tag.ts, search.ts, canvas.ts, base.ts — same pattern.

src/commands/serve.ts — import("apps/web")? No — instead, it spawns next start from the bundled standalone Next output, threading INDX_* env. This keeps the CLI binary lean and lets serve work both inside and outside the Docker image.

src/commands/mcp.ts — serve --stdio | serve --http | tools | call. --stdio calls createMcpServer from packages/mcp and wires it to stdio via the SDK’s transport. tools and call exist for testing — call runs a tool against a local vault.

src/commands/schema.ts — openapi | mcp | patch-ops | events. Reads from @indx/shared directly; no network, no vault — pure introspection.

5.3 Anti-patterns enforced

output.ts must never write to stdout outside its two functions; console.log is banned (lint rule).
No verb prompts. Destructive verbs (note delete, vault import) check flags.yes; missing --yes → exit 4 with code: needs_yes.
The CLI never reads ~/.indxrc or any home config in v1.

6. `packages/mcp`

6.1 Structure

src/index.ts — createMcpServer(deps: { vault: VaultHandle; scopes: Scope[] }) returns an McpServer from @modelcontextprotocol/sdk. The function:

Registers identity (§6.2).
Registers tools, resources, prompts, each filtered by scopes.
Returns the server. The transport (stdio or HTTP) is wired by the caller.

src/identity.ts — name, version (from package.json), and the exact instructions string from MCP spec §2. Editing this string is a docs change, not a code change — keep them in sync.

src/registry.ts — helper defineTool({ name, schemaIn, schemaOut, scopes, handler }). Inputs are validated with Zod (re-using @indx/shared); the schema is also converted to JSON Schema for the tool catalog — exactly the same converter as openapi/builder.ts so the tool schemas and the API schemas can never drift.

6.2 Tools

One file per noun matching MCP §3.1–3.3. Each handler is literally one or two lines:

defineTool({
  name: "note_patch",
  scopes: ["vault:write"],
  schemaIn: PatchToolInput,
  schemaOut: Note,
  handler: (deps, input) => deps.vault.notes.patch(deps.actor, input)
});

Errors thrown by core surface as { isError: true, content: [{ type: "text", text: <code+message> }], structuredContent: { code, message, details } }. The text block is a one-line summary the agent can quote without parsing.

A safe flag on read-only tools enables the 60s (tool, input) replay window described in MCP §3.

6.3 Resources

src/resources/index.ts — registers a single vault:// URI namespace with a router that dispatches based on the path:

URI pattern	Backed by
`vault://<path>`	`vault.notes.read` (or canvas/base/attachment based on extension)
`vault://search?q=...`	`vault.search.search` (live every read)
`vault://graph`	`vault.links.graph()`
`vault://graph/backlinks/<path>`	`vault.links.backlinks`
`vault://graph/forward/<path>`	`vault.links.forward`
`vault://tags`	`vault.tags.list`
`vault://canvas/<path>`	`vault.canvas.read`
`vault://base/<path>`	`vault.bases.read`

Subscribe support uses the event bus: a subscription on vault://Projects/** filters note.* events by glob.

6.4 Prompts

One file per prompt. Each exports { name, description, arguments, build(args, deps): { messages, tools }}. The tools field is an allow-list of tool names — when an agent runs the prompt, only those tools are advertised on its turn.

6.5 AI tools and resources

tools/ai.ts registers ai_status, ai_summarize, ai_ask, ai_toc, ai_relate. Each tool’s handler is a one-liner over deps.vault.ai.<op>; the implementation lives entirely in @indx/core/src/ai. Registration is gated: a tool is added to the catalog iff (a) the AI runtime is enabled (a chat provider is configured, or for ai_relate an embeddings provider is configured), and (b) the connecting token’s scopes admit it. ai_toc carrying a write payload is registered as a write tool — invisible to a vault:read-only session, exactly like other write tools per MCP §6.2.

resources/ai.ts registers two URI patterns (ai://summary/<scope>, ai://moc/<scope>) that re-derive on subscribed scope changes by piggy-backing on the same event bus that drives vault:// subscriptions: a note.updated event whose path is in the cited set re-runs the derivation lazily on the next resources/read.

Streaming for ai_ask and large ai_summarize is wired through notifications/progress per MCP convention; the final tool result still carries the complete structuredContent.

7. Data and config files

Dockerfile (already present) — multi-stage:

node:24-alpine build stage → pnpm install, pnpm build, copy apps/web/.next/standalone.
gcr.io/distroless/nodejs24-debian12 runtime → copies the standalone dir, the prebuilt packages/cli/dist, the shared node_modules, and a tiny init script that runs node apps/web/server.js. The indx binary is on PATH via a symlink.

Healthcheck: HEALTHCHECK CMD node -e "fetch('http://127.0.0.1:'+process.env.INDX_PORT+'/v1/health').then(r=>process.exit(r.ok?0:1))" (or equivalent — keep dependency-free).

docker-compose.yml (already present) — one service, vault volume mount, env from .env. Don’t add Postgres/Redis/etc.

.env.example (already present) — already correct; touch only when adding env vars.

tsconfig.base.json (already present) — strict, NodeNext, noUncheckedIndexedAccess. Per-package configs extends this; paths are not used.

.gitignore — must include vault/, node_modules/, .next/, dist/, .indx/ (because dev runs may produce one inside the repo).

8. Cross-cutting policies

8.1 Schema-first contract flow

Adding any new operation is one PR with a strict order:

Zod schema in @indx/shared/src/schemas/<domain>.ts.
Route entry in @indx/shared/src/openapi/routes.ts.
Implementation in @indx/core (engine method on the matching API object + tests).
Three thin adapters:
- HTTP route in apps/web/src/app/api/v1/....
- CLI verb in packages/cli/src/commands/....
- MCP tool in packages/mcp/src/tools/....
Smoke test at the surface level in tests/.

Adapters that are not one-liners are a smell: business logic has leaked out of core. Push it back.

8.2 ETag scheme

ETag = first 16 hex chars of xxhash64(bytes-on-disk). Computed by core.hash.etag().
If-Match is a strong match. Mismatch → EtagMismatch (409, exit 4).
If-None-Match is supported on GET; matches return 304 with no body and the same headers.
Move/rename produces a new ETag (the bytes are byte-identical but the metadata is part of the resource identity, not the content hash; we use etag = xxhash64(bytes) | "@" | path for list endpoints’ ETags only — single-resource ETags are content-only).

8.3 Idempotency cache layout

Key: (token_id, method, path, idempotency_key).
Value: { status, body, body_hash, created_at }.
TTL: 24h.
Eviction: LRU at 10 000 entries.
Same key + different body → 409 idempotency_key_reused (no replay; protects against client bugs).

8.4 Atomic write pipeline (recap)

adapter validates input
  → core.notes.write(actor, input)
       → md.parse(combined)                (errors: ParseFailed)
       → if ifMatch: hash compare         (errors: EtagMismatch)
       → fs.writeBytesAtomic               (errors: IOError → Internal)
       → index.upsertNote                  (transaction)
       → events.publish + log
       → return read(path)                 (read-after-write)
adapter renders response with new ETag header

This pipeline is the only way bytes change in the vault. There is no “fast path” for the UI or any other surface.

8.5 Path safety

Every adapter calls paths.normalizeVaultPath on every user-supplied path before passing it to core.
Core’s fs.ts also re-validates — defense in depth.
Linux container, case-sensitive matching by default; case_insensitive_match config falls back to a folded lookup after the exact lookup fails (not as a default, to avoid surprise on agent retries).

8.6 Error mapping (canonical)

Core error	HTTP status	API code	CLI exit	MCP `isError`
`NotFound`	404	`not_found`	3	true
`EtagMismatch`	409	`etag_mismatch`	4	true
`AlreadyExists`	409	`already_exists`	4	true
`Gone`	410	`gone`	3	true
`ParseFailed`	422	`parse_failed`	6	true
`ValidationFailed`	400	`validation_failed`	6	true
`Forbidden`	403	`forbidden`	5	true
`Unauthorized`	401	`unauthorized`	5	true
`ReadOnly`	423	`readonly`	4	true
`Reindexing`	503	`reindexing`	8	true
`RateLimited`	429	`rate_limited`	7	true
`Internal`	500	`internal`	1	true
`AiUnavailable`	503	`ai_unavailable`	8	true
`AiProviderError`	502	`ai_provider_error`	8	true
`AiQuotaExceeded`	429	`ai_quota_exceeded`	7	true
`AiGroundingFailed`	422	`ai_grounding_failed`	6	true
`AiInputTooLarge`	422	`ai_input_too_large`	6	true
`AiSchemaInvalid`	422	`ai_schema_invalid`	6	true
`AiApplyConflict`	409	`ai_apply_conflict`	4	true

Adapters never invent codes; they only translate.

8.7 Logging

One pino logger per package, child-logged from core’s root logger.
Format: JSON to stdout. Fields: level, time, surface, request_id, actor, op, path, latency_ms.
Never log full note bodies. Log path + ETag + size.

8.8 Testing layout

Unit (per package, src/__tests__/): AST patches, link resolution, frontmatter round-trip, error mapping, scope check.
Integration (tests/): a real on-disk vault under tmp/, full boot of apps/web with next start --port=$port, calls via fetch. Same suite reused via the CLI and MCP adapters.
Property (tests/round-trip.test.ts): walks tests/corpus/, parse → serialize → assert byte-identical.
Agent eval: not in CI; runs in a separate workflow against the 50-task benchmark. Pass rate gates a release.

8.9 What stays out of v1

This is the negative checklist — if a PR touches one of these, it’s the wrong PR for v1:

Plugin system / Obsidian plugin compat.
Multi-tenant OIDC (token list is fine; per-user vaults are not).
Real-time multi-user collab.
Encryption at rest.
Vault sync.
Attachment writes (read is in scope).
Visual canvas editor (read-only viewer + patch ops are in scope).
Visual base builder.
Diff/rollback UI (activity feed is in scope; rollback is v1.1).

9. Implementation order (suggested phasing)

A reasonable sequence that keeps the system runnable at every step:

Schemas & errors — finish @indx/shared (canvas, base, link, tag, vault, http) + core/errors.ts.
Vault open & FS — core/config.ts, core/paths.ts, core/hash.ts, core/vault/{handle,fs,walk}.ts. openVault opens, walks, and closes; no parsing yet.
Markdown core — core/md/* (parse, serialize, frontmatter, outline, links, tags, blocks). Round-trip tests pass on a 100-file corpus.
Index — core/index/* (db, schema, notes, links, tags, fts). Reindex of the corpus succeeds; searchFts returns sane results.
Notes API — core/vault/note.ts (read, write, patch, delete, move) wired to index + events. Unit tests cover atomicity and ETag.
Web routes — notes — apps/web/src/app/api/v1/notes/*, plus lib/server/*. End-to-end smoke: curl PUT/PATCH/GET/DELETE.
CLI — notes — packages/cli/src/commands/note.ts + transports. Same operations via indx note.
Search — core/search/*, then /v1/search, indx search. Lexical first; semantic is gated on embeddings provider.
Links & tags — core/links/*, core/tags-side helpers, then routes + CLI.
Canvas — core/canvas/*, core/vault/canvas.ts, then routes + CLI.
Bases — core/base/*, then routes + CLI.
Events & SSE — core/events/*, /v1/events, indx vault index --watch.
MCP — packages/mcp/*, apps/web/src/app/mcp/route.ts, indx mcp serve --stdio.
Web UI — (ui)/ route group, components, activity panel.
Auth & rate-limit — core/auth/*, core/ratelimit/*, middleware.
Idempotency — core/idempotency/*, plumbed through the write routes.
OpenAPI live — /openapi.json, indx schema openapi. Sanity-check against route handlers in CI.
AI runtime — core/ai/*, /v1/ai/*, indx ai *, ai_* MCP tools, ai://summary/ai://moc resources. Lexical-only ai_ask lands first; ai_toc (note mode) lands without provider; chat-dependent paths gate on provider config.
Docker & deploy — finalize Dockerfile, healthcheck, image-size budget.
Agent eval — wire the 50-task benchmark behind pnpm bench:agent.

Each step should be a small PR that ends with the system runnable end-to-end at its current scope.

10. Glossary of internal terms

Adapter — surface code (HTTP route handler, CLI command, MCP tool) that translates transport in/out and calls a single core method.
Engine / core — @indx/core. The only place that touches files, the database, or the index.
Surface — one of {Web UI, REST API, CLI, MCP}.
Actor — { kind, id } describing who performed a write. Recorded on every event.
Patch — an AST-aware structured edit op (SPEC §6.2).
ETag — content-addressed strong tag for a resource (xxhash64-derived).
Cursor — opaque pagination token; never inspectable by clients.
Scope — capability granted by a token (vault:read, vault:write, vault:admin, path:<glob>).

This plan is intended to be executable: each section lists files, contracts, and policies that can land as small, independently-reviewable PRs while preserving the invariants in §0. When a future change needs to break one of those invariants, update this document in the same PR.