AI Assistant

The AI assistant is a slide-out chat panel that answers reader questions from a tenant’s own docs and OpenAPI spec. It is an agentic “Ask Assistant”: the model navigates the docs to find answers, cites its sources, and admits when it doesn’t know.

Agentic retrieval, not single-shot RAG

A naïve docs bot does one top-k vector lookup and stuffs the results into a prompt. Papervine’s assistant instead lets the model decide how to search, iterating with tools until it can answer — effectively a virtual filesystem over the vector store that the model navigates with filesystem-like tools.

This is cheap to build because the tools are thin wrappers over capabilities Papervine already has — the content loader, the navigation tree, search, and the OpenAPI pipeline:

User question ──▶ /api/assistant  (Vercel AI SDK streamText, configured model)
                    │  loop: the model calls tools until it can answer
                    ├─ search_docs(query)   → search index (titles / headings / body)
                    ├─ read_page(slug)      → full MDX for a page
                    ├─ list_pages()         → the navigation tree
                    └─ search_api(query)    → OpenAPI operations
                    ▼
                 streamed answer + Sources (cited page hrefs / #anchors)

Why tools instead of pure top-k:

• Multi-step retrieval handles vague questions — the model can refine its own search.
• It can read a whole page when a snippet isn’t enough.
• One tool layer, many surfaces — the same tools power the in-docs chat and the read MCP server.

Model & routing

The model and how it’s reached are configuration, not code — set via environment, resolved in one place (src/lib/ai-model.ts) that both the assistant and the editor agent use. Papervine isn’t tied to any single provider:

• PAPERVINE_AI_MODEL — a provider/model id (e.g. anthropic/claude-haiku-4-5, google/gemini-3.1-flash-lite, openai/gpt-5-nano).
• AI_ROUTING=gateway (default) — route through the Vercel AI Gateway: one key (AI_GATEWAY_API_KEY, or Vercel OIDC when deployed there) unlocks nearly every provider and model without per-provider SDKs.
• AI_ROUTING=direct — bypass the gateway and call a provider’s SDK directly with that provider’s own key (ANTHROPIC_API_KEY, GOOGLE_GENERATIVE_AI_API_KEY, OPENAI_API_KEY). Use this to bring your own keys or avoid the gateway entirely.
• A local endpoint — an ollama/, lmstudio/, or local/ model id runs against your own hardware and pays nobody. See Local AI models.

Individual surfaces can override the model: PAPERVINE_AI_MODEL_AUTOMATIONS applies only to automation runs, which write documentation that lands in Git — worth a stronger model than the high-volume assistant needs. Anything without an override falls back to PAPERVINE_AI_MODEL.

Because the assistant is agentic (it makes several model calls per question), pick a model that’s both capable at tool use and not rate-limited by your provider tier. See .env.example for the documented defaults and trade-offs.

Current-page context

The page the reader is currently on is injected into the system prompt as starting context, so a question like “how do I do this?” resolves locally without the model having to first re-discover where the reader is.

Guardrails

• Answer only from tool results, and cite every claim.
• When confidence is low, say “I don’t know” and surface the tenant’s configurable deflection email / support contact rather than guessing.
• Per-tenant rate limits and token budgets cap runaway cost.

The Ask-Assistant UI

The chat UI is built on AI Elements (shadcn/ui + Vercel AI SDK components) rather than hand-rolled, and themed with the same CSS variables as the docs site so it matches the surrounding theme:

• Conversation / Message — the transcript.
• Response — streaming markdown, including code blocks.
• Sources — the cited pages and #anchors under each answer, navigable.
• PromptInput — the “Ask a question…” box, with file/image attach for multi-modal input.
• Suggestions — starter and follow-up questions (see starter questions).
• Reasoning / Chain of Thought — optional, to surface the tool-call and search steps.

The panel slides out from the right (expand + close), opened by an Ask Assistant button in the navbar, and carries the disclaimer “Responses are generated using AI and may contain mistakes.”

Invocation

The assistant offers several invocation surfaces:

Indexing and freshness

The assistant retrieves over the same content the rest of the site indexes:

• Published pages + OpenAPI specs are indexed at sync time; changed pages are re-indexed on publish.
• Hidden / noindex pages are excluded by default. Setting docs.json’s seo.indexing: "all" includes them (Papervine already parses hidden / noindex frontmatter).

Surfaces beyond the docs site

The assistant is not confined to the docs panel:

• /api/assistant is an SSE endpoint (AI SDK data stream), with a compatible alias path so existing integrations port over.
• It is embeddable on any external site via a <script> snippet — see the embeddable widget; Slack and Discord bots are planned.
• The tool layer is the read MCP server. search_docs / read_page / list_pages / search_api are exactly the tools a generated read-MCP exposes, so the in-docs assistant and the MCP server are one implementation behind two transports.

Assistant settings (control plane)

Docs owners manage the assistant from a dashboard page (Automate → Assistant; see the control plane overview). The top of the page shows three overview cards — Total questions, Answered properly, Not answered — each with a month-over-month delta, plus a link through to Analytics.

Settings are grouped:

• Status & control — an Assistant Status enable/disable toggle (Active/Inactive). This is an operational kill switch held in the database, not docs.json, so it takes effect instantly without a Git commit.
• Response handling — Deflection — a support email the assistant points readers at when it can’t answer, plus an optional “Contact support” help button in the widget. Drives the low-confidence guardrail above.
• Search domains — extra domains to include as retrieval context beyond the tenant’s own docs (e.g. a marketing site), added/removed with validation.
• Bot protection — an invisible CAPTCHA (hCaptcha) toggle on the public /api/assistant endpoint to limit automated abuse and runaway token cost.
• Starter questions — up to 3 suggested questions shown in the chat empty state (rendered via AI Elements Suggestions).
• Plan / credits — trial banner, per-tenant credit metering, and rate limits. Some controls are plan-gated by config (not hardcoded), so an operator running their own deployment gets everything.

Where each setting lives