Local AI models

Papervine’s AI surfaces (the assistant, the editor agent, and automation runs) resolve their model in one place, from environment variables. That resolution accepts a local endpoint as readily as a hosted provider, so a cost-sensitive setup can run everything on its own hardware.

Point Papervine at your server

Set a model id whose prefix names a local runtime, and Papervine sends its requests there instead of to a vendor:

# Ollama (default endpoint http://localhost:11434/v1)
PAPERVINE_AI_MODEL=ollama/qwen3

# LM Studio (default endpoint http://localhost:1234/v1)
PAPERVINE_AI_MODEL=lmstudio/qwen3

# Anything else that speaks the OpenAI API — vLLM, llama.cpp, LiteLLM, a remote GPU box
PAPERVINE_AI_MODEL=local/qwen3
AI_BASE_URL=http://gpu-box.lan:8000/v1

AI_BASE_URL overrides the default endpoint for any of these prefixes, and is required for the generic local/ one. No API key is needed; set AI_LOCAL_API_KEY only if your server demands one.

Running Ollama

The repository ships an opt-in Compose profile, so a plain docker compose up never pulls multi-gigabyte model weights:

docker compose --profile local-ai up -d
docker compose --profile local-ai exec ollama ollama pull qwen3

LM Studio has no container: start its local server from the app and point AI_BASE_URL at it (or use the lmstudio/ prefix, which assumes the default port).

Thinking models: off by default

Most capable open models released recently — Qwen3, Gemma, DeepSeek-R1 and friends — think before they answer, emitting a long internal monologue that never reaches the reader. On a laptop that reasoning dominates everything: measured on an 8B-class Qwen, the same one-sentence question took 40 seconds with thinking and 1.9 seconds without, having produced 3,800 characters of reasoning to deliver a 212-character answer. Multiply that across the two dozen steps of an agent run and a workable feature becomes an unusable one.

Worse, the thinking can consume the entire response: the model finishes its monologue and returns empty content. That’s what an assistant answer looks like when it shows its tool steps and then simply stops.

So Papervine asks local models not to think. Set AI_LOCAL_REASONING=1 to opt back in on hardware that can afford it. Hosted models are unaffected — their reasoning is fast and priced accordingly.

Choose a model that can use tools

This is the part that decides whether running models locally works for you. Papervine’s AI is agentic, not single-shot: the assistant calls searchDocs and readPage before answering, and an automation run may take a couple dozen tool-calling steps to read pages, edit them, and report what it changed.

Small models are unreliable at multi-step tool use — they hallucinate arguments, skip calls, or loop. Pick a model with genuine tool-calling support (recent Qwen, Llama, and Mistral instruct models are the usual choices), and give it enough context length for a documentation page plus its instructions. If runs come back with no changes when changes were needed, or with mangled edits, the model is the first thing to suspect.

For calibration, an 8B-class Qwen on a 36 GB Apple-silicon laptop: a small find-and-fix-a-broken-link task over two pages completes correctly in well under a minute, including recovering from one malformed path argument along the way. A full automation run over a real site is a different scale — a dozen or more pages read across up to two dozen reasoning steps takes many minutes, where a hosted model takes well under one. Expect wobble-then-recover behavior rather than crisp precision, and expect both accuracy and patience to be tested as the task grows.

That difference suggests choosing per surface rather than all-or-nothing:

Surface	Local?	Why
Assistant, editor agent	Comfortable	A handful of short calls per question — well within a laptop’s reach
Automation runs	Demanding	Up to two dozen reasoning steps over your whole site, with the model resident the entire time

Switching between local and hosted is two lines of environment configuration, so it costs nothing to keep both configured and flip as the task demands.

Credits and metering

Usage of a locally-run model is rated at zero credits. Papervine still records a usage event per run so you can see token volumes, but nothing is debited and the credit gate never refuses work — you’re not paying anyone, so the meter has nothing to meter.

Automations need an executor too

The assistant and editor agent run inside the app, so a local model is all they need. Automations additionally require a background executor to run jobs and fire schedules. Without one, the Automations page says so plainly and nothing else in the product is affected — automations are the only feature that stops working.

Can a scheduled run reach a model on my machine?

It depends on where the worker runs, not where the schedule lives. Scheduling and execution are separate: a scheduler decides when, a worker does the work, and only the worker talks to your model.

• Local development — the worker runs on your own machine, so it reaches localhost:11434 normally. Scheduled automations firing from a hosted scheduler still execute locally against your local model.
• A worker you run yourself — put it on the same network as your inference server, and everything stays inside your infrastructure.
• A hosted worker — a worker running in someone else’s cloud cannot reach a model on your private network. That deployment shape needs a hosted model (or an inference endpoint you expose deliberately).