Skip to content

Provider Expansion: OpenAI, Anthropic, Billing-Verifiable Hosts, Gateways, and Local

Status: STARTED (provider facts refreshed October 22, 2026; Responses adapters, governed OpenRouter preview, explicitly gated Codex experiment, and host-native handoff shipped; full recipe promotions remain eval-gated) ROADMAP anchor: Active Queue #26. Companion to 2.0-backend-freedom.md (routing architecture); this doc is the concrete provider catalog and delivery plan that routing will route over.

Principles (set by the operator, June 2026)

  1. No reason this must be Grok. Grok + Gemini is the measured default, not a dependency. Every pipeline role (reasoning, writing, utility) should be servable by any provider whose models meet the role's bar.
  2. Support whatever someone has. A user arrives with an OpenAI key, or an Anthropic key, or a corporate Bedrock/Foundry gateway, or sanctioned agent-host allocation, or just a gaming GPU and Ollama. Each of those should produce a report once its recipe is validated. Absence of any of them is silent, never an error.
  3. Account capacity is a first-class capacity source. Sanctioned agent hosts should be evaluated as ways to spend capacity the user already has. They become Primr runners only through official automation, connector, CLI, hook, or agent-skill surfaces that can accept bounded stage packets and return structured outputs. They are never treated as hidden API keys or repo-owned accounts.
  4. Mac/Linux/Windows equally. Local inference integrates over HTTP to OpenAI-compatible servers only; no platform-specific code. (vLLM has no Windows support; that is the server operator's concern, not primr's, because primr only speaks the protocol.)
  5. Current catalogs, evaluated defaults. Verified model generations below; never register a superseded model as a default. Re-verify IDs at implementation time against provider docs (the registry refresh is cheap; shipping stale defaults is not).
  6. Test the fit before the run. Before any local model is used: verify the server is reachable, the model is actually installed, AND it fits currently-available memory (a one-token probe call; on a memory error, step down to the next-smaller candidate). Learned live: a 27B model that "exists" but needs 19.5 GiB when 11 GiB is free produces per-call failures, not a clean error.
  7. Default to least incremental spend that passes quality. A free or billing-verifiable tier must exist, even if measurably worse and labeled that way. DDG search is already free, scraping is already local; with local inference the whole run is $0 API plus electricity, which changes what "run it on the whole portfolio" costs. For users already paying for an agent subscription, primr-zero is the supported plan-native path after the host is verified not to bill API usage or overages. An embedded host runner may join routing only when its billing basis can be proven or the operator explicitly acknowledges potentially metered API use. If no validated zero-incremental route is configured, the default should be the best validated sub-dollar direct API recipe. Premium recipes are opt-in and must document the marginal quality or coverage they buy.
  8. Caching must prove savings before it runs. Provider prompt caching is a cost optimization only when a paid write is likely to be reused inside the provider's supported cache window. Do not enable cache writes, pre-warming, background refresh, keepalive requests, or long TTLs until the estimator accounts for write/read pricing and usage records expose cache reads and writes by provider.

Current state (what already exists in the codebase)

  • Provider ABC with xAI, Gemini, OpenAI, Anthropic, OpenRouter, Ollama, Foundry, and Bedrock providers; pick_model_for_role keeps the measured xAI/Gemini route first, then gives explicitly enabled OpenRouter precedence over unpromoted direct OpenAI and Anthropic recipes. Current GPT-6 and Claude 5.5/Fable 5.1 candidates are registered, but they have not been validated as full-pipeline recipes. As of 1.37.0, ai/providers/azure_foundry.py and ai/providers/bedrock.py also exist as real Provider classes registered in KNOWN_PROVIDERS — wired for credential validation (primr keys test) and doctor only; full-pipeline model→provider routing for them is still tracked under the gateway phase (Phase C) below.
  • OpenAICompatibleProvider supports explicit Responses and Chat Completions API styles. Direct OpenAI and xAI use Responses with store=false; the openai_compatible_client.chat_completion helper remains for local Chat Completions servers. Bedrock and Foundry have separate provider classes.
  • ai/local_inference.py (shipped with the calibration local judge): fail-open /v1/models detection, env-chain base URL, family-preference model pick. Missing: the memory fit-check from principle 5.
  • Eval + calibration instruments (Version Plan step 1) to judge any new recipe honestly.
  • ai/capability_routing.py now provides the first backend-freedom router slice: pure StageRequirements matching over supplied backend capability rows, profile-specific cloud/agent/hybrid/local ranking, host-runner opt-in checks, and API-credit handoff guards.
  • ai/stage_routing.py now provides the first runtime bridge from declared production stages to the capability router. fast.scrape_summary, fast.source_relevance, and fast.hiring_signals use it behind --inference cloud|hybrid, append capped body-free stage_routes records to _run_state.json, and still execute through existing provider seams with the legacy utility models preserved as fallback.
  • The typed host-account runner contract now exists in ai/host_agent_runner.py: bounded HostAgentStagePacket, billing policy, normalized result/provenance, and a prompt renderer that fences evidence with the existing content-sanitizer. The first concrete official-host process runner uses codex exec for an unpromoted fast.source_relevance pilot. The public CLI remains --inference cloud|hybrid; Codex becomes eligible only for an explicitly acknowledged, single-company hybrid experiment because authentication does not prove whether execution is plan-backed or API-key billed. Other stages still use their declared backends. Separately, primr prep plus primr-zero provides a host-native evidence handoff without passing subscription credentials into Primr. Neither path treats subscription credentials as interchangeable with API keys.
  • OpenRouterProvider is a Standard-pipeline preview behind both OPENROUTER_API_KEY and PRIMR_OPENROUTER_ENABLED=1. Its curated role recipe is estimate-bound, applies per-request price ceilings, denies data-collection providers, defaults to zero-data-retention endpoints, and records response-level exact cost. Custom model slugs require explicit finite prices. It remains pending representative report-quality evaluation.

OpenRouter gateway preview

  • OpenRouter exposes an OpenAI-compatible endpoint and a public model catalog. Primr keeps gateway-qualified model names separate from direct-provider rows so estimates and accounting retain the correct billing boundary.
  • The September 1, 2026 curated recipe uses Gemini 2.5 Flash Lite for utility, GPT-4.1 Mini for writing, and DeepSeek V3.2 for reasoning. The catalog rates and context limits are recorded in the central model registry.
  • The default utility model remains google/gemini-3.1-flash-lite. The October 22 gateway catalog still lists the three Gemini 2.5 base slugs, now without the previously announced retirement dates. Their explicit overrides are available again in the catalog; the separately absent batch slugs retain their earlier dates. This does not promote the old utility default or prove account-specific access.
  • Provider routing sends registered prompt/completion max_price ceilings, require_parameters=true, data_collection=deny, and ZDR by default. See OpenRouter's provider routing, zero-data retention, usage accounting, and models API.
  • The route is operationally complete across CLI and agent transports, but it is deliberately labeled preview until a representative full-report eval supports quality promotion.

Verified provider facts (October 22, 2026)

The catalog covers supported text/report execution and curated gateway models. It is a dated snapshot, not a promise to list every audio, image, restricted, preview, or alternate-endpoint model a vendor offers. Registration allows explicit selection; it does not establish account access, report quality, or a promotion.

Rates below are USD per million standard text tokens. The estimator supports input, output, cached reads, declared long-context tiers, and conservative per-token write-price allowances when write-free usage cannot be established. Batch, priority, regional contracts, explicit cache writes/storage, and tool fees must be checked separately for the selected execution path. Explicit cache writes remain disabled until their estimation and metering gates are complete. No paid evaluation was performed for this catalog refresh.

Primary sources are linked inline with the integration facts they support:

OpenAI (direct)

Model Input Output Cached input
gpt-6-astra $10 $50 $1
gpt-6.1-sol $2 $10 $0.10
gpt-6-sol $2 $10 $0.20
gpt-6-luna $0.10 $0.50 $0.01

These have 922,000 maximum input tokens and 128,000 maximum output tokens inside the 1,050,000 shared context. Prompts above 272,000 tokens reprice the full request at twice input/cache rates and 1.5 times output rates. Direct OpenAI uses Responses, including for GPT-6.1 Sol tool calling. GPT-6.1 Sol does not accept none or minimal reasoning effort. - Historical registrations remain selectable where available. GPT-5.6 Sol and its unsuffixed alias now cost $4/$20, cached $0.40; the provider guarantees its promotion at least through November 21, without publishing an exact end date. Do not invent a scheduled price change. Terra remains $2/$12 and Luna $0.20/$1.20. The o4-mini cached-input rate is $0.275, not an inferred $0.11. - Shutdowns rechecked 2026-10-27 against the deprecations page: o4-mini and gpt-image-1 shut down on 2026-10-23, with GPT-5.6 Terra and gpt-image-2.5-flare (or -sunburst) as the named substitutes. OpenAI-only reasoning now routes to Terra and icon generation requests Flare; o4-mini stays priced for history. gpt-5.4-nano shuts down on 2027-04-01 (replacement gpt-6-luna) and gpt-4.1-mini on 2027-01-14. On 2026-10-28 the gateway's openai/gpt-4.1-mini route was marked unavailable ahead of that date and stays priced for history; openai/gpt-6-luna replaced it as the default OpenRouter writer. GPT-6.1 Terra ($1/$5, cached $0.05; $2/$7.50 above 272K) is registered directly and on the gateway and is the OpenAI-only reasoner. - primr doctor compares every registered OpenAI ID and the image default with the optional shutdown_date (YYYY-MM-DD or null) on the free List models rows. It warns on past or scheduled shutdowns the registry lacks and on date mismatches. The call passes the SSRF guard and is bounded to 10 seconds. Models missing from the list are named separately from recorded retirements, because absence can also mean the key lacks access. It is report-only; registry edits stay manual and dated. The installed SDK (openai 2.38.0) does not type the field, so it is read as an extra attribute. - Web search: web_search in the Responses API is $10 per 1k calls plus search-content tokens at model rates. Build new work on Responses; Chat Completions is still supported, but the Assistants API shuts down on 2026-08-26. - Batch: OpenAI documents 50% lower costs and a 24-hour completion window. Primr does not currently route production stages through Batch. - Caching: GPT-5.6 cache writes cost 1.25x uncached input and current explicit controls use prompt_cache_options. Current direct OpenAI requests use explicit-only mode without breakpoints, preventing automatic paid writes and prewarming. Foundry applies the same policy for declared GPT-5.6/6 pricing references on Responses. Quotes include maximum published write-price allowances where absence of writes cannot be proven. Token-derived recorded costs remain approximations when write usage or lifetime is unavailable; a quote allowance is not actual spend. Explicit write usage/TTL metering and savings validation are still prerequisites to enabling a caching optimization. Older retention-field assumptions must not drive new behavior. See the OpenAI model guide. - Deep research: OpenAI still documents o3-deep-research and o4-mini-deep-research on Responses. They are provider-expansion candidates, not a wired Primr Premium route.

Google baseline

  • Primr Premium uses Gemini Deep Research through background Interactions with storage enabled. Current responses are parsed from steps[*].model_output.content; the legacy outputs shape remains only for persisted-job compatibility. See Google's Deep Research guide and Interactions overview.
  • File Search uploads are long-running operations and must complete before an interaction references the store. Imported store data persists until manual deletion; raw File API uploads are deleted after 48 hours. See Google's File Search guide.
  • Gemini 4 Carbon is the direct Pro default: $2/$10, cached $0.10, through 2026-11-30, then $4/$20, cached $0.20. Gemini 4 Argon stays registered at the same schedule and is not the default. Gemini 3.8 Flash is the Flash default at $0.75/$3.75, cached $0.075, through December 31, then $1.50/$7.50, cached $0.15. Gemini 3.5 Flash-Lite is the lite default. Flash 3.5 and 3.7 are provider aliases; their first observed accounting cutover stays October 20 when the catalog is rechecked. Primr omits unsupported sampling controls on these request paths, records explicit thinking levels, and applies Google's published January 1, 2027 price change to 3.6 and 3.7 by pricing date. Promotion still requires representative quality and cost evaluation. See the latest-model migration guide and pricing page.

xAI

Grok 4.7 is the current selectable flagship, with a 500K context. Standard input/output/cache rates are $2/$6/$0.50 below 200,000 prompt tokens and $4/$12/$1 at or above 200,000, applied to the full request. Since 2026-10-28 it is the default Grok reasoner in every tier; see grok-default-routing.md.

The release notes retire grok-imagine-image-quality on 2026-11-02 and serve grok-imagine-image-2.0 at low quality in its place. Opt-in icon generation requests image 2.0 with explicit medium quality (the highest documented value, rechecked 2026-10-27) rather than the auto default, which generates at low. Per-image prices are not part of the text-token registry. Live request acceptance is unverified.

OpenRouter

The gateway models API is the authority for its own slugs, prices, context tiers and availability. Direct-provider rates are not copied into gateway rows. For example, gateway Gemini 3.8 Flash is now $0.825/$4.125, cached $0.0825, while direct Google remains $0.75/$3.75.

The curated catalog includes GPT-6/6.1, Claude 5.5/Fable 5.1, Gemini 4 Argon, Grok 4.7, the October 17 DeepSeek V4.1 Flash snapshot and MiniMax M3.1 Flash. Existing DeepSeek, GLM, Qwen and Llama rows were rechecked against that endpoint. Three absent historical routes reject new work without inventing retirement dates. Their prior prices remain usable for historical accounting.

Scheduled DeepSeek V4 rates vary within the UTC day. Until time-of-day pricing is represented by the estimator, their rows use the highest published schedule rate as a conservative estimate and request ceiling, not a claim of the current spot rate. Native billed cost remains authoritative when every call reports it. Gateway min_prompt_tokens thresholds are interpreted inclusively for conservative estimates at the boundary; direct-provider thresholds follow their own docs. Gateway ceilings include the highest declared prompt tier and retain privacy, zero-data-retention and required-parameter controls.

Bedrock and Foundry

AWS pricing and its public pricing-table metered-unit maps independently verify current Bedrock rates. Nova 2 Lite is $0.30/$2.50 globally and $0.33/$2.75 for US geo inference. Current Claude 5.5 and Fable 5.1 global rates match the direct table below; US geo rates are 10% higher, including cache and Haiku's long-context tier. Explicit global and US inference profile IDs are registered. Converse omits invalid Claude sampling controls. These registrations do not promise support for mantle-only APIs or all AWS regions, and the provider still requires configured AWS region and access.

Foundry deployments are operator-owned names with contract, region and service-tier pricing. They still require a verified AZURE_FOUNDRY_PRICE_AS catalog reference or explicit finite input/output prices. GPT-6 references are now available; a direct-provider reference must only be used when it matches the deployment's actual offer. No universal Azure price or account eligibility is inferred. For a deployment with Responses support, set AZURE_OPENAI_API_STYLE=responses explicitly. This reuses the existing Responses adapter and preserves the deployment name, output-token field and store=false; the default remains Chat Completions for existing deployments. Pricing-reference metadata never silently selects an endpoint.

Local Ollama

Current local candidates include Qwen 3.8 27B and Gemma 4 12B, 26B and 31B. They carry $0 model API prices only for local execution, not cloud tags. The catalog's 256K context requires matching runtime configuration; 16K is a conservative configured generation cap, not an asserted provider maximum. No model was downloaded or inferred, and registration does not establish that it fits this machine.

Anthropic (direct)

Model Input Output Cached input
claude-fable-5-1 $10 $50 $0.25
claude-opus-5-5 $4 $20 $0.20
claude-sonnet-5-5 $2 $10 $0.10
claude-haiku-5-5 $0.10 $0.50 $0.01

All have 1M context and 128K output. Haiku 5.5 prompts above 100,000 tokens reprice the full request at $0.50/$2.50, cached $0.05. Sonnet 5's launch rate became its standard $2/$10, cached $0.20; a guessed post-intro increase must not drive estimates. The evaluated legacy routing IDs remain pinned. - Current Claude families reject sampling parameters and manual thinking budgets. Use adaptive thinking and output_config.effort; Opus 5 and Fable 5 cannot disable thinking. They reject assistant-prefill final turns. The newer tokenizer can count about 30% more tokens for the same text, so price-per-token savings do not establish report cost or quality savings. - Retired Haiku 3.5 remains available for historical cost lookup, but new work is blocked from its documented February 19 retirement. Active Sonnet 4.6 is not marked retired. - Web search: server-side web_search_20260209 ($10/1k) plus web_fetch_20260209 (free per-call, token costs only). Both Messages-API tools. - Batch: 50%, inline JSON (no file upload), most complete within 1 hour. - Caching: explicit cache_control, reads at 0.1x, writes at 1.25x/2x; minimum cacheable prefix 2,048-4,096 tokens by model. - No deep-research API product; the agentic loop equivalent is DIY or the Managed Agents beta. - Prompt caching requires cost-aware handling. Anthropic's current docs price 5-minute cache writes above normal input, 1-hour writes higher again, and cache hits lower than base input. Automatic caching exists on the Claude API, Claude Platform on AWS, and Microsoft Foundry beta; Bedrock and Vertex AI do not support automatic caching. Implementation must therefore model writes, reads, TTL choice, gateway support, and cache-hit expectations before enabling this for any Primr stage.

Billing-verifiable agent runners

These are not provider SDKs. They are host-agent execution surfaces that can consume already-paid or already-allocated capacity only when official auth, automation, and trustworthy billing provenance support that claim. Otherwise the operator must explicitly acknowledge that metered API billing may apply.

  • Candidate host surfaces are tracked as sanctioned execution hosts, not provider APIs. A host can become a Primr runner only where it provides an official task, connector, action, CLI, hook, or skill interface that can be invoked with a bounded packet and audited result.
  • Authentication must come from the user's own configured host surface. Primr must not ship repo-owned tokens, account ids, private endpoint knowledge, or hidden credential reuse. Authentication type alone is not billing proof. If a host can transition from plan allocation to API credit spend, that transition must remain explicit in the host and visible in Primr's estimate.
  • Integration shape for primr: emit a bounded stage packet, call the host runner through its official CLI/SDK/automation surface, parse structured output, and validate with the same structural checks and semantic evals as direct API recipes. The host runner never owns the pipeline loop, URL fetch policy, disk writes, or completion decision.
  • Non-goals: no unofficial Max/ChatGPT proxies, browser-session scraping, reverse-engineered endpoints, or "subscription as hidden API key" behavior. If an official surface cannot run a stage reliably, use direct API, gateway, or local inference instead.

AWS Bedrock (gateway)

  • The bedrock-mantle.{region}.api.aws endpoint (console GA June 2026) serves OpenAI Chat Completions + Responses AND native Anthropic Messages. Auth: a plain Bedrock API key (AWS_BEARER_TOKEN_BEDROCK for boto3, or as the api_key against the OpenAI-compatible endpoint). No SigV4 needed for primr's path.
  • Claude Fable 5 was day-one on Bedrock; GPT-5.4/5.5 are GA (Responses only). Whether Claude works through the Chat Completions shape is contradicted within AWS's own docs: TEST EMPIRICALLY, else use the anthropic SDK against mantle's /anthropic/v1/messages.
  • Pricing: provider list price on global endpoints, ~10% premium regional.
  • Gaps that shape the recipe: no Anthropic web_search through Bedrock (Nova-only web grounding); batch lags newest models (no Fable 5 / Opus 4.7+ / GPT-5.5 batch yet).

Microsoft Foundry (gateway)

  • The /openai/v1/ endpoint (GA, no api-version param) works with the stock openai SDK: set base URL + API key, done. DeepSeek/Grok/MAI models route through the same endpoint; Claude (preview) uses a separate /anthropic/v1/messages endpoint billed via Marketplace at Anthropic list prices.
  • Web search: web_search tool in the plain Responses API, $14/1k (Bing-grounded; data leaves the compliance boundary).
  • Do NOT adopt azure-ai-inference (retires 2026-08-26); Microsoft's own recommendation is the stock openai package.
  • Batch: Azure Global Batch 50% but gpt-5.4/5.5 not yet in the batch table; no batch for non-OpenAI models.

Local (Ollama / LM Studio / llama.cpp / vLLM)

  • 2026 reality: small-active-parameter MoE models give ~30B quality at ~3-4B decode speed. Role fits on the 16-24 GB tier: reasoning qwen3.6:35b (3B active) or gpt-oss:20b; writing gemma4:31b or qwen3.6:27b; utility qwen3.5:4b/:9b; judging granite4.1-guardian:8b (purpose-built yes/no judge). All Apache 2.0.
  • Honest quality bar: best 24 GB-runnable models measure roughly HALF of frontier on long-form writing (EQ-Bench Longform ~40-42 vs ~79-80), with repetition/slop growth past a few thousand words. primr's section-by-section writing + refine loop is the right shape to partially compensate, and the free tier ships labeled as "free, slower, weaker writing" for this measured hardware/model generation. That label is not a permanent product judgment: as desk-side AI appliances and workstation-class local models improve, local profiles should be re-evaluated and promoted stage by stage when they meet the same quality bar as paid routes.
  • Integration footguns to handle in code:
  • Ollama's /v1 endpoint cannot set context per request and silently front-truncates beyond the loaded context. The local profile must verify effective context (instruct users on OLLAMA_CONTEXT_LENGTH, or create a model variant with num_ctx baked in) and refuse corpus-stage calls when the window is too small to be honest.
  • Synthesis-usable context is far below advertised for most local models (NoLiMa); keep primr's chunk-summarize-then-synthesize shape, never one giant prompt.
  • Prefill dominates on Mac/CPU (minutes for a 60K-token corpus); progress display should set expectations per stage.
  • KV-cache memory: recommend OLLAMA_KV_CACHE_TYPE=q8_0 + OLLAMA_FLASH_ATTENTION=1 in docs.
  • Throughput envelope (Q4, 20k-word report): ~2-3 min decode on a 4090 with a 30B-A3B MoE; ~5-6 min on an M4 Max; 20-40 min CPU-only MoE. Viable everywhere; the free tier is real, and its quality should be treated as a moving benchmark rather than a fixed ceiling.

Delivery plan (phased, each phase independently shippable)

Phase A: first-class OpenAI and Anthropic recipes (1.x)

  1. Registry refresh - PARTIAL: GPT-5.6 and current Claude-family model rows are registered at current prices. The finer-grained capability flags remain queued: supports_temperature, tokenizer_multiplier, web_search_tool (none | openai_responses | anthropic_messages | xai_browse), and requires_responses_api.
  2. Sampling-param guard: the LLM call seam consults supports_temperature before setting sampling params (Fable 5 / Opus 4.7+ reject them). The Anthropic provider already omits temperature, top_p, and top_k on every model, because SDK 2 sends API version 2026-10-21 and that version rejects them.
  3. Web-search capability routing: the browse/search stage dispatches to the provider-native tool (OpenAI Responses web_search, Anthropic web_search_20260209) when the recipe's provider has one; DDG remains the free default everywhere.
  4. Recipe validation: one cheap live run per recipe (openai-only key, anthropic-only key), eval-scored with the step-1 instruments, before the README/dry-run advertise them. Estimator entries for both. Each recipe exits validation with a default class: sub-dollar default candidate, specialty fallback, or premium-only.

Phase A.1: provider prompt-caching research and cost-safe design

  1. Research current provider behavior for Anthropic, OpenAI, Gemini, xAI, Bedrock, Foundry, and Vertex. Capture which APIs support explicit breakpoints, automatic caching, cache TTLs, cache diagnostics, and gateway parity.
  2. Add model/provider metadata for cache write multiplier, cache read multiplier, minimum cacheable prefix, supported TTLs, breakpoint limits, and automatic-caching availability.
  3. Extend estimates and usage records before enabling cache controls. A route must show projected savings from repeated static prefixes; otherwise caching stays off.
  4. Disallow paid background cache pre-warming, keepalive refresh loops, max-token workarounds, and 1-hour TTL defaults. Long TTLs require explicit operator opt-in and a cost estimate.
  5. Add tests proving cost accounting covers cache writes and reads, and proving caching is disabled for one-off or volatile prompts that would pay repeated writes without hits.

Phase B: billing-verifiable host runners

  1. Foundation seam - SHIPPED: HostAgentRunner accepts a typed stage packet (role, prompt, evidence bundle, output schema, budget/plan policy) and returns structured text plus runner metadata. It is covered with fake runner tests.
  2. First official-host transport - EXPERIMENTAL, EVAL PENDING: the Codex CLI adapter handles fast.source_relevance through documented codex exec automation. It fails closed against silent API fallback when the eval harness selects the internal agent profile. A single-company hybrid command can now opt in only with --acknowledge-host-agent-may-bill; Primr records potentially metered billing and pending-eval status and rejects batch fan-out. Additional hosts and stages remain eval-gated.
  3. Host-native handoff - SHIPPED: primr prep emits a bounded evidence packet under a hard no-model-call policy, and primr-zero lets the surrounding host research and synthesize from its own plan allowance after the host is verified not to bill API usage or overages. This is not an internal stage runner and is labeled host-assisted.
  4. Add capability probes before any runner. A surface is eligible only if it can be invoked through official automation, returns or stores schema-constrained output, exposes enough provenance to stamp the sidecar, and can be bounded by wall-clock plus task-count policy. If any of those are missing, it remains an operating host for Primr, not an internal stage runner.
  5. Budget model: API dollars are unknown unless the host exposes trustworthy billing provenance. The preflight estimate may show "host plan usage" only when that basis is proven; otherwise it must show unknown or potentially metered billing, bounded stage count, wall-clock cap, optional token/task ceilings, and an acknowledgment gate. Any handoff to API credits must remain explicit in the host, never hidden by primr.
  6. Validation: one cheap or verified plan-backed recipe eval per runner on the standing corpus. It must pass the same label calibration and trust checks as direct provider recipes before README promotion. A promoted runner may outrank paid API stages only when zero incremental API spend is proven. An explicitly acknowledged, potentially metered runner remains a distinct opt-in route; wall-clock, plan-limit, and billing uncertainty still appear in the estimate.

Phase C: gateway support (Bedrock + Foundry)

Near-zero new code by design: both are base URL + API key against the existing OpenAI-compatible provider.

  1. Env plumbing + docs: PRIMR_GATEWAY=bedrock|foundry style profile (or plain LOCAL_LLM_BASE_URL-pattern envs) mapping to the right base URL shape, key env, and model-ID dialect (deployment names on Foundry, anthropic.-prefixed IDs on Bedrock).
  2. Doctor checks: reachability + a one-token probe per configured gateway.
  3. Honest docs on gateway gaps: no Anthropic web_search through either (those recipes run DDG-only browse or pair a direct key for search); batch lag on newest models; Bedrock Claude-via-ChatCompletions must be verified empirically per AWS's self-contradictory docs.

Phase D: the $0 local profile (with 2.0 backend freedom)

  1. --inference local execution profile per the backend-freedom design: local models for every stage with a local equivalent; stages with none (Deep Research) are skipped with clear logging.
  2. Fit-check before use (principle 5), generalizing what the calibration local judge needs today: model installed (from /v1/models), memory headroom (one-token probe; on the Ollama out-of-memory error, step down to the next-smaller installed candidate), effective context window adequate for the stage.
  3. Context guard for corpus stages (refuse silent truncation).
  4. Role mapping defaults from the June-2026 research (MoE-first), but ALWAYS selected from what is actually installed; nothing hardcoded.
  5. Eval the local recipe with the step-1 instruments and publish the honest quality delta next to the $0 API price tag. Local mode is allowed to be slower or weaker, but it must never silently upgrade itself to paid cloud; any paid fallback is a separate operator choice. Re-run the local eval when new local model generations or desktop-class AI hardware materially change the capability envelope, and let passing stages move up the default route.
  6. For a single RTX 4090 or comparable 24 GB GPU, run the focused 4090-report-race stage eval before the broader 4090-top10 sweep. The first question is pragmatic: does the user's local box already clear the local-vs-sub-dollar bar for a production-adjacent stage?

Sequencing vs the Version Plan

Phase A remains 1.x-compatible incremental work. Registry refresh and the Responses transport are shipped; capability flags, provider-native search routing, recipe estimates, and eval promotion remain. Phase B's typed runner and first gated Codex transport are shipped experiments and do not block direct API recipes. Phase C is config/docs heavy and can land after A. Phase D is the backend-freedom pillar (2.0) and depends on the shipped evaluation instruments plus per-stage routing requirements.

Validation protocol (cost-disciplined)

  • Everything testable offline stays offline: registry flags, sampling guards, dispatch seams, fit-check logic all get injected-seam unit tests.
  • One cheap live run per new recipe, ever, before it is advertised (precedent: the v1.24.0 cross-provider eval).
  • Local recipes validated with agreement-style checks against a cloud reference (the calibration --judge-compare pattern) before being trusted for judge/utility roles, and with the eval scorecard for writing roles.