Provider Expansion: OpenAI, Anthropic, Billing-Verifiable Hosts, Gateways, and Local¶
Status: STARTED (provider facts refreshed October 22, 2026; Responses
adapters, governed OpenRouter preview, explicitly gated Codex experiment, and
host-native handoff shipped; full recipe promotions remain eval-gated)
ROADMAP anchor: Active Queue #26. Companion to
2.0-backend-freedom.md (routing architecture);
this doc is the concrete provider catalog and delivery plan that routing
will route over.
Principles (set by the operator, June 2026)¶
- No reason this must be Grok. Grok + Gemini is the measured default, not a dependency. Every pipeline role (reasoning, writing, utility) should be servable by any provider whose models meet the role's bar.
- Support whatever someone has. A user arrives with an OpenAI key, or an Anthropic key, or a corporate Bedrock/Foundry gateway, or sanctioned agent-host allocation, or just a gaming GPU and Ollama. Each of those should produce a report once its recipe is validated. Absence of any of them is silent, never an error.
- Account capacity is a first-class capacity source. Sanctioned agent hosts should be evaluated as ways to spend capacity the user already has. They become Primr runners only through official automation, connector, CLI, hook, or agent-skill surfaces that can accept bounded stage packets and return structured outputs. They are never treated as hidden API keys or repo-owned accounts.
- Mac/Linux/Windows equally. Local inference integrates over HTTP to OpenAI-compatible servers only; no platform-specific code. (vLLM has no Windows support; that is the server operator's concern, not primr's, because primr only speaks the protocol.)
- Current catalogs, evaluated defaults. Verified model generations below; never register a superseded model as a default. Re-verify IDs at implementation time against provider docs (the registry refresh is cheap; shipping stale defaults is not).
- Test the fit before the run. Before any local model is used: verify the server is reachable, the model is actually installed, AND it fits currently-available memory (a one-token probe call; on a memory error, step down to the next-smaller candidate). Learned live: a 27B model that "exists" but needs 19.5 GiB when 11 GiB is free produces per-call failures, not a clean error.
- Default to least incremental spend that passes quality. A free or
billing-verifiable tier must exist, even if measurably worse and labeled that way.
DDG search is already free, scraping is already local; with local inference
the whole run is $0 API plus electricity, which changes what "run it on the
whole portfolio" costs. For users already paying for an agent subscription,
primr-zerois the supported plan-native path after the host is verified not to bill API usage or overages. An embedded host runner may join routing only when its billing basis can be proven or the operator explicitly acknowledges potentially metered API use. If no validated zero-incremental route is configured, the default should be the best validated sub-dollar direct API recipe. Premium recipes are opt-in and must document the marginal quality or coverage they buy. - Caching must prove savings before it runs. Provider prompt caching is a cost optimization only when a paid write is likely to be reused inside the provider's supported cache window. Do not enable cache writes, pre-warming, background refresh, keepalive requests, or long TTLs until the estimator accounts for write/read pricing and usage records expose cache reads and writes by provider.
Current state (what already exists in the codebase)¶
ProviderABC with xAI, Gemini, OpenAI, Anthropic, OpenRouter, Ollama, Foundry, and Bedrock providers;pick_model_for_rolekeeps the measured xAI/Gemini route first, then gives explicitly enabled OpenRouter precedence over unpromoted direct OpenAI and Anthropic recipes. Current GPT-6 and Claude 5.5/Fable 5.1 candidates are registered, but they have not been validated as full-pipeline recipes. As of 1.37.0,ai/providers/azure_foundry.pyandai/providers/bedrock.pyalso exist as realProviderclasses registered inKNOWN_PROVIDERS— wired for credential validation (primr keys test) anddoctoronly; full-pipeline model→provider routing for them is still tracked under the gateway phase (Phase C) below.OpenAICompatibleProvidersupports explicit Responses and Chat Completions API styles. Direct OpenAI and xAI use Responses withstore=false; theopenai_compatible_client.chat_completionhelper remains for local Chat Completions servers. Bedrock and Foundry have separate provider classes.ai/local_inference.py(shipped with the calibration local judge): fail-open/v1/modelsdetection, env-chain base URL, family-preference model pick. Missing: the memory fit-check from principle 5.- Eval + calibration instruments (Version Plan step 1) to judge any new recipe honestly.
ai/capability_routing.pynow provides the first backend-freedom router slice: pureStageRequirementsmatching over supplied backend capability rows, profile-specific cloud/agent/hybrid/local ranking, host-runner opt-in checks, and API-credit handoff guards.ai/stage_routing.pynow provides the first runtime bridge from declared production stages to the capability router.fast.scrape_summary,fast.source_relevance, andfast.hiring_signalsuse it behind--inference cloud|hybrid, append capped body-freestage_routesrecords to_run_state.json, and still execute through existing provider seams with the legacy utility models preserved as fallback.- The typed host-account runner contract now exists in
ai/host_agent_runner.py: boundedHostAgentStagePacket, billing policy, normalized result/provenance, and a prompt renderer that fences evidence with the existing content-sanitizer. The first concrete official-host process runner usescodex execfor an unpromotedfast.source_relevancepilot. The public CLI remains--inference cloud|hybrid; Codex becomes eligible only for an explicitly acknowledged, single-company hybrid experiment because authentication does not prove whether execution is plan-backed or API-key billed. Other stages still use their declared backends. Separately,primr prepplusprimr-zeroprovides a host-native evidence handoff without passing subscription credentials into Primr. Neither path treats subscription credentials as interchangeable with API keys. OpenRouterProvideris a Standard-pipeline preview behind bothOPENROUTER_API_KEYandPRIMR_OPENROUTER_ENABLED=1. Its curated role recipe is estimate-bound, applies per-request price ceilings, denies data-collection providers, defaults to zero-data-retention endpoints, and records response-level exact cost. Custom model slugs require explicit finite prices. It remains pending representative report-quality evaluation.
OpenRouter gateway preview¶
- OpenRouter exposes an OpenAI-compatible endpoint and a public model catalog. Primr keeps gateway-qualified model names separate from direct-provider rows so estimates and accounting retain the correct billing boundary.
- The September 1, 2026 curated recipe uses Gemini 2.5 Flash Lite for utility, GPT-4.1 Mini for writing, and DeepSeek V3.2 for reasoning. The catalog rates and context limits are recorded in the central model registry.
- The default utility model remains
google/gemini-3.1-flash-lite. The October 22 gateway catalog still lists the three Gemini 2.5 base slugs, now without the previously announced retirement dates. Their explicit overrides are available again in the catalog; the separately absent batch slugs retain their earlier dates. This does not promote the old utility default or prove account-specific access. - Provider routing sends registered prompt/completion
max_priceceilings,require_parameters=true,data_collection=deny, and ZDR by default. See OpenRouter's provider routing, zero-data retention, usage accounting, and models API. - The route is operationally complete across CLI and agent transports, but it is deliberately labeled preview until a representative full-report eval supports quality promotion.
Verified provider facts (October 22, 2026)¶
The catalog covers supported text/report execution and curated gateway models. It is a dated snapshot, not a promise to list every audio, image, restricted, preview, or alternate-endpoint model a vendor offers. Registration allows explicit selection; it does not establish account access, report quality, or a promotion.
Rates below are USD per million standard text tokens. The estimator supports input, output, cached reads, declared long-context tiers, and conservative per-token write-price allowances when write-free usage cannot be established. Batch, priority, regional contracts, explicit cache writes/storage, and tool fees must be checked separately for the selected execution path. Explicit cache writes remain disabled until their estimation and metering gates are complete. No paid evaluation was performed for this catalog refresh.
Primary sources are linked inline with the integration facts they support:
OpenAI (direct)¶
- The current model catalog and standard prices are:
| Model | Input | Output | Cached input |
|---|---|---|---|
gpt-6-astra |
$10 | $50 | $1 |
gpt-6.1-sol |
$2 | $10 | $0.10 |
gpt-6-sol |
$2 | $10 | $0.20 |
gpt-6-luna |
$0.10 | $0.50 | $0.01 |
These have 922,000 maximum input tokens and 128,000 maximum output tokens
inside the 1,050,000 shared context. Prompts above 272,000 tokens reprice the
full request at twice input/cache rates and 1.5 times output rates. Direct
OpenAI uses Responses, including for GPT-6.1 Sol tool calling. GPT-6.1 Sol
does not accept none or minimal reasoning effort.
- Historical registrations remain selectable where available. GPT-5.6 Sol and
its unsuffixed alias now cost $4/$20, cached $0.40; the provider guarantees
its promotion at least through November 21, without publishing an exact end
date. Do not invent a scheduled price change. Terra remains $2/$12 and Luna
$0.20/$1.20. The o4-mini cached-input rate is $0.275, not an inferred $0.11.
- Shutdowns rechecked 2026-10-27 against the
deprecations page:
o4-mini and gpt-image-1 shut down on 2026-10-23, with GPT-5.6 Terra and
gpt-image-2.5-flare (or -sunburst) as the named substitutes. OpenAI-only
reasoning now routes to Terra and icon generation requests Flare; o4-mini
stays priced for history. gpt-5.4-nano shuts down on 2027-04-01 (replacement
gpt-6-luna) and gpt-4.1-mini on 2027-01-14. On 2026-10-28 the gateway's
openai/gpt-4.1-mini route was marked unavailable ahead of that date and
stays priced for history; openai/gpt-6-luna replaced it as the default
OpenRouter writer. GPT-6.1 Terra ($1/$5, cached $0.05; $2/$7.50 above 272K)
is registered directly and on the gateway and is the OpenAI-only reasoner.
- primr doctor compares every registered OpenAI ID and the image default
with the optional shutdown_date (YYYY-MM-DD or null) on the free
List models
rows. It warns on past or scheduled shutdowns the registry lacks and on
date mismatches. The call passes the SSRF guard and is bounded to 10
seconds. Models missing from the list are named separately from recorded
retirements, because absence can also mean the key lacks access. It is
report-only; registry edits stay manual and dated.
The installed SDK (openai 2.38.0) does not type the field, so it is read as
an extra attribute.
- Web search: web_search in the Responses API is
$10 per 1k calls
plus search-content tokens at model rates. Build new work on Responses; Chat
Completions is still supported, but the
Assistants API shuts down on 2026-08-26.
- Batch: OpenAI documents
50% lower costs and a 24-hour completion window.
Primr does not currently route production stages through Batch.
- Caching: GPT-5.6 cache writes cost 1.25x uncached input and current explicit
controls use prompt_cache_options. Current direct OpenAI requests use
explicit-only mode without breakpoints, preventing automatic paid writes and
prewarming. Foundry applies the same policy for declared GPT-5.6/6 pricing
references on Responses. Quotes include maximum published write-price allowances where absence of
writes cannot be proven. Token-derived recorded costs remain approximations
when write usage or lifetime is unavailable; a quote allowance is not actual
spend. Explicit write usage/TTL metering and savings validation
are still prerequisites to enabling a caching optimization. Older
retention-field assumptions must not drive new behavior. See the
OpenAI model guide.
- Deep research: OpenAI still documents
o3-deep-research
and o4-mini-deep-research
on Responses. They are provider-expansion candidates, not a wired Primr
Premium route.
Google baseline¶
- Primr Premium uses Gemini Deep Research through background Interactions with
storage enabled. Current responses are parsed from
steps[*].model_output.content; the legacyoutputsshape remains only for persisted-job compatibility. See Google's Deep Research guide and Interactions overview. - File Search uploads are long-running operations and must complete before an interaction references the store. Imported store data persists until manual deletion; raw File API uploads are deleted after 48 hours. See Google's File Search guide.
- Gemini 4 Carbon is the direct Pro default: $2/$10, cached $0.10, through 2026-11-30, then $4/$20, cached $0.20. Gemini 4 Argon stays registered at the same schedule and is not the default. Gemini 3.8 Flash is the Flash default at $0.75/$3.75, cached $0.075, through December 31, then $1.50/$7.50, cached $0.15. Gemini 3.5 Flash-Lite is the lite default. Flash 3.5 and 3.7 are provider aliases; their first observed accounting cutover stays October 20 when the catalog is rechecked. Primr omits unsupported sampling controls on these request paths, records explicit thinking levels, and applies Google's published January 1, 2027 price change to 3.6 and 3.7 by pricing date. Promotion still requires representative quality and cost evaluation. See the latest-model migration guide and pricing page.
xAI¶
Grok 4.7 is the current selectable
flagship, with a 500K context. Standard input/output/cache rates are $2/$6/$0.50
below 200,000 prompt tokens and $4/$12/$1 at or above 200,000, applied to the full
request. Since 2026-10-28 it is the default Grok reasoner in every tier; see
grok-default-routing.md.
The release notes retire
grok-imagine-image-quality on 2026-11-02 and serve grok-imagine-image-2.0 at
low quality in its place. Opt-in icon generation requests image 2.0 with
explicit medium quality (the highest documented value, rechecked 2026-10-27)
rather than the auto default, which generates at low. Per-image prices are
not part of the text-token registry. Live request acceptance is unverified.
OpenRouter¶
The gateway models API is the authority for its own slugs, prices, context tiers and availability. Direct-provider rates are not copied into gateway rows. For example, gateway Gemini 3.8 Flash is now $0.825/$4.125, cached $0.0825, while direct Google remains $0.75/$3.75.
The curated catalog includes GPT-6/6.1, Claude 5.5/Fable 5.1, Gemini 4 Argon, Grok 4.7, the October 17 DeepSeek V4.1 Flash snapshot and MiniMax M3.1 Flash. Existing DeepSeek, GLM, Qwen and Llama rows were rechecked against that endpoint. Three absent historical routes reject new work without inventing retirement dates. Their prior prices remain usable for historical accounting.
Scheduled DeepSeek V4 rates vary within the UTC day. Until time-of-day pricing
is represented by the estimator, their rows use the highest published schedule
rate as a conservative estimate and request ceiling, not a claim of the current
spot rate. Native billed cost remains authoritative when every call reports it.
Gateway min_prompt_tokens thresholds are interpreted inclusively for conservative
estimates at the boundary; direct-provider thresholds follow their own docs.
Gateway ceilings include the highest declared prompt tier and retain privacy,
zero-data-retention and required-parameter controls.
Bedrock and Foundry¶
AWS pricing and its public pricing-table metered-unit maps independently verify current Bedrock rates. Nova 2 Lite is $0.30/$2.50 globally and $0.33/$2.75 for US geo inference. Current Claude 5.5 and Fable 5.1 global rates match the direct table below; US geo rates are 10% higher, including cache and Haiku's long-context tier. Explicit global and US inference profile IDs are registered. Converse omits invalid Claude sampling controls. These registrations do not promise support for mantle-only APIs or all AWS regions, and the provider still requires configured AWS region and access.
Foundry deployments
are operator-owned names with contract, region and service-tier pricing. They
still require a verified AZURE_FOUNDRY_PRICE_AS catalog reference or explicit
finite input/output prices. GPT-6 references are now available; a direct-provider
reference must only be used when it matches the deployment's actual offer. No
universal Azure price or account eligibility is inferred. For a deployment with
Responses support,
set AZURE_OPENAI_API_STYLE=responses explicitly. This reuses the existing
Responses adapter and preserves the deployment name, output-token field and
store=false; the default remains Chat Completions for existing deployments.
Pricing-reference metadata never silently selects an endpoint.
Local Ollama¶
Current local candidates include Qwen 3.8 27B and Gemma 4 12B, 26B and 31B. They carry $0 model API prices only for local execution, not cloud tags. The catalog's 256K context requires matching runtime configuration; 16K is a conservative configured generation cap, not an asserted provider maximum. No model was downloaded or inferred, and registration does not establish that it fits this machine.
Anthropic (direct)¶
- Current models and API identifiers have these standard rates:
| Model | Input | Output | Cached input |
|---|---|---|---|
claude-fable-5-1 |
$10 | $50 | $0.25 |
claude-opus-5-5 |
$4 | $20 | $0.20 |
claude-sonnet-5-5 |
$2 | $10 | $0.10 |
claude-haiku-5-5 |
$0.10 | $0.50 | $0.01 |
All have 1M context and 128K output. Haiku 5.5 prompts above 100,000 tokens
reprice the full request at $0.50/$2.50, cached $0.05. Sonnet 5's launch
rate became its standard $2/$10, cached $0.20; a guessed post-intro increase
must not drive estimates. The evaluated legacy routing IDs remain pinned.
- Current Claude families reject sampling parameters and manual thinking
budgets. Use adaptive thinking and output_config.effort; Opus 5 and Fable
5 cannot disable thinking. They reject assistant-prefill final turns.
The newer tokenizer can count about 30% more tokens for the same text, so
price-per-token savings do not establish report cost or quality savings.
- Retired Haiku 3.5
remains available for historical cost lookup, but new work is blocked from
its documented February 19 retirement. Active Sonnet 4.6 is not marked retired.
- Web search: server-side web_search_20260209 ($10/1k) plus
web_fetch_20260209 (free per-call, token costs only). Both Messages-API
tools.
- Batch: 50%, inline JSON (no file upload), most complete within 1 hour.
- Caching: explicit cache_control, reads at 0.1x, writes at 1.25x/2x;
minimum cacheable prefix 2,048-4,096 tokens by model.
- No deep-research API product; the agentic loop equivalent is DIY or the
Managed Agents beta.
- Prompt caching requires cost-aware handling. Anthropic's current docs price
5-minute cache writes above normal input, 1-hour writes higher again, and
cache hits lower than base input. Automatic caching exists on the Claude API,
Claude Platform on AWS, and Microsoft Foundry beta; Bedrock and Vertex AI do
not support automatic caching. Implementation must therefore model writes,
reads, TTL choice, gateway support, and cache-hit expectations before enabling
this for any Primr stage.
Billing-verifiable agent runners¶
These are not provider SDKs. They are host-agent execution surfaces that can consume already-paid or already-allocated capacity only when official auth, automation, and trustworthy billing provenance support that claim. Otherwise the operator must explicitly acknowledge that metered API billing may apply.
- Candidate host surfaces are tracked as sanctioned execution hosts, not provider APIs. A host can become a Primr runner only where it provides an official task, connector, action, CLI, hook, or skill interface that can be invoked with a bounded packet and audited result.
- Authentication must come from the user's own configured host surface. Primr must not ship repo-owned tokens, account ids, private endpoint knowledge, or hidden credential reuse. Authentication type alone is not billing proof. If a host can transition from plan allocation to API credit spend, that transition must remain explicit in the host and visible in Primr's estimate.
- Integration shape for primr: emit a bounded stage packet, call the host runner through its official CLI/SDK/automation surface, parse structured output, and validate with the same structural checks and semantic evals as direct API recipes. The host runner never owns the pipeline loop, URL fetch policy, disk writes, or completion decision.
- Non-goals: no unofficial Max/ChatGPT proxies, browser-session scraping, reverse-engineered endpoints, or "subscription as hidden API key" behavior. If an official surface cannot run a stage reliably, use direct API, gateway, or local inference instead.
AWS Bedrock (gateway)¶
- The
bedrock-mantle.{region}.api.awsendpoint (console GA June 2026) serves OpenAI Chat Completions + Responses AND native Anthropic Messages. Auth: a plain Bedrock API key (AWS_BEARER_TOKEN_BEDROCKfor boto3, or as theapi_keyagainst the OpenAI-compatible endpoint). No SigV4 needed for primr's path. - Claude Fable 5 was day-one on Bedrock; GPT-5.4/5.5 are GA (Responses
only). Whether Claude works through the Chat Completions shape is
contradicted within AWS's own docs: TEST EMPIRICALLY, else use the
anthropicSDK against mantle's/anthropic/v1/messages. - Pricing: provider list price on global endpoints, ~10% premium regional.
- Gaps that shape the recipe: no Anthropic web_search through Bedrock (Nova-only web grounding); batch lags newest models (no Fable 5 / Opus 4.7+ / GPT-5.5 batch yet).
Microsoft Foundry (gateway)¶
- The
/openai/v1/endpoint (GA, no api-version param) works with the stockopenaiSDK: set base URL + API key, done. DeepSeek/Grok/MAI models route through the same endpoint; Claude (preview) uses a separate/anthropic/v1/messagesendpoint billed via Marketplace at Anthropic list prices. - Web search:
web_searchtool in the plain Responses API, $14/1k (Bing-grounded; data leaves the compliance boundary). - Do NOT adopt
azure-ai-inference(retires 2026-08-26); Microsoft's own recommendation is the stockopenaipackage. - Batch: Azure Global Batch 50% but gpt-5.4/5.5 not yet in the batch table; no batch for non-OpenAI models.
Local (Ollama / LM Studio / llama.cpp / vLLM)¶
- 2026 reality: small-active-parameter MoE models give ~30B quality at
~3-4B decode speed. Role fits on the 16-24 GB tier:
reasoning
qwen3.6:35b(3B active) orgpt-oss:20b; writinggemma4:31borqwen3.6:27b; utilityqwen3.5:4b/:9b; judginggranite4.1-guardian:8b(purpose-built yes/no judge). All Apache 2.0. - Honest quality bar: best 24 GB-runnable models measure roughly HALF of frontier on long-form writing (EQ-Bench Longform ~40-42 vs ~79-80), with repetition/slop growth past a few thousand words. primr's section-by-section writing + refine loop is the right shape to partially compensate, and the free tier ships labeled as "free, slower, weaker writing" for this measured hardware/model generation. That label is not a permanent product judgment: as desk-side AI appliances and workstation-class local models improve, local profiles should be re-evaluated and promoted stage by stage when they meet the same quality bar as paid routes.
- Integration footguns to handle in code:
- Ollama's
/v1endpoint cannot set context per request and silently front-truncates beyond the loaded context. The local profile must verify effective context (instruct users onOLLAMA_CONTEXT_LENGTH, or create a model variant withnum_ctxbaked in) and refuse corpus-stage calls when the window is too small to be honest. - Synthesis-usable context is far below advertised for most local models (NoLiMa); keep primr's chunk-summarize-then-synthesize shape, never one giant prompt.
- Prefill dominates on Mac/CPU (minutes for a 60K-token corpus); progress display should set expectations per stage.
- KV-cache memory: recommend
OLLAMA_KV_CACHE_TYPE=q8_0+OLLAMA_FLASH_ATTENTION=1in docs. - Throughput envelope (Q4, 20k-word report): ~2-3 min decode on a 4090 with a 30B-A3B MoE; ~5-6 min on an M4 Max; 20-40 min CPU-only MoE. Viable everywhere; the free tier is real, and its quality should be treated as a moving benchmark rather than a fixed ceiling.
Delivery plan (phased, each phase independently shippable)¶
Phase A: first-class OpenAI and Anthropic recipes (1.x)¶
- Registry refresh - PARTIAL: GPT-5.6 and current Claude-family model rows
are registered at current prices. The finer-grained capability flags remain
queued:
supports_temperature,tokenizer_multiplier,web_search_tool(none | openai_responses | anthropic_messages | xai_browse), andrequires_responses_api. - Sampling-param guard: the LLM call seam consults
supports_temperaturebefore setting sampling params (Fable 5 / Opus 4.7+ reject them). The Anthropic provider already omitstemperature,top_p, andtop_kon every model, because SDK 2 sends API version 2026-10-21 and that version rejects them. - Web-search capability routing: the browse/search stage dispatches to
the provider-native tool (OpenAI Responses
web_search, Anthropicweb_search_20260209) when the recipe's provider has one; DDG remains the free default everywhere. - Recipe validation: one cheap live run per recipe (openai-only key, anthropic-only key), eval-scored with the step-1 instruments, before the README/dry-run advertise them. Estimator entries for both. Each recipe exits validation with a default class: sub-dollar default candidate, specialty fallback, or premium-only.
Phase A.1: provider prompt-caching research and cost-safe design¶
- Research current provider behavior for Anthropic, OpenAI, Gemini, xAI, Bedrock, Foundry, and Vertex. Capture which APIs support explicit breakpoints, automatic caching, cache TTLs, cache diagnostics, and gateway parity.
- Add model/provider metadata for cache write multiplier, cache read multiplier, minimum cacheable prefix, supported TTLs, breakpoint limits, and automatic-caching availability.
- Extend estimates and usage records before enabling cache controls. A route must show projected savings from repeated static prefixes; otherwise caching stays off.
- Disallow paid background cache pre-warming, keepalive refresh loops, max-token workarounds, and 1-hour TTL defaults. Long TTLs require explicit operator opt-in and a cost estimate.
- Add tests proving cost accounting covers cache writes and reads, and proving caching is disabled for one-off or volatile prompts that would pay repeated writes without hits.
Phase B: billing-verifiable host runners¶
- Foundation seam - SHIPPED:
HostAgentRunneraccepts a typed stage packet (role, prompt, evidence bundle, output schema, budget/plan policy) and returns structured text plus runner metadata. It is covered with fake runner tests. - First official-host transport - EXPERIMENTAL, EVAL PENDING: the Codex CLI adapter
handles
fast.source_relevancethrough documentedcodex execautomation. It fails closed against silent API fallback when the eval harness selects the internal agent profile. A single-company hybrid command can now opt in only with--acknowledge-host-agent-may-bill; Primr records potentially metered billing and pending-eval status and rejects batch fan-out. Additional hosts and stages remain eval-gated. - Host-native handoff - SHIPPED:
primr prepemits a bounded evidence packet under a hard no-model-call policy, andprimr-zerolets the surrounding host research and synthesize from its own plan allowance after the host is verified not to bill API usage or overages. This is not an internal stage runner and is labeled host-assisted. - Add capability probes before any runner. A surface is eligible only if it can be invoked through official automation, returns or stores schema-constrained output, exposes enough provenance to stamp the sidecar, and can be bounded by wall-clock plus task-count policy. If any of those are missing, it remains an operating host for Primr, not an internal stage runner.
- Budget model: API dollars are unknown unless the host exposes trustworthy billing provenance. The preflight estimate may show "host plan usage" only when that basis is proven; otherwise it must show unknown or potentially metered billing, bounded stage count, wall-clock cap, optional token/task ceilings, and an acknowledgment gate. Any handoff to API credits must remain explicit in the host, never hidden by primr.
- Validation: one cheap or verified plan-backed recipe eval per runner on the standing corpus. It must pass the same label calibration and trust checks as direct provider recipes before README promotion. A promoted runner may outrank paid API stages only when zero incremental API spend is proven. An explicitly acknowledged, potentially metered runner remains a distinct opt-in route; wall-clock, plan-limit, and billing uncertainty still appear in the estimate.
Phase C: gateway support (Bedrock + Foundry)¶
Near-zero new code by design: both are base URL + API key against the existing OpenAI-compatible provider.
- Env plumbing + docs:
PRIMR_GATEWAY=bedrock|foundrystyle profile (or plainLOCAL_LLM_BASE_URL-pattern envs) mapping to the right base URL shape, key env, and model-ID dialect (deployment names on Foundry,anthropic.-prefixed IDs on Bedrock). - Doctor checks: reachability + a one-token probe per configured gateway.
- Honest docs on gateway gaps: no Anthropic web_search through either (those recipes run DDG-only browse or pair a direct key for search); batch lag on newest models; Bedrock Claude-via-ChatCompletions must be verified empirically per AWS's self-contradictory docs.
Phase D: the $0 local profile (with 2.0 backend freedom)¶
--inference localexecution profile per the backend-freedom design: local models for every stage with a local equivalent; stages with none (Deep Research) are skipped with clear logging.- Fit-check before use (principle 5), generalizing what the
calibration local judge needs today: model installed (from
/v1/models), memory headroom (one-token probe; on the Ollama out-of-memory error, step down to the next-smaller installed candidate), effective context window adequate for the stage. - Context guard for corpus stages (refuse silent truncation).
- Role mapping defaults from the June-2026 research (MoE-first), but ALWAYS selected from what is actually installed; nothing hardcoded.
- Eval the local recipe with the step-1 instruments and publish the honest quality delta next to the $0 API price tag. Local mode is allowed to be slower or weaker, but it must never silently upgrade itself to paid cloud; any paid fallback is a separate operator choice. Re-run the local eval when new local model generations or desktop-class AI hardware materially change the capability envelope, and let passing stages move up the default route.
- For a single RTX 4090 or comparable 24 GB GPU, run the focused
4090-report-racestage eval before the broader4090-top10sweep. The first question is pragmatic: does the user's local box already clear the local-vs-sub-dollar bar for a production-adjacent stage?
Sequencing vs the Version Plan¶
Phase A remains 1.x-compatible incremental work. Registry refresh and the Responses transport are shipped; capability flags, provider-native search routing, recipe estimates, and eval promotion remain. Phase B's typed runner and first gated Codex transport are shipped experiments and do not block direct API recipes. Phase C is config/docs heavy and can land after A. Phase D is the backend-freedom pillar (2.0) and depends on the shipped evaluation instruments plus per-stage routing requirements.
Validation protocol (cost-disciplined)¶
- Everything testable offline stays offline: registry flags, sampling guards, dispatch seams, fit-check logic all get injected-seam unit tests.
- One cheap live run per new recipe, ever, before it is advertised (precedent: the v1.24.0 cross-provider eval).
- Local recipes validated with agreement-style checks against a cloud
reference (the calibration
--judge-comparepattern) before being trusted for judge/utility roles, and with the eval scorecard for writing roles.