Model Onboarding¶
How to add a new model to primr - Grok, Gemini, Claude, OpenAI, or any future provider - without breaking the cost estimator, the pipeline circuit breakers, or the eval harness.
The model data source of truth is src/primr/config/model_registry.py.
src/primr/config/models.py is the stable selection and cost facade and
re-exports registry symbols for compatibility. Pricing, fallback chains,
dry-run estimates, eval profiles, and doctor checks consume those two layers.
This playbook is intentionally short. Each step is a checkbox you should be able to tick off without guessing. The worked example at the bottom is the April 2026 Grok 4.3 onboarding.
The five-step process¶
1. Verify identity against official provider sources¶
Do not trust blog posts, search snippets, or third-party pricing pages. Start with the provider's current model page, pricing page, and API reference. Record the exact canonical model ID, aliases, context and output limits, supported API shape, reasoning controls, tool support, storage defaults, and all token, cache, long-context, and tool-call prices.
An authenticated /models request is an optional availability check. It proves
what the current account can see, not the model's capabilities or pricing. It
must remain an auth-only preflight and must not generate tokens.
For xAI:
For Google:
For OpenAI:
Capture: model ID string, context window, max output tokens, multimodal support, whether reasoning is always-on, and whether tiered or cached pricing is published. If pricing isn't surfaced via the API, take it from the provider's official docs page - never from third-party comparison sites.
2. Register the model in ModelRegistry¶
Add a ModelConfig entry alongside the existing siblings in
src/primr/config/model_registry.py. Required fields:
name- exact API ID returned by step 1display_name- human-friendly label used in dry-run outputprovider-xai/google/anthropic/openaicost_per_1m_input_tokens,cost_per_1m_output_tokens- base pricingmax_input_tokens,max_output_tokenssupports_thinking,supports_tools,supports_multimodal
Optional fields (use them when the provider publishes them - leave None
otherwise):
cost_per_1m_input_tokens_high,cost_per_1m_output_tokens_high,tier_threshold_tokens- tiered pricing above a context thresholdcost_per_1m_input_tokens_cached- discount rate for prompt-cache hitsprice_change_date,cost_per_1m_input_tokens_after,cost_per_1m_output_tokens_after, andcost_per_1m_input_tokens_cached_after- a provider-published future price change that must take effect by pricing date without a code release
The estimator consumes these fields through
PrimrModels.calculate_cost_breakdown(). Keep the registry explicit: if a
provider publishes long-context, cache-read, or cache-write economics, record
the read and tier metadata before using the model in a default route. Do not
assume prompt-cache savings in dry-run output unless historical usage records
show cached input tokens for that mode.
Add the new entry to PrimrModels.ALL_MODELS in
src/primr/config/models.py so
get_model_config, get_price, and calculate_cost can find it.
If the new model is meant to replace an existing default, add a new
constant (e.g. GROK_MODEL_43) but keep the old constant
(GROK_MODEL_420) as a back-compat alias pointing at the legacy registration.
Don't delete the old registration - in-flight runs and resumed jobs read the
old model ID from _run_state.json.
3. Wire the model into the pipeline¶
Find every place the old model name is referenced and decide whether the new model should replace it. The three routing surfaces are:
PrimrModels.get_grok_models(tier)- returns(reasoning_model, writing_model)forGrokTier.FAST/HYBRID/MAX. Update this if the new model belongs in one of those tiers.src/primr/pipeline/model_breaker.py::ANALYSIS_FALLBACK_CHAIN(andPREMIUM_FALLBACK_CHAIN) - ordered tuples used by the model circuit breaker. Newer flagships go at the front, older flagships drop to the next position.- Utility-tier audit -
src/primr/ai/llm.py::_get_model_for_typeresolves utility-tier calls (scraping summaries, link selection, generic "fast" tasks) based on which provider key is set. When you onboard a new flagship for the analysis tier, ask whether the new provider also has a cheap-and-cheerful SKU that could replace the utility-tier model from a different provider. A standard pipeline that requires keys from multiple providers is fragile: a hang on the cheap utility call stalls the whole run even when the flagship is healthy. The April 2026 Grok 4.3 onboarding surfaced exactly this - a stalled Gemini Flash link-selection call hung the rerun until utility-tier dispatch was rewired to the then-current Grok 4.1 NR. The current utility primary is the dated Grok 4.20 non-reasoning model, so treat the 4.1 example as onboarding history rather than live routing guidance.
Also grep for the legacy model ID across the codebase:
Decide for each hit whether it should switch. For test fixtures that just need a registered model, leave them alone - they're testing breaker logic, not naming conventions.
4. Add validation tests¶
In tests/test_utils/test_cost_estimator.py, add (at minimum):
- A pricing assertion:
ModelRegistry.NEW_MODEL.cost_per_1m_input_tokens == X - An always-on / reasoning assertion if applicable
- A tiered-pricing assertion if the model is tiered
- A cache-discount assertion:
calculate_cost(name, ..., cached_input_tokens=N) < calculate_cost(name, ...)if the model has a published cache rate - A
get_grok_models(tier)assertion for whichever tier the new model lives in - An updated cost-range test for any tier whose price moved
Then run the focused suite:
python -m pytest tests/test_utils/test_cost_estimator.py \
tests/test_pipeline/test_model_breaker.py \
tests/test_ai/ -x --tb=short -q
A model upgrade should never need test changes outside cost / model_breaker / ai tests. If it does, you're probably refactoring something else by accident.
5. Eval-gate the promotion¶
Don't promote a new model to default on vendor-recommendation alone. Run the existing eval harness on the standard 3-5 company corpus and let the scorecard decide.
# Generate reports under three labelled folders
primr "Acme Corp" https://acme.com --grok-tier hybrid --output-dir eval_corpus/hybrid
primr "Acme Corp" https://acme.com --grok-tier max --output-dir eval_corpus/max
# (after wiring the new model, regenerate the same companies)
# Score them
primr eval --eval-root eval_corpus
Decision criteria - write these down before you look at the scorecard:
- Utility-per-dollar must be ≥ the current default
- Hallucination rate (judge overlay) must be ≤ the current default
- Drift markers (
**What to validate:leakage etc.) must not regress
Only flip the default in step 3 (or PrimrModels.GROK_MODEL_43 etc.) once
those criteria pass. If they don't, leave the new model registered but not
default - the eval result is the artifact, the model stays available behind
an explicit flag for users who want to opt in.
For utility-stage backend-freedom work, route promotions should also flow
through ai/stage_routing.py and the declared row in
core/stage_inventory.py. The current runtime pilots are
fast.scrape_summary, fast.source_relevance, and fast.hiring_signals
behind --inference cloud|hybrid; they record capped body-free
stage_routes entries in _run_state.json. Do not add a stage-local provider
dispatch path to test a new model, and do not infer
actual token/cache/cost fields until provider usage seams expose stage-scoped
counters.
Adding a new provider (OpenAI, Ollama, Anthropic, …)¶
The five-step process above covers adding a model under a provider primr
already speaks. Adding a whole new provider is a different shape: it
goes through the provider abstraction in src/primr/ai/providers/. The
work splits into two cases.
Case A: OpenAI SDK-compatible endpoint¶
Primr supports two transport shapes through this family. Direct OpenAI and xAI
use POST /v1/responses; local Ollama, vLLM, llama.cpp, and similar servers use
POST /v1/chat/completions unless that backend has verified Responses support.
The normalized messages input is translated at the adapter boundary.
- Add a
ProviderEntrytoKNOWN_PROVIDERSinsrc/primr/ai/providers/registry.pywith the provider's name, env var, description, and roles. - Add a branch to
build_providerthat constructs anOpenAICompatibleProviderwith the rightbase_url,api_key_env, and explicitapi_style. Useresponsesonly after the provider is verified to implement its request and response contract. For Ollama, set anapi_key_defaultplaceholder so the SDK does not reject an unset key. - Add model entries for each model you want to call in
src/primr/config/models.py(ModelRegistry). Setprovider="openai"/"ollama"/etc. and add the entry toALL_MODELS. - Add a routing branch in
primr.ai.routing.get_provider_for_modelthat maps the new provider name to the right singleton accessor. - Decide role policy in
pick_model_for_role- should the utility tier prefer the new provider over Grok/Gemini when its key is set? This is where multi-provider preference order lives.
Case B: Different SDK shape (Anthropic Claude)¶
Anthropic's Messages API uses different field names (system is
top-level, cache_control blocks, etc.). It needs its own provider class.
- Write
src/primr/ai/providers/anthropic.pywith anAnthropicProvider(Provider)class that translates the OpenAI-shapemessageslist into Anthropic's format and calls the SDK. - Implement
chat()to return aChatResponse. Pass through provider-specific kwargs (cache_control,extended_thinking_budget) via**provider_kwargs. Don't try to flatten them into a generic shape - the abstraction is intentionally permissive. - Raise
QuotaExhaustedfor daily-limit errors so callers can add UI without reaching into provider internals. - Wire into the registry the same way as Case A: a
ProviderEntry, abuild_providerbranch, and a routing branch. - Add a test file alongside
test_providers.pycovering message translation, retry behavior, and quota classification.
What the abstraction normalizes vs. doesn't¶
Normalized (every provider returns the same shape, supports the same interface):
chat(messages, *, model, temperature, max_tokens, retries, **kwargs)- Returns
ChatResponsewith text, input/output tokens, cached input, and any available exact cost or response-status metadata - Retry/backoff on transient errors
QuotaExhaustedfor daily quotais_available()reports whether the env key is set- Per-model usage accounting via
get_usage_by_model()
Not normalized - passed through as **provider_kwargs, providers ignore
what they don't recognize:
- Gemini
thinking_levelvalues supported by the selected model - Anthropic
cache_controlblocks,extended_thinking_budget - OpenAI/xAI
reasoning_effort, tools, storage policy, and normalizedresponse_format(translated to Responsestext.formaton the wire) - Chat Completions backends:
top_p,seed,stop, and related controls
This is deliberate: a lowest-common-denominator API would lose features that matter (Gemini's deep thinking, Anthropic's cache control). Callers that need a provider-specific feature pass the kwarg; callers that don't care get a clean, uniform interface.
Request compatibility is still model-specific. Every selectable Gemini path
must use the shared sampling-capability policy so 3.5 Flash-Lite, 3.6 Flash,
3.7 Flash, and later families do not receive removed temperature, top_p,
or top_k fields. Convert only model-supported thinking levels through the
SDK enum and test direct clients as well as the provider adapter before adding
an eval profile. Anthropic requests omit those three fields for every model.
SDK 2 sends API version 2026-10-21, and that version rejects them.
Doc updates¶
When the default changes, update these in the same PR so the docs stay coherent with the code:
README.md- modes table cost figures, headline cost in the ledeAGENTS.md- Quick Start cost line;clients/primr-operator.md- agent-facing cost guidanceROADMAP.md- strike through the "evaluate new model" entry, add aChangeloglinedocs/ARCHITECTURE.md- model-pricing table if present
Worked example: Grok 4.3 (April 2026)¶
xAI released grok-4.3 on 2026-04-30 with configurable reasoning effort,
$1.25/$2.50 pricing, $0.20 cached input, 1M context, and multimodal input.
Step 1 - verify: xAI's official Grok 4.3 model page and pricing table confirm the API ID, reasoning-effort controls, context, and current pricing. Model availability in a specific account remains a credential-scoped preflight fact.
Step 2 - register: Added GROK_4_3 = ModelConfig(...) in
src/primr/config/model_registry.py, including the published $0.20 cached-input
rate and inclusive >=200k prices of $2.50 input, $0.40 cached input, and $5.00
output per million tokens. Added it to ALL_MODELS and introduced
GROK_MODEL_43; GROK_MODEL_420 remains a compatibility constant for the
dated Grok 4.20 reasoning ID.
Step 3 - wire: at that point get_grok_models() mapped FAST and HYBRID to
Grok 4.3 reasoning plus grok-4.20-non-reasoning writing, while MAX mapped to
Grok 4.5 for both roles. FAST used lower reasoning effort. The estimator
labels were Grok 4.3 hybrid and Grok 4.5 max. That wiring is the
historical example. The live map is the current-routing note below.
Step 4 - tests: test_cost_estimator.py got six new assertions
(test_grok_43_pricing, _tiered_pricing, _always_on_reasoning,
_in_all_models, _cache_discount, plus updated tier-routing and
cost-range tests). Existing 4.20 tests rewritten to assert legacy behavior
(still registered, still works) rather than current default.
Step 5 - eval: at the time, the 4.3 default was a vendor recommendation
plus mechanical wiring, and the eval sweep was still open. That sweep is
closed history. Do not revert get_grok_models() to 4.3. The live defaults
are the current-routing note below.
Current routing (2026-11-14): every Grok tier uses grok-4.7 for
reasoning and writing. The analysis fallback chain is grok-4.7, Claude
Sonnet 5.5, GPT-6.1 Terra, then Gemini 3.8 Flash. The Pro default is
gemini-4-carbon. See docs/design/grok-default-routing.md.
Current follow-up:
- The published inclusive >=200k tier is represented in the registry and cost tests.
- Every xAI tier, including writing when Gemini is unavailable, uses
grok-4.7. With Gemini configured, routed Flash-Lite writing remainsgemini-3.5-flash-lite. Gemini Pro usesgemini-4-carbon. - The Responses adapters normalize cached-input usage. xAI's
cost_in_usd_ticksis the exact-cost authority when present; registry token pricing remains the conservative fallback.