Skip to content

Research Continuity: Compounding Across Company Runs

Status: IN PROGRESS as a supporting capability, not an independent product pillar. Layer 1 storage, company track/list/show profiles, bounded run-pointer storage, and profile-plus-hypotheses export are shipped; clear, automatic pipeline attachment, and the shadow run-ledger history remain planned. The indexed claim/proposition store is eligible for 2.0 only if representative evaluation shows that it improves a later company report. Strategy Delta follows in 2.x after that promotion. ROADMAP anchors: "Cross-Run Research Memory", "Strategy Delta Mode", "Watch / Delta as a Consumed Primitive" (Considered for Later); per-user cache (#12, shipped v1.30.0).

Motivation

Each run starts cold. Researching 50 companies in one industry rediscovers the same context 50 times; re-running a tracked company regenerates what hasn't changed. Continuity is useful only when it makes the next Strategic Overview or YAML-defined strategy document sharper, fresher, cheaper, or faster. A generic memory, claim-graph, or corpus product is outside Primr's boundary.

Immediate next slice

Memory is deliberately after label calibration and job-scoped artifact resources. Prior-run claims should not be able to re-enter a new report as fresh findings unless their provenance and confidence are explicit. The next build slice finishes layer 1 only: automatic pipeline attachment, clear, freshness, retention docs, and body-free pointers/hashes for the shadow run-scoped epistemic ledger. The ledger itself follows the source-receipt and evidence-anchor contract in run-epistemic-ledger.md and does not drive report prose before representative evaluation.

Design: three layers, built strictly in order

Layer 1 - persistent company tracking + run-ledger history (filesystem, no new deps)

Shipped foundation:

  • ResearchMemory() now defaults to <per-user data dir>/research_memory instead of a project-local or logs folder. PRIMR_DATA_DIR relocates that durable state.
  • Memory writes reject secret-like values before YAML is written, reusing the same masking patterns used by persistent logs.
  • primr doctor File Locations reports both the user data root and research memory path.
  • primr company track <name> <url> creates or updates a local JSON profile under <per-user data dir>/company_profiles, rejects URL userinfo and secret-like values, and records retention/classification metadata plus a bounded body-free run-pointer list for later pipeline wiring.
  • primr company list and primr company show <name> provide deterministic local inventory/readback without paid calls or network access.
  • primr company export <name> writes a local JSON + Markdown bundle under the profile's exports/ folder. It includes profile metadata, stored run pointers when present, persisted research-memory hypotheses with confidence tags, and explicit flagged gaps for missing run-history or claim-store data. It is an OKF-shaped precursor, not a conformant OKF v0.2 bundle: the shipped export is a shallow profile/hypothesis Markdown/JSON pair without the required root bundle or full linked claim graph. Conformant OKF export waits for layer 2; see open-knowledge-format.md.
  • Research memory (agentic.memory.ResearchMemory) already tracks hypotheses; layer 1 gives it a stable per-company home and CLI surface
  • Remaining layer-1 work automatically attaches owned run pointers plus the body-free digest and schema metadata for source_index.json, findings.jsonl, and strategic_inferences.jsonl when those shadow artifacts exist. Raw statements and evidence do not enter profile metadata.

Layer 2 - indexed claim/proposition store + priming (2.0 candidate)

  • SQLite (stdlib, no new service) ingests immutable source receipts, evidence-bound finding instances, strategic inferences, verification events, confidence history, typed relations, and observed/valid time directly from run ledgers. It does not re-extract canonical memory from narrative reports.
  • Assertion/finding instances remain immutable and source-specific. A semantic proposition may reconcile many instances without destructively collapsing their provenance or time.
  • Embeddings: local sentence-transformers when available, Gemini embedding API otherwise - chosen per the backend-freedom profile. Embeddings nominate possible proposition matches only; model- or human-adjudicated relations own same-proposition, supports, contradicts, qualifies, supersedes, different-scope, and orthogonal decisions.
  • primr memory search "<query>", primr memory timeline "<Company>"
  • Priming: at run start, query for related companies/industry claims and inject as clearly-provenance-marked context into the analysis stage (fenced as data; prior-run claims are never silently re-asserted as fresh findings - they enter as "prior research suggests, validate:" items)
  • The store's portable form is a lossless OKF v0.2 projection. SQLite remains canonical cross-run memory and OKF remains interchange; see open-knowledge-format.md.

Layer 3 - Strategy Delta Mode (2.x)

  • Given prior run ledgers plus the claim/proposition store, gather fresh signals and reconcile new, strengthened, weakened, contradicted, qualified, superseded, stale, and unchanged records. Every delta row traces to old and new finding/inference IDs plus evidence anchors.
  • The first promotion emits the delta alongside a full refreshed report. Only after reconciliation is human-adjudicated may the section evidence-usage graph drive selective regeneration. This prevents citation renumbering or a missing dependency from masquerading as strategic change.
  • Stays inside the single-job model: a delta run is one job, not a daemon. The watch/poll loop that triggers delta runs lives on the consumer side (ROADMAP: "Watch / Delta as a Consumed Primitive")
  • Change detection trusts the ledger and its evidence grades, which is why the v1.40 calibration corpus and the ledger's shadow evaluation are prerequisites: a delta gated on uncalibrated claims propagates noise.

Data governance (ships WITH layer 1, not later)

Memory creates a new surface the existing threat model doesn't cover:

  • Locality: all stores under the per-user data dir; nothing leaves the machine except via explicit export
  • Deletion: primr memory clear [--company X] removes claims + profile; documented retention default (keep until cleared)
  • Classification: profile folders are operator data about third parties - the docs say so plainly, and doctor File Locations lists the store
  • No secrets in memory entries (existing rule, enforced at the write seam with the same scanner used for research memory)

Validation cost

Layer 1 and the deterministic parts of layer 2 are free to validate with schema, idempotency, property, retention, deletion, and mocked-priming tests. Semantic reconciliation requires agreement-validated fixtures and any paid judge or report run remains estimate-gated. Layer 3 needs an explicitly approved tracked-company pair to validate reconciliation and cost before any selective-regeneration claim.

Exit criteria

  • For 2.0 promotion: owned run ledgers ingest idempotently; the claim/proposition store answers provenance-bearing search and timeline queries; priming, clear, export, retention, classification, and no-secret behavior work as documented; store -> OKF -> store is lossless for supported fields; and blinded representative evaluation shows that priming improves a later company report without freshness, uncertainty, privacy, or cost regression.
  • Invariant pinned by tests: priming may inform analysis but a prior-run claim can never appear in a new report without fresh attribution
  • Strategy Delta and selective regeneration are not required for the 2.0 release. They have separate 2.x acceptance criteria.

Explicitly not

  • No daemon, no scheduler, no push notifications - consumers poll/trigger
  • No cloud sync of memory; export is the only egress
  • No cross-user/multi-tenant store (single-operator tool)
  • No semantic identity from hashes or embeddings alone
  • No report prose as canonical memory