State Machine Specifications¶
This document provides state machine specifications for Primr's tier escalation and job lifecycle management.
Overview¶
Primr uses formal state machines to manage: 1. Tier Escalation: The process of trying progressively more capable scraping methods 2. Job Lifecycle: The states a research job goes through from creation to completion
These state machines provide: - Clear definition of valid states and transitions - Transition guards for conditional logic - State invariants for correctness guarantees - Event emission for monitoring and debugging
Tier Escalation State Machine¶
States¶
| State | Description |
|---|---|
IDLE |
No scraping in progress |
ATTEMPTING |
Currently trying a scraping tier |
ESCALATING |
Current tier failed, preparing to try next |
SUCCEEDED |
Scraping completed successfully |
FAILED |
All tiers exhausted without success |
BLOCKED |
Hard block detected (e.g., CAPTCHA, IP ban) |
State Diagram¶
stateDiagram-v2
[*] --> IDLE
IDLE --> ATTEMPTING : start_scrape
ATTEMPTING --> SUCCEEDED : scrape_success
ATTEMPTING --> ESCALATING : soft_block
ATTEMPTING --> FAILED : all_tiers_exhausted
ATTEMPTING --> BLOCKED : hard_block
ESCALATING --> ATTEMPTING : try_next_tier
ESCALATING --> FAILED : no_more_tiers
SUCCEEDED --> IDLE : reset
FAILED --> IDLE : reset
BLOCKED --> IDLE : reset
Transitions¶
| From | To | Trigger | Description |
|---|---|---|---|
| IDLE | ATTEMPTING | start_scrape |
Begin scraping with first tier |
| ATTEMPTING | SUCCEEDED | scrape_success |
Content extracted successfully |
| ATTEMPTING | ESCALATING | soft_block |
Soft block detected (WAF, rate limit) |
| ATTEMPTING | FAILED | all_tiers_exhausted |
No more tiers to try |
| ATTEMPTING | BLOCKED | hard_block |
Hard block detected (CAPTCHA, IP ban) |
| ESCALATING | ATTEMPTING | try_next_tier |
Move to next tier |
| ESCALATING | FAILED | no_more_tiers |
No more tiers available |
| SUCCEEDED | IDLE | reset |
Reset for next URL |
| FAILED | IDLE | reset |
Reset for next URL |
| BLOCKED | IDLE | reset |
Reset for next URL |
State Invariants¶
| State | Invariant |
|---|---|
| IDLE | No active scraping operation |
| ATTEMPTING | Current tier is set and valid |
| ESCALATING | Previous tier recorded, next tier available |
| SUCCEEDED | Content is non-empty and valid |
| FAILED | All tiers have been attempted |
| BLOCKED | Block reason is recorded |
Usage Example¶
from primr.utils.state_machine import create_tier_state_machine, TierState
# Create state machine
sm = create_tier_state_machine()
assert sm.state == TierState.IDLE
# Start scraping
sm.transition("start_scrape")
assert sm.state == TierState.ATTEMPTING
# Soft block detected - escalate
sm.transition("soft_block")
assert sm.state == TierState.ESCALATING
# Try next tier
sm.transition("try_next_tier")
assert sm.state == TierState.ATTEMPTING
# Success!
sm.transition("scrape_success")
assert sm.state == TierState.SUCCEEDED
# Reset for next URL
sm.transition("reset")
assert sm.state == TierState.IDLE
Job Lifecycle State Machine¶
States¶
| State | Description |
|---|---|
PENDING |
Job created, waiting to start |
RUNNING |
Job is actively executing |
PAUSED |
Job temporarily suspended |
COMPLETED |
Job finished successfully |
FAILED |
Job encountered an error |
CANCELLED |
Job was cancelled by user |
State Diagram¶
stateDiagram-v2
[*] --> PENDING
PENDING --> RUNNING : start
PENDING --> CANCELLED : cancel
RUNNING --> PAUSED : pause
RUNNING --> COMPLETED : complete
RUNNING --> FAILED : fail
RUNNING --> CANCELLED : cancel
PAUSED --> RUNNING : resume
PAUSED --> CANCELLED : cancel
COMPLETED --> [*]
FAILED --> [*]
CANCELLED --> [*]
Transitions¶
| From | To | Trigger | Description |
|---|---|---|---|
| PENDING | RUNNING | start |
Begin job execution |
| PENDING | CANCELLED | cancel |
Cancel before starting |
| RUNNING | PAUSED | pause |
Temporarily suspend |
| RUNNING | COMPLETED | complete |
Finish successfully |
| RUNNING | FAILED | fail |
Error occurred |
| RUNNING | CANCELLED | cancel |
User cancellation |
| PAUSED | RUNNING | resume |
Continue execution |
| PAUSED | CANCELLED | cancel |
Cancel while paused |
Terminal States¶
The following states are terminal (no further transitions):
- COMPLETED
- FAILED
- CANCELLED
State Persistence¶
Job state machines support persistence for crash recovery:
from primr.utils.state_machine import JobStateMachine, JobState
# Create and run job
sm = JobStateMachine("job-123")
sm.transition("start")
# Save state
sm.save("jobs/job-123.json")
# Later, recover state
recovered = JobStateMachine.load("jobs/job-123.json")
assert recovered.state == JobState.RUNNING
assert recovered.job_id == "job-123"
Persistence Format¶
{
"job_id": "job-123",
"state": "running",
"created_at": "2026-02-02T10:30:00",
"history": [
{
"from_state": "pending",
"to_state": "running",
"trigger": "start",
"timestamp": "2026-02-02T10:30:00",
"context": {}
}
]
}
Usage Example¶
from primr.utils.state_machine import create_job_state_machine, JobState
# Create job
sm = create_job_state_machine("research-abc123")
assert sm.state == JobState.PENDING
# Start execution
sm.transition("start")
assert sm.state == JobState.RUNNING
assert sm.is_active
# Pause for rate limiting
sm.transition("pause")
assert sm.state == JobState.PAUSED
# Resume
sm.transition("resume")
assert sm.state == JobState.RUNNING
# Complete successfully
sm.transition("complete")
assert sm.state == JobState.COMPLETED
assert sm.is_terminal
Event Listeners¶
Both state machines support event listeners for monitoring:
from primr.utils.state_machine import create_job_state_machine, StateChangeEvent
def on_state_change(event: StateChangeEvent):
print(f"Job transitioned: {event.from_state} -> {event.to_state}")
print(f"Trigger: {event.trigger}")
print(f"Time: {event.timestamp}")
sm = create_job_state_machine("job-123")
sm.add_listener(on_state_change)
sm.transition("start")
# Output:
# Job transitioned: JobState.PENDING -> JobState.RUNNING
# Trigger: start
# Time: 2026-02-02 10:30:00
Error Handling¶
Invalid transitions raise InvalidTransitionError:
from primr.utils.state_machine import (
create_job_state_machine,
InvalidTransitionError,
)
sm = create_job_state_machine("job-123")
try:
sm.transition("complete") # Can't complete from PENDING
except InvalidTransitionError as e:
print(f"Invalid: {e.from_state} -> {e.to_state} via '{e.trigger}'")
# Output: Invalid: JobState.PENDING -> None via 'complete'
Integration with Telemetry¶
State transitions can be traced using the telemetry system:
from primr.utils.state_machine import create_job_state_machine, StateChangeEvent
from primr.utils.telemetry import TelemetrySystem
telemetry = TelemetrySystem()
def trace_state_change(event: StateChangeEvent):
telemetry.record_event(
"state_transition",
{
"from_state": event.from_state.value,
"to_state": event.to_state.value,
"trigger": event.trigger,
}
)
sm = create_job_state_machine("job-123")
sm.add_listener(trace_state_change)
Version History¶
| Version | Date | Changes |
|---|---|---|
| 1.0 | Feb 2026 | Initial specification |