Numbers only when they are measured.
We publish a result with its method, dataset, baseline, repeat count and limits. Until then, this page shows the standard we hold ourselves to, not a claim.
How we measure
- Task success
- All mandatory business conditions pass. Severe failures are reported separately.
- Permission compliance
- Forbidden accesses or actions that were attempted, blocked or completed.
- Consistency
- Repeated trials per task, so one success is not mistaken for reliable behaviour.
- Recovery
- Success after a controlled timeout, stale record, tool error or checkpoint restart.
- Human burden
- Review time, interventions, escalations and rework.
- Economics
- Total operating expense divided by accepted outcomes, including failed runs.
- Latency
- Completion-time distribution under a stated load, with p50 and p95 where meaningful.
What zero failures does and does not show
With zero failures in 300 independent, representative trials, the exact one-sided 95% upper bound on the failure rate is about 1%. This is arithmetic on a sample size, not a measured result on any client workflow.
Limit: Real tasks may be correlated or unrepresentative. The correct statement is "zero observed failures on this suite under these conditions", not "safe".
How to read a Chainfore number
Every figure we publish carries five fields: method, dataset or task distribution, baseline, repeat count, and limits. If a figure you see from us is missing one, tell us; it should not have been published.
Questions buyers ask
What does Chainfore measure for an agent in production?
Seven measures: task success, permission compliance, consistency, recovery, human burden, economics and latency.
What does zero failures in 300 trials show?
The exact one-sided 95% upper bound on the failure rate is about 1%. It is arithmetic on a sample size, not a measured result on any client workflow; the correct statement is "zero observed failures on this suite under these conditions".
What must every published Chainfore figure include?
Its method, dataset or task distribution, baseline, repeat count and limits.
Work with the programme
If you have a workflow and a question, write to research@chainfore.com.
Research collaboration →