Skip to content

Engineering evidence levels: claimed, observed, and verified

Learn how claimed, observed, and verified evidence differ, what each level establishes, and why an evidence label must reflect its source.

Engineering evidence is useful only when its label matches how it was produced. A person saying a task is complete, a connector finding a commit, and an independent check confirming a result provide different confidence. Keeping them distinct lets teams see what is known, where it came from, and what remains to be checked.

L0 Claimed: an actor reports what happened

Claimed evidence records a statement from a person or agent, such as "implementation complete" or "test run passed." It helps report progress and intent. By itself, it does not establish that a commit exists or a test ran. Keep the source visible so readers do not mistake an assertion for an independent observation.

L1 Observed: a trusted source reads an artifact

Observed evidence comes from a connector that reads an artifact directly, such as a repository event or check result. It establishes that the connected source reported that artifact; it does not necessarily show that the artifact satisfies a requirement. Retain its source and reference so a person can inspect the underlying event.

L3 Verified: an independent system checks an outcome

Verified evidence is produced when an independent system checks a defined outcome. A test can verify the cases represented by its assertions. The result is only as broad as the check: a passing test does not prove every behavior, deployment condition, or requirement. Record the check, its result, and the expectation it was meant to assess.

Preserve provenance and trust boundaries

  • Store the source, time, and artifact reference with each evidence record.
  • Do not let the actor being assessed promote its own claim into independent evidence.
  • Keep observations and checks scoped to the workspace and system that produced them.
  • Treat missing evidence as a question for review, not permission to mutate external systems.

Use evidence to guide review

Evidence levels do not replace engineering judgment. They help a team ask who reported this, which system observed it, what a check covered, and whether the result matches the requirement. When a work item lacks evidence its policy expects, make that gap visible and assign follow-up work.

This post was last updated on .

Frequently asked questions

What is the difference between claimed and observed evidence?

Claimed evidence is a statement from a person or agent. Observed evidence is an artifact read directly by a trusted connector, with its source retained.

What does verified evidence mean?

Verified evidence means an independent system checked a defined outcome. It only supports the behavior and conditions covered by that check.

Can an AI agent mark its own work as verified in OROTOV?

No. Human and agent sources are limited to L0 Claimed evidence. Higher evidence levels require trusted connector or reconciler paths.

Does OROTOV implement every evidence level?

No. The current public workflow uses L0 Claimed, L1 Observed, and L3 Verified evidence. L2 and L4 are not implemented in this release.