Rate each criterion from 0 to 5, cap that rating by the strength of its evidence, and multiply by its weight.
Weighted points = criterion weight × evidence-capped rating ÷ 5
Overall score = sum of the ten weighted-point results
For example, a rating of 4 on the 20-point economics criterion contributes 16 points.
What does each rating mean?
| Rating | Interpretation |
|---|---|
| 0 | Absent, fundamentally unacceptable or unsupported by any usable evidence |
| 1 | Material weaknesses; largely assertions or an incomplete operating approach |
| 2 | Some credible capability, but important requirements or tests remain unmet |
| 3 | Meets the buyer’s pre-agreed baseline requirements in the tested scope |
| 4 | Meets baseline and the pre-agreed stronger-performance or resilience tests |
| 5 | Strong performance sustained across repeated comparable operating periods, with effective monitoring and corrective action |
Use the criterion-specific anchors below. Award 2 when there is more than an assertion but the requirements for 3 are not met; award 4 only when all requirements for 3 and the agreed additional tests are met. Do not average a serious weakness away inside a criterion.
How should evidence limit the rating?
| Evidence available | Maximum rating |
|---|---|
| No usable evidence | 0 |
| Sales assertion, testimonial or unsupported questionnaire response | 1 |
| Credible documents, designs or isolated demonstrations, but no representative buyer verification | 2 |
| Buyer-verifiable representative testing, relevant control validation and reproducible records | 4 |
| Repeated operating evidence under the same acceptance standard, including relevant stress and change conditions | 5 |
Use final rating = lower of performance rating and evidence cap. A polished demonstration cannot earn 4 simply because its output looks good.
For an unbuilt plan, report forecast performance separately from evidence-backed readiness. Apply the same rule to a supplier proposal and an internal automation business case. Historical incumbent results count only if they can be reconstructed under the same definitions.
What should the total score trigger?
| Evidence-backed score | Proposed action, after gates pass |
|---|---|
| 80–100 | Eligible for a staged production decision within the tested scope; not automatic approval |
| 65–79.9 | Extend the controlled pilot or limited operation; close named gaps before expansion |
| Below 65 | Redesign, narrow the scope or retain the comparator rather than scaling |
Additionally, require ratings of at least 3 for technical quality, traceability, authority/control and security. Any proposed deployment must also meet its explicit quality and service thresholds, even if the weighted total is high.
These bands are decision rules proposed by this desk, not empirically validated industry cut-offs. Test whether the preferred option changes when disputed ratings move by one point or the committee changes legitimate priorities.