AI Assurance & Evaluation
AI claims need evidence.
HARIS develops methods and technical infrastructure for testing AI capability claims against evidence and stating what the results do and do not support.
This work is for organisations whose decisions depend on whether a capability claim withstands scrutiny.
1Evidence before reliance
A capability claim extends only as far as the evidence behind it.
AI performance can vary materially with the task, data, language, operating environment and system configuration. Evidence from one setting does not automatically establish capability in another.
Figure 1. The evidence envelope. Illustrative — not derived from a specific engagement.
Any conclusion issued by HARIS must be traceable to the system examined, the evidence relied upon and the conditions of evaluation.
“Supports Arabic” is a claim.
What it means in practice —
- for which uses,
- in which forms of the language,
- under which conditions —
has to be demonstrated.
The same applies whenever a system is expected to work beyond the conditions in which it was built and tested.
Figure 2. A claim, the conditions under which it has to be demonstrated, and the demand that follows.
Those limits apply to HARIS first.
2 Independent scrutiny
The people who judge an AI system shouldn’t be the people who built or sold it.
When a capability claim comes from the organisation that built or supplies the system, independent scrutiny can provide a separate basis for judging whether that claim is supported.
For assurance intended for external reliance, independence must be demonstrable in the way the work is structured and governed — not merely asserted.
HARIS applies the same standard to its own assurance work.
01
Fees are not contingent on the result.
02
The commissioning party may dispute facts and identify errors for correction, but has no editorial veto over findings.
03
Before HARIS accepts assurance intended for external reliance, it assesses relevant conflicts and prior involvement. Where HARIS cannot demonstrate the required independence, it does not take the assurance role.
04
Where the evidence does not support the capability claim under examination, HARIS says so.
Our work
Technical evaluation
Assurance
Evaluation infrastructure
HARIS does not issue certificates or seals. A HARIS
conclusion does not authorise deployment or
procurement, and does not replace the institution’s
own decision.
How an assurance engagement is run
HARIS assesses relevant conflicts and prior involvement before accepting the engagement and before assigning people to the assurance work.
The claim and evaluation boundary are defined.
For confirmatory work, the basis is recorded before results are assessed.
Evidence is gathered and the agreed tests are run.
Before a report intended for external reliance is issued, a reviewer who did not carry out the evaluation or prepare the proposed findings challenges the evidence, methods, findings and stated limits.
The report states what the evidence does and does not support.
The institution retains responsibility for its own deployment, procurement and operational decisions.
criteria, reference material and test conditions
a reviewer who did not carry out the evaluation
what the evidence does and does not support; scope, uncertainty and limits stated
its own deployment, procurement and operational decisions
Figure 3. The structure of an assurance engagement. Each layer is bounded by the one outside it; the conclusion cannot exceed the evidence.
For institutions
Each evaluation begins with a defined question and stated scope. For confirmatory work, the evaluation basis, including criteria, reference material and test conditions, is established before results are assessed. HARIS is responsible for the integrity and stated boundaries of any conclusion issued in its name. Each engagement concludes in a written report: what was examined, under what conditions, what the results do and do not support, and the limits of the conclusion.
Universities, research groups, engineers and specialist practitioners may contribute where their expertise is required. Formal collaborations are set out in written agreements covering responsibilities and any relevant intellectual property, confidentiality and publication arrangements.
Institutional engagements involving confidential material are governed by written confidentiality terms, with data-handling and security arrangements agreed before that material is shared. HARIS does not identify a confidential engagement publicly without the organisation’s agreement.
Where access to a system or its evidence is restricted, the conclusion is narrowed to what could be examined rather than widened by assumption. Adverse and contradictory evidence stays in the report; it is not removed.
- What was examined
- Under what conditions
- What the results do and do not support
- The limits of the conclusion
About HARIS
HARIS Group Ltd evaluates AI capability claims for organisations that must decide whether to rely on those claims, and may commission applied research and technology development with universities and specialist partners.
HARIS takes its name from the Arabic word for one who keeps watch. The name reflects the task: examining what a system does, rather than what is claimed for it.