How To Use AI TruthCheck™
Upload an AI-generated report and a primary research report that AITruthCheck™ will use to score the answers to every research question for Accuracy, Completeness, and Thoroughness, treating your primary research as the verified reference standard.
AI TruthCheck™ benchmarks an AI-generated competitive intelligence report against a primary research report whose research content is treated as the definitional reference standard — 100% accurate and complete by design — and every AI answer is scored against it for Accuracy (semantic alignment, not keyword matching) and Completeness (full coverage of what the primary research found), then rolled up into a Thoroughness score and a plain-English confidence classification.
From Two Reports To A Scored Comparison
AI TruthCheck™ needs two things to run: an AI-generated report and a primary research report the AI report will be scored against. Follow this sequence to go from upload to a scored, decision-ready comparison.
Upload both reports together
AITruthCheck™ requires both an AI-generated report and a primary research report in the same submission. Generate the AI report using AI As Analyst (or supply your own AI report— see the formatting section in How To Use AI As Analyst), and pair it with a primary research report that contains insights into the AI report Scope of Work questions.
AI report format is validated automatically
AITruthCheck™ automatically performs a formatting validation scan to ensure your AI report meets the required structure (see the formatting section of How To Use AI As Analyst) for details on AI report formatting requirements), and provides a specific, itemized list of any syntax that requires modification.
Primary report format is checked for Q&A format
AITruthCheck™ checks whether your primary research report is already structured in Q&A format. If it's a free-form narrative report instead, you'll be shown how many questions were matched and told that AITruthCheck™ will attempt to extract answers from narrative text but results may be less comprehensive than with a Q&A-formatted report (see the section "Q&A-Formatted vs. Free-Flow Primary Research" below) and be asked to confirm whether its "OK To Proceed With This Primary Report". From there you can choose to proceed or go back to the primary report and edit it to conform to Q&A format.
Review your cost estimate and confirm
AI TruthCheck™ displays the cost of performing the AITruthCheck™ analysis and asks you to confirm if you want to proceed.
Every question is scored against your primary research as the reference standard
Your primary research is treated as definitionally 100% accurate and complete for comparison purposes. Each AI answer is scored for Accuracy (does it capture the same meaning and intent, not just the same keywords) and Completeness (does it cover every dimension the primary research covered) — writing style and verbosity are never penalized.
Receive your scored comparison report
For every question you'll get: Accuracy %, Completeness %, a combined Thoroughness %, a plain-English confidence band classification, page references into both the AI report and the primary report, an Advisory Note, and a short narrative impression of the AI Outline for that question.
What's In Your Comparison Report
Every question is scored the same way, so results are consistent across sections and across reports. Here's what each column tells you.
| Column | What It Tells You |
|---|---|
| Accuracy % | How closely the AI's answer matches the meaning and intent of the primary research — semantic alignment, not keyword overlap. |
| Completeness % | How fully the AI's answer covers every dimension the primary research addressed. Anything present in the primary research but missing from the AI answer counts against this score. |
| Thoroughness % | A combined decision-readiness score derived from Accuracy and Completeness together. |
| Band Classification | A plain-English confidence label for the answer, applied per our governed scoring rubric — a fast way to gauge decision-readiness without reading every percentage. |
| Page References | The exact page in the AI report and the exact page in the primary report where each answer can be found — so nothing is a black-box score. |
| Advisory Note | A short, specific note on what's missing, vague, or hedged in the AI's answer relative to the primary research. |
| Impressions of AI Outline | A brief narrative assessment of how the AI performed on that specific question. |
Benefits of Using AI TruthCheck™
- Scores meaning, not wording. Accuracy is measured on semantic alignment with your primary research, not keyword overlap — an AI answer that says the same thing in different words still scores well; writing style and verbosity are never penalized.
- Your Scope of Work is extracted automatically. There's no separate SOW upload to keep in sync — AI TruthCheck™ reconstructs the exact question set directly from your validated AI report, so the benchmark is always aligned to what was actually asked.
- Free-form primary research still works. If your primary report isn't already structured in Q&A format, the AITruthCheck™ analysis will still work — you're not required to hand-format it yourself before benchmarking.
- Converts a vague sense of "AI seems okay" into a numeric, auditable Accuracy / Completeness / Thoroughness score for every single question.
Two Files, Every Time
AITruthCheck™ always requires both files together — there's no mode that scores an AI report on its own. If you don't yet have a primary research report to benchmark against, that's a separate step to complete first.
- An AI-generated report — from AI As Analyst, or your own AI tool (see formatting requirements below)
- A primary research report believed to contain accurate answers to relevant SOW questions. A structured Q&A format is ideal, but a free-form narrative report is also accepted
Q&A-Formatted vs. Free-Flow Primary Research
Your primary research report can arrive in either format — AI TruthCheck™ accepts both. Here's what each one actually is, and how it's processed differently.
| Q&A-Formatted | Free-Flow Narrative |
|---|---|
| Each SOW question appears as its own marker, immediately followed by a human analyst's answer to that specific question. | Findings may be written as ordinary prose or report sections — with no requirement for explicit question markers. |
| Matched paragraphs are directly assigned to their question — a fast, deterministic match, not an interpretive one. | Every paragraph is evaluated by AITruthCheck™ to assess the extent to which its content is relevant to an SOW question. In this free-flow format, the judgement of a human analyst is replaced with AITruthCheck's™ relevance assessment |
| Best for analysts following a structured template question-by-question. | Best for narrative-style deliverables, interview writeups, or reports not originally built against your SOW's exact structure. |
Ready to see how your AI-generated intelligence actually holds up? Launch AI TruthCheck™ now and run your first comparison.
