validate.science
Claim-Level Epistemic Risk Assessment
Epistemic Risk Assessment Report
Coffee Consumption Causes Improved Cognitive Function: A Cross-Sectional Study
Generated: 2/2/2026, 8:48:33 PM
Introduction
This report provides a claim-level epistemic risk assessment of the analyzed scientific document.
Each claim extracted from the document has been evaluated against the evidence presented to identify potential instances of overreach—where claims may exceed what the evidence actually supports.
The assessment focuses on three primary failure modes: causal claims from correlational evidence, overgeneralization beyond sample scope, and underpowered claims from small samples.
Executive Summary
Risk Distribution
● High: 1
● Medium: 0
● Low: 1
All Claims
| # | Claim | Risk Level | Score | Failure Modes |
|---|
| 1 | Participants who reported drinking 3 or more cups of coffee daily showed 15% higher scores on standardized cognitive assessments compared to non-coffee drinkers. | low | 30% | None |
| 2 | The 15% improvement in cognitive scores demonstrates that coffee consumption leads to better brain function. | high | 85% | Causal from Correlation |
Flagged Claims Details
1. The 15% improvement in cognitive scores demonstrates that coffee consumption leads to better brain function.
Risk Score: 85%
Failure Modes: Causal from Correlation
Evidence:
We conducted a cross-sectional survey of 45 participants aged 25-55 years.
N=45
Evidence:
This 15% difference was statistically significant (p=0.03, N=45).
N=45
p=0.03
Explanation:
This claim makes a causal assertion ("leads to") based on cross-sectional observational data that can only show correlation. The study design cannot establish that coffee causes cognitive improvement - the correlation could be due to confounding factors (e.g., higher education levels, healthier lifestyles) or reverse causation (people with better cognitive function may prefer coffee).
Evidence Extracted
The following 2 statistical evidence items were extracted from the document:
1
We conducted a cross-sectional survey of 45 participants aged 25-55 years.
N=45
cross-sectional
2
This 15% difference was statistically significant (p=0.03, N=45).
N=45
p=0.03
Appendix: Methodology
How This Report Was Generated
1
Document Processing
PDF text extracted with section boundaries preserved.
2
Claim Extraction
Atomic, testable claims identified using large language model analysis.
3
Claim Classification
Each claim classified by type, strength language, and population scope.
4
Evidence Extraction
Statistical evidence extracted including sample sizes and p-values.
5
Claim-Evidence Matching
Semantic similarity used to match claims to their supporting evidence.
6
Burden-of-Proof Check
Deterministic rules applied to detect epistemic overreach.
7
Risk Scoring
Epistemic risk score computed based on failure modes.
Failure Mode Definitions
| Causal from Correlation | Claim asserts causation, but evidence is correlational/observational. |
| Overgeneralization | Claim makes broad assertions from a narrow or small sample. |
| Underpowered | Claim makes strong assertions with inadequate sample size. |
| Insufficient Evidence | No matching evidence found to evaluate this claim. |
