Extracting Claims

/paper-rag:extract-claims reads an ingested paper, writes down what the paper found as short standalone claims, checks each one against the source text, and stores them beside the paper's chunks. Claims are what /paper-rag:ask searches first and what the dashboard's Claims view lists. This guide covers what a claim is, the path from chunks to stored claims, how to run it, and what to do when a claim is rejected.

What is a claim?

A claim is one self-contained, atomic assertion that a paper itself makes or reports: a finding, an effect size, a comparison, a method result, or a stated conclusion. Most come from the Results and Discussion, plus the abstract.

A good claim is:

• Standalone. It names the population, exposure or intervention, comparator and outcome explicitly (no "it", "this", "the drug") and includes the numbers the source gives: effect size, confidence interval, p, n.
• Faithful. It says only what the paper states and keeps the paper's hedging ("associated with", "suggests"). No outside knowledge.
• Grounded. It lists the chunk ids that support it (source_chunk_ids) and a verbatim evidence_span quoted from one of them.

What is not a claim: background and citations of other work, methods without results, and boilerplate. This is a rule in the extraction prompt, not something the code enforces, so a few background statements can still get through. Expect roughly 5 to 25 claims per paper.

An example:

{
  "text": "Autistic adults showed increased frontal cortical thickness versus controls (Cohen's d = 0.41).",
  "source_chunk_ids": ["31978945::text::4"],
  "evidence_span": "Autistic adults showed increased cortical thickness in frontal regions (Cohen's d = 0.41, 95% CI 0.15-0.67).",
  "section": "Results",
  "population": "autistic adults",
  "comparator": "non-autistic controls",
  "outcome": "frontal cortical thickness",
  "direction": "increase",
  "effect_value": "0.41",
  "effect_measure": "Cohen's d",
  "uncertainty_interval": "95% CI 0.15-0.67"
}

Claims do not depend on any question. They are extracted once per paper, from whatever abstract and full text is available, before anyone asks anything. Later, when you ask a question, /paper-rag:ask searches those stored claims for the ones that bear on it, so a claim can be used to answer questions the extraction never knew about.

A claim is a pointer to the right paper and passage. When you ask a question, the answer is written from the source chunks the claim cites, not from the claim's own wording.

The path from chunks to stored claims

Only the Claude steps are judgement; the guards are plain string and number checks. A rejected batch writes nothing, and an accepted one replaces the paper's previous claims.

1. Read. claims.py chunks <doc_key> prints the paper's abstract and text chunks with their ids. Figure chunks are not sources.
2. Extract. Claude writes the claims, each with its source_chunk_ids, a verbatim evidence_span, and the structured fields the chunk states.
3. Verify. Claude re-reads only each claim's cited chunks and judges whether they say it: population, direction, magnitude and hedging must all match. A partly supported claim is corrected and an unsupported one is dropped.
4. Check. claims.py add runs the guards on every claim before writing anything.
5. Store. The paper's existing claims are deleted, the new ones are written as claim rows, and the BM25 index is rebuilt.
6. Use. Claims are searched by /paper-rag:ask and listed in the dashboard.

Running it

/paper-rag:extract-claims <doc_key|pmid>
/paper-rag:extract-claims --all-missing

• One paper: pass its PMID or doc_key. It prints the number of claims stored.
• --all-missing: lists the papers that have none (claims.py status --missing) and runs the same procedure for each, in parallel batches of 5, one Claude agent per paper. It reports how many succeeded and which failed.
• After ingest: /paper-rag:ingest tells Claude to run extraction after a successful ingest, unless you ask for a bare ingest. Running scripts/ingest.py directly never extracts claims.

Extraction is done by Claude, so it costs tokens and takes time per paper. There is no timing guarantee. In one batch of 12 papers, individual agents took roughly one to three minutes each.

Abstract-only and full-text papers

Claims are extracted from whatever source chunks exist. A metadata-only ingest has only the abstract chunk, so its claims come from the abstract alone. Each claim records this in source_level:

source_level	Cited chunks
abstract	only the abstract chunk
fulltext	only full-text chunks
mixed	both

A paper with full text can still have abstract claims, so the field describes the claim, not the paper.

Re-extracting

Run /paper-rag:extract-claims <doc_key> again any time. It replaces the paper's claims.

If you add full text to a paper that was ingested metadata-only, re-ingest it with --force and then re-extract. --force deletes the paper's existing claims, so re-extraction is required, not optional. /paper-rag:attach-pdf does not help: it only stores the file for the dashboard and adds no chunks.

The claim record

Every claim is stored as a claim row: the claim text is the embedded document, and the rest is metadata.

Field	Meaning
text	The claim as one readable sentence.
source_chunk_ids	Ids of the paper's own chunks that support it.
evidence_span	Verbatim quote from one of those chunks.
section	Section of the first source chunk, unless the claim sets one.
source_level	abstract, fulltext or mixed (derived, never set by hand).
population, intervention, comparator, outcome	Who, what, versus what, measured how.
direction	increase, decrease, no_difference or mixed.
effect_value, effect_measure, uncertainty_interval	e.g. 0.41, Cohen's d, 95% CI 0.15-0.67.
study_design	e.g. randomized trial, cohort.

The structured fields are optional. If the chunk does not state one, it is left out (values like unknown are dropped). They are for display and comparison and are not used for filtering, except that source_level and the other metadata can be filtered with /paper-rag:ask --type claim --where '{"source_level": "abstract"}'.

The guards

claims.py add checks the whole batch before writing anything. One bad claim rejects all of them.

Error	Meaning
unknown_source_chunk_ids	A cited id is not one of this paper's abstract or text chunks.
missing_evidence_span	The claim has no quote.
evidence_span_not_in_source	The quote is not found in a cited chunk. Whitespace and case are ignored, but nothing else, so no paraphrase and no ....
numbers_not_in_source	A multi-digit or decimal number in the claim text, effect_value or uncertainty_interval does not appear in the cited chunks. Single digits are not checked.
invalid_direction	direction is not one of the four allowed values.
invalid_claim	Missing text or source_chunk_ids.

The usual fixes: quote a shorter stretch of the sentence, reword the claim so it uses the source's own numbers, or add the chunk that contains a missing number to source_chunk_ids. Watch for text the chunker stores differently from how it reads: a number spelled out in the source ("Twenty-one" for 21) or markdown in a quoted statistic (*p* for p).

--skip-span-check and --skip-number-check turn a guard off. Use them only when a quote or number is legitimately not verbatim, such as a computed difference, and say so.

Where claims show up

• Search. /paper-rag:ask runs a claim search and a chunk search, attaches each claim's source chunks, and drops chunks the claims already cover. See Asking Questions.
• Dashboard. The Claims view lists every claim with its direction, effect and evidence, and each paper has a Claims tab. Re-run /paper-rag:dashboard after extracting. See Dashboard.
• Coverage. claims.py status lists papers with their claim counts, and --missing shows those with none.

For the concept itself and how claims relate to the other row types, see Concepts; for every flag, see Commands.