Tutorial

A systematic review, from saved search to PRISMA flow

You'll run a PubMed search you can reproduce, screen the hits with a written reason for every decision, get full text, render a PRISMA 2020 flow record from what's on disk, and then run an appraised review over the included set. Commands used: /ref:project, /ref:search-pubmed, /ref:update-queries, /ref:add, /ref:screen, /ref:fetch, /ref:attach, /ref:review --prisma, and /ref:review.

The journey

deterministic, writes committed state network acquisition, per paper your judgement

Each step writes one kind of record, and the PRISMA flow is just a count over those records. That's why the order matters less than you might expect: you can screen before adding, or add before screening. The flow reports whatever is actually on disk.

The example project is tdcs-review, the saved queries are tdcs-wm and tdcs-wm-adhd, and the PMIDs 38000101…38000107 are placeholders. Blocks labelled output from a scratch library were produced by running the plugin's scripts against a throwaway library. In those runs, PMID lists we supplied by hand stood in for the PubMed MCP call.

Steps

Create the review project

Screening decisions belong to a project. The same paper can be included in one review and excluded from another without conflict.

output from a scratch library
```
› /ref:project create tdcs-review --scope "tDCS and working memory"
{
  "slug": "tdcs-review",
  "scope": "tDCS and working memory",
  "questions": []
}
```

Search PubMed into a saved query

Give Claude a question or an explicit PubMed expression. Claude writes the boolean query, runs it through the PubMed MCP tool, and then saves the exact expression, the source, a timestamp and the result PMIDs. Pass --create on the first run of a new slug.

output from a scratch library

› /ref:search-pubmed "Does tDCS improve working memory?" --slug tdcs-wm --create
# Claude translates the question into this expression, runs it, then saves the run:
{
  "run_id": "run-004c50fe5cfa",
  "query": "(\"transcranial direct current stimulation\"[MeSH] OR tDCS) AND \"working memory\"",
  "source": "pubmed",
  "retrieved_at": "2026-09-15T10:15:58.976036+00:00",
  "pmids": ["38000101", "38000102", "38000103", "38000104", "38000105"]
}
› /ref:search-pubmed "tDCS AND ADHD" --slug tdcs-wm-adhd --create
{
  "run_id": "run-d5d8e9f70664",
  "query": "tDCS AND ADHD",
  …
  "pmids": ["38000102", "38000106", "38000107"]
}

Why the run is frozen. The script doesn't parse PICO and doesn't call PubMed. It stores exactly what it's handed. Later, --query tdcs-wm --run run-004c50fe5cfa always means these five PMIDs, so a comparison or flow built on that run can't quietly change membership.

Re-run the search when you choose to

/ref:update-queries re-runs the stored expression. It doesn't re-derive a query from your original question. It appends a new run and lists what changed since the previous one. Nothing re-runs on a schedule.

output from a scratch library

› /ref:update-queries tdcs-wm
{
  "run": {
    "run_id": "run-9ff45f72904c",
    "query": "(\"transcranial direct current stimulation\"[MeSH] OR tDCS) AND \"working memory\"",
    "source": "pubmed",
    "retrieved_at": "2026-09-15T10:15:59.067749+00:00",
    "pmids": ["38000101", "38000102", "38000103", "38000104", "38000105", "38000106"]
  },
  "added": ["38000106"],
  "removed": []
}

Add the hits you want a record for

A search run is only a list of PMIDs. /ref:add is still the only way a paper enters the library. The PRISMA "reports sought" count comes from meta.json, so add every paper you include.

output from a scratch library

› /ref:add 38000101 38000102 38000103
38000101: added (citekey=rivera2023transcranial)
38000102: added (citekey=lindqvist2024working)
38000103: added (citekey=chen2024stimulation) — no abstract available — metadata-only record, not fabricated (§3a)

Screen, with a reason every time

Every decision needs a --reason. Decisions are included, excluded or pending. --run is optional and links the decision to the search run the paper came from. If you screen a PMID that isn't a project member yet, it becomes one.

output from a scratch library
```
› /ref:screen --project tdcs-review --pmid 38000101 --decision included --reason "RCT, adult population, WM outcome"
{
  "pmid": "38000101",
  "decision": "included",
  "reason": "RCT, adult population, WM outcome",
  "timestamp": "2026-09-15T10:15:59.382605+00:00"
}
› /ref:screen --project tdcs-review --pmid 38000102 --decision included --reason "cohort, WM outcome"
› /ref:screen --project tdcs-review --pmid 38000103 --decision excluded --reason "wrong outcome"
› /ref:screen --project tdcs-review --pmid 38000104 --decision excluded --reason "wrong outcome"
› /ref:screen --project tdcs-review --pmid 38000199 --decision pending --reason "sent by a colleague"
```
History, not state. screening.jsonl is append-only. Change your mind and the new line wins, but the old decision stays on file. Word reasons consistently, because the flow groups exclusions by their exact reason text.

Pending still counts as screened. The flow counts every PMID with any latest decision, pending included. 38000199 above also never appeared in a saved run, so the flow will flag it as unevidenced instead of silently counting it.
Get full text for the included papers

Fetch works through PMC OA, then Unpaywall, then publisher HTML, and falls back to abstract-only. Attach is for PDFs you already have. Both report one result per PMID, and one failure never blocks the rest.

illustrative output
```
› /ref:fetch 38000101 38000102
# one line per PMID: acquired / oa_location_found / abstract_only / failed,
# plus any diagnostic: lines about incomplete conversion
› /ref:attach 38000102 ~/Downloads/lindqvist-2024.pdf
# per pair: attached / duplicate_noop / refused (identity check failed) / failed
› /ref:extract 38000101 38000102
# full-tier claims, which the appraisal in step 8 reads
```
Abstract-only is not a failure. For PRISMA, a paper without full text counts as a report not retrieved, which is exactly what the flow should say.
Render the PRISMA 2020 flow

Name the saved queries that feed this review. A project has no stored link to its queries, so you pass them each time. --query slug:run_id pins a specific run, and a bare slug uses the latest run.

output from a scratch library (trimmed)
```
› /ref:review --prisma --project tdcs-review --query tdcs-wm --query tdcs-wm-adhd
{
  "status": "created",
  "snapshot_id": "prisma-0c8322f37193",
  "manifest": {
    …
    "flow": {
      "identified": {
        "per_source": { "pubmed": { "count": 9, "runs": [ …tdcs-wm: 6…, …tdcs-wm-adhd: 3… ] } },
        "total_raw": 9,
        "note": null
      },
      "duplicates_removed": 2,
      "unresolved_query_specs": [],
      "screened": 5,
      "excluded": { "count": 2, "by_reason": { "wrong outcome": 2 } },
      "unevidenced_screened_pmids": ["38000199"],
      "reports": {
        "sought": 2,
        "not_retrieved": 2,
        "not_retrieved_pmids": ["38000101", "38000102"],
        "unevidenced_sought_pmids": []
      },
      "included": {
        "publications_count": 2,
        "publications": ["38000101", "38000102"],
        "studies_count": "unknown",
        "studies_note": "studies/studies.jsonl not available -- included-studies count is unknown, not assumed equal to included-publications count",
        …
      }
    }
  }
}
```
This scratch run happened before step 6 had acquired anything, so both included papers read as not retrieved. Here's the same data as a flow diagram:

final included count unknown or unevidenced: reported, not balanced away

Each box names the committed file its count comes from. The script never estimates. A count with no evidence behind it reads "unknown", and a PMID that can't be traced to a run is listed by ID. That's the honest version of PRISMA's arithmetic.

Leave out --query and the top of the flow becomes unknown. total_raw and duplicates_removed read "unknown", with the note no --query given: identified counts are unknown, not zero. Screening, sought, retrieved and included still render from what's committed.

The flow is frozen. Running it again returns the same snapshot until you ask for a refresh:

output from a scratch library (trimmed)
```
› /ref:review --prisma --project tdcs-review
{
  "status": "reused_frozen_snapshot",
  "snapshot_id": "prisma-0c8322f37193",
  …
}
› /ref:review --prisma --project tdcs-review --query tdcs-wm --query tdcs-wm-adhd --refresh
# status "refreshed", a new snapshot id, plus included_added / included_removed
# against the prior snapshot
```
Appraise the included set

Default-mode /ref:review takes any selector and a --batch label. --screened is project-scoped, so pair it with --project. The appraisal needs full text, and a wholly abstract-tier set is refused before any rating is drafted:

output from a scratch library
```
› /ref:review --project tdcs-review --screened included --batch wm-appraisal
{
  "status": "refused",
  "batch": "wm-appraisal",
  "reason": "selected set is wholly abstract-tier -- appraisal needs full-text detail; not proceeding (§8 phase-10 gate)",
  "by_extraction_tier": { "abstract": 2, "full": 0, "unavailable": 0, "missing_record": 0 }
}
```
Once step 6 has given at least one included paper full-tier claims, the same command drafts per-paper risk-of-bias ratings and a set-level GRADE rating. RCTs get RoB 2, cohort and case-control studies get Newcastle-Ottawa, and meta-analyses get AMSTAR-2. Any abstract-tier paper left in the set is marked insufficient_information rather than dropped.

illustrative output
```
› /ref:review --project tdcs-review --screened included --batch wm-appraisal
{
  "status": "created",
  "batch": "wm-appraisal",
  "manifest": { "selector_expression": "--project tdcs-review --screened included", "pmids": [ … ], … },
  "appraisals": { "38000101": { … each domain: rating, claim_ids, note, review_status: "model_draft" … } },
  "grade": {
    "baseline": …,
    "factors": {
      …
      "publication_bias": { "downgrade": false, "not_assessed": true,
        "reason": "not assessable from a curated single-library snapshot (no funnel-plot/search-completeness data)" }
    },
    "downgrades_applied": …,
    "certainty": …
  }
}
```
Every rating is a draft. Ratings are derived deterministically from claim fields, and most methodological domains land on insufficient_information because the claim schema doesn't capture blinding or allocation concealment. To record your own judgement on a domain, ask Claude to review it. It runs verify.py review-appraisal … --decision accept|edit|reject --rationale "…", and a --refresh then shows human_confirmed, human_edited or human_rejected. The GRADE rating belongs to the whole set, lives in grade.json, and has no per-paper review path.

What you now have

<LIBRARY>/
  queries/
    tdcs-wm.yaml                         # two immutable runs: the original and the manual re-run
    tdcs-wm-adhd.yaml                    # one run
  papers/38000101/ … 38000103/
    meta.json · authorship.json · funding.json · raw/<sha256>/response.json
    acquisitions.json · versions/<id>/  # after fetch/attach and extract
  projects/tdcs-review/
    project.yaml · papers.yaml           # membership, latest screening mirrored per paper
    screening.jsonl                      # append-only decision history with reasons
    prisma/
      latest.json                        # → prisma-0c8322f37193
      prisma-0c8322f37193/manifest.json    # query specs, data cutoff, full flow
      prisma-0c8322f37193/flow.md · flow.csv
    reviews/wm-appraisal/                # only once the set has full-tier papers
      manifest.json · grade.json · appraisals/<pmid>.json

Next, the PI report tutorial uses the same review machinery for identity and funding evidence. For selectors and frozen sets, see Concepts. The full command reference is at /ref:screen and /ref:review.

The journey

Steps

Create the review project

Search PubMed into a saved query

Re-run the search when you choose to

Add the hits you want a record for

Screen, with a reason every time

Get full text for the included papers

Render the PRISMA 2020 flow

Appraise the included set

What you now have