ref-manager · PLAN.md §3, §5c, §7a–b

Paper Directory Anatomy

Every paper lives in one PMID-keyed directory. Raw snapshots are immutable; derived records are versioned behind an atomic pointer; user notes and review overlays sit outside the regeneration path entirely. Amber pins mark where the actual manuscript content — original PDF, extracted text, extracted figures — physically lives.

papers/<pmid>/ meta.json identity: PMID · stable citekey raw/<sha256>/ immutable snapshots PDF A JATS · HTML · metadata never rewritten; evidence for every claim acquisitions.json origin · hash · availability convert + extract cached by hash VERSIONS — DERIVED RECORDS versions/a41f/ superseded — retained for reproducibility versions/c7e2/ manifest.json source.md B claims.json figures/ C figures.json · paper.md active — validated; sets extraction_tier versions/f09d/ staging — invisible until validated current.json atomic pointer points to validate → atomic swap USER-OWNED notes.md annotations.json regeneration never edits state.json resumable stage checkpoints REVIEW & OBSERVATION authorship.json funding.json corrections.json citations.json overlays & dated observations — never rewrite raw extraction accept/edit/reject applies to claims.json at read time library level, outside the paper dir ↓ committed version → FTS5, in one transaction index/catalog.sqlite rebuildable via /ref:index --rebuild
active validated version staging / pending validation superseded, retained
A raw/<sha256>/<file>.pdforiginal manuscript as fetched — byte-identical, never modified B versions/<id>/source.mdextracted full text of the active version, with source-locator mapping back to the raw PDF/JATS C versions/<id>/figures/extracted figure images; captions, locators, and hashes live alongside in figures.json
Writes flow one way: the immutable PDF in raw/ (A) is converted and extracted into a staging version, producing extracted text in source.md (B) and extracted figures (C); only after validation does current.json swap atomically, and only committed versions reach the index. corrections.json holds accept/edit/reject decisions on claims and author matches — applied as a read-time overlay, never a rewrite of raw extraction.

Full layout

<LIBRARY>/papers/<pmid>/       # pure PMID dir — D11: every paper has a PMID
  meta.json                    # bibliographic authority, stable citekey, status + checked_at
  acquisitions.json            # origin, URL/item ID, fetched_at, media type, hash, availability
  raw/<sha256>/                # immutable snapshots: metadata response, JATS, HTML, Papers text
    *.pdf                      # (A) original manuscript, byte-identical as fetched
  current.json                 # atomic pointer → active validated version
  versions/<id>/               # versioned derived records
    manifest.json              # input hashes, schema/tool/model/prompt versions, completeness
    source.md                  # (B) extracted full text + source-locator mapping
    claims.json                # extraction records, provenance, evidence tier
    figures/                   # (C) extracted figure images
    figures.json               # figure IDs, captions, locators, hashes, optional vision interpretation
    paper.md                   # generated OKF Reference view
  notes.md                     # user-authored; extraction never touches
  annotations.json             # imported annotations, keyed by stable source IDs
  authorship.json              # ordered authors, list completeness, identity links, role evidence
  funding.json                 # funding observations, grant links, locators and review states
  citations.json               # dated cited-by observations: source, query, count, coverage
  corrections.json             # accept/edit/reject overlays on claims, mappings, identity, funding
  state.json                   # resumable stage checkpoints + errors

Papers handoff

<LIBRARY>/ — the authority meta.json citekey · PMID · DOI · PMCID raw/<sha256>/*.pdf copied, never moved notes.md user content — first export only project membership → tags (--tags-from project) exports/papers/<batch>/ references.bib Papers' own dialect note · keywords · local-url <citekey>.pdf or <Author>/<Journal>-<Year>.pdf attached via local-url manifest.json frozen selector + PMIDs file hashes · pushed-note hash Papers.app — read-only $.article · $.ext_ids matched on PMID, else DOI $.user_data.notes the Notes column · 76 items $.user_data.tags JSON array · 29 items $.user_data.annotations sha256 + page rects · 324 / 5 export you import app UI highlights never pushed /ref:fetch · /ref:pull-annotations full text, highlights, and margin notes travel inbound only — read from a consistent snapshot, never written
Two directions, deliberately asymmetric. Outbound is files only: a BibTeX batch in the app's own dialect plus PDF copies, imported by you through its UI — the live database is never written (§7). Inbound is where user content comes from, which is why an outbound note push is allowed only on a paper's first export and refused afterwards unless forced (D28). Highlights have no outbound path at all: they are anchored to a file hash and page rectangles that are only valid against Papers' own copy.

What each field maps to, read off Papers' own export matched against its live library:

LibraryBibTeX fieldPapers
meta.jsontitle, author, journal, volume, number, pages, issn, year, doi, pmid, pmcid$.article · $.ext_ids
raw/<sha256>/*.pdflocal-url = {file://localhost/…}attached file
notes.md (--notes)note$.user_data.notes
project / tags (--tags)keywords$.user_data.tags
— never writtenrating$.user_data.rating
annotations.jsonno channel$.user_data.annotations (inbound only)

Selecting papers

One grammar, shared by every command that works on a set — compare, summarize, review, ask, export, and the batch acquisition commands. There is no free-text topic argument: it would re-resolve on every run, so a saved artifact could silently change membership underneath its conclusions.

SelectorResolves toFrom
<pmid…>exactly those papers — the base casephase 1
--project <slug>project membership, narrowed by --question <qid>phase 1
--screened <state>recorded screening decisionsphase 2
--read / --queue <state>human reading state, never model promotionphase 1
--query <slug> [--run <id>]one immutable saved search runphase 2
--search "<expr>"FTS hits, resolved once and frozenphase 2
--study <id>every publication of one investigationphase 5
--concept <id>a graph concept, via its alias tablephase 8
--from-file <path>a PMID list from a filephase 1

ID formats (§3d). Anything you type is a lowercase slug — a–z, 0–9, hyphen, 1–64 chars — minted by you, with duplicates refused rather than silently suffixed. --question is scoped to its project; every other typed ID is library-global. IDs the system references instead (version, claim, table, brief, report) are opaque, never typed, and always surfaced by a listing command. External codes — MeSH, ORCID, DOI, award numbers — are aliases, never identities; PMID is the one deliberate exception, since v1 ingests nothing without one.

Selectors combine with AND, and --tier / --exclude refine any of them. Before expensive work runs, the set resolves to an explicit PMID list reported with counts by extraction tier, verification state, and retraction status. That list is then frozen into the artifact manifest alongside the selector expression, so explicit refresh reports added, removed, and changed-evidence papers rather than quietly substituting a different set.

Planned commands

Bring papers in

/ref:init [path]
set the library root; recorded in ~/.config/ref-manager/config.json, never hardcoded
/ref:add <pmid…>
the only way in — resolve metadata, dedupe by PMID, allocate a citekey, abstract-tier extract
/ref:fetch <pmid…>
optional full text for one or more PMIDs: PMC JATS → Unpaywall PDF → publisher HTML → Papers fulltext; convert, extract figures/captions; reports a per-PMID result
/ref:attach <pmid> <path> […]
verify identity and preserve one or more local PDFs as immutable acquisitions, each PMID paired with its own path
/ref:extract <pmid…>
full-tier extraction for one or more PMIDs, fanned out per paper; each commits independently once adequate converted full text exists

Find

/ref:search <query>
lexical FTS5 search over indexed passages; --scope notes searches notes, annotations, and project comments, labeled as personal content
/ref:ask <question>
expand terms (MeSH, synonyms), retrieve passage + graph evidence, answer with [^pmid] citations
/ref:search-pubmed <question>
parse into PICO, run PubMed, log include/exclude screening decisions
/ref:update-queries
manually rerun saved queries/*.yaml, dedupe against prior hits — never scheduled
/ref:related <pmid>
snowball — backward references from full text, forward citations via ELink

Organize & review

/ref:project
manage questions, chapter/experiment scope, membership, relevance, and project-specific screening
/ref:queue
track reading state — to_screen / to_read / reading / read — and priority, separate from extraction tier
/ref:verify
accept, edit, or reject claims, concept mappings, identity matches, and grant links as evidence-backed overlays
/ref:audit [--citations]
re-check retraction/errata status across every PMID; --citations appends a dated PMC cited-by observation per paper

Write & compare

/ref:summarize <selector>
prose narrative across the selected set; a one-paper set degrades to a single-paper summary
/ref:review <selector>
appraised synthesis over the set — GRADE-style certainty and risk-of-bias, plus evidence tables (full tier)
/ref:review --prisma --project <slug>
PRISMA 2020 flow record rendered from saved query runs, screening decisions, and acquisition availability; unevidenced counts stay unknown
/ref:compare <selector>
editable evidence matrix with source-linked cells and study-level grouping; no appraisal needed
/ref:methods <topic>
retrieve protocols, datasets, software, instruments, and analysis choices with evidence locators
/ref:check-citations <text>
check a paragraph's assertions against evidence; flag unsupported wording or conflicting results
/ref:brief
save an answer, evidence snapshot, and open questions as a versioned research brief
/ref:cite <citekey>
inline @citekey while writing
/ref:export --bib / --csl / --papers
emit BibTeX or CSL-JSON; --papers <selector> writes a Papers-dialect batch plus PDF copies into the folder the app imports from, optionally carrying notes.md and tags (--notes, --tags)

Explore the graph

/ref:gaps <selector>
structural graph query — single-study claims, unresolved contradictions, untested concept pairs
/ref:hypothesize --concept <id>
Swanson ABC traversal over the concept graph; candidates re-checked against PubMed

People, grants & reports

/ref:person
manage name variants, ORCID, affiliation history, and confirmed publication matches
/ref:publications --person <id> --role <role>
filter ordered author lists by independently evidenced shared/corresponding roles
/ref:publications --coauthors --since <date>
coauthor list for NSF COA / NIH conflict forms; incomplete author lists surface as gaps rather than a short list
/ref:grant
manage funders, awards, approved aliases, aims, and evidence-backed publication links
/ref:discover --person / --grant <id>
run an explicit PubMed search and queue portfolio candidates for confirmation
/ref:report --person <id> --from <date> --to <date>
export publication lists by period, role, lab, or grant, with evidence and unresolved-match appendices
/ref:report --citations
include dated cited-by counts labelled as PMC-indexed coverage, never as total citations (D25)

Maintain

/ref:note <pmid>
write free-text thoughts to notes.md, kept distinct from model extraction
/ref:open <pmid>
push the acquired PDF into Papers.app for reading and annotating
/ref:pull-annotations <pmid>
import Papers highlights and margin notes into annotations.json — the inbound direction for user content; highlights never travel back out
/ref:index [--rebuild]
reconstruct catalog.sqlite from committed records; incomplete staging is ignored
/ref:status
report library location, config, and pending/failed stage state