The standalone repo doubles as a personal reference manager: search, tag, rate, and export what you've registered.

What The Registry Can Do For You

Everything hangs off one file, data/papers/registry.jsonl. Each capability below is a slash command that reads or writes it, or a read-only index derived from it.

Using It As A Reference Manager

Day-to-day use is six commands, in this order. Everything below expands on one step.

Two housekeeping commands run alongside, not in the flow: /deep-research:reindex rebuilds the derived indexes from registry.jsonl after a bulk import or a corrupt database, and /deep-research:doctor checks the asset store for missing or corrupt files. Neither is required for normal use.

# register, then tag/rate for later retrieval
/deep-research:pool-add --repo ~/lit/ --pmid 38214501
/deep-research:annotate tag  --repo ~/lit/ --evidence-id pmid:38214501 --add to-read
/deep-research:annotate rate --repo ~/lit/ --evidence-id pmid:38214501 --stars 4

# find it later by tag, rating, or free text
/deep-research:search --repo ~/lit/ --tag to-read --min-rating 4 --q "biomarker"

# ask the library a question instead of re-reading everything
/deep-research:ask --repo ~/lit/ "does exercise reduce depressive symptoms in adolescents?"

Every command above has an equivalent direct script call (registry.py, annotations.py, ask.py, …) — the slash command is the one to reach for; the script underneath is documented at each section below and in full in Command reference.

Worked Example: Build A Publication Repo, Then Export To ReadCube

Start to finish: a fresh repo, papers added three different ways, and a bundle ReadCube can import. Nothing here needs a full review run.

1. Create the repo

One command, once, per library. This is the prerequisite for every other command on this page.

/deep-research:init ~/research/my-lit-repo

{
  "status": "ok",
  "command": "init",
  "repo_root": "/Users/you/research/my-lit-repo",
  "created_dirs": ["data/papers", "data/sources/assets", "data/sources/snapshots", "runs", "projects", "exports/wiki", "exports/bib"]
}

--repo ~/research/my-lit-repo is what every other command on this page takes from here on. Already have a wiki-manager wiki with papers in it? /deep-research:init ~/research/my-lit-repo --from-wiki ~/wiki/ seeds the new repo's registry from it instead of starting empty.

2. Add papers

All intake paths converge on the same registry record — it does not matter which one you use, and duplicates merge by identifier or by PDF sha256 rather than double-storing.

# one paper by PMID
/deep-research:pool-add --repo ~/research/my-lit-repo --pmid 38214501

# a PubMed search, first 15 hits, human studies only
/deep-research:pool-add --repo ~/research/my-lit-repo \
  --query "CBT AND adolescent depression" --n 15 --human-only

# explicit list of PMIDs in one call
/deep-research:pool-add --repo ~/research/my-lit-repo --pmid 38214501,39012233,31234567

pool-add covers PMIDs and bounded PubMed queries. A DOI-only add, a PDF you already have, a bulk .bib file, or a whole folder of PDFs have no slash-command wrapper yet — call the registry script directly for those:

python3 scripts/registry.py add         --repo ~/research/my-lit-repo --doi 10.1001/jama.2023.24567
python3 scripts/registry.py add-pdf     --repo ~/research/my-lit-repo --file ~/Downloads/smith2024.pdf
python3 scripts/registry.py import-bib  --repo ~/research/my-lit-repo --file refs.bib
python3 scripts/registry.py import-folder --repo ~/research/my-lit-repo --dir ~/Downloads/papers/ --recursive

Check whether a paper is already in before re-fetching it, and confirm what landed:

/deep-research:pdf-lookup --repo ~/research/my-lit-repo --doi 10.1001/jama.2023.24567
/deep-research:search     --repo ~/research/my-lit-repo --limit 20

3. (Optional) Tag and rate as you go

/deep-research:annotate tag  --repo ~/research/my-lit-repo --evidence-id pmid:38214501 --add diagnostics
/deep-research:annotate rate --repo ~/research/my-lit-repo --evidence-id pmid:38214501 --stars 5

4. Export to ReadCube

Publishes a self-contained local bundle — RIS bibliography, primary PDFs, manifest, import report — that the ReadCube desktop app imports. This never touches wiki state and is separate from the OKF export below.

# everything registered, blocking if any record has no resolvable PDF
/deep-research:export --repo ~/research/my-lit-repo \
  --all --destination ~/Desktop/readcube-import --require-pdfs

# just the papers you tagged, with a BibTeX file alongside the RIS
/deep-research:export --repo ~/research/my-lit-repo \
  --evidence-id pmid:38214501 --evidence-id pmid:39012233 \
  --destination ~/Desktop/readcube-import --bib

# re-run later — only publishes what changed since the last successful export
/deep-research:export --repo ~/research/my-lit-repo \
  --all --destination ~/Desktop/readcube-import --incremental

The bundle lands at <destination>/<batch-name>-<unique-id>/: references.ris, PDFs/, manifest.json, and a README.md that walks through the actual ReadCube import — that last step is manual, deep-research never confirms the import itself. Full flags, bundle layout, and the refmgr-backed alternative source: ReadCube export.

What's New Vs. What Already Existed

Search (/deep-research:search)

One command, three layers, all combinable in a single call:

• Facet filters — --journal, --year, --status, --extraction-status, --appraisal-status — plain in-memory filters over fields the registry already carries.
• Annotation filters — --tag, --min-rating — join read-only against annotations.jsonl.
• Keyword search — --q "text" — a stdlib-only, lowercase AND-of-terms match over title, abstract, journal, the extraction's claim sentences and outcome names, extraction narrative, appraisal rationale (scope it to one review with --project), and the paper's full text. Claims are searched ahead of raw full text, so a snippet quotes the extracted finding whenever both match.
• Controlled vocabulary — --mesh, --author, --article-type — case-insensitive substring match against MeSH headings, author names and article types, so --mesh depress finds "Depressive Disorder, Major". /deep-research:search's underlying script has a facets --scheme mesh mode that lists what the library actually holds, with paper counts.
• Similarity ranking — --similar-to <evidence-id> — re-ranks whatever the filters above surfaced by cosine similarity, using the cache built by /deep-research:embed index.

/deep-research:search --repo ~/lit/ --tag to-read --min-rating 4 --q "biomarker"

Direct script call: registry.py search --repo ~/lit/ --tag to-read --min-rating 4 --q "biomarker"; facets via registry.py facets --repo ~/lit/ --scheme mesh.

Annotations (/deep-research:annotate)

Tags, a 1-5 star rating, and a free-text note per paper — your own opinion, kept out of the canonical registry and out of project-scoped appraisal on purpose, the same way appraisal already stays separate from the registry. An annotation never requires the paper to be registered first.

/deep-research:annotate tag  --repo ~/lit/ --evidence-id pmid:38214501 --add to-read
/deep-research:annotate rate --repo ~/lit/ --evidence-id pmid:38214501 --stars 4
/deep-research:annotate note --repo ~/lit/ --evidence-id pmid:38214501 --set "Good discussion of threshold selection."
/deep-research:annotate show --repo ~/lit/ --evidence-id pmid:38214501
/deep-research:annotate list --repo ~/lit/ --tag to-read --min-rating 3

Direct script call: annotations.py tag|rate|note|show|list with the same flags.

Semantic Similarity (/deep-research:embed)

Build a cached embedding per paper from its title, abstract, and extraction narrative, then ask for nearest neighbors by cosine similarity — no re-embedding at query time. similar starts from a paper you already have; query starts from a question. Neither ranks across two embedding models — query says so and stops rather than mixing them.

/deep-research:embed index   --repo ~/lit/
/deep-research:embed similar --repo ~/lit/ --evidence-id pmid:38214501 --k 5
/deep-research:embed query   --repo ~/lit/ --text "does X reduce Y in older adults?" --k 10

# or fold similarity into a faceted/keyword search once the index exists:
/deep-research:search --repo ~/lit/ --similar-to pmid:38214501 --limit 5

Direct script call: embeddings.py index|similar|query, pure Python underneath (no re-embedding at query time except for query's own question).

Asking The Library (/deep-research:ask)

The library already holds extracted, span-verified papers. Asking it something should not require a new pipeline run — so this retrieves from what is there, and answers only from evidence that still verifies.

/deep-research:ask --repo ~/lit/ "does exercise reduce depressive symptoms in adolescents?"

Direct script call: ask.py retrieve --repo ~/lit/ --question "...".

Retrieval is hybrid: bm25 over the full-text chunk index, plus cosine similarity over the embedding cache when one exists, fused by reciprocal rank. Every returned claim and passage carries the (source_id, start, end) span it came from, re-checked against the snapshot store at retrieval time; anything that no longer verifies is moved to unverified and must not be cited.

Saved Searches And Alerts (/deep-research:alerts)

Save a PubMed query, then re-run it whenever you want to know what has appeared since last time. Storage is refmgr's existing saved-searches table — no new file, no daemon, no background polling: run is a command you (or your own cron) invoke.

/deep-research:alerts save --repo ~/lit/ --name cbt --query "CBT AND adolescents"
/deep-research:alerts run  --repo ~/lit/ --register

Direct script call: alerts.py save|list|delete|run.

Each rerun is bounded by entry date — records added to PubMed since that search last ran — because the question is "what is new to me", and a 2019 paper indexed last week is new to you. Hits are diffed against registry.jsonl, so "new" means the repo has never seen it. Without --register nothing is added and the same papers will surface again next time; with it they are registered through the normal path, mirrored and indexed like any other record.

Indexes (/deep-research:reindex)

registry.jsonl is the source of truth. The SQLite library at data/refmgr/ is an index built from it — every mutating registry command mirrors its changed records into refmgr and reindexes them, and the full-text chunk index is built from the stored snapshots. Both are rebuildable, so a lost or corrupt database is repaired, not restored:

/deep-research:reindex --repo ~/lit/                 # both indexes
/deep-research:reindex --repo ~/lit/ --chunks-only   # full text only

Direct script call: registry.py reindex --repo ~/lit/ [--papers-only|--chunks-only].

This is also the backfill path for a repo whose papers were registered before the mirror existed, and it is safe to re-run: an unchanged snapshot is skipped rather than re-split. If the index is missing or unusable, search --q falls back to scanning snapshot bodies — slower, same answers.

Health Check (/deep-research:doctor)

Content-addressed storage fails quietly: a PDF deleted out from under the database stays invisible until an export or a reader tries to open it. doctor looks for that — missing asset files, bytes that no longer hash to their name, attachments pointing at nothing, and stale or orphaned index rows. It is read-only and repairs nothing.

/deep-research:doctor --repo ~/lit/          # fast: trusts file sizes
/deep-research:doctor --repo ~/lit/ --deep   # exact: re-hashes every asset

Direct script call: registry.py doctor --repo ~/lit/ [--deep].

It exits non-zero only for actual data loss — missing or corrupt assets, dangling attachments — so it is usable from cron. Stale index rows are reported but stay "healthy", because reindex fixes those and nothing is at risk. A missing original cannot be rebuilt from the registry, so that finding means restore-from-backup or re-import, which is your decision, not the script's.

Manual-Retrieval Gaps (/deep-research:missing-fulltext)

Every automated acquisition rung can still come up empty — paywalled, no OA copy, institutional access needed. This is the punch list for a human to go finish the job: every registered paper with no PDF stored (asset_status: missing), one row each, with a link straight to the paper.

/deep-research:missing-fulltext --repo ~/lit/
/deep-research:missing-fulltext --repo ~/lit/ --status included   # only papers actually kept

Direct script call: registry.py missing-fulltext --repo ~/lit/ [--status <s>] [--limit N].

Rendered as a table:

The link prefers doi.org, falls back to PubMed, and falls back again to a plain "nothing to link to" when the record has neither. Once a copy is found by hand, attach it to the same record — this never creates a duplicate:

python3 scripts/registry.py add-pdf --repo ~/lit/ --file ~/Downloads/paper.pdf --doi 10.1001/jama.2023.24567

OKF Knowledge-Graph Export (/deep-research:okf-export)

Pick N papers out of the registry by evidence_id and promote them into an existing OKF wiki bundle, reusing okf.py promote's validator unmodified rather than hand-writing a second concept-emission path. Each evidence_id must already have an extraction on file — run /deep-research:extract first if not (it fetches full text and promotes the extraction without paying for an appraisal or a narrative summary this export doesn't need).

/deep-research:okf-export --repo ~/lit/ \
  --evidence-id pmid:38214501 --evidence-id doi:10.1001/jama.2023.24567 \
  --wiki ~/wiki/ --project diagnostics-review

Direct script call: research.py okf-export with the same flags.

End-To-End Example

All slash commands, in order. Two steps have no wrapper yet (DOI-only add, PDF add) — the script call is inlined for those, per Command reference.

$ /deep-research:init ~/lit/
{"status": "ok", "command": "init", "repo_root": "/home/you/lit"}

$ /deep-research:pool-add --repo ~/lit/ --pmid 38214501
{"status": "ok", "evidence_id": "pmid:38214501", ...}

$ python3 scripts/registry.py add --repo ~/lit/ --doi 10.1001/jama.2023.24567   # no wrapper: DOI-only add
{"status": "ok", "evidence_id": "doi:10.1001/jama.2023.24567", ...}

$ python3 scripts/registry.py add-pdf --repo ~/lit/ --file ~/Downloads/smith2024.pdf   # no wrapper: PDF add
{"status": "ok", "evidence_id": "pmid:39012233", ...}

$ /deep-research:annotate tag  --repo ~/lit/ --evidence-id pmid:38214501 --add diagnostics
$ /deep-research:annotate tag  --repo ~/lit/ --evidence-id pmid:38214501 --add to-read
$ /deep-research:annotate rate --repo ~/lit/ --evidence-id pmid:38214501 --stars 5
$ /deep-research:annotate note --repo ~/lit/ --evidence-id doi:10.1001/jama.2023.24567 \
    --set "Good external validation cohort."

$ /deep-research:search --repo ~/lit/ --journal "JAMA" --min-rating 3
{"status": "ok", "results": [ ... ]}

$ /deep-research:search --repo ~/lit/ --q "biomarker threshold" --tag diagnostics
{"status": "ok", "results": [ ... ]}

$ /deep-research:embed index --repo ~/lit/
{"status": "ok", "command": "index", "embedded": 3, "skipped_current": 0}

$ /deep-research:embed similar --repo ~/lit/ --evidence-id pmid:38214501 --k 5
{"status": "ok", "results": [
  {"evidence_id": "pmid:39012233", "title": "...", "score": 0.81},
  {"evidence_id": "doi:10.1001/jama.2023.24567", "title": "...", "score": 0.74}
]}

$ /deep-research:extract --repo ~/lit/ --pmid 38214501
{"status": "ok", ...}   # needed before okf-export: it requires an extraction on disk
                        # (extract-only — no appraisal, no narrative summary paid for)

$ /deep-research:okf-export --repo ~/lit/ \
    --evidence-id pmid:38214501 --wiki ~/wiki/
{"status": "ok", "command": "okf-export", "promote_exit_code": 0, ...}

Storage Added