Scripture corpus

Nothing in this repository may contain Bible text until a translation is licensed for this use. The audit engine is built to fail closed: with no corpus, any artifact containing Scripture returns SCRIPTURE_UNVERIFIED and publication is blocked.

Installing a translation

Do not hand-write corpus files. Use the importer, which normalises the source, checks every reference against the parser the audit engine actually uses, and records the licence:

python tools/corpus/ingest.py install \
  --id WEB --name "World English Bible" \
  --license "Public Domain" --license-url https://ebible.org/web/ \
  --source /path/to/web.json

It accepts four source shapes (nested book/chapter/verse, flat "John 3:16" keys, a list of {book, chapter, verse, text} records, and verse ranges which it splits). Anything else is refused rather than guessed at.

Safety behaviour:

• A corpus missing books is refused unless --allow-partial is passed.
• A translation whose licence is not recognisably public domain is refused unless --confirm-licensed is passed, with a reminder that spoken narration usually needs a separate permission.
• Numbering gaps are warnings (a partial corpus legitimately starts mid-chapter); non-numeric chapters, empty verses and unparseable references are errors and stop the install.
• --dry-run reports coverage without writing anything.

Verify an installed corpus at any time:

python tools/corpus/ingest.py check --id WEB

Required layout

data/bible/
├── manifest.json
└── <translation>.json

manifest.json:

{
  "translations": [
    {
      "id": "WEB",
      "name": "World English Bible",
      "license": "Public Domain",
      "license_url": "https://...",
      "file": "web.json",
      "verified_against_source": "2026-11-01"
    }
  ]
}

<translation>.json (book → chapter → verse → text, as plain strings):

{
  "John": {
    "3": {
      "16": "For God so loved the world ...",
      "17": "..."
    }
  }
}

Installed: Berean Standard Bible

./tools/corpus/fetch_bsb.sh           # fetch, hash-verify, install, verify
./tools/corpus/fetch_bsb.sh --check   # verify an existing install

The BSB is dedicated to the public domain by the Berean Bible Translation Committee. The text is fetched from the publisher at bereanbible.com and the SHA-256 is pinned in the script, so a changed upstream file stops the install instead of silently replacing the corpus.

Current install: 66 books, 1,189 chapters, 31,086 verses, 725,243 words, zero structural errors.

16 verses are deliberately empty in this edition, because the critical text omits them: Matthew 17:21, 18:11, 23:14; Mark 7:16, 9:44, 9:46, 11:26, 15:28; Luke 17:36, 23:17; John 5:4; Acts 8:37, 15:34, 24:7, 28:29; Romans 16:24. The importer records them in the manifest as omitted_verses, and the audit engine reports them as omitted by this edition rather than as a wrong reference.

Do not use a mirror. A widely-mirrored "BSB" JSON on GitHub turned out to be a different edition: 803 verses differ in wording and book naming differs for the numbered books. Only the publisher's file is authoritative here.

Why the hash is pinned

A Bible corpus is not a dependency to upgrade casually. If the hash changes, read the diff, confirm the publisher still dedicates the text to the public domain, and update SHA256 in the script deliberately — with the date recorded in the manifest.

Licensing notes

• Public-domain translations (e.g. WEB, KJV in most jurisdictions) can be stored directly; keep the licence record in the manifest.
• Modern translations (NIV, ESV, NASB, CSB, NLT) require a licence for storage and for spoken narration. Spoken quotation often requires additional permission beyond print. Confirm with the publisher before any narration is generated.
• Book names must match the parser (Genesis, 1 Samuel, Psalms, Song of Songs, Revelation, …). Use Psalms for the book; Psalm 23 is normalised automatically.
• Chapter/verse keys are strings.

Why this blocks everything

The brief requires verified quotations, and YouTube policy requires accurate attribution. A system that "quotes from memory" will eventually misquote Scripture on camera. The gate exists so that cannot happen silently.

Verification behaviour

1. Every reference in the content is parsed and resolved.
2. Declared quotations are normalised (case, punctuation, curly quotes) and compared token-wise against the corpus; below 85% token coverage is a blocker.
3. Paraphrase must be labelled as paraphrase in the artifact; a paraphrase submitted as quotation is rejected.

Verifying an installed corpus

data/bible/manifest.json is git-ignored; manifest.example.json shows the shape. After any copy between machines, run:

python3 tools/corpus/verify.py         # report
python3 tools/corpus/verify.py --write # also record the installed file hash

It checks 66 books, contiguous chapter numbering, contiguous verse numbering within a chapter, empty verses, manifest agreement, and the recorded file hash. Exit code 1 means Scripture verification must not be trusted.

Current state (BSB, verified 2026-11-01): 66 books, 1,189 chapters, 31,086 verses, 0 empty verses, 16 critical-text omissions recorded. file_sha256 is the durable anchor because the recorded source was read from a temporary directory.