br-locleg corpus explorer

OCR progress

Documents by extraction status across nightly refreshes. Watch the pending OCR band drain into extracted as the backfill proceeds. Each bar is one refresh snapshot; the history survives the nightly rebuild of the serving database.

Documents by status over time (10 snapshots)

0 2,500,000 5,000,000 7,500,000 10,000,000 07-06 · extracted: 3,224,665 07-06 · pending OCR: 2,284,486 07-06 · corrupt: 209 07-06 · other: 625,491 07-14 · extracted: 3,224,665 07-14 · pending OCR: 2,284,486 07-14 · corrupt: 209 07-14 · other: 625,491 07-14 · extracted: 3,224,665 07-14 · pending OCR: 2,284,486 07-14 · corrupt: 209 07-14 · other: 625,491 07-14 · extracted: 3,224,665 07-14 · pending OCR: 2,284,486 07-14 · corrupt: 209 07-14 · other: 625,491 07-15 · extracted: 3,224,665 07-15 · pending OCR: 2,284,486 07-15 · corrupt: 209 07-15 · other: 625,491 07-17 · extracted: 4,289,412 07-17 · pending OCR: 1,219,706 07-17 · corrupt: 242 07-17 · other: 625,491 07-18 · extracted: 4,412,954 07-18 · pending OCR: 1,096,159 07-18 · corrupt: 247 07-18 · other: 625,491 07-19 · extracted: 4,580,512 07-19 · pending OCR: 928,595 07-19 · corrupt: 253 07-19 · other: 625,491 07-20 · extracted: 4,751,297 07-20 · pending OCR: 757,804 07-20 · corrupt: 259 07-20 · other: 625,491 07-21 · extracted: 4,921,194 07-21 · pending OCR: 587,906 07-21 · corrupt: 260 07-21 · other: 625,491 07-06 07-14 07-14 07-14 07-15 07-17 07-18 07-19 07-20 07-21
extracted pending OCR corrupt other

Current snapshot

StatusDocumentsShare
extracted 4,921,194 80.2%
pending_ocr 587,906 9.6%
unsupported_format 480,498 7.8%
not_a_document 92,695 1.5%
unreachable 39,107 0.6%
skipped_robots 12,256 0.2%
failed 935 0.0%
corrupt 260 0.0%

Recent refreshes

The engine-swap early-warning metrics: when the search-index rebuild leaves the overnight window or the database size balloons, it shows here first.

Refresh (as of, UTC) Build wall Index build DB size Record rows Indexed docs
2026-07-21 07:56 1276.7s 1129.8s 40171.2 MB 6,513,560 4,921,194
2026-07-20 07:53 1174.4s 1023.2s 38907.4 MB 6,513,560 4,751,297
2026-07-19 07:54 1114.6s 992.1s 37618.7 MB 6,513,560 4,580,512
2026-07-18 07:53 1004.7s 887.3s 36452.2 MB 6,513,560 4,412,954
2026-07-17 13:50 610.7s 541.2s 35532.6 MB 6,513,560 4,289,412
2026-07-15 09:48 1062.8s 897.4s 27724.1 MB 6,513,560 3,224,665
2026-07-14 23:58 846.2s 735.3s 27795.7 MB 6,513,560 3,224,665
2026-07-14 21:59 654.1s 543.0s 27821.6 MB 6,513,560 3,224,665
2026-07-14 21:19 624.5s 549.6s 27843.9 MB 6,513,560 3,224,665
2026-07-06 01:02 811.1s 735.6s 28867.8 MB 6,513,560 3,224,665