br-locleg corpus explorer

OCR progress

Documents by extraction status across nightly refreshes. Watch the pending OCR band drain into extracted as the backfill proceeds. Each bar is one refresh snapshot; the history survives the nightly rebuild of the serving database.

Documents by status over time (31 snapshots)

0 2,500,000 5,000,000 7,500,000 10,000,000 07-06 · extracted: 3,224,665 07-06 · pending OCR: 2,284,486 07-06 · corrupt: 209 07-06 · other: 625,491 07-14 · extracted: 3,224,665 07-14 · pending OCR: 2,284,486 07-14 · corrupt: 209 07-14 · other: 625,491 07-14 · extracted: 3,224,665 07-14 · pending OCR: 2,284,486 07-14 · corrupt: 209 07-14 · other: 625,491 07-14 · extracted: 3,224,665 07-14 · pending OCR: 2,284,486 07-14 · corrupt: 209 07-14 · other: 625,491 07-15 · extracted: 3,224,665 07-15 · pending OCR: 2,284,486 07-15 · corrupt: 209 07-15 · other: 625,491 07-17 · extracted: 4,289,412 07-17 · pending OCR: 1,219,706 07-17 · corrupt: 242 07-17 · other: 625,491 07-18 · extracted: 4,412,954 07-18 · pending OCR: 1,096,159 07-18 · corrupt: 247 07-18 · other: 625,491 07-19 · extracted: 4,580,512 07-19 · pending OCR: 928,595 07-19 · corrupt: 253 07-19 · other: 625,491 07-20 · extracted: 4,751,297 07-20 · pending OCR: 757,804 07-20 · corrupt: 259 07-20 · other: 625,491 07-21 · extracted: 4,921,194 07-21 · pending OCR: 587,906 07-21 · corrupt: 260 07-21 · other: 625,491 09-29 · extracted: 5,509,089 09-29 · corrupt: 271 09-29 · other: 625,491 09-29 · extracted: 5,509,089 09-29 · corrupt: 271 09-29 · other: 625,491 09-29 · extracted: 5,509,089 09-29 · corrupt: 271 09-29 · other: 625,491 09-30 · extracted: 5,509,089 09-30 · corrupt: 271 09-30 · other: 625,491 10-01 · extracted: 5,509,089 10-01 · corrupt: 271 10-01 · other: 625,491 10-02 · extracted: 5,509,089 10-02 · corrupt: 271 10-02 · other: 625,491 10-03 · extracted: 5,509,089 10-03 · corrupt: 271 10-03 · other: 625,491 10-05 · extracted: 5,509,089 10-05 · corrupt: 271 10-05 · other: 625,491 10-06 · extracted: 5,509,089 10-06 · corrupt: 271 10-06 · other: 625,491 10-07 · extracted: 5,509,089 10-07 · corrupt: 271 10-07 · other: 625,491 10-08 · extracted: 5,509,089 10-08 · corrupt: 271 10-08 · other: 625,491 10-09 · extracted: 5,509,089 10-09 · corrupt: 271 10-09 · other: 625,491 10-10 · extracted: 5,509,089 10-10 · corrupt: 271 10-10 · other: 625,491 10-11 · extracted: 5,509,089 10-11 · corrupt: 271 10-11 · other: 625,491 10-12 · extracted: 5,509,089 10-12 · corrupt: 271 10-12 · other: 625,491 10-13 · extracted: 5,509,089 10-13 · corrupt: 271 10-13 · other: 625,491 10-15 · extracted: 5,509,089 10-15 · corrupt: 271 10-15 · other: 625,491 10-16 · extracted: 5,509,089 10-16 · corrupt: 271 10-16 · other: 625,491 10-17 · extracted: 5,509,089 10-17 · corrupt: 271 10-17 · other: 625,491 10-18 · extracted: 5,509,089 10-18 · corrupt: 271 10-18 · other: 625,491 10-19 · extracted: 5,509,089 10-19 · corrupt: 271 10-19 · other: 625,491 07-06 07-14 07-18 07-21 09-29 10-02 10-06 10-09 10-12 10-16 10-19
extracted pending OCR corrupt other

Current snapshot

StatusDocumentsShare
extracted 5,509,089 89.8%
unsupported_format 480,498 7.8%
not_a_document 92,695 1.5%
unreachable 39,107 0.6%
skipped_robots 12,256 0.2%
failed 935 0.0%
corrupt 271 0.0%

Recent refreshes

The engine-swap early-warning metrics: when the search-index rebuild leaves the overnight window or the database size balloons, it shows here first.

Refresh (as of, UTC) Build wall Index build DB size Record rows Indexed docs
2026-10-19 07:47 823.9s 677.0s 44346.1 MB 6,513,560 5,509,089
2026-10-18 07:51 894.9s 687.6s 44368.9 MB 6,513,560 5,509,089
2026-10-17 07:52 976.2s 753.2s 44401.2 MB 6,513,560 5,509,089
2026-10-16 07:51 912.4s 705.3s 44323.1 MB 6,513,560 5,509,089
2026-10-15 07:51 891.9s 694.4s 44335.6 MB 6,513,560 5,509,089
2026-10-13 07:51 919.6s 692.5s 44341.4 MB 6,513,560 5,509,089
2026-10-12 07:47 835.6s 678.5s 44422.7 MB 6,513,560 5,509,089
2026-10-11 07:51 924.8s 682.5s 44409.8 MB 6,513,560 5,509,089
2026-10-10 07:51 959.8s 729.9s 44398.3 MB 6,513,560 5,509,089
2026-10-09 07:50 930.3s 688.3s 44382.8 MB 6,513,560 5,509,089
2026-10-08 07:51 961.1s 723.3s 44345.6 MB 6,513,560 5,509,089
2026-10-07 07:51 926.5s 710.6s 44347.7 MB 6,513,560 5,509,089
2026-10-06 07:51 895.5s 703.5s 44373.1 MB 6,513,560 5,509,089
2026-10-05 07:47 874.8s 708.5s 44374.4 MB 6,513,560 5,509,089