旧OCR文本阻碍语言模型训练，FineBooks希望大规模解决

The FineBooks project, a collaboration between Hugging Face and EleutherAI, benchmarked 14 open-source OCR models on over 2,000 historical book pages to evaluate how well they can convert scanned texts into clean training data for AI language models. Smaller models frequently outperformed larger ones, with the top-performing model achieving over 97 percent character accuracy at a cost of less than two U.S. dollars per thousand pages. While the researchers consider the output quality sufficient for AI training purposes, they note that the models remain too error-prone for use in scholarly or scientific applications. The FineBooks project from Hugging Face and EleutherAI tested 14 open-weight OCR models on more than 2,000 pages from historical books. The best models already produce text good enough for AI training, but they aren't ready for scholarly use. Training open-source AI language models on public-domain books means dealing with bad text. Libraries extracted those texts from scans years ago using optical character recognition, and the results are often full of errors. The Talkie project put a number on the potential damage: a language model trained on OCR text learned at only 30 percent the efficiency of one trained on human transcriptions of the same books. FineBooks , a collaboration between Hugging Face and EleutherAI, tested whether current open-source OCR models can solve this problem. The team ran 14 open-weights models on 2,165 historical book pages and published the results as a leaderboard . The best models hit character accuracy above 97 percent at less than two dollars per thousand pages. Ad When EleutherAI and its partners released Common Pile last year, the largest openly licensed training corpus to date, it contained about 300,000 public-domain books with text from older OCR runs. The FineBooks authors say reprocessing those books with better models is one of the most effective ways to improve open AI training datasets. Ad DEC_D_Incontent-1 The project picked the Biodiversity Heritage Library (BHL) as its first target, which holds more than 300,000 digitized natural history documents totaling over 64 million pages . The BHL offers its collection as a bulk download via AWS . Measuring OCR quality requires pages with known correct transcriptions. The team used work from the IMPACT project and BHL-Europe : between 2011 and 2012, experts transcribed six BHL volumes in English, French, German, and Latin with an error rate of about one character per 2,000. That data is available under a CC-BY license in a GitHub repository and forms the basis of the new ground-truth dataset . Ad All 14 models are freely available and run on local hardware without an API key. The metric is Character Error Rate (CER), the share of incorrectly recognized characters. The leaderboard splits results into a "diplomatic" variant that counts modernization of archaic characters like the long s (ſ) as an error and a "reading" variant that tolerates such changes. The leading dots.mocr model uses just 3 billion parameters, while Qwen3.5-9B scores lower despite being nearly three times as large. OvisOCR2 takes second place with only 0.9 billion parameters at 46 cents per thousand pages. Model size and OCR quality don't correlate for historical documents. Ad DEC_D_Incontent-2 The evaluation covers only Antiqua typefaces in four languages. It doesn't account for Fraktur , non-Latin scripts, or handwriting, and FineBooks is limited to single-column book pages. The team plans to reprocess about 200,000 public-domain BHL documents with one of the top models and release the text as an open dataset. New models are added to the leaderboard on a rolling basis, and the evaluation framework is openly available. Ad