Benchmark study · October 2026
PDF to text benchmark: PDF2TEXT vs Tesseract and pdftotext
In short
- We ran PDF2TEXT's production pipeline and three free approaches on the same 82 hand-checked real-world pages as our accuracy benchmark, and scored all four with the same code: pdftotext (the PDF's own text layer), Tesseract 4.1.1 OCR, and the text layer with Tesseract as a fallback.
- On the 50 held-out pages, PDF2TEXT found 98.4% of the numbers on the page (95% interval 96.5–99.5%), and 99.1% of the numbers it returned were correct. The free method that found the most was Tesseract: 77.0% (61.7–90.1%).
- pdftotext was as precise as PDF2TEXT: 99.1% of the numbers it returned were correct. Scans and photos have no text layer, though, so it found only 46.9%.
- The gap is on scanned PDFs: on the held-out scans, Tesseract found 62.4% of the numbers and PDF2TEXT 98.2%. On digital PDFs the text layer was nearly as good: 99.1% correct and 92.1% found, against 99.6% and 99.4%.
- Free tools came out ahead in places. Across all 11 plain-text pages, every number the text layer returned was correct (PDF2TEXT: 92.9%). On the 5 pages mixing digital and scanned content, Tesseract found 95.3% of the numbers to PDF2TEXT's 88.2%. And for words found, Tesseract (88.2%) was close to PDF2TEXT (93.0%); the interval of the difference includes zero.
- We built the pipeline being tested. The free tools' settings were picked on separate development pages by a rule fixed in advance, and the commands to re-run them are below. PDF2TEXT's figures are the published ones (all pages: 98.1% correct, 97.3% found).
- Numbers found on the 50 held-out pages, PDF2TEXT
- 98.4%
- Best free method: Tesseract, 77.0%
- Numbers correct on the same pages, PDF2TEXT
- 99.1%
- Best free method: pdftotext, 99.1%, though it found 46.9%
Results
The 50 held-out pages
“Correct” is the share of values in a method's output that appear on the page. “Found” is the share of values on the page that appear in the output. F1 combines the two. Under each figure: its 95% interval.
| Method | Numbers correct | Numbers found | Numbers F1 |
|---|---|---|---|
| PDF2TEXT production pipeline | 99.1% 97.6–99.8 | 98.4% 96.5–99.5 | 98.7% 97.2–99.6 |
| Text layer, else Tesseract | 89.3% 78.2–98.0 | 75.6% 57.3–89.8 | 81.9% 67.7–93.1 |
| Tesseract 4.1.1 OCR | 87.9% 78.4–96.1 | 77.0% 61.7–90.1 | 82.1% 70.6–92.6 |
| PDF text layer (pdftotext) | 99.1% 95.3–100.0 | 46.9% 17.5–73.7 | 63.7% 29.6–84.6 |
| Method | Numbers correct | Numbers found |
|---|---|---|
| PDF2TEXT | 99.1%, 95% interval 97.6 to 99.8 | 98.4%, 95% interval 96.5 to 99.5 |
| Text layer or Tesseract | 89.3%, 95% interval 78.2 to 98.0 | 75.6%, 95% interval 57.3 to 89.8 |
| Tesseract | 87.9%, 95% interval 78.4 to 96.1 | 77.0%, 95% interval 61.7 to 90.1 |
| pdftotext | 99.1%, 95% interval 95.3 to 100.0 | 46.9%, 95% interval 17.5 to 73.7 |
| Method | Words correct | Words found | Dates correct | Dates found |
|---|---|---|---|---|
| PDF2TEXT | 96.6% 94.3–99.0 | 93.0% 89.2–95.8 | 99.7% 95.8–100.0 | 100.0% 100.0–100.0 |
| Text layer or Tesseract | 90.8% 84.6–94.8 | 79.0% 63.8–89.3 | 97.4% 92.2–100.0 | 89.7% 59.6–97.7 |
| Tesseract | 89.4% 85.1–92.6 | 88.2% 83.0–92.0 | 97.4% 92.5–100.0 | 91.0% 72.1–98.6 |
| pdftotext | 93.6% 86.6–98.2 | 53.4% 28.3–71.8 | 100.0% 100.0–100.0 | 54.5% 0.0–92.5 |
| Compared with | Numbers correct | Numbers found | Numbers F1 | Pages better / same / worse |
|---|---|---|---|---|
| Text layer or Tesseract | +9.7 +1.4 to +20.7 | +22.8 +8.9 to +40.0 | +16.8 +6.0 to +30.3 | 21 / 22 / 1 |
| Tesseract | +11.1 +3.2 to +20.7 | +21.4 +8.5 to +35.6 | +16.6 +6.5 to +27.4 | 25 / 18 / 1 |
| pdftotext | -0.1 -1.7 to +3.2 | +51.5 +24.9 to +80.6 | +35.0 +14.3 to +68.7 | 28 / 15 / 1 |
Differences in percentage points, with 95% intervals from the same resampled documents. Pages compare each page's numbers F1 and leave out the 6 held-out pages with no numbers.
Where each method works
By kind of page
Numbers correct / found for each group, on the held-out pages and on all pages. Groups with fewer than 5 pages are marked: too few pages to conclude anything.
Digital PDFs: when a PDF carries its own text, reading it is free and nearly exact. Every number pdftotext missed on the held-out digital files came from pages whose text layer was missing or had been added by a scanner's OCR, not from real digital text.
Scanned PDFs: this is where the methods part ways. Tesseract returns more wrong numbers and misses more of them. 2 held-out scans are rotated, but rotation doesn't explain the gap: reading them in the best of the four orientations (a choice no user could make in advance) moves Tesseract's held-out figures from 87.9% correct and 77.0% found to only 88.6% and 77.5%.
Plain text documents: on the held-out ones every method but pdftotext found almost every number (Tesseract: 98.6%). Across all pages this is PDF2TEXT's weakest group, and there the text layer returned fewer wrong numbers than PDF2TEXT did.
Other languages: on the held-out pages in Arabic, Persian, Russian, Chinese and Japanese, PDF2TEXT was 96.9% correct and found 97.8%; the text layer with Tesseract as fallback 91.1% and 91.6%.
Phone photos: none are among the held-out pages. On the 3 development photos, Tesseract found 27.1% of the numbers and PDF2TEXT 92.3%, too few pages to conclude.
| Pages | PDF2TEXT | Text layer or Tesseract | Tesseract | pdftotext |
|---|---|---|---|---|
| Kind of file | ||||
| Digital PDFs (23) | 99.6 / 99.4 | 99.0 / 92.1 | 96.2 / 89.9 | 99.1 / 92.1 |
| Scanned PDFs (24) | 98.4 / 98.2 | 77.0 / 61.0 | 77.3 / 62.4 | – / 0.0 |
| PDFs mixing digital and scanned pages (3) Too few pages to conclude | 100.0 / 79.6 | 100.0 / 10.2 | 93.8 / 91.8 | 100.0 / 10.2 |
| Layout | ||||
| Financial tables (statements, invoices) (19) | 99.2 / 98.7 | 87.8 / 76.1 | 86.8 / 78.9 | 100.0 / 48.4 |
| Plain text documents (8) | 100.0 / 100.0 | 100.0 / 99.3 | 100.0 / 98.6 | 100.0 / 54.0 |
| Forms and other layouts (23) | 98.5 / 97.1 | 90.5 / 68.0 | 87.6 / 66.4 | 96.1 / 41.0 |
| Language | ||||
| Languages written in Latin script (38) | 99.3 / 98.4 | 89.1 / 73.7 | 88.4 / 78.5 | 100.0 / 44.4 |
| Arabic, Persian, Russian, Chinese, Japanese, Bengali (12) | 96.9 / 97.8 | 91.1 / 91.6 | 83.6 / 63.6 | 94.4 / 68.0 |
| Pages | PDF2TEXT | Text layer or Tesseract | Tesseract | pdftotext |
|---|---|---|---|---|
| Kind of file | ||||
| Digital PDFs (39) | 99.5 / 99.2 | 98.9 / 92.2 | 96.6 / 92.9 | 98.9 / 92.2 |
| Scanned PDFs (35) | 97.1 / 97.3 | 82.0 / 59.5 | 80.1 / 60.7 | – / 0.0 |
| PDFs mixing digital and scanned pages (5) | 100.0 / 88.2 | 100.0 / 43.5 | 96.4 / 95.3 | 100.0 / 5.9 |
| Phone photos (3) Too few pages to conclude | 96.4 / 92.3 | 86.6 / 27.1 | 86.6 / 27.1 | – / 0.0 |
| Layout | ||||
| Financial tables (statements, invoices) (32) | 98.7 / 97.5 | 90.8 / 70.5 | 90.2 / 73.3 | 99.4 / 43.8 |
| Plain text documents (11) | 92.9 / 94.3 | 100.0 / 42.1 | 83.3 / 43.0 | 100.0 / 21.5 |
| Forms and other layouts (39) | 98.6 / 97.7 | 93.0 / 75.8 | 90.2 / 75.3 | 97.0 / 39.9 |
| Language | ||||
| Languages written in Latin script (65) | 98.8 / 97.6 | 91.7 / 71.5 | 90.9 / 75.1 | 99.5 / 40.9 |
| Arabic, Persian, Russian, Chinese, Japanese, Bengali (17) | 93.6 / 94.9 | 92.5 / 52.2 | 78.4 / 42.7 | 95.3 / 41.5 |
| Method | Numbers correct | Numbers found | Words correct | Words found |
|---|---|---|---|---|
| PDF2TEXT | 98.1% 96.6–99.5 | 97.3% 95.6–98.8 | 96.2% 93.8–98.6 | 90.2% 86.7–93.3 |
| Text layer or Tesseract | 91.8% 84.4–97.5 | 69.1% 51.2–85.8 | 91.8% 88.0–94.7 | 75.9% 62.9–86.5 |
| Tesseract | 89.8% 83.0–95.4 | 71.0% 53.7–87.7 | 90.1% 87.0–92.9 | 87.4% 82.2–91.5 |
| pdftotext | 99.0% 97.0–100.0 | 41.0% 22.4–61.6 | 93.6% 89.0–97.0 | 47.1% 30.5–61.9 |
Includes the development pages that PDF2TEXT was tuned on and the free tools' settings were chosen on.
| Method | Numbers correct | Numbers found | Words correct | Words found |
|---|---|---|---|---|
| PDF2TEXT | 97.0% 94.6–99.6 | 96.0% 94.0–98.7 | 95.7% 91.0–99.1 | 86.2% 79.9–91.8 |
| Text layer or Tesseract | 95.7% 92.6–98.4 | 61.2% 36.6–93.0 | 93.5% 88.8–96.9 | 71.3% 51.0–92.3 |
| Tesseract | 92.7% 86.0–97.0 | 63.7% 38.1–96.1 | 91.2% 86.0–95.5 | 86.3% 76.7–94.6 |
| pdftotext | 98.7% 96.7–100.0 | 33.8% 14.9–61.8 | 93.7% 85.9–98.7 | 37.9% 20.3–58.0 |
The pages used to tune PDF2TEXT and to choose the free tools' settings, so these figures flatter every method.
Method
What was compared
All four methods read the same 82 pages and were scored by the same code. Versions and settings are exact.
PDF2TEXT production pipeline
- The outputs behind our accuracy page, produced by the production pipeline on September 30, 2026. Nothing was re-run for this study.
- Pages rendered at 200 dpi. For PDFs, the pipeline also reads the file's own text layer.
- It returns tables, form fields and text blocks. For scoring, these are flattened to text by the same code that scores every method.
Text layer, else Tesseract
- What a careful free setup does: use the PDF's text layer when it has at least 1 non-space character, otherwise run Tesseract with the settings below.
- The threshold was chosen on the development pages from 6 candidates; 45 of the 82 pages used the text layer.
Tesseract 4.1.1 OCR
- Tesseract 4.1.1 with Leptonica 1.82.0 (Ubuntu 22.04), LSTM engine (--oem 1), page segmentation mode 3, tessdata_best 4.1.0 models, no preprocessing.
- PDF pages rendered at 300 dpi with pdftoppm 22.02.0; photos read as uploaded.
- Languages: the document's own language, as a user would pick it, plus English, French, German, Spanish, Portuguese, Italian and Dutch on Latin-script pages, or plus English on other scripts.
- Chosen from 48 set-ups on the development pages: Ubuntu's default models (tessdata_fast 4.0.0) or tessdata_best 4.1.0, page segmentation modes 3, 4 and 6, with or without Otsu binarization, four language choices. Binarization never helped.
- A median of 15.7 seconds per page on one CPU thread of our shared server, not counting rendering. Context only, not a speed test.
PDF text layer (pdftotext)
- pdftotext 22.02.0 (poppler) with -layout, one page at a time.
- Photos and scans without a text layer give no text, which is what a user gets. Scans that carry a scanner's hidden OCR layer give that layer.
How the pages were chosen and checked
- 68 real-world documents (82 pages): bank statements, invoices, receipts, forms, letters, scans and phone photos, sampled to match the mix of files people upload. They are the pages behind our accuracy benchmark.
- 39 pages come from digital PDFs, 35 from scanned PDFs, 5 from PDFs mixing both, and 3 are phone photos. 17 pages come from documents in Arabic, Persian, Russian, Chinese, Japanese or Bengali.
- Every page's expected text was checked by hand against the source. For 29 digital pages it started from the PDF's own text layer (22 needed corrections); the other 53 pages were transcribed by hand.
- Characters the checker could not read are left out of the scoring for every method.
Development and held-out pages
- Before any tuning, the pages were split by document: 32 development pages (32 documents) and 50 held-out pages (36 documents).
- PDF2TEXT's pipeline was tuned on the development pages only.
- The free tools' settings were chosen on the same development pages, by a rule written down before any of them was scored: the highest F1 on numbers. We tried 48 Tesseract set-ups and 6 text-layer thresholds, then ran the chosen ones once on the held-out pages.
- Neither side was tuned on the held-out pages, so they are the fair comparison. The other figures are shown for completeness.
How scores and intervals are computed
- Each method's output is compared with the page's checked text as a bag of words, numbers and dates, so layout and reading order don't count.
- Numbers are compared by their digits (1,234.56, 1.234,56 and 1234.56 match) and one-digit numbers are ignored; dates by day, month and year in either order; words after Unicode normalization and case folding. Digits count as written, so Arabic-Indic digits must be read as Arabic-Indic digits.
- Correct and found are pooled over pages, so every value counts once. F1 is 2 × correct × found ÷ (correct + found).
- 95% intervals come from 10,000 bootstrap resamples of whole documents (pages of one document move together), with a fixed seed. A difference between two methods uses the same resamples for both.
- PDF2TEXT's tables and fields are flattened with column headers, cell values, form labels and values. Names the pipeline makes up for fields are left out, because they are not page text.
Limitations
- 82 pages is a small set. Intervals are wide for small groups, and groups under 5 pages are marked as too few pages to conclude.
- The pages follow what people upload to PDF2TEXT, with many bank statements and invoices. Results on another mix of documents will differ.
- We compared Tesseract 4.1.1, the version Ubuntu 22.04 ships, not Tesseract 5; newer versions and other open-source engines may do better.
- No paid OCR APIs or commercial converters were compared.
- Table structure is not scored. A value counts wherever it appears, so tools that scramble a table lose nothing for it, and PDF2TEXT gets no credit for keeping rows and columns together.
- On digital PDFs the expected text started from the PDF's text layer, which favours pdftotext there, and PDF2TEXT, which also reads that layer.
- PDF2TEXT leaves some page text out on purpose (page furniture around tables), which lowers its words found compared with a full-text dump.
- The text-layer threshold the development pages picked (1 character) did slightly worse on the held-out pages than the 20 characters PDF2TEXT itself uses: with 20, numbers would be 89.4% correct and 77.2% found. We report the pre-chosen one; PDF2TEXT's lead in numbers F1 is +15.8 points (+5.1 to +28.7) with either.
- We built the pipeline being tested and ran this study ourselves. To limit that bias, the free tools' settings were picked by a rule fixed in advance on separate pages, and the commands are below.
Reproduce
How to reproduce the free baselines
The pages themselves are private real-world documents, so they are not published. These are the exact commands, to run on your own pages.
1. The PDF text layer
One page at a time; N is the page number.
pdftotext -layout -enc UTF-8 -f N -l N document.pdf page_N.txt
2. Render a PDF page for OCR
Photos go to Tesseract as they are.
pdftoppm -r 300 -png -singlefile -f N -l N document.pdf page_N
3. Tesseract models
The LSTM models of the tessdata_best 4.1.0 release, one file per language.
curl -LO https://github.com/tesseract-ocr/tessdata_best/raw/4.1.0/eng.traineddata
# likewise fra, deu, spa, por, ita, nld and the document's language, into tessdata_best/
4. Tesseract OCR
One thread per page. On Latin-script pages, the document's language plus the set below; on other scripts, the document's language plus English (Russian: rus+ukr+eng).
OMP_THREAD_LIMIT=1 tesseract page_N.png stdout --tessdata-dir tessdata_best --oem 1 --psm 3 -l eng+fra+deu+spa+por+ita+nld --dpi 300
5. Text layer, else Tesseract
Keep the text layer when it has at least 1 non-space character; otherwise use the Tesseract output.
if [ $(tr -d '[:space:]' < page_N.txt | wc -m) -ge 1 ]; then cat page_N.txt; else tesseract page_N.png stdout ...; fi
98% of numbers correct across 82 hand-checked real-world pages
The full results for PDF2TEXT, by kind of page, are on the accuracy benchmark page. Results on your documents may differ: check the preview against the original.
Try it on a document