Community post

How to Collect Evidence for Document Analysis: Using Docling to Keep Layout Errors Outside the Knowledge Base

Design testing materials based on reading order, table structure, and source identification to avoid merely checking whether the conversion is successful.

Successful document conversion does not mean the data is ready for use

The official Docling documentation introduces capabilities such as parsing multi-format documents, understanding PDF layouts, and structured export. For knowledge bases or document organization applications, the parsing results serve as essential inputs for subsequent processes. If the reading order, header relationships, or footnotes change during retrieval and generation stages, work may continue on incorrect materials, yet the final answer will still appear smooth.

This paper proposes a design approach for small-scale testing collections, suitable for using when changing parsing configurations or updating processing workflows. It is not a test ranking for a specific version, and no large number of documents are required at the start. Select publicly available materials that can illustrate specific issues, which usually helps determine where errors occur and whether fixes are effective.

Classify by layout difficulties rather than just file format

Even for PDFs, single-column manuals, double-column papers, and scanned tables have different challenges. It is recommended to group samples based on layout features: multi-column, cross-page tables, rotated pages, many footnotes, images closely attached to text, and pages containing formulas or code. Select a small number of publicly available representative materials from each group and note why they were chosen.

File formats still need to be preserved, but they cannot replace difficult features. Otherwise, a collection mainly consisting of simple documents may appear well-performing overall, but hides complex pages that the team often handles. For structural issues that occur less frequently in actual business but have significant impact, at least one case should be retained so they are not overlooked during future updates.

Define partial expected results for the samples

There is no need to manually rewrite the entire result for each long document. First, select key pages and areas. Clearly define which layer a title belongs to, the order between two paragraphs, which header a certain value belongs to, and what a footnote explains. These partial expectations are easier to review than a vague “readable” criterion.

Assume a table spans two pages, with the same header on the second page. The expectation should indicate whether the same logical table is maintained, how the repeated header is handled, and whether units remain clear. For example, image descriptions in double-column papers cannot be inserted into sentences in the other column. Writing these requirements as specific examples helps different reviewers use similar judgment criteria.

Record accurate text and structure separately

If words in a paragraph are recognized correctly but the order is wrong, the meaning will still be altered. It is recommended to record issues such as missing text, reading order, title relationships, table structure, and source positioning separately. For scanned materials, include unreadable areas; for materials with embedded text, focus on whether extraction and layout combination are correct.

Each defect record should include the original file identity, page number or area, observed results, expected results, and impact description. Do not save only a screenshot marked with a circle without explaining what it affects subsequent tasks. This allows maintainers to arrange fixes based on impact, and determine whether a improvement merely makes the display more attractive or actually preserves the relationships in the data.

Keep the export results traceable to their source

Docling offers unified document representations and various export formats. The choice of format depends on subsequent tasks. Readable text and results with retained structure can be saved together, but it must be clear that they originate from the same processing. Do not present the edited content as unprocessed original text after manual modifications.

If you plan to submit paragraphs to the knowledge base, it is recommended to retain document versions, sections, and location clues for each segment. When generating responses, returning a file name does not mean the reader can find the exact source of the conclusion. For tables or formulas with complex structures, allow the application to keep the entry to the original page instead of forcing all content to be explained in plain text.

Manage changes in the parsing process through regression checks

When adjusting recognition methods, processing models, or splitting rules, recheck using the same set of materials. Record which cases improved, which deteriorated, and which environments remain unverified. Assign priorities to different issues, but do not rely solely on a overall average value to decide whether to update, as serious errors may be concentrated in a few key pieces of data.

The final review material should allow another person to recheck: the original text must be accessible, the processing configuration should be explained, the expected results should be specific enough, and the reasoning for judgment should be available. Such a collection not only helps select parsing tools but also enables the team to develop a common understanding of document quality. The credibility of the knowledge base often starts with these less obvious upstream checks.


Original source: docling-project/docling official repository ↗. Verification date: November 13, 2026.

Editor’s note: This article is original content organized by COTCC’s official side, written with AI assistance and verified against official sources. The solutions, examples, and acceptance suggestions are the editor’s design; they do not claim to have been fully deployed or performance tested. Specific features, licenses, and costs shall be based on the current project descriptions.

Discussion 0

Oldest first

No public comments yet. Start the discussion.

Publish