Table Extraction in Content Capture Domain — Why, What and How..

IBM’s Automated Document Processing (ADP) component in CloudPak for Business Automation (CP4BA) enables table extraction from documents through a combination of AI-driven techniques, OCR (Optical Character Recognition), and deep learning models.

This article introduces you to the generic concept of extraction of tables from documents in ADP.

Lets start with “Why” part..

Have you ever noticed that nearly all receipts from your daily activities — whether for shopping, fuel, pharmacy purchases, or other services — are structured as tables? This is because tables inherently capture relationships within data, aligning with the way humans naturally visualize and interpret information.

In the enterprise world, tables play a crucial role across various domains, such as supply chain management (including logistics and procurement). In the past, bookkeeping relied on storing physical receipts and manual processes for business operations.

Imagine how much simpler it would be if invoices and other documents were scanned and fed into in a system, that triggers appropriate processes seamlessly based on its content — without human intervention.

A key enabler of this automation is the efficient extraction of tables from documents, which helps streamline business workflows.

Now, coming to the “What” part..

A table is essentially a structured arrangement of data in rows and columns, typically with headers that define relationships within the data. It may also include an optional summary to consolidate key information and additional properties that provide further context.

A sample table can be seen as below:

• Table Headers: Usually found in the top row, specifying the meaning of each column.
• Summary Data (Optional): Aggregated information, such as totals.
• Additional Data (Optional): Metadata or properties related to the table.

Tables come in various complexities — from simple grids with clearly separated rows, columns, and headers to more intricate designs featuring subsections, vertical or wrapped headers, multiple headers in the same column, or even layouts without visible grid lines. Furthermore, tables can appear in different document formats, such as scanned or printed documents, which introduce additional challenges.

Factors like watermarks, document skew, blurring, or poor scan quality can complicate table extraction, making it a complex task in itself.

And finally the “How”..

The “Definition”..!!

Documents such as an invoice, contains specific “fields” that define its structure. For example, an invoice typically includes fields like Invoice Date, Invoice Number, Address, Itemized Table and Totals. IBM’s Automated Document Processing (ADP) provides a type library (aka ontology) that defines various fields generally seen in various documents. These fields can be:

• Simple types (e.g., “Numeric” for Totals, “Date” for Due Dates)
• Complex types (e.g., “Composite fields” like Address, which consists of Name, Street, City, State, Country, Zip Code, and Phone)
• Tables, representing itemized details of data within the document

All these fields are grouped under different Document Classes, that define the types of input documents. ADP also allows users to create, modify, or delete these type definitions and document classes as needed.

Defining Tables in ADP — ADP enables users to define tables with — Table Headers, Summary Data (Optional), Additional Data (Optional)

The “Extraction”..!!

When a document is ingested into the system, the following steps occur:

1. OCR Processing: Extracts text from the document.
2. Document Classification: The document is categorized into the correct Document Class.
3. Data Extraction: Relevant fields and tables are identified and extracted.
4. Downstream Processing: The extracted data is passed to other components in CP4BA for further automation.

Machine learning models assist in both document classification and data extraction.

Users can leverage the Document Processing Designer to train data extraction models by specifying the field locations within sample documents, and also enrich the extracted values with converters and formatters. By training the model with sample documents, ADP enhances accuracy and automates data extraction, enabling seamless integration into business workflows.

Stay tuned.. An in-depth into ADP’s table extraction capabilities..!!