Resume Insights with LlamaIndex: Structured Data Extraction from Unstructured Documents

Introduction

Knowledge extraction from unstructured sources is paramount in enterprise environments that possess thousands of documents in a diversity of formats. PDFs in particular contain valuable information often in diverse formats and styles. Enter Large Language Models (LLMs) and LlamaIndex.

LlamaIndex is a powerful tool that brings vertical domain knowledge and LLMs together, enabling developers to create sophisticated knowledge-retrieval systems (Talk to your data). Let’s leverage LlamaIndex to create a Resume Insights application. The application will demonstrate how to extract structured domain knowledge from unstructured documents (PDFs), and indexing them to facilitate performant queries.

**STOP THIS MADNESS AND SHOW ME THE CODE!**

Resume Insights Application

To the end of this article you’ll be familiar with:

• Document processing steps (parsing, splitting, node creation)
• LlamaIndex core operations (embedding generation, indexing, query engine setup)
• The querying and extraction process
• The role of the Pydantic models in structuring the output

Abstract

When working with LlamaIndex, you rapidly cross paths with two concepts: Pydantic models and Document Parsing. This article explores how to build an Index for unstructured data that later can be queried using natural language (to get structured output).

We’ll be working on a Resume Insights application that follow this workflow:

1. The user uploads their resume.
2. The resume is parsed (document understanding).
3. The PDF resume is split into sentences. The sentences are turned into embeddings.
4. An index is built using the sentence embeddings.
5. A query engine is built to efficiently query the document.
6. Structured results (candidate’s insights) are shown to the user.

Pydantic extraction

Let’s start a bit shy and create our Pydantic model:

from pydantic import BaseModel, Field
from typing import Optional, List

class Candidate(BaseModel):
    name: Optional[str] = Field(None, description="The full name of the candidate")
    email: Optional[str] = Field(None, description="The email of the candidate")
    age: Optional[int] = Field(
        None,
        description="The age of the candidate. If not explicitly stated, estimate based on education or work experience.",
    )
    skills: Optional[list[str]] = Field(
        None, description="A list of skills possessed by the candidate"
    )

The Pydantic extractor/model would be used to instruct the query engine about their output. Pydantic is a Python library that will help us with structured extraction in a type-safe way. The extractor is a regular Python class with a few “decorators” from the Pydantic library. As long as we follow the Pydantic syntax, we could mold the output’s shape to get the knowledge we need from the documents we provided.

Querying the model using the extractor

LlamaIndex provides useful APIs for working with LLMs. As an example, the as_structured_llmAPI “Return a structured LLM around a given object”. That given object is the Pydantic extractor we just defined.

# Guide the LLM to generate the right information.
sllm = llm.as_structured_llm(output_cls=Candidate)

# If for instance, we want the LLM to generate synthetic data
input_msg = ChatMessage.from_str("Generate the candidate details.")

The Pydantic model (Candidate) shapes the result from the query engine. It includes name, email, age and skills.

But so far there is no data extraction, nor documents have been provided to the LLM for that matter. This is simple synthetic data generation.

Things start to get complicated when customization comes into play. We want to issue queries around our documents (vertical domain knowledge). If there is missing data, we’ll let the LLM fill into the details that the Candidate model is missing or requiring.

Document Parsing and Loading

Let’s explore a more practical way to get structured outputs from PDFs (unstructured documents). But First come first, let’s start loading the documents:

# Load documents
documents = SimpleDirectoryReader(
    input_files=["Resume.pdf"], file_extractor=file_extractor
).load_data()

To query a PDF, the application must understand its structure, metadata, and content. This process is known as parsing, hence the need for a parser.

# Set up the LlamaParse parser
parser = LlamaParse(
    result_type="text",  # "markdown" and "text" are available
    api_key=LLAMA_CLOUD_API_KEY,
    verbose=True,
)

# Configure the file extractor
file_extractor = {".pdf": parser}

Document Indexing

The sweetness of LlamaIndex is its remarkable ability to build indexes around data and LLMs together for efficient querying.

index = VectorStoreIndex.from_documents(documents)

That’s the value LlamaIndex brings to the table, a way to stick proprietary data, authoritative sources to LLMs so we can build powerful knowledge-retrieval applications. This process is also known as RAG (Retrieval Augmented Generation).

LlamaIndex comes with powerful defaults, so defining an LLM or embedding model is not strictly necessary. However, we can customize these defaults if needed.

LLM

llm = Gemini(model_name="models/gemini-1.5-flash-002", api_key=GOOGLE_API_KEY)

Embedding model

embed_model = GeminiEmbedding(
    model_name="models/text-embedding-004", api_key=GOOGLE_API_KEY
)

Prompt Engineering + Pydactic Models

Then let’s use the Candidate model to instruct the LLM how the output must be structure (Maybe this is how LlamaExtract works under the hood):

output_schema = Candidate.model_json_schema()
prompt = f"""
        Extract the following information from the resume:
        {output_schema}
        Provide the result in a structured JSON format. Please remove any ```json ``` characters from the output.
        """

With this approach, the as_structured_llm(output_cls=Candidate)function call is no longer necessary. And for some elusive reason, it wouldn’t work anyway!

Settings

The Settings object allows for dynamic updates to our index behavior. It replaces the previously ServiceContext object used to that matter. Attributes like the LLM and the embedding model are loaded when they are actually required by the underlying module.

Settings.embed_model = embed_model
Settings.llm = llm

Let’s use Gemini

# LLM query model and embedding model definition
llm = Gemini(model_name="models/gemini-1.5-flash-002", api_key=GOOGLE_API_KEY)

embed_model = GeminiEmbedding(
    model_name="models/text-embedding-004", api_key=GOOGLE_API_KEY
)

Indexes

Finally, we come to the very core of LlamaIndex: indexes optimized for retrieval to Large Language Models.

Vector Store Indexes

The vector store index store each Node and a corresponding embedding in a Vector Store.

index = VectorStoreIndex.from_documents(documents)

By default VectorStoreIndex stores everything in memory SimpleVectorStorebut you can change that behavior!

https://docs.llamaindex.ai/en/stable/module_guides/indexing/index_guide/#querying_1

Vector Store

A database that handles vector data (embeddings).

Node Parsers and Text Splitters

Node parsers and text splitters are closely related but have nuanced distinctions. Both play crucial roles in preparing documents for indexing and retrieval in LlamaIndex.

Text Splitters

Text splitters break down the text into logical units (words, paragraphs, sentences, documents, etc.). The text splitting strategy defines the granularity of the content that will be embedded and indexed. Common text splitting strategies include:

• Sentence splitting: Breaks text into individual sentences.
• Semantic splitting: Meaning based segmentation, aiming to split text into units that preserve semantic context.
• Paragraph splitting: Separates text based on paragraph breaks.

The choice of text splitting strategy can significantly impact the quality of retrieval and the ability to extract relevant information. Node parsers will consider documents metadata in the generated nodes.

from llama_index.node_parser import SentenceSplitter

SentenceSplitter(chunk_size=1024, chunk_overlap=20)

In this example, we’re using a SentenceSplitter with a chunk size of 1024 characters and an overlap of 20 characters between chunks. This overlap helps maintain context between chunks.

Node Parsers

Node parsers take the concept of text splitting a step further. They not only split the text but also create structured “nodes” that preserve metadata and relationships within the document. Node parsers are particularly important for structured data extraction because:

• Metadata preservation: Node parsers can maintain important metadata such as the source of the information, its position in the original document, and any associated tags or categories.
• Relationship maintenance: They can maintain relationships between different parts of the document, which is essential for accurate information retrieval and context understanding.

Having said that, it looks like a bit misleading if I can do the following

# Node parsers and text splitters are conflated in LlamaIndex.
Settings.node_parser = SentenceSplitter(chunk_size=1024, chunk_overlap=20)

Conflation of Node Parsers and Text Splitters in LlamaIndex

In LlamaIndex, node parsers and text splitters are often conflated, which can lead to some confusion for users. This conflation is not accidental but rather a design choice that simplifies the API and workflow (IMAO).

LlamaIndex Limitations

Disperse information and cross-node reasoning remains a challenge for RAG systems and frameworks. Therefore, inferring years of experience and skills proficiency from fragmented data are not possible.

Long Rank Dependencies

An educated guess can be provided to age and skills proficiency by calculating the number of years a candidate has worked. But this analysis depends on the ability to cross data between long distant sections and paragraphs.

1. Fragmentation of Context: The candidate’s work experience is splitted across varios nodes. This fragmentation prevents the model from drawing connections from disperse information.
2. Lack of Cross-Node Reasoning: LlamaIndex process nodes independently meaning it does not support reasoning across sections to infer relationships such calculating the candidate’ age or skill development over time.

Streamlit Integration

Streamlit makes the perfect front-end companion for data application apps. Given how easy it makes to provide an interface for data apps, we can iterate and prototype applications much faster than with any other tool in the market.

[embed]

Resume Insights Stack

Resume Insights Stack

Conclusion

This approach offers several advantages:

• Type-safe: The use of Pydanticensures that the output is always structured and validated.
• Scalable: This structure can handle any number of documents returned from the LLM
• Customizable: You can easily adjust the Pydanticmodel and/or the LLM to better fit your needs.

Resources

[embed]A Simple Guide to Structured Outputs - LlamaIndex A Guide to Building a Full-Stack LlamaIndex Web App with Delphicdocs.llamaindex.ai

[embed]Pydantic Program - LlamaIndex Tip The Pydantic Program is a lower-level abstraction for structured output extraction. The default way to perform…docs.llamaindex.ai

[embed]Migrating from ServiceContext to Settings - LlamaIndex Introduced in v0.10.0, there is a new global Settings object intended to replace the old ServiceContext configuration…docs.llamaindex.ai

[embed]Node Parser Usage Pattern - LlamaIndex Node parsers are a simple abstraction that take a list of documents, and chunk them into Node objects, such that each…docs.llamaindex.ai

[embed]What is the difference between LlamaIndex text splitters and node parsers? · run-llama llama_index… Aren't they both the same thing - given a document, chunks them down into nodes.github.com

[embed]ChatGPT ChatGPT helps you get answers, find inspiration and be more productive. It is free to use and easy to try. Just ask and…chatgpt.com