Introduction to Fundamental Operations of Polars Dataframe

Photo by Markus Spiske on Unsplash

Over the past decade, Pandas has long been the “king” of Python data analysis. But by 2025, a rising challenger named Polars has quickly emerged. More and more data scientists and engineers working on the front lines are shifting their attention from Pandas to this younger, faster tool.

Polars is a high-performance DataFrame library, often regarded as an alternative to Pandas, with significant performance advantages when handling large-scale data.

In this post we will introduce the fundamental operations of the Polars library’s DataFrame, including creating DataFrames, selecting and filtering columns, sorting, and slicing data.

Creating a DataFrame from a Dictionary

First, import the polars library and abbreviate it as pl. Then, create a dictionary containing five columns: name, age, company, position, and city. Use pl.DataFrame() to convert the dictionary into a DataFrame. The printed result shows the shape of the data (3 rows and 5 columns) and the data type of each column.

The code is as follows:

import polars as pl

# Create a dictionary where the keys are column names and the values are column data
data_dict = {
    'name': ['Alex Johnson', 'Emily Carter', 'Michael Brown'],
    'age': [28, 32, 25],
    'company': ['TechNova', 'DataWorks', 'InsightLab'],
    'position': ['Data Analyst', 'Software Engineer', 'Project Manager'],
    'city': ['New York', 'San Francisco', 'Chicago']
}

# Convert the dictionary into a DataFrame
df = pl.DataFrame(data_dict)
print(df)

Then after code execution:

shape: (3, 5)
┌───────────────┬─────┬────────────┬────────────────────┬───────────────┐
│ name          │ age │ company    │ position           │ city          │
│ ---           │ --- │ ---        │ ---                │ ---           │
│ str           │ i64 │ str        │ str                │ str           │
├───────────────┼─────┼────────────┼────────────────────┼───────────────┤
│ Alex Johnson  │ 28  │ TechNova   │ Data Analyst       │ New York      │
│ Emily Carter  │ 32  │ DataWorks  │ Software Engineer  │ San Francisco │
│ Michael Brown │ 25  │ InsightLab │ Project Manager    │ Chicago       │
└───────────────┴─────┴────────────┴────────────────────┴───────────────┘

How to Create a DataFrame from a CSV File？

# Suppose we have a file named data.csv
# The content is as follows:
# name,age,company,position,city
# Alex Johnson,28,TechNova,Data Analyst,New York
# Emily Carter,32,DataWorks,Software Engineer,San Francisco
# Michael Brown,25,InsightLab,Project Manager,Chicago

df_csv = pl.read_csv('data.csv')
print(df_csv.head(2))  # View the first two rows

Then after code execution:

shape: (2, 5)
┌───────────────┬─────┬────────────┬───────────────────┬───────────────┐
│ name          │ age │ company    │ position          │ city          │
│ ---           │ --- │ ---        │ ---               │ ---           │
│ str           │ i64 │ str        │ str               │ str           │
├───────────────┼─────┼────────────┼───────────────────┼───────────────┤
│ Alex Johnson  │ 28  │ TechNova   │ Data Analyst      │ New York      │
│ Emily Carter  │ 32  │ DataWorks  │ Software Engineer │ San Francisco │
└───────────────┴─────┴────────────┴───────────────────┴───────────────┘

In this code :

• Use pl.read_csv() to read a CSV file.
• The head(2) method displays only the first two rows of the data.

How to Select and Filter Columns in a DataFrame？

Select columns from a Dataframe:

import polars as pl

# Create a dictionary where the keys are column names and the values are column data
data_dict = {
    'name': ['Alex Johnson', 'Emily Carter', 'Michael Brown'],
    'age': [28, 32, 25],
    'company': ['TechNova', 'DataWorks', 'InsightLab'],
    'position': ['Data Analyst', 'Software Engineer', 'Project Manager'],
    'city': ['New York', 'San Francisco', 'Chicago']
}

# Convert the dictionary into a DataFrame
df = pl.DataFrame(data_dict)

# Select a single column
names = df['name']
print(names)

# Select multiple columns
subset = df.select(['name', 'age'])
print(subset)

Output for selecting the single column name:

shape: (3,)
Series: 'name' [str]
[
 "Alex Johnson"
 "Emily Carter"
 "Michael Brown"
]

Output for selecting multiple columns name and age:

shape: (3, 2)
┌───────────────┬─────┐
│ name          │ age │
│ ---           │ --- │
│ str           │ i64 │
├───────────────┼─────┤
│ Alex Johnson  │ 28  │
│ Emily Carter  │ 32  │
│ Michael Brown │ 25  │
└───────────────┴─────┘

In the above code:

• Use df[‘column_name’] to select a single column.
• Use the select() method to select multiple columns.

Filtering columns using regular expressions:

First, use with_columns() to add a new column id. *pl.col(‘^n.$’)** uses a regular expression to match all column names starting with n. In this case, only the name column is matched.

The code is as follows:

# Add a new column
df = df.with_columns(pl.lit(1).alias('id'))

# Use a regular expression to select all columns that start with 'n'
filtered_cols = df.select(pl.col('^n.*$'))
print(filtered_cols)

Code output is:

shape: (3, 1)
┌───────────────┐
│ name          │
│ ---           │
│ str           │
├───────────────┤
│ Alex Johnson  │
│ Emily Carter  │
│ Michael Brown │
└───────────────┘

Data Sorting and Slicing

Data sorting:

# Sort the DataFrame by the 'age' column in descending order
df_sorted = df.sort('age', descending=True)
print(df_sorted)

Code output is：

shape: (3, 5)
┌───────────────┬─────┬────────────┬───────────────────┬───────────────┐
│ name          │ age │ company    │ position          │ city          │
│ ---           │ --- │ ---        │ ---               │ ---           │
│ str           │ i64 │ str        │ str               │ str           │
├───────────────┼─────┼────────────┼───────────────────┼───────────────┤
│ Emily Carter  │ 32  │ DataWorks  │ Software Engineer │ San Francisco │
│ Alex Johnson  │ 28  │ TechNova   │ Data Analyst      │ New York      │
│ Michael Brown │ 25  │ InsightLab │ Project Manager   │ Chicago       │
└───────────────┴─────┴────────────┴───────────────────┴───────────────┘

The sort() method sorts the DataFrame, and descending=True indicates descending order.

Data slicing:

# Get the first two rows
print(df.head(2))

# Get the last row
print(df.tail(1))

Code output is:

shape: (2, 5)
┌───────────────┬─────┬────────────┬───────────────────┬───────────────┐
│ name          │ age │ company    │ position          │ city          │
│ ---           │ --- │ ---        │ ---               │ ---           │
│ str           │ i64 │ str        │ str               │ str           │
├───────────────┼─────┼────────────┼───────────────────┼───────────────┤
│ Alex Johnson  │ 28  │ TechNova   │ Data Analyst      │ New York      │
│ Emily Carter  │ 32  │ DataWorks  │ Software Engineer │ San Francisco │
└───────────────┴─────┴────────────┴───────────────────┴───────────────┘

shape: (1, 5)
┌───────────────┬─────┬────────────┬────────────────────┬───────────────┐
│ name          │ age │ company    │ position           │ city          │
│ ---           │ --- │ ---        │ ---                │ ---           │
│ str           │ i64 │ str        │ str                │ str           │
├───────────────┼─────┼────────────┼────────────────────┼───────────────┤
│ Michael Brown │ 25  │ InsightLab │ Project Manager    │ Chicago       │
└───────────────┴─────┴────────────┴────────────────────┴───────────────┘

head(n) retrieves the first n rows of data, and tail(n) retrieves the last n rows of data.

slice() method:

# Take 2 rows starting from the first row
df_slice = df.slice(1, 2)  # Equivalent to df[1:3]

Explicitly specifies the starting position and number of rows (offset = starting index, length = number of rows*), and compatible with LazyFrame.

Conditional filtering:

Find all users whose age is greater than 25.

older_users = df.filter(pl.col('age') > 25)
print(older_users)

Output:

shape: (2, 5)
┌───────────────┬─────┬────────────┬───────────────────┬───────────────┐
│ name          │ age │ company    │ position          │ city          │
│ ---           │ --- │ ---        │ ---               │ ---           │
│ str           │ i64 │ str        │ str               │ str           │
├───────────────┼─────┼────────────┼───────────────────┼───────────────┤
│ Alex Johnson  │ 28  │ TechNova   │ Data Analyst      │ New York      │
│ Emily Carter  │ 32  │ DataWorks  │ Software Engineer │ San Francisco │
└───────────────┴─────┴────────────┴───────────────────┴───────────────┘

Code explanation:

pl.col(‘age’): Selects the column named age.

25: Applies a condition to filter rows where the age is greater than 25.

filter(): Keeps only the rows that meet the condition and returns a new DataFrame.

Summary

Polars is a high-performance DataFrame library designed for working with structured data.

Built from the ground up in Rust, Polars is tightly integrated with system-level performance. Its vectorized and columnar processing enables cache-efficient algorithms and high-speed computation on modern processors.

If you’re already familiar with Pandas DataFrames, you’ll find Polars easy to use — it’s arguably the most promising library to replace Pandas. Why not give it a try in your next data analysis project?