How to Perform Rolling Statistics with Pandas for Time Series Data Analysis

Photo by Chris Liverani on Unsplash

In data analysis and processing, rolling statistics are a common and valuable technique — especially when working with time series data.

Rolling statistics calculate metrics within a moving window, allowing us to observe trends and changes over time. The Pandas library provides powerful window functions that make it easy to perform various rolling statistical operations.

In this post we will explain how to use window functions in Python Pandas for rolling statistics, covering core concepts, function usage, and practical code examples to help you understand and apply these techniques effectively.

Introduction to Window Functions

Window functions are functions applied over a subset of the data, known as a “window.” These functions compute statistics within a moving window and return the result. In Pandas, key window functions include:

rolling(): for sliding window calculations

expanding(): for cumulative calculations

ewm(): for exponentially weighted calculations

Window functions are useful in many scenarios, especially in the following cases:

Smoothing data: Techniques like moving averages help eliminate short-term fluctuations and highlight long-term trends.

Capturing local patterns: Rolling windows can reveal local characteristics of data over different time periods.

Real-time data processing: When handling real-time data, window functions allow us to calculate dynamic statistics within a moving window.

Types of Window Functions in Pandas

Pandas offers three main types of window functions, each suited for different use cases:

rolling(): Performs calculations over a fixed-size sliding window.

expanding(): Performs cumulative calculations as the window expands with more data.

ewm(): Computes exponentially weighted moving averages, giving more importance to recent observations.

1. Sliding Window Function: rolling()

The rolling() function is one of the most commonly used window functions. It allows us to compute statistics — such as moving averages or moving standard deviations — within a fixed-size rolling window.

The example below demonstrates how to use the rolling() function to calculate the moving average of a time series:

import pandas as pd
import numpy as np

# Create new time series data (e.g., February 2023)
data = {
    'date': pd.date_range(start='2025-02-01', periods=10, freq='D'),
    'value': [12, 18, 21, 17, 25, 29, 33, 28, 24, 20]
}
df = pd.DataFrame(data)

# Set the date column as the index
df.set_index('date', inplace=True)

# Calculate the moving average with a window size of 4
df['rolling_mean'] = df['value'].rolling(window=4).mean()

print(df)

In this example, a time series dataset containing dates and values was created, and the rolling() function was used to calculate the moving average with a window size of 4. The result is as follows:

            value  rolling_mean
date                           
2025-02-01     12           NaN
2025-02-02     18           NaN
2025-02-03     21           NaN
2025-02-04     17         17.00
2025-02-05     25         20.25
2025-02-06     29         23.00
2025-02-07     33         26.00
2025-02-08     28         28.75
2025-02-09     24         28.50
2025-02-10     20         26.25

In the output above, the first three rows return NaN because there are not enough data points within the window. Starting from the fourth row, the average of each consecutive group of four values is calculated.

The moving standard deviation is another commonly used rolling statistic that helps analyze data volatility over different time periods.

import pandas as pd

# use the above time series data sample
df['rolling_std'] = df['value'].rolling(window=4).std()

print(df)

In this example, a new column rolling_std is added to calculate the moving standard deviation with a window size of 4. The result is as follows:

            value  rolling_std
date                           
2025-02-01     12          NaN
2025-02-02     18          NaN
2025-02-03     21          NaN
2025-02-04     17       3.7081
2025-02-05     25       3.4034
2025-02-06     29       4.0415
2025-02-07     33       3.5590
2025-02-08     28       2.2174
2025-02-09     24       3.5590
2025-02-10     20       3.5590

As shown, starting from the fourth row, the rolling_std column displays the standard deviation within each window, which helps in understanding the degree of fluctuation in the data.

2. Using expanding() for Cumulative Statistics

In addition to rolling(), Pandas also provides the expanding() function, which is used to compute cumulative statistics. As the amount of data increases, the window used by expanding() gradually expands until it includes all available data.

import pandas as pd

# use the above time series data sample
df['expanding_mean'] = df['value'].expanding().mean()

print(df)

In this example, the expanding() function is used to calculate the cumulative average. The output is as follows:

            value  expanding_mean
date                              
2025-02-01     12           12.00
2025-02-02     18           15.00
2025-02-03     21           17.00
2025-02-04     17           17.00
2025-02-05     25           18.60
2025-02-06     29           20.33
2025-02-07     33           22.14
2025-02-08     28           22.88
2025-02-09     24           23.00
2025-02-10     20           22.70

As shown, the expanding_mean column displays the cumulative average of all data from the first row up to the current row.

3. Using ewm() for Exponentially Weighted Calculations

The ewm() function is used to calculate exponentially weighted moving statistics, which assign higher weights to more recent data points during computation. This approach is especially useful in time series analysis because it can more sensitively reflect recent trends in the data.

import pandas as pd

# use the above time series data sample
df['ewm_mean'] = df['value'].ewm(span=4, adjust=False).mean()

print(df)

In this example, the exponentially weighted moving average is calculated using ewm(). The result is as follows:

            value  ewm_mean
date                        
2025-02-01     12  12.0000
2025-02-02     18  14.8000
2025-02-03     21  17.7200
2025-02-04     17  17.2320
2025-02-05     25  20.5392
2025-02-06     29  24.3235
2025-02-07     33  28.5941
2025-02-08     28  28.3565
2025-02-09     24  26.6139
2025-02-10     20  23.6483

In the output, the ewm_mean column shows the exponentially weighted moving average for each row, where more recent data have a greater influence on the result.

Handling Missing Values Using Window Functions

In real-world data processing, missing values (NaN) are often encountered. Pandas’ window functions can automatically handle these missing values, but sometimes special treatment is required.

import pandas as pd
import numpy as np

# Create data containing missing values
data = {
    'date': pd.date_range(start='2025-06-01', periods=10, freq='D'),
    'value': [20, np.nan, 25, np.nan, 35, 45, np.nan, 50, 40, 30]
}
df = pd.DataFrame(data)
df.set_index('date', inplace=True)

# Calculate rolling mean while skipping missing values
df['rolling_mean'] = df['value'].rolling(window=3, min_periods=1).mean()

print(df)

In this example, the data contains missing values. By setting min_periods=1, the calculation is performed as long as there is at least one valid data point within the window. The output is as follows:

            value  rolling_mean
date                           
2025-06-01   20.0    20.000000
2025-06-02    NaN    20.000000
2025-06-03   25.0    22.500000
2025-06-04    NaN    25.000000
2025-06-05   35.0    30.000000
2025-06-06   45.0    40.000000
2025-06-07    NaN    40.000000
2025-06-08   50.0    47.500000
2025-06-09   40.0    45.000000
2025-06-10   30.0    40.000000

As shown, the rolling mean can still be calculated correctly even in the presence of missing values.

Advanced Usage of Rolling Windows

Beyond simple statistical calculations, Pandas’ rolling() function also supports applying custom functions within the sliding window.

import pandas as pd

# Create new time series data (e.g., February 2023)
data = {
    'date': pd.date_range(start='2025-02-01', periods=10, freq='D'),
    'value': [12, 18, 21, 17, 25, 29, 33, 28, 24, 20]
}
df = pd.DataFrame(data)

# Set the date column as the index
df.set_index('date', inplace=True)

df['custom_rolling'] = df['value'].rolling(window=3).apply(lambda x: x.max() - x.min())

print(df)

In this example, the difference between the maximum and minimum values within each window is calculated. The result is as follows:

            value  custom_rolling
date                            
2025-02-01     12             NaN
2025-02-02     18             NaN
2025-02-03     21             9.0
2025-02-04     17             4.0
2025-02-05     25             8.0
2025-02-06     29             12.0
2025-02-07     33             4.0
2025-02-08     28             5.0
2025-02-09     24             9.0
2025-02-10     20             8.0

Custom rolling window functions allow us to perform more complex computations within the window, greatly enhancing the flexibility and functionality of Pandas.

In this post we delved into how to use window functions in Python’s Pandas library to perform rolling statistics, which are particularly important operations in data analysis.

By thoroughly introducing key functions such as rolling(), expanding(), and ewm(), the post demonstrated how to calculate moving averages, moving standard deviations, and handle trend changes in time series data.

With code examples, it also explained how to customize rolling window functions to meet more complex computational needs. Whether for time series analysis or smoothing data fluctuations, Pandas’ window functions provide powerful support.