Mastering Non-Linear Data: Why Splines Outperform Linear Models

Learn how to use piece-wise polynomials and knots to build more flexible, disciplined, and accurate machine learning models.

Splines can fit curve data | Image generated by AI. Google Gemini, 2026.

We all know that Linear models can be…well, stiff. Have you ever looked at a scatter plot and realized a straight line just isn’t going to cut it? We’ve all been there.

Real-world data is always challenging. Most of the time, feels like the exception is the rule. The data you get in your job is nothing like those beautiful linear datasets that we used during years of training in the academy.

For example, you’re looking at something like “Energy Demand vs. Temperature.” It’s not a line; it’s a curve. Usually, our first instinct is to reach for Polynomial Regression. But that’s a trap!

If you’ve ever seen a model curve go wild at the edges of your graph, you’ve witnessed the “Runge Phenomenon.” High-degree polynomials are like a toddler with a crayon, since they’re too flexible and have no discipline.

That’s why I am going to show you this option called Splines. They’re a neat solution: more flexible than a line, but far more disciplined than a polynomial.

Splines are mathematical functions defined by polynomials, and used to smooth a curve.

Instead of trying to fit one complex equation to your entire dataset, you break the data into segments at points called knots. Each segment gets its own simple polynomial, and they’re all stitched together so smoothly you can’t even see the seams.

The Problem with Polynomials

Imagine we have a non-linear trend, and we apply a polynomial x² or x³ to it. It looks okay locally, but then we look at the edges of your data, and the curve is going way off. According to Runge’s Phenomenon, high-degree polynomials are global have this problem where one weird data point at one end can pull the entire curve out of whack at the other end.

Example of low versus high-degree polynomials. Image by the author.

Why Splines are the “Just Right” Choice

Splines don’t try to fit one giant equation to everything. Instead, they divide your data into segments using points called knots. We have some advantages of using knots.

• Local Control: What happens in one segment stays in that segment.
• Smoothness: They use “B-splines” (Basis splines) to ensure that where segments meet, the curve is perfectly smooth.
• Stability: Unlike polynomials, they don’t go wild at the boundaries.

Enough talk, now let’s implement this solution.

Implementing it with Scikit-Learn

Scikit-Learn’s SplineTransformer is the go-to choice for this. It turns a single numeric feature into multiple basis features that a simple linear model can then use to learn complex, non-linear shapes.

Let’s import some modules.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import SplineTransformer
from sklearn.linear_model import Ridge
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import GridSearchCV

Next, we create some curved oscilating data.

# 1. Create some 'wiggly' synthetic data (e.g., seasonal sales)
rng = np.random.RandomState(42)
X = np.sort(rng.rand(100, 1) * 10, axis=0)
y = np.sin(X).ravel() + rng.normal(0, 0.1, X.shape[0])

# Plot the data
plt.figure(figsize=(12, 5))
plt.scatter(X, y, color='gray', alpha=0.5, label='Data')
plt.legend()
plt.title("Data")
plt.show()

Plot of the data

Ok. Now we will create a pipeline that runs the SplineTranformer with default sets, followed by a Ridge Regression.

# 2. Build a pipeline: Splines + Linear Model
# n_knots=5 (default) creates 4 segments; degree=3 makes it a cubic spline
model = make_pipeline(
    SplineTransformer(n_knots=5, degree=3), 
    Ridge(alpha=0.1)
    )

Next, we will tune the number of knots for our model.

# We tune 'n_knots' to find the best tune
param_grid = {'splinetransformer__n_knots': range(3, 12)}
grid = GridSearchCV(model, param_grid, cv=5)
grid.fit(X, y)

print(f"Best knot count: {grid.best_params_['splinetransformer__n_knots']}")

Best knot count: 8

Then, we retrain our model with the best knot count, predict, and plot the data.

# 2. Build the optimized Spline
model = make_pipeline(
    SplineTransformer(n_knots=8, degree=3), 
    Ridge(alpha=0.1)
    ).fit(X, y)

# 3. Predict and Visualize
y_plot = model.predict(X)

# Plot
plt.figure(figsize=(12, 5))
plt.scatter(X, y, color='gray', alpha=0.5, label='Data')
plt.plot(X, y_plot, color='teal', linewidth=3, label='Spline Model')
plt.plot(X, y_plot_10, color='purple', linewidth=2, label='Polynomial Fit (Degree 20)')
plt.legend()
plt.title("Splines: Flexible yet Disciplined")
plt.show()

Here is the result. With splines, we have better control and a smoother model, escaping the problem at the ends.

Comparison of a high-degree polynomial (degree=20) vs. splines. Image by the author.

We are comparing a polynomial model of degree=20 with the spline model. One can argue that lower degrees can do a much better modeling of this data, and they would be correct. I have tested up to the 13th degree, and it fits well with this dataset.

However, that is exactly the point of this article. When the model is not fitting too well to the data, and we need to keep increasing the polynomial degree, we certainly will fall into the wild edges problem.

Real-Life Applications

Where would you actually use this in business?

• Time-Series Cycles: Use extrapolation='periodic' for features like "hour of day" or "month of year." It ensures the model knows that 11:59 PM is right next to 12:01 AM. With this argument, we tell the SplineTransformer that the end of our cycle (hour 23) should wrap around and meet the beginning (hour 0). Thus, the spline ensures that the slope and value at the end of the day perfectly match the start of the next day.
• Dose-Response in Medicine: Modeling how a drug affects a patient. Usually not a straight line!
• Income vs. Experience: Salary often grows quickly early on and then plateaus; splines capture this “bend” perfectly.

Before You Go

We’ve covered a lot here, from why polynomials can be a “wild” choice to how periodic splines solve the midnight gap. Here’s a quick wrap-up to keep in your back pocket:

• The Golden Rule: Use Splines when a straight line is too simple, but a high-degree polynomial starts oscillating and overfitting.
• Knots are Key: Knots are the “joints” of your model. Finding the right number via GridSearchCV is the difference between a smooth curve and a jagged mess.
• Periodic Power: For any feature that cycles (hours, days, months), use extrapolation='periodic'. It ensures the model understands that the end of the cycle flows perfectly back into the beginning.
• Feature Engineering > Complex Models: Often, a simple Ridge regression combined with SplineTransformer will outperform a complex "Black Box" model while remaining much easier to explain to your boss.

Summary Splines vs other models | Image generated by AI. google Gemini, 2026.

If you liked this content, find more about my work and my contacts on my website.

[embed]Gustavo R Santos I am a Data Scientist specializing in data analysis, machine learning, and visualization using Python, Spark, SQL, and…gustavorsantos.me

GitHub Repository

Here is the complete code of this exercise, and a couple of extras.

[embed]Studying/Python/sklearn/SplineTransformer.ipynb at master · gurezende/Studying This is a repository with my tests and studies of new packages - Studying/Python/sklearn/SplineTransformer.ipynb at…github.com

References

[embed]SplineTransformer Gallery examples: Time-related feature engineering Plot classification probability Visualizing the probabilistic…scikit-learn.org

[embed]Runge's phenomenon - Wikipedia In the mathematical field of numerical analysis, Runge's phenomenon (German: ) is a problem of oscillation at the edges…en.wikipedia.org