AutoML for time series: definitely a good idea

Approaches for time series forecasting using AutoML and example of the forecast obtained in the automated way

AutoML for time series forecasting (image by author)

At the moment, Data Science has become a valuable part of the IT industry and provides helpful features for business. Data scientists collect and filter data. They train a large number of ML models, perform validation and choose the best one. And the main reason for doing all this activity is to make a product for business, which will be the most effective (bring more money). These activities can be presented as a machine learning workflow.

Meanwhile, some of the stages from year to year go deeply into automatization. Primarily, such automation touches the dullest parts of the workflow. So, it allows data scientists to spend their saved time on more exciting tasks.

http://memegenerator.net/

Let’s imagine that we have got a task: to build a machine learning model, then deploy it to a web application. This model must predict, for example, how much money the customer will pay in an online shop today (or whatever else). But before we even start train our model, there is a need to overcome several steps:

• collect data from various sources;
• perform preprocessing operations such as normalisation, encoding etc.;
• feature engineering, remove features or synthes new ones;
• perform filtering, e.g. remove outliers;

Machine learning pipelines Such multi-step sequences are called “pipelines”. Working with pipelines is harder than with single machine learning models due to their complex structure. Moreover, the probability of failure becomes higher for such multi-step constructions. To control and manage complicated systems, which comprise several ML operations, specific instruments are built — MLFlow, Apache AirFlow. They are similar to workflow management systems (WMS) for data science and machine learning world. The reason for you to use them is to simplify the management of ML pipelines.

http://memegenerator.net/

It is likely to be a much more ambitious task to generate pipelines in an automatic way. There are several frameworks, which can do that. Open-source libraries are TPOT, AutoGluon, MLJAR, or H2O. These frameworks are trying toя solve the task “generate pipeline, which will give the lowest error on the final forecast”. In the vast majority of cases, the structure of the pipeline is fixed and only hyperparameters are changing. But several frameworks are able to find pipeline structure on their own. This optimization task (finding a pipeline with variable structure) is usually solved using evolutionary algorithms, for example, TPOT and FEDOT frameworks. There are also proprietary SaaS products, such as DataRobot, GoogleAutoTables, and Amazon SageMaker.

As a rule, AutoML libraries and services effectively solve only widespread tasks: classification and regression on tabular data. Less often, AutoML supports text and image processing, time series forecasting tasks. In this article, we will not consider the pros and cons of AutoML libraries, but we will analyze their capabilities in time series forecasting.

Automatic machine learning for time series

Despite the fact that the time series forecasting task is quite a popular task in science and business, most of the open-source AutoML libraries do not provide instruments for generating pipelines for time series processing. There may be several reasons for this, one of which is the difficulty in adapting the current library functionality for predicting series without refactoring the tools for other tasks (classification and regression).

The point is that the processing of time series data differs from the usual set of actions when solving the regression problem. The differences begin with the partitioning of the original sample: for example, it does not make sense to mix the data in random order to validate the time series model. Features are formed differently for time series: as a rule, machine learning models are not trained on the original series, because it is required to transform it into another representation. It can be problematic to implement such constructions in an already existing AutoML project with its own features. Most likely, that is why developers often refuse from time series ( so that “not to chase two birds with one stone “) and concentrate on processing specific data types: only tabular, or only text.

Some researchers who decide to support time series prediction are limited to this type of data. A good open-source example is AutoTS framework. Such libraries usually use “classical” statistical models for forecasting, e.g. AR or ARIMA. “Inside” the framework, these models are tuned, and then the best one is selected (by error metric on validation sample), but no new models are generated here. Also, pmdarima library is based on the same logic.

Another way is to adapt the ready-made AutoML-tool to forecasting. Regression models are well suited to this. For example, this logic is used in H2O, where in its commercial version of the product developers provided such a possibility. However, judging by some examples of using the open-source version, the user will have to preprocess raw series manually. Such a reduced toolkit may not be enough for the complete work.

What functions would we like to have in AutoML-toolkit?

Although the task of predicting the behavior of a one-dimensional array looks quite trivial, there are a variety of tools that engineers would like to have when working with a time series. For instance:

• To be able to build interpretable model ensembles (e.g., so that the first model reproduces the high-frequency component of the time series, the second is able to simulate the low-frequency component, and a third one combines their predictions);
• To be able to adjust hyperparameters in time series pipelines;
• Use exogenous (auxiliary) time series;
• Apply specific preprocessing methods (from smoothing by moving average to Box-Cox transformation);
• Apply in-sample and out-of-sample forecasting;
• And what if the time series contain gaps, how to fix them?

It is quite a difficult task to take into account all the listed possibilities in one framework and not to be limited with time series only.

Such requirements for AutoML did not just appear by chance. We work in Natural Systems Simulations (NSS lab), and over the past few years, we have been involved in dozens of various research and business projects in machine learning, data-driven modeling, and optimization. In practice, we often have to solve similar problems, including spatio-temporal data prediction.

Who we are and why we worry about AutoML

Some time ago we got interested in AutoML research, and the team got fresh ideas for experiments. Eventually, the ideas formed into a fully-fledged open-source AutoML framework that we develop and maintain in the lab — FEDOT.

We also would like to automatically solve more than one problem within a single pipeline, i.e. a regression problem together with a time series prediction problem, or a regression and classification problem simultaneously. For example, in the task of credit scoring, i.e. binary classification of potential borrowers, you can use not only the questionnaire features but also the upcoming values of cash flow, which can be tried to predict as a time series. In our opinion, it would be very promising to obtain a baseline for such tasks automatically.

We were also interested in investigating methods for identifying pipelines of variable structure and using multicriteria optimization to optimize not only the modeling error but also the complexity of the solution found (it can be estimated, for example, through the total number of blocks in the pipelines).

We have obtained interesting results in some of these problems, and if interested, scientific papers on them can be found here and here. Below, we’ll focus on the problem of predicting time series and discuss how you can use AutoML in such tasks and FEDOT in particular.

Time series forecasting using FEDOT framework

Task which FEDOT can solve (image by author)

As you may know from our previous post, FEDOT framework can produce ML pipelines for various tasks in an automatic way using evolutionary algorithms. Below we demonstrate how time series of traffic volume can be forecasted by this AutoML tool on the example of FEDOT (version 0.3.0). This version can easily be installed by command

pip install fedot==0.3.0

Jupyter notebook with code and plots available via link.

We have time series with traffic volume, which shows traffic along Interstate 94 between Minneapolis and St Paul in Minnesota in passenger car units. The data contains hourly traffic volume values with 2 peaks per day: morning and evening. Also for that data it is typical to have a decrease in traffic on weekends.

Let’s imagine that we have to forecast 144 elements for traffic volume (we chose this value rather randomly, we didn’t want the forecast horizon to be too large or too short). It means that there is a need to find out how many passengers will pass on this road in the next six days.

Imports

# FEDOT api
from fedot.api.main import Fedot

# Tasks to solve
from fedot.core.repository.tasks import Task, TaskTypesEnum, TsForecastingParams

# Input data for fit and predict
from fedot.core.data.data import InputData

# Train and test split
from fedot.core.data.data import train_test_data_setup

Firstly, we need to define task and forecasting horizon

forecast_length = 144
task = Task(TaskTypesEnum.ts_forecasting,
            TsForecastingParams(forecast_length=forecast_length))

Prepare input data for model

# Load data from csv file and wrap it into InputData structure
input_data = InputData.from_csv_time_series(task, 'ts_traffic.csv', target_column='value')

# Divide into train and test
train_data, test_data = train_test_data_setup(input_data)

Launch model: fit it and make a prediction

# Init model for the time series forecasting
model = Fedot(problem='ts_forecasting',task_params=task.task_params)

# Run AutoML model design
chain = model.fit(features=train_data)

# Use model to obtain forecast
forecast = model.predict(features=test_data)

The obtained forecast can be seen in the graph below:

Forecast of AutoML FEDOT model (image by author)

It is clear from the plot that the obtained model is good at guessing patterns in the data and can approximate both intra-day fluctuations (morning and evening rush hours) and a decrease in traffic on weekends. And it takes only two minutes of computations to get such a result!

Let’s take a look at the produced pipeline:

Machine learning pipeline, which was obtained by AutoML framework FEDOT (image by author)

As can be seen from the plot, the resulting chain consists of two operations. Operation “lagged” provides lagged (surprisingly) transformation of time series. We will talk about it in our next post.

Conclusion

Today, we’ve covered several automated machine learning technologies. We discussed how to predict time series and identify forecasting models in automatic mode. We have checked the capabilities of the AutoML solution using a simple example.

In the following articles, we will take a closer look at all the difficulties in forecasting time series using AutoML. We will analyze in detail the capabilities of the FEDOT framework on real-world data.

A few useful links:

• Github repository with FEDOT framework
• Notebooks with examples

Mikhail Sarafanov, Pavel Vychuzhanin, and Nikolay Nikitin worked on the article.