Predicting Electricity with the SKTime Python Package

NYISO, or the New York Independent System Operator, is responsible for managing the high-voltage power grid in the state of New York. As part of its operations, NYISO collects and publishes a variety of data sources related to the performance and operation of the power grid. We are going to use SKTime to predict it.

We’ll only need two packages:

NYISO Data via Microprediction

We obtain electricity data as follows:

In addition to real-time energy data, NYISO also publishes a variety of historical data sources related to the operation of the power grid. You could try to find those, but here I think it is simpler to just use the microprediction platform. There are many electricity timeseries there, such as this one.

Data published by NYISO includes data on energy consumption, energy prices, and power plant operations, among other metrics. NYISO also publishes a variety of reports and analysis on the performance of the power grid, which can be useful for understanding long-term trends and patterns in energy usage and production.

So in theory NYISO provides a rich set of data sources for understanding the operation and performance of the power grid in New York. These data sources can be a valuable resource for researchers, analysts, and other stakeholders in the energy industry, and can help to inform policy and decision-making related to energy production, consumption, and distribution.

Prediction

First let’s import the ThetaForecaster:

And we can fit like so:

Here is a step-by-step explanation of the code:

1. The code creates a pandas series y from the provided YS list of data points. The series index is a PeriodIndex with hourly frequency starting from January 2021. This creates a time series y with hourly frequency.
2. The last timestamp in the index of the time series y is obtained using last_t = y.index[-1].
3. Next, the code calculates the next timestamp next_t which is one hour after the last timestamp using next_t = last_t.to_timestamp() + datetime.timedelta(hours=1).
4. The code creates a ThetaForecaster model forecaster with a seasonality parameter sp of 1.
5. The time series y is then fitted to the forecaster using forecaster.fit(y).
6. The code sets the number of future time steps to predict as k = 3.
7. The code creates a ForecastingHorizon object fh which represents the future time steps to predict. Here, it is set to 3 hourly time steps from the next_t.
8. The forecaster is then used to predict the future values for the k time steps using y_pred = forecaster.predict(fh).

The ForecasterHorizon object is used to specify the time steps for which the predictions are to be made.

Rolling Predictions

Now, let’s make predictions in an online fashion:

The code iterates over the first 50 data points of the YS list and adds each point to the y_buffer. For each iteration, the code creates a pandas series yt with hourly frequency using the current values in y_buffer and adds it to the list of xs which will store the predicted values.The code creates a ForecastingHorizon object fh which represents the future time steps to predict. Here, it is set to k hourly time steps from the next_t.

The forecaster is then used to predict the future value for the next k time steps using x = forecaster.predict(fh). The predicted value x[0] is appended to the xs list.

And then we plot, naturally:

Aside: What is the Theta Method Really?

I recommend “Unmasking the Theta Method”, a research paper published by Professor Rob Hyndman and Dr. Anastasios Panagiotelis in 2018. The paper proposes a new method for estimating the parameters of the Theta model, which is a popular method for time series forecasting, and also puts it in context.

The Theta method is a variant of the exponential smoothing method that combines a linear trend component and a seasonal component to forecast future values of a time series. However, the existing methods for estimating the parameters of the Theta model have some limitations, such as the inability to estimate the variance of the errors and the assumption of equal variances across time.

The proposed method in the “Unmasking the Theta Method” paper addresses these limitations by estimating the parameters of the Theta model using a maximum likelihood approach. This allows for the estimation of the variance of the errors and the use of unequal variances across time. The authors also proposed a new method for selecting the optimal smoothing parameters in the Theta model.

Can you do better?

Microprediction.org offers prizes for predicting electricity consumption and production time series data. These prizes are aimed at encouraging the development of more accurate and efficient models for predicting energy usage, which can ultimately help to reduce waste and improve overall efficiency in the energy sector.

To participate, users can submit predictions for a variety of different time series, including hourly and sub-hourly data on energy usage and production from various locations around the world. The submissions are then evaluated based on their accuracy and efficiency, with cash prizes awarded to the top performers. By all means discuss in the microprediction slack.