Neural Prophet

A simple time series forecasting framework

Oskar Triebe
Stanford University
Oct 12th, 2021
A time series is
a sequence of data points
that occur in successive order
over some period of time.
 Prophet

When to use NeuralProphet

Task: Forecasting. Predict future of observed variable.

Data: 100 to millions of samples. Unidistant, real-valued.

Dynamics: Must: Future values depend on past values.

Ideal: Seasonal, trended, repeating events, correlated variables.

Applications: Human behavior, energy, traffic, sales, environment, server load, ...
Overview What is

Motivation
The Neural & Prophet
Model Components
Model Use
Example
Outlook
 Motivation

Time series forecasting is messy.

We need hybrid models to bridge the gap.
Traditional Methods Deep Learning

LSTM
(S)ARIMA(X) Seasonal + Trend
NeuralProphet
Decomposition Prophet AR-Net
(V)ARMA(X) Transformer
DeepAR
Exponential ES-RNN
GARCH Smoothing
Holt-Winters N-BEATS

Gaussian WaveNet
(S)Naïve Process
Dynamic Linear
HMM
Models
Causal
(T)BATS
Convolutions
Other ML
 Motivation

Chasm Bridge
Time series - applied ML NeuralProphet

Need expertise in both Abstracts time series and

domains ML knowledge
 Prophet

Facebook Prophet is the most used

forecasting package since 2017.

Quick from data to predictions.

Gentle learning curve.

Taylor, S. J., & Letham, B. (2017).
Forecasting at scale, PeerJ.
Customizable. https://peerj.com/preprints/3190/
Prophet is an interpretable and decomposable model. Prophet

Trend

Events

Seasonality

Regressors
X(t)
NeuralProphet is more than the Neural evolution of Prophet. Prophet

Prophet has three major shortcomings: NeuralProphet solves these:

1. Missing local context for predictions 1. Support for auto-regression and covariates.

2. Acceptable forecast accuracy 2. Hybrid model (linear <> Neural Network)

3. Framework is difficult to extend (Stan) 3. Python package based on PyTorch using

standard deep learning methods.
 Model

Current Model Components

S Seasonality _ Sparsity / Regularization

T Trend NN Nonlinear (deep) layers
E/H Events / Holidays {} Global Modelling
X Regressors ? Uncertainty
AR Autoregression
Cov Covariates
Trend & Seasonality are identical to Prophet. Model

Piecewise linear trend Seasonality

● N changepoints ● N Fourier terms
● Segment-wise independent ● Automatic yearly, weekly, daily
● Automatic changepoint detection ● Optional multiplicative mode
● Optional logistic growth
Events, Holidays and Regressors are identical to Prophet Model

Events / Holidays (Future-Known) Regressors

● Automatic for given country ● Single weight
● Various user-defined formats ● Real-valued regressor
● Optional multiplicative mode ● Optional multiplicative mode
We model Covariates and Auto-Regression as AR-Nets. Model

Auto-Regression (Lagged) Covariates

● By default AR-Net(0) ● By default AR-Net(0) with y as target)
● Depth customizable AR-Net(n) ● Depth customizable AR-Net(n)
● Optional auto-AR via sparsification ● Optional auto-lags via sparsification

AR-Net(0) AR-Net(n)
Interpretable Non-linear modeling
 Use

A user-friendly Python package

Gentle learning curve.

Get results first, learn, and improve.

NeuralProphet has smart defaults.

Advanced features are optional.

Preprocessing happens mostly behind the scenes. Use

Missing Data is is automatically filled in:

1. bi-directional linear interpolation
2. centred rolling average

Data is automatically normalized:

Hyperparameters have smart defaults. Use

Loss Function is Huber loss,

unless user-defined.

The learning rate is approximated

with a learning-rate range test.

Batch size and epochs are approximated

from the dataset size.

We use one-cycle policy

with AdamW as optimizer for simplicity.
There is a utility for that. Use

Visualize:
- Plot past and future predictions
- Decompose forecast components
- Interpret model parameters
- Plot most recent prediction
- Inspect a particular forecast horizon

Other:
- Simple Split, cross validation, double cross validation
- Control logger verbosity
- Make fit reproducible
- ...
 Example

Examples

Yosemite Temperature prediction with Trend, Seasonality and Auto-Regression:

- 1 step ahead
- 36 steps ahead (with extras: AR Sparsity, 24 hours ahead)
Yosemite Temperature - Predict next observation. Example: Yos1

Dataset:
Observed temperature in Yosemite Valley,
measured every 5 min over two months.
Visualize model parameters with one line. Example: Yos1
Yosemite Temperature - Predict next 36 observations. Example: Yos36

Dataset:
Observed temperature in Yosemite Valley,
measured every 5 min over two months.
Analyze a specific forecast horizon. Sparsify. Example: Yos36
Predict. See how new data impacts the forecast. Example: Yos36
Want to forecast a larger horizon? Example: Yos24
 Outlook

Outlook

We are extending the framework to suit more forecasting needs.

Pull Request pending:

- Global modelling
- Quantile estimation (Uncertainty Interval)
- Classification
- Better documentation
We are working hard to extend the framework. Join us! Outlook

Extensions [upcoming] Improvements [upcoming]

● Anomaly Prediction ● Infuse Deep Learning
& Semi-Supervised Learning
● Faster Training Time
● Hierarchical Forecasting & GPU support
& Global Modelling
● Improved UI
● Attention: Automatic Multimodality
● Diagnostic Tools for Deep Dives
& Dynamic Feature Importance

● Quantifiable and Explainable Uncertainty

Anything trainable by gradient descent can be added as module

Upgrade Prophet. Outlook
THANK YOU, dear collaborator, supporter and advisor! Team

Gonzague Henri

Ram Rajagopal

Christoph Bergmeir

Alessandro Panella
Oskar Triebe Hansika Hewamalage Polina Pilyugina
Evgeny Burnaev

Caner Komurlu

Italo Lima

Abishek Sriramulu
Lluvia Ochoa Nikolay Laptev Mateus De Castro Ribeiro
Bernhard Hausleitner
 Github: github.com/ourownstory/neural_prophet
Docs: Neuralprophet.com

Connect with me: linkedin.com/in/oskartriebe/

triebe [at] stanford.edu

Thank You
Appendix: AR-Net
 Model: AR-Net

AR-Net is a Neural Network

for autoregressive time series.

AR-Net(0) AR-Net(n)
Interpretable Stronger modeling ability
Skip estimating the AR process order. Model: AR-Net

Trained with SGD (Adam)

Automatic AR-lag selection, yet faster. Model: AR-Net

Automatic Sparsity Quadratically faster

Sparse AR-Net surpasses Classic AR and scales to large orders. Model: AR-Net

Closeness to true coefficients

MSE loss on forecast target

Appendix: Model Use Details
Few lines from data to forecast. Hands-on

skip data preparation: Just create a DataFrame

with desired columns
Validate model performance without extra lines. Hands-on
Trend Hands-on
Seasonality Hands-on
Events can be added in different forms. Hands-on
Auto-Regression Hands-on

Autoregression,
here with sparsity

Recommended example notebook:

https://github.com/ourownstory/neural_prophet/blob/master/example_notebooks/autoregression_yosemite_temps.ipynb
Focus on a specific forecast. Hands-on
Lagged Covariates & Future Regressors Hands-on

Covariates and regressors are

similarly added

Make it non-linear.