ARIMA MODELS FOR TIME

SERIES DATA
Autoregressive Moving Average
Models
➢p and q in an ARMA model are determined from
the patterns of the sample autocorrelations and
partial autocorrelations and the values of the
model selection criteria that are discussed in a
later section
➢In practice, the values of p and q each rarely
exceed 2.
 ARIMA models for time series
data
➢Autoregressive (AR) models can be coupled with
moving average (MA) models to form a general
and useful class of time series models called
Autoregressive Moving Average (ARMA) models.
➢These can be used when the data are
stationary.

11/30/2020 1
 ARIMA models for time series
data
➢This class of models can be extended to non-
stationary series by allowing the differencing of
the data series.
➢These are called Autoregressive Integrated
Moving Average(ARIMA) models.
➢There are a large variety of ARIMA models.

11/30/2020 1
 ARIMA models for time series
data
➢The general non-seasonal model is known as
ARIMA (p, d, q):
▪ p is the number of autoregressive terms.
▪ d is the number of differences.
▪ q is the number of moving average terms.

11/30/2020 1
 ARIMA models for time series
data
➢A white noise model is classified as ARIMA (0,
0, 0)
▪ No AR part since yt does not depend on yt-1.
▪ There is no differencing involved.
▪ No MA part since yt does not depend on et-1.

11/30/2020 1
 ARIMA models for time series
data
➢A random walk model is classified as ARIMA (0,
1, 0)
▪ There is no AR part.
▪ There is no MA part.
▪ There is one difference.

11/30/2020 1
 ARIMA models for time series
data
➢Note that if any of p, d, or q are equal to zero,
the model can be written in a shorthand
notation by dropping the unused part.
➢Example
▪ ARIMA(2, 0, 0) = AR(2)
▪ ARIMA (1, 0, 1) = ARMA(1, 1)

11/30/2020 1
 An autoregressive model of order
one AR(1)
➢The basic form of an ARIMA (1, 0, 0) or AR(1) is:

▪ 𝑌𝑡 = 𝜙0 + 𝜙1 𝑌𝑡−1 + ℰ𝑡
▪ Observation yt depends on yt-1.
▪ The value of autoregressive coefficient 1 is between
–1 and 1.

11/30/2020 1
 An autoregressive model of order
one
➢The time plot of an AR(1) model varies with the
parameter 1..
▪ When 1= 0, yt is equivalent to a white noise series.
▪ When 1= 1, yt is equivalent to a random walk series
▪ For negative values of 1, the series tends to oscillate
between positive and negative values.
➢The following slides show the time series, ACF
and PACF plot for an ARIMA(1, 0, 0) time series
data.

11/30/2020 1
 Mixtures ARMA models
➢Basic elements of AR and MA models can be
combined to produce a great variety of models.
➢The following is the combination of MA(1) and AR(1)
models
𝑌𝑡 = 𝜙0 + 𝜙1 𝑌𝑡−1 + ℰ𝑡 − 𝜔1 𝜀𝑡−1
➢This is model called ARMA(1, 1) or
➢ ARIMA (1, 0, 1)
➢The series is assumed stationary in the mean and
in the variance.
11/30/2020 1
 Mixtures ARIMA models
➢If non-stationarity is added to a mixed ARMA
model, then the general ARIMA (p, d, q) is
obtained.
➢The equation for the simplest ARIMA (1, 1, 1) is
given below.
𝑌𝑡 = 𝜙0 + 𝜙1 𝑌𝑡−1 + 𝜙2 𝑌𝑡−2 + ℰ𝑡 − 𝜔1 𝜀𝑡−1

11/30/2020 1
 Mixtures ARIMA models
➢The general ARIMA (p, d, q) model gives a
tremendous variety of patterns in the ACF and
PACF, so it is not practical to state rules for
identifying general ARIMA models.
➢In practice, it is seldom necessary to deal with
values p, d, or q that are larger than 0, 1, or 2.
➢It is remarkable that such a small range of
values for p, d, or q can cover such a large range
of practical forecasting situations.

11/30/2020 1
 Seasonality and ARIMA models
➢The ARIMA models can be extended to handle
seasonal components of a data series.
➢The general shorthand notation is
➢ ARIMA (p, d, q)(P, D, Q)s
▪ Where s is the number of periods per season.

11/30/2020 1
 Seasonality and ARIMA models
➢The general ARIMA(1,1,1)(1,1,1)4 can be
written as

yt = (1 + 1 ) yt −1 − 1 yt −2 + (1 + 1 ) yt −4 − (1 + 1 + 1 + 11 ) yt −6
− 1 yt −8 + (1 + 11 ) yt −9 − 11 yt −10 + et − 1et −1 − 1et −4 + 11et −5
➢Once the coefficients 1, Ф1, θ1, and 1 have
been estimated from the data, the above
equation can be used for forecasting.
➢

11/30/2020 1
 Seasonality and ARIMA models
➢The seasonal lags of the ACF and PACF plots
show the seasonal parts of an AR or MA model.
➢Examples:
▪ Seasonal MA model:
▪ ARIMA(0,0,0)(0,0,1)12
▪ will show a spike at lag 12 in the ACF but no other significant spikes.
▪ The PACF will show exponential decay in the seasonal lags i.e. at lags 12, 24, 36,…

11/30/2020 1
 Seasonality and ARIMA models
➢Seasonal AR model:
▪ ARIMA(0,0,0)(1,0,0)12
▪ will show exponential decay in seasonal lags of the ACF.
▪ Single significant spike at lag 12 in the PACF.

11/30/2020 1
 Implementing the model –
Building Strategy
➢The Box –Jenkins approach uses an iterative
model-building strategy that consist of
▪ Selecting an initial model (model identification)
▪ Estimating the model coefficients (parameter
estimation)
▪ Analyzing the residuals (model checking)

11/30/2020 1
 Implementing the model –
Building Strategy
➢If necessary, the initial model is modified and
the process is repeated until the residual
indicate no further modification is necessary. At
this point the fitted model can be used for
forecasting.

11/30/2020 1
 Model identification
➢The following approach outlines an approach to
select an appropriate model among a large
variety of ARIMA models possible.
▪ Plot the data
▪ Identify any unusual observations
▪ If necessary, transform the data to stabilize the variance

11/30/2020 1
 Model identification
➢Check the time series plot, ACF, PACF of the
data (possibly transformed) for stationarity.
➢IF
▪ Time plot shows the data scattered horizontally
around a constant mean
▪ ACF and PACF to or near zero quickly
➢Then, the data are stationary.

11/30/2020 1
 Model identification
➢Use differencing to transform the data into a
stationary series
▪ For no-seasonal data take first differences
▪ For seasonal data take seasonal differences
➢Check the plots again if they appear non-
stationary, take the differences of the
differenced data.

11/30/2020 1
 Model identification
➢When the stationarity has been achieved, check
the ACF and PACF plots for any pattern
remaining.
➢There are three possibilities
▪ AR or MA models
▪ No significant ACF after time lag q indicates MA(q) may be
appropriate.
▪ No significant PACF after time lag p indicates that AR(p) may
be appropriate.

11/30/2020 1
 Model identification
➢Seasonality is present if ACF and/or PACF at the
seasonal lags are large and significant.
➢If no clear MA or AR model is suggested, a
mixture model may be appropriate.

11/30/2020 1
 Model identification
➢Example
▪ Non seasonal time series data.
▪ The following example looks at the number of users logged
onto an internet server over a 100 minutes period.
▪ The time plot, ACF and PACF is reported in the following three
slides.

11/30/2020 1
 Model identification

Time Series Plot of Number of Users

240

220

200
Number of Users

180

160

140

120

100

80
1 10 20 30 40 50 60 70 80 90 100
Minutes

11/30/2020 1
 Model identification

Autocorrelation Function for Number of Users

(with 5% significance limits for the autocorrelations)

1.0
0.8
0.6
0.4
Autocorrelation

0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

2 4 6 8 10 12 14 16 18 20
Lag

11/30/2020 1
 Model identification

Partial Autocorrelation Function for Number of Users

(with 5% significance limits for the partial autocorrelations)

1.0
0.8
0.6
Partial Autocorrelation

0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

2 4 6 8 10 12 14 16 18 20
Lag

11/30/2020 1
 Model identification
➢The gradual decline of ACF values indicates
non-stationary series.
➢The first partial autocorrelation is very dominant
and close to 1, indicating non-stationarity.
➢The time series plot clearly indicates non-
stationarity.
➢We take the first differences of the data and
reanalyze.
11/30/2020 1
 Model identification

Time Series Plot of first difference

15

10

5
first difference

-5

-10

-15
1 10 20 30 40 50 60 70 80 90 100
Minutes

11/30/2020 1
 Model identification

Autocorrelation Function for first difference

(with 5% significance limits for the autocorrelations)

1.0
0.8
0.6
0.4
Autocorrelation

0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

2 4 6 8 10 12 14 16 18 20
Lag

11/30/2020 1
 Model identification

Partial Autocorrelation Function for first difference

(with 5% significance limits for the partial autocorrelations)

1.0
0.8
0.6
Partial Autocorrelation

0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

2 4 6 8 10 12 14 16 18 20
Lag

11/30/2020 1
 Model identification
➢ACF shows a mixture of exponential decay and
sine-wave pattern
➢PACF shows three significant PACF values.
➢This suggests an AR(3) model.
➢This identifies an ARIMA(3,1,0).

11/30/2020 1
 Model identification
➢Example
▪ A seasonal time series.
▪ The following example looks at the monthly industry sales (in
thousands of francs) for printing and writing papers between
the years 1963 and 1972.
▪ The time plot, ACF and PACF shows a clear seasonal pattern in
the data.
▪ This is clear in the large values at time lag 12, 24 and 36.

11/30/2020 1
 Model identification

Time Series Plot of Sales

1100

1000

900

800

700
Sales

600

500

400

300

200
Month Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan
Year 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972

11/30/2020 1
 Model identification

Autocorrelation Function for Sales

(with 5% significance limits for the autocorrelations)

1.0
0.8
0.6
0.4
Autocorrelation

0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

1 5 10 15 20 25 30 35 40
Lag

11/30/2020 1
 Model identification

Partial Autocorrelation Function for Sales

(with 5% significance limits for the partial autocorrelations)

1.0
0.8
0.6
Partial Autocorrelation

0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

2 4 6 8 10 12 14 16 18 20
Lag

11/30/2020 1
 Model identification
➢We take a seasonal difference and check the
time plot, ACF and PACF.
➢ The seasonally differenced data appears to be
non-stationary (the plots are not shown), so we
difference the data again.
➢ the following three slides show the twice
differenced series.

11/30/2020 1
 Model identification

Time Series Plot of first difference of seasonal

200
first difference of seasonal

100

-100

-200
Month Jan Jan Jan Jan Jan Jan Jan Jan Jan Jan
Year 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973

11/30/2020 1
 Model identification

Autocorrelation Function for first difference of seasonal

(with 5% significance limits for the autocorrelations)

1.0
0.8
0.6
0.4
Autocorrelation

0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

1 5 10 15 20 25 30 35 40
Lag

11/30/2020 1
 Model identification

Partial Autocorrelation Function for first difference of seasonal

(with 5% significance limits for the partial autocorrelations)

1.0
0.8
0.6
Partial Autocorrelation

0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0

1 5 10 15 20 25 30 35 40
Lag

11/30/2020 1
 Model identification
➢The PACF shows the exponential decay in
values.
➢The ACF shows a significant value at time lag 1.
▪ This suggest a MA(1) model.
➢The ACF also shows a significant value at time
lag 12
▪ This suggest a seasonal MA(1).

11/30/2020 1
 Model identification
➢Therefore, the identifies model is
ARIMA (0,1,1)(0,1,1)12.
➢This model is sometimes is called the “airline
model” because it was applied to international
airline data by Box and Jenkins.
➢It is one of the most commonly used seasonal
ARIMA model.

11/30/2020 1
 Summary
➢The process of identifying an ARIMA model
requires experience and good judgment. The
following guidelines can be helpful.
▪ Make the series stationary in mean and variance
▪ Differencing will take care of non-stationarity in the mean.

11/30/2020 1