🪴 Freeman's Second Brain

Search

SearchSearch

ARIMA

Aug 27, 2024, 6 min read

ARIMA

Models Family:

AR, Auto Regressive Process:

  • Assumes stationary process, is related to the past values ()
  • future is related to the past

MA, Moving- Average process:

I: intergrated (differencing Method):

SARIMA: S for seasonality (differencing Method ):

SARIMAX: add ex. regressor to the model

VAR: vector time series predictors

Def:

  • Let {} be an AR(P) process if where {}

Stationary Condition:

stationary process

  1. Backward Shift Operator B for AR:
    • no practical meaning
    • a variable that can apply to which could give us
    • ex:

consider Ar(p): =>let =>gnerating funciton of AR(P) where is the generating function, polynomial function.

  1. Backward Shift Operator B for MA:

if MA apply a backshift operator B:

apply on for the noise part, since MA is measure related to the past errors. rewrite : =>

In summary:

note: The stationarity condition for an AR(p) process is that all the roots of the characteristic polynomial should lie outside the unit circle in the complex plane. A complex number for the process to be stationary.
  • AR(p):
    • if
  • MA(q):
    • All MA(q) are stationary
    • if we can find a
  • when = proof: for a funciton to have an inverse function and be able to express as . the roots B for have to be outside the unit circle:

Stationary condition for AR(p); , the generating function has all its zero solutions lie outside the unit circle in a complex plane

Invertible condition for MA(q)

  • A given data only observe not . we need to define what method we use to track the noise
  • how do we use observable {} to get unobserved {}?
  • rewrite the model only use the observed value. turn MA(q) as AR, then we can use only () data to estimate the MA(q) model

Invertible Condition: An MA(q) model can be estimate only if it satisfy the invertible condition:

  • the zero solutions of the generating function are all outside the unit circle

ARMA process

Def A stationary process ARMA(P,q), it usually describe a stationary process, where the is from 2 part of the past(AR, MA), past from the error and measurement

Invertibility Condition

The invertibility condition is a critical aspect for the Moving Average (MA) component of an ARMA model. It ensures that the MA model can be inverted to become an AR model.

Definition

All the zero solutions for the generating function must lie outside the unit circle.

ARMA Model

An ARMA model combines both autoregressive (AR) and moving average (MA) components.

Equation

The ARMA(p, q) model can be written as:

White Noise

The error terms are assumed to be white noise with a mean of zero and a variance of , denoted as:

Conditions for ARMA Model

An ARMA(p, q) model must satisfy both:

  1. The stationarity condition for the AR component.
  2. The invertibility condition for the MA component.

Characteristic Polynomials

The characteristic polynomial for AR is given by and for MA by . The solutions to these polynomials (roots) determine the stationarity and invertibility of the model.

note: All ARMA models assume stationary condition and invertible condition are satisfy

Modeling :

model estimation

  • css-mle (conditional sum of square max likelihood)
    • Algorithm kalman filter
    • think it as a linear model , we want to estimate the parameters:
    • use MLE to maximize
        • where is a auto-covarince matrix that contain the autocovarince for
        • L3P36

Model estimation:

ARIMA (df, order = (p,0,q)) parameters estimated:

Model estimtaed;

In sampel Estimation:

given data and create estimation for the data in sample;

DataEsimtaionError
  • the true predictions observed values -> in sample estimation works well
  • the predictions errors -> in sample estimation works well
  • every step has a support error

out of sample forecast:

Notes: depends on the orders (p,q), the forecast will quickly lose the support of real values and predictor errors.
  • ARIMA models are better at short-term forecast, not long-term
  • With time step going further, it would lost the support
  • long- term, all the forecast converge to the mean( flat-line), because we are losing the error term to 0 with the losing support of the error

order selection for ARMA (p,q)

Use plots to check is the data is stationary or not

  • time series plot
  • ACF plot
  • PACF plot only work for pure AR(p) or MA(q), but not for broader cases. ACF and PACF plots if we can’t not see a cut off from both tails off, other order selection model

Use AIC/BIC

  • normally AIC and BIC only works for the same model class, in this cases. only compare models in ARIMA family. choose the model with the smallest AIC/BIC value
note:

fit the model with AIC the data to calculate AIC/BIC. No train-test method should be combinated here.

Combine Train - validation - Test split/ Cross -Validation and predictions performance metrics

  • RMSE
  • MAE
  • MAPE
Only measures the predictor error, can be used universally to compare different model approaches.

forecast the unknown future

order selection: choose p & q

prediction perfomance metrocs

TAGS:

#timeseries#models

Graph View

Backlinks