In time series forecasting, the least squares method or one of its variants is commonly used. These methods predict a single value, and the difference between this prediction and the actual value is treated as the error. In this setup, both over-prediction and under-prediction are equally penalized, proportional to their absolute values.
However, this type of modeling doesn’t tell us much about how confident the model is in its prediction. If we had a confidence interval around the predicted value, we could make more informed decisions — especially in real-world scenarios where underestimation or overestimation can have severe consequences.
In a previous post, I talked about quantile forecasting as a way to address this issue. Instead of predicting a single point, it estimates an interval for the target variable. I explained how quantile loss (also called pinball loss) helps define a confidence interval around predictions. I explained the non-parametric approach and how it doesn’t assume a specific distribution for the data.
Now, as promised, I’m turning to parametric modeling, where we assume a particular conditional distribution for the target variable and train the model to predict the parameters of that distribution.
In this post, I’ll explain how this works — and introduce a loss function for training such models: the Continuous Ranked Probability Score (CRPS).
