A post by Ben Griffiths, PhD student on the Compass programme.

My area of research is studying Quantile Generalised Additive Models (QGAMs), with my main application lying in energy demand forecasting. In particular, my focus is on developing faster and more stable fitting methods and model selection techniques. This blog post aims to briefly explain what QGAMs are, how to fit them, and a short illustrative example applying these techniques to data on extreme rainfall in Switzerland. I am supervised by Matteo Fasiolo and my research is sponsored by Électricité de France (EDF).

Quantile Generalised Additive Models

QGAMs are essentially the result of combining quantile regression (QR; performing regression on a specific quantile of the response) with a generalised additive model (GAM; fitting a model assuming additive smooth effects). Here we are in the regression setting, so let be the conditional c.d.f. of a response, , given a -dimensional vector of covariates, . In QR we model the th quantile, that is, .

This might be useful in cases where we do not need to model the full distribution of and only need one particular quantile of interest (for example urban planners might only be interested in estimates of extreme rainfall e.g. ). It also allows us to make no assumptions about the underlying true distribution, instead we can model the distribution empirically using multiple quantiles.

We can define the th quantile as the minimiser of expected loss

w.r.t. , where

is known as the pinball loss (Koenker, 2005).

We can approximate the above expression empirically given a sample of size , which gives the quantile estimator, where

where is the th vector of covariates, and is vector of regression coefficients.

So far we have described QR, so to turn this into a QGAM we assume has additive structure, that is, we can write the th conditional quantile as

where the additive terms are defined in terms of basis functions (e.g. spline bases). A marginal smooth effect could be, for example

where are unknown coefficients, are known spline basis functions and is the basis dimension.

Denote the vector of basis functions evaluated at , then the design matrix is defined as having th row , for , and is the total basis dimension over all . Now the quantile estimate is defined as . When estimating the regression coefficients, we put a ridge penalty on to control complexity of , thus we seek to minimise the penalised pinball loss

where is a vector of positive smoothing parameters, is the learning rate and the ‘s are positive semi-definite matrices which penalise the wiggliness of the corresponding effect . Minimising with respect to given fixed and leads to the maximum a posteriori (MAP) estimator .

There are a number of methods to tune the smoothing parameters and learning rate. The framework from Fasiolo et al. (2021) consists in:

1. calibrating by Integrated Kullback–Leibler minimisation
2. selecting by Laplace Approximate Marginal Loss minimisation
3. estimating by minimising penalised Extended Log-F loss (note that this loss is simply a smoothed version of the pinball loss introduced above)

For more detail on what each of these steps means I refer the reader to Fasiolo et al. (2021). Clearly this three-layered nested optimisation can take a long time to converge, especially in cases where we have large datasets which is often the case for energy demand forecasting. So my project approach is to adapt this framework in order to make it less computationally expensive.

Application to Swiss Extreme Rainfall

Here I will briefly discuss one potential application of QGAMs, where we analyse a dataset consisting of observations of the most extreme 12 hourly total rainfall each year for 65 Swiss weather stations between 1981-2015. This data set can be found in the R package gamair and for model fitting I used the package mgcViz.

A basic QGAM for the 50% quantile (i.e. ) can be fitted using the following formula

where is the intercept term, is a parametric factor for climate region, are smooth effects, is the Annual North Atlantic Oscillation index, is the metres above sea level, is the year of observation, and and are the degrees east and north respectively.

After fitting in mgcViz, we can plot the smooth effects and see how these affect the extreme yearly rainfall in Switzerland.

From the plots observe the following; as we increase the NAO index we observe a somewhat oscillatory effect on extreme rainfall; when increasing elevation we see a steady increase in extreme rainfall before a sharp drop after an elevation of around 2500 metres; as years increase we see a relatively flat effect on extreme rainfall indicating the extreme rainfall patterns might not change much over time (hopefully the reader won’t regard this as evidence against climate change); and from the spatial plot we see that the south-east of Switzerland appears to be more prone to more heavy extreme rainfall.

We could also look into fitting a 3D spatio-temporal tensor product effect, using the following formula

where is the tensor product effect between , and . We can examine the spatial effect on extreme rainfall over time by plotting the smooths.

There does not seem to be a significant interaction between the location and year, since we see little change between the plots, except for perhaps a slight decrease in the south-east.

Finally, we can make the most of the QGAM framework by fitting multiple quantiles at once. Here we fit the first formula for quantiles , and we can examine the fitted smooths for each quantile on the spatial effect.

Interestingly the spatial effect is much stronger in higher quantiles than in the lower ones, where we see a relatively weak effect at the 0.1 quantile, and a very strong effect at the 0.9 quantile ranging between around -30 and +60.

The example discussed here is but one of many potential applications of QGAMs. As mentioned in the introduction, my research area is motivated by energy demand forecasting. My current/future research is focused on adapting the QGAM fitting framework to obtain faster fitting.

References

Fasiolo, M., S. N. Wood, M. Zaffran, R. Nedellec, and Y. Goude (2021). Fast calibrated additive quantile regression. Journal of the American Statistical Association 116 (535), 1402–1412.

Koenker, R. (2005). Quantile Regression. Cambridge University Press.