Asymmetric heavy-tailed robust loss function for regression, regularisation and fast computation

Abstract

Robust inference for outliers to statistics and adversarial samples in deep learning is not only 'a new tune of an old song', yet also the current hot research topic in both statistics and artificial intelligence. Asymmetric distributions, including heavy-tailed distributions in regression models and data analysis have been challenging and are required to take into account the robustness of methods. Bayesian inference or Bayesian analysis, as one of the most popular statistics methods, has been widely used in all research elds, including science, social science and engineering recently. Many regression models face challenging issues in high-dimensional and computational problems, which has attracted substantial research in the literature recent years. Therefore, this thesis aims to develop novel Bayesian methods in parametric statistical inference to address these issues via three attempts. The rst attempt is to employ Bayesian variable selection with quantile-dependent prior for the fractional polynomial (FP) model, with a medical application in the analysis of blood pressure (BP) amongst United States adults. Whilst the FPs act as a concise and accurate formula for examining smooth relationships between BP measures and risk factors of cardiovascular disease, conditional quantile functions with FPs provide comprehensive relationships, including median and extremely high BP measures. The second attempt is to propose a new asymmetric Huberised loss function taking account of robustness, asymmetry and heavy tails. This motivates the development of robust Bayesian regularisation for a high-dimensional setting. The former has its corresponding probability distribution with the normal scale-mixture property. This leads to a by-product of the research, that is, a new Bayesian Huberised regularised quantile regression, which is derived by adopting the Markov chain Monte Carlo (MCMC) method. Finally, the third attempt is to revisit the work of the previous attempt addressing the computational issue. The MCMC method is a popular technique for full Bayesian probabilistic models, however, it faces the high computational cost when the amount of data increases. Alternative to the MCMC method, variational inference is the approximate-based technique to tackle the computational issue, and is utilised to propose variational Bayesian Huberised Lasso quantile regression and variational Bayesian Huberised adaptive Lasso quantile regression for high-dimensional and computational problems.