A weighted quantile regression approach for complex high-dimensional heterogeneous data

Abstract

With the development of digital intelligent technology, many problems arise, such as information flooding, computing power expansion, data heterogeneity, and complexity, which bring great challenges to the theories of data modeling. To this end, from the perspective of the mode, this paper proposes the concept of the optimal quantile level and mode-based weighted quantile regression (MWQR) to maximize the utilization of sample information. The proposed MWQR method is superior to the existing methods in the following aspects: (1) The proposed method is suitable for complex and high-dimensional heterogeneous data, the robustness can be ensured even when the error term is thick-tailed and skewed; (2) The MWQR method solves the 问题 of subjectivity in choosing quantile levels in quantile regression; (3) By assigning different weights to different quantile levels, estimation efficiency is greatly improved and computation time is reduced; (4) The entire conditional distribution of response variables can be investigated effectively in the MWQR method. Considering the advantages of the MWQR method, we apply it to partial linear additive models and propose two algorithms for robust coefficient estimation and variable selection, the consistency and asymptotic distribution of estimators are also demonstrated. The numerical simulation results and empirical study of the “implicit guarantee" of urban investment bonds and plasma β-carotene concentration problems further show that the proposed method can well explore the intrinsic structure of data, significantly improves computational efficiency, and has broad applicability.

摘要 随着数字化智能技术的发展,信息泛滥、算力膨胀、数据异构性及混杂性等问题频现,给数据建模的理论方法带来极大挑战.本文从众数角度出发,提出最优分位水平概念和基于众数的加权分位回归(mode-based weighted quantile regression, MWQR) 方法, 以求最大程度利用样本信息. 与已有估计方法相比, MWQR方法具有如下优势: (1) 适用于复杂高维异质性数据,在误差分布厚尾和偏态时仍能保证稳健性; (2) 解决了分位回归建模中分位水平主观选择的问题; (3)通过赋予不同分位水平不同权重, 极大提升估计效率,减少运算时间; (4) 有效探测响应变量的条件分布.鉴于MWQR方法的优势, 本文进一步将其应用于部分线性可加模型, 提出两种算法进行变量选择和系数估计,并探究理论性质. 数值模拟及城投债“隐性担保”和血浆β-胡萝卜素浓度两组实际数据分析,表明该方法能很好地挖掘数据内蕴结构,显著提高运算效率,具有广泛的应用价值.