Arbeitspapier
A lava attack on the recovery of sums of dense and sparse signals
Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of non-zero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small non-zero parameters. We consider a generalization of these two basic models, termed here a "sparse + dense" model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein's unbiased estimator for lava's prediction risk. A simulation example compares the performance of lava to lasso, ridge, and elastic net in a regression example using data-dependent penalty parameters and illustrates lava's improved performance relative to these benchmarks.
- Language
-
Englisch
- Bibliographic citation
-
Series: cemmap working paper ; No. CWP56/15
- Classification
-
Wirtschaft
- Subject
-
high-dimensional models
penalization
shrinkage
non-sparse signal recovery
- Event
-
Geistige Schöpfung
- (who)
-
Chernozhukov, Victor
Hansen, Christian
Liao, Yuan
- Event
-
Veröffentlichung
- (who)
-
Centre for Microdata Methods and Practice (cemmap)
- (where)
-
London
- (when)
-
2015
- DOI
-
doi:10.1920/wp.cem.2015.5615
- Handle
- Last update
-
10.03.2025, 11:43 AM CET
Data provider
ZBW - Deutsche Zentralbibliothek für Wirtschaftswissenschaften - Leibniz-Informationszentrum Wirtschaft. If you have any questions about the object, please contact the data provider.
Object type
- Arbeitspapier
Associated
- Chernozhukov, Victor
- Hansen, Christian
- Liao, Yuan
- Centre for Microdata Methods and Practice (cemmap)
Time of origin
- 2015
Other Objects (12)
A lava attack on the recovery of sums of dense and sparse signals
L1-Penalised quantile regression in high-dimensional sparse models
Inference for high-dimensional sparse econometric models
Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors
Robust inference in high-dimensional approximately sparse quantile regression models
Valid post-selection inference in high-dimensional approximately sparse quantile regression models
Simultaneous inference for best linear predictor of the conditional average treatment effect and other structural functions
Posterior inference in curved exponential families under increasing dimensions
On the computational complexity of MCMC-based estimators in large samples
Post-l1-penalized estimators in high-dimensional linear regression models
Quantile models with endogeneity
Inference for extremal conditional quantile models, with an application to market and birthweight risks
A lava attack on the recovery of sums of dense and sparse signals
L1-Penalised quantile regression in high-dimensional sparse models
Inference for high-dimensional sparse econometric models
Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors
Robust inference in high-dimensional approximately sparse quantile regression models
Valid post-selection inference in high-dimensional approximately sparse quantile regression models
Simultaneous inference for best linear predictor of the conditional average treatment effect and other structural functions
Posterior inference in curved exponential families under increasing dimensions
On the computational complexity of MCMC-based estimators in large samples
Post-l1-penalized estimators in high-dimensional linear regression models
Quantile models with endogeneity
Inference for extremal conditional quantile models, with an application to market and birthweight risks
A lava attack on the recovery of sums of dense and sparse signals
L1-Penalised quantile regression in high-dimensional sparse models
Inference for high-dimensional sparse econometric models
Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors
Robust inference in high-dimensional approximately sparse quantile regression models
Valid post-selection inference in high-dimensional approximately sparse quantile regression models
Simultaneous inference for best linear predictor of the conditional average treatment effect and other structural functions
Posterior inference in curved exponential families under increasing dimensions
On the computational complexity of MCMC-based estimators in large samples
Post-l1-penalized estimators in high-dimensional linear regression models
Quantile models with endogeneity