Arbeitspapier
An Information Theoretic Comparison of Model Selection Criteria
Information theory offers a coherent perspective on model selection. As in Rissanen's original application of information theory to model selection, our perspective arises from viewing a model as a component of a compressed representation of data in a two-part code. The first part of such a code is an explicit representation of the model used to compress the data. Simpler models have shorter representations. The second part is the encoded data itself. Models which fit better compress the data into shorter sequences. The objective is to choose the model which produces the shortest total message length, requiring an explicit trade-off of model complexity (length of the first part) versus goodness-of-fit (length of second part). In adidtion to Rissanen's MDL criterion, this perspective illuminates the properties of numerous model selection criteria, including AIC, Cp, BIC, RIC, and EBIC. We show that each corresponds to a specific way of coding the model parameters. By selecting the model that minimizes the total message length, our representations of these criteria reproduce their more familiar definitions. Examples from wavelets illustrate the use of these methods.
- Language
-
Englisch
- Bibliographic citation
-
Series: Discussion Paper ; No. 1180
- Classification
-
Wirtschaft
- Event
-
Geistige Schöpfung
- (who)
-
Foster, Dean P.
Stine, Robert A.
- Event
-
Veröffentlichung
- (who)
-
Northwestern University, Kellogg School of Management, Center for Mathematical Studies in Economics and Management Science
- (where)
-
Evanston, IL
- (when)
-
1997
- Handle
- Last update
-
10.03.2025, 11:45 AM CET
Data provider
ZBW - Deutsche Zentralbibliothek für Wirtschaftswissenschaften - Leibniz-Informationszentrum Wirtschaft. If you have any questions about the object, please contact the data provider.
Object type
- Arbeitspapier
Associated
- Foster, Dean P.
- Stine, Robert A.
- Northwestern University, Kellogg School of Management, Center for Mathematical Studies in Economics and Management Science
Time of origin
- 1997
Other Objects (12)
Model selection criteria for factor-augmented regressions
Model selection and inference : a practical information theoretic approach
Comparison of Bayesian model selection criteria and conditional Kolmogorov test as applied to spot asset pricing models
Selection criteria in regime switching conditional volatility models
Empirical likelihood-based model selection criteria for moment condition models
Model selection and multimodel inference : a practical information theoretic approach
Proof-theoretic versus Model-theoretic Consequence
Model theoretic hygiene
Model theoretic logics
Observation error model selection by information criteria vs. normality testing
Exploratory model comparison : interactive model ensemble selection and management
Automata-Theoretic Criteria for Feature Interactions in Telecommunications Systems
Model selection criteria for factor-augmented regressions
Model selection and inference : a practical information theoretic approach
Comparison of Bayesian model selection criteria and conditional Kolmogorov test as applied to spot asset pricing models
Selection criteria in regime switching conditional volatility models
Empirical likelihood-based model selection criteria for moment condition models
Model selection and multimodel inference : a practical information theoretic approach
Proof-theoretic versus Model-theoretic Consequence
Model theoretic hygiene
Model theoretic logics
Observation error model selection by information criteria vs. normality testing
Exploratory model comparison : interactive model ensemble selection and management
Automata-Theoretic Criteria for Feature Interactions in Telecommunications Systems
Model selection criteria for factor-augmented regressions
Model selection and inference : a practical information theoretic approach
Comparison of Bayesian model selection criteria and conditional Kolmogorov test as applied to spot asset pricing models
Selection criteria in regime switching conditional volatility models
Empirical likelihood-based model selection criteria for moment condition models
Model selection and multimodel inference : a practical information theoretic approach
Proof-theoretic versus Model-theoretic Consequence
Model theoretic hygiene
Model theoretic logics
Observation error model selection by information criteria vs. normality testing
Exploratory model comparison : interactive model ensemble selection and management