Using Model Selection Algorthims to Obtain Reliable Coefficient Estimates
This review surveys a number of common Model Selection Algorithms (MSAs), discusses how they relate to each other, and identifies factors that explain their relative performances. At the heart of MSA performance is the trade-off between Type I and Type II errors. Some relevant variables will be mistakenly excluded, and some irrelevant variables will be retained by chance. A successful MSA will find the optimal trade-off between the two types of errors for a given data environment. Whether a given MSA will be successful in a given environment depends on the relative costs of these two types of errors. We use Monte Carlo experimentation to illustrate these issues. We confirm that no MSA does best in all circumstances. Even the worst MSA in terms of overall performance – the strategy of including all candidate variables – sometimes performs best (viz., when all candidate variables are relevant). We also show how (i) the ratio of relevant to total candidate variables and (ii) DGP noise affect relative MSA performance. Finally, we discuss a number of issues complicating the task of MSAs in producing reliable coefficient estimates.
Subjectsmodel selection algorithms
Showing items related by title, author, creator and subject.
A Numerical Modelling Study of Tropical Cyclone Sidr (2007): Sensitivity Experiments Using the Weather Research and Forecasting (WRF) Model Shepherd, Tristan James (University of Canterbury. Geography, 2008)The tropical cyclone is a majestic, yet violent atmospheric weather system occurring over tropical waters. Their majesty evolves from the significant range of spatial scales they operate over: from the mesoscale, to the ...
Ho, Kwok Yan (Daniel) (University of Canterbury. Civil Engineering, 2007)This thesis examines the prediction of opposed flow flame spread in the Fire Dynamics Simulator version 4 (FDS4) Computational Fluid Dynamics (CFD) model by adapting the Lateral Ignition Flame Transport (LIFT) test ...
Effect of varying hysteresis models and damage models on damage assessment of r/c structures under standard design level earthquakes obtained using a new scaling method Dong, Ping (University of Canterbury, 2003)A seismic damage assessment of a real ductile framed structure of reinforced concrete requires not only realistic damage indices for members, storeys and the whole structure, but also a hysteresis rule simulating ...