A Penalized Approach To Mixed Model Selection Via Cross Validation
DOWNLOAD
Download A Penalized Approach To Mixed Model Selection Via Cross Validation PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get A Penalized Approach To Mixed Model Selection Via Cross Validation book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages. If the content not found or just blank you must refresh this page
A Penalized Approach To Mixed Model Selection Via Cross Validation
DOWNLOAD
Author : Jingwei Xiong
language : en
Publisher:
Release Date : 2017
A Penalized Approach To Mixed Model Selection Via Cross Validation written by Jingwei Xiong and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2017 with Linear models (Statistics) categories.
A linear mixed model is a useful technique to explain observations by regarding them as realizations of random variables, especially when repeated measurements are made to statistical units, such as longitudinal data. However, in practice, there are often too many potential factors considered affecting the observations, while actually, they are not. Therefore, statisticians have been trying to select significant factors out of all the potential factors, where we call the process model selection. Among those approaches for linear mixed model selection, penalized methods have been developed profoundly over the last several decades. In this dissertation, to solve the overfitting problem in most penalized methods and improve the selection accuracy, we mainly focus on a penalized approach via cross-validation. Unlike the existing methods using the whole data to fit and select models, we split the fitting process and selection into two stages. More specifically, an adaptive lasso penalized function is customized in the first stage and marginal BIC criterion is used in the second stage. We consider that the main advantage of our approach is to reduce the dependency between models construction and evaluation. Because of the complex structure of mixed models, we adopt a modified Cholesky decomposition to reparameterize the model, which in turn significantly reduces the dimension of the penalized function. Additionally, since random effects are missing, there is no closed form for the maximizer of the penalized function, thus we implement EM algorithm to obtain a full inference of parameters. Furthermore, due to the computation limit and moderately small samples in practice, some noisy factors may still remain in the model, which is particularly obvious for fixed effects. To eliminate the noisy factors, a likelihood ratio test is employed to screen the fixed effects. Regarding the overall process, we call it adaptive lasso via cross-validation. Additionally, we demonstrate that the proposed approach possesses selection and estimation consistency simultaneously. Moreover, simulation studies and real data examples are both provided to justify the method validity. At the very end, a brief conclusion is drawn and some possible further improvements are discussed.
Shrinkage Parameter Selection In Generalized Linear And Mixed Models
DOWNLOAD
Author : Erin K. Melcon
language : en
Publisher:
Release Date : 2014
Shrinkage Parameter Selection In Generalized Linear And Mixed Models written by Erin K. Melcon and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2014 with categories.
Penalized likelihood methods such as lasso, adaptive lasso, and SCAD have been highly utilized in linear models. Selection of the penalty parameter is an important step in modeling with penalized techniques. Traditionally, information criteria or cross validation are used to select the penalty parameter. Although methods of selecting this have been evaluated in linear models, general linear models and linear mixed models have not been so thoroughly explored.This dissertation will introduce a data-driven bootstrap (Empirical Optimal Selection, or EOS) approach for selecting the penalty parameter with a focus on model selection. We implement EOS on selecting the penalty parameter in the case of lasso and adaptive lasso. In generalized linear models we will introduce the method, show simulations comparing EOS to information criteria and cross validation, and give theoretical justification for this approach. We also consider a practical upper bound for the penalty parameter, with theoretical justification. In linear mixed models, we use EOS with two different objective functions; the traditional log-likelihood approach (which requires an EM algorithm), and a predictive approach. In both of these cases, we compare selecting the penalty parameter with EOS to selection with information criteria. Theoretical justification for both objective functions and a practical upper bound for the penalty parameter in the log-likelihood case are given. We also applied our technique to two datasets; the South African heart data (logistic regression) and the Yale infant data (a linear mixed model). For the South African data, we compare the final models using EOS and information criteria via the mean squared prediction error (MSPE). For the Yale infant data, we compare our results to those obtained by Ibrahim et al. (2011).
Linear Mixed Model Selection Via Minimum Approximated Information Criterion
DOWNLOAD
Author : Olivia Abena Atutey
language : en
Publisher:
Release Date : 2020
Linear Mixed Model Selection Via Minimum Approximated Information Criterion written by Olivia Abena Atutey and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2020 with Linear models (Statistics) categories.
The analyses of correlated, repeated measures, or multilevel data with a Gaussian response are often based on models known as the linear mixed models (LMMs). LMMs are modeled using both fixed effects and random effects. The random intercepts (RI) and random intercepts and slopes (RIS) models are two exceptional cases from the linear mixed models that are taken into consideration. Our primary focus in this dissertation is to propose an approach for simultaneous selection and estimation of fixed effects only in LMMs. This dissertation, inspired by recent research of methods and criteria for model selection, aims to extend a variable selection procedure referred to as minimum approximated information criterion (MIC) of Su et al. (2018). Our contribution presents further use of the MIC for variable selection and sparse estimation in LMMs. Thus, we design a penalized log-likelihood procedure referred to as the minimum approximated information criterion for LMMs (lmmMAIC), which is used to find a parsimonious model that better generalizes data with a group structure. Our proposed lmmMAIC method enforces variable selection and sparse estimation simultaneously by adding a penalty term to the negative log-likelihood of the linear mixed model. The method differs from existing regularized methods mainly due to the penalty parameter and the penalty function.With regards to the penalty function, the lmmMAIC mimics the traditional Bayesian information criterion (BIC)-based best subset selection (BSS) method but requires a continuous or smooth approximation to the L0 norm penalty of BSS. In this context, lmmMAIC performs sparse estimation by optimizing an approximated information criterion, which substantially requires approximating that L0 norm penalty of BSS with a continuous unit dent function. A unit dent function, motivated by bump functions called mollifiers (Friedrichs, 1944), is an even continuous function with a [0, 1] range. Among several unit dent functions, incorporating a hyperbolic tangent function is most preferred. The approximation changes the discrete nature of the L0 norm penalty of BSS to a continuous or smooth one making our method less computationally expensive. Besides, the hyperbolic tangent function has a simple form and it is much easier to compute its derivatives. This shrinkage-based method fits a linear mixed model containing all p predictors instead of comparing and selecting a correct sub-model across 2p candidate models. On this account, the lmmMAIC is feasible for high-dimensional data. The replacement, however, does not enforce sparsity since the hyperbolic tangent function is not singular at its origin. To better handle this issue, a reparameterization trick of the regression coefficients is needed to achieve sparsity.For a finite number of parameters, numerical investigations demonstrated by Shi and Tsai (2002) prove that traditional information criterion (IC)-based procedure like BIC can consistently identify a model. Following these suggestions of consistent variable selection and computational efficiency, we maintain the BIC fixed penalty parameter. Thus, our newly proposed procedure is free of using the frequently applied practices such as generalized cross validation (GCV) in selecting an optimal penalty parameter for our penalized likelihood framework. The lmmMAIC enjoys less computational time compared to other regularization methods.We formulate the lmmMAIC procedure as a smooth optimization problem and seek to solve for the fixed effects parameters by minimizing the penalized log-likelihood function. The implementation of the lmmMAIC involves an initial step of using the simulated annealing algorithm to obtain estimates. We proceed using these estimates as starting values by applying the modified Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm until convergence. After this step, we plug estimates obtained from the modified BFGS into the reparameterized hyperbolic tangent function to obtain our fixed effects estimates. Alternatively, the optimization of the penalized log-likelihood can be solved using generalized simulation annealing.Our research explores the performance and asymptotic properties of the lmmMAIC method by conducting extensive simulation studies using different model settings. The numerical results of our simulations for our proposed variable selection and estimation method are compared to other standard LMMs shrinkage-based methods such as Lasso, ridge, and elastic net. The results provide evidence that lmmMAIC is more consistent and efficient than the existing shrinkage-based methods under study. Furthermore, two applications with real-life examples are illustrated to evaluate the effectiveness of the lmmMAIC method.
Discrepancy Based Model Selection Criteria Using Cross Validation
DOWNLOAD
Author : Simon Lee Davies
language : en
Publisher:
Release Date : 2002
Discrepancy Based Model Selection Criteria Using Cross Validation written by Simon Lee Davies and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2002 with Linear models (Statistics) categories.
An important component of any linear modeling problem consists of determining an appropriate size and form of the design matrix. Improper specification may substantially impact both estimators of the model parameters and predictors of the response variable: underspecification may lead to results which are severely biased, whereas overspecification may lead to results with unnecessarily high variability. Model selection criteria provide a powerful and useful tool for choosing a suitable design matrix. Once a setting has been proposed for an experiment, data can be collected, leading to a set of competing candidate models. One may then attempt to select an appropriate model from this set using a model selection criterion. In this thesis we establish four frameworks which initialize with previously proposed model selection criteria targeting well-known traditional discrepancies, namely the Kullback-Leibler discrepancy, the Gauss discrepancy, the transformed Gauss discrepancy, and the Kullback symmetric discrepancy. These criteria are developed using the bias adjustment approach. Prior work has focused on finding approximately or exactly unbiased estimators of these discrepancies. We expand on this work to additionally show that the criteria which are exactly unbiased serve as the minimum variance unbiased estimators. In many situations, the predictive ability of a candidate model is its most important attribute. In light of our interest in this property, we also concentrate on model selection techniques based on cross validation. New cross validation model selection criteria that serve as counterparts to the standard bias adjusted forms are introduced, together with descriptions of the target discrepancies upon which they are based. We then develop model selection criteria which are minimum variance unbiased estimators of the cross validation discrepancies. Furthermore, we argue that these criteria serve as approximate minimum variance unbiased estimators of the corresponding traditional discrepancies. We propose a general framework to unify and elucidate part of our cross validation criterion development. We show that for the cross validation analogue of a traditional discrepancy, we can always find a "natural" criterion which serves as an exactly unbiased estimator. We study how the cross validation criteria compare to the standard bias adjusted criteria as selection rules in the linear regression framework. This is done by concluding our development of each of the four frameworks with simulation results which illustrate how frequently each criterion identifies the correctly specified model among a sequence of nested fitted candidate models. Our results indicate that the cross validation criteria tend to outperform their bias adjusted counterparts. We close by evaluating the performance of all the model selection criteria considered throughout our work by investigating the results of a simulation study compiled using a sample of data from the Missouri Trauma Registry.
Bayesian Models
DOWNLOAD
Author : N. Thompson Hobbs
language : en
Publisher: Princeton University Press
Release Date : 2015-08-04
Bayesian Models written by N. Thompson Hobbs and has been published by Princeton University Press this book supported file pdf, txt, epub, kindle and other format this book has been release on 2015-08-04 with Science categories.
Bayesian modeling has become an indispensable tool for ecological research because it is uniquely suited to deal with complexity in a statistically coherent way. This textbook provides a comprehensive and accessible introduction to the latest Bayesian methods—in language ecologists can understand. Unlike other books on the subject, this one emphasizes the principles behind the computations, giving ecologists a big-picture understanding of how to implement this powerful statistical approach. Bayesian Models is an essential primer for non-statisticians. It begins with a definition of probability and develops a step-by-step sequence of connected ideas, including basic distribution theory, network diagrams, hierarchical models, Markov chain Monte Carlo, and inference from single and multiple models. This unique book places less emphasis on computer coding, favoring instead a concise presentation of the mathematical statistics needed to understand how and why Bayesian analysis works. It also explains how to write out properly formulated hierarchical Bayesian models and use them in computing, research papers, and proposals. This primer enables ecologists to understand the statistical principles behind Bayesian modeling and apply them to research, teaching, policy, and management. Presents the mathematical and statistical foundations of Bayesian modeling in language accessible to non-statisticians Covers basic distribution theory, network diagrams, hierarchical models, Markov chain Monte Carlo, and more Deemphasizes computer coding in favor of basic principles Explains how to write out properly factored statistical expressions representing Bayesian models
Mixed Effects Models For The Population Approach
DOWNLOAD
Author : Marc Lavielle
language : en
Publisher: CRC Press
Release Date : 2014-07-14
Mixed Effects Models For The Population Approach written by Marc Lavielle and has been published by CRC Press this book supported file pdf, txt, epub, kindle and other format this book has been release on 2014-07-14 with Mathematics categories.
Wide-Ranging Coverage of Parametric Modeling in Linear and Nonlinear Mixed Effects ModelsMixed Effects Models for the Population Approach: Models, Tasks, Methods and Tools presents a rigorous framework for describing, implementing, and using mixed effects models. With these models, readers can perform parameter estimation and modeling across a whol
Some Problems In Model Selection
DOWNLOAD
Author : Chenlei Leng
language : en
Publisher:
Release Date : 2004
Some Problems In Model Selection written by Chenlei Leng and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2004 with categories.
A Computational Approach To Statistical Learning
DOWNLOAD
Author : Taylor Arnold
language : en
Publisher: CRC Press
Release Date : 2019-01-23
A Computational Approach To Statistical Learning written by Taylor Arnold and has been published by CRC Press this book supported file pdf, txt, epub, kindle and other format this book has been release on 2019-01-23 with Business & Economics categories.
A Computational Approach to Statistical Learning gives a novel introduction to predictive modeling by focusing on the algorithmic and numeric motivations behind popular statistical methods. The text contains annotated code to over 80 original reference functions. These functions provide minimal working implementations of common statistical learning algorithms. Every chapter concludes with a fully worked out application that illustrates predictive modeling tasks using a real-world dataset. The text begins with a detailed analysis of linear models and ordinary least squares. Subsequent chapters explore extensions such as ridge regression, generalized linear models, and additive models. The second half focuses on the use of general-purpose algorithms for convex optimization and their application to tasks in statistical learning. Models covered include the elastic net, dense neural networks, convolutional neural networks (CNNs), and spectral clustering. A unifying theme throughout the text is the use of optimization theory in the description of predictive models, with a particular focus on the singular value decomposition (SVD). Through this theme, the computational approach motivates and clarifies the relationships between various predictive models. Taylor Arnold is an assistant professor of statistics at the University of Richmond. His work at the intersection of computer vision, natural language processing, and digital humanities has been supported by multiple grants from the National Endowment for the Humanities (NEH) and the American Council of Learned Societies (ACLS). His first book, Humanities Data in R, was published in 2015. Michael Kane is an assistant professor of biostatistics at Yale University. He is the recipient of grants from the National Institutes of Health (NIH), DARPA, and the Bill and Melinda Gates Foundation. His R package bigmemory won the Chamber's prize for statistical software in 2010. Bryan Lewis is an applied mathematician and author of many popular R packages, including irlba, doRedis, and threejs.
Adaptive Model Selection
DOWNLOAD
Author : Yongli Zhang
language : en
Publisher:
Release Date : 2007
Adaptive Model Selection written by Yongli Zhang and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 2007 with categories.
Neural Network Model Selection Using Asymptotic Jackknife Estimator And Cross Validation Method
DOWNLOAD
Author :
language : en
Publisher:
Release Date : 1993
Neural Network Model Selection Using Asymptotic Jackknife Estimator And Cross Validation Method written by and has been published by this book supported file pdf, txt, epub, kindle and other format this book has been release on 1993 with categories.
Two theorems and a lemma are presented about the use of jackknife estimator and the cross-validation method for model selection. Theorem 1 gives the asymptotic form for the jackknife estimator. Combined with the model selection criterion, this asymptotic form can be used to obtain the fit of a model. The model selection criterion we used is the negative of the average predictive likelihood, the choice of which is based on the idea of the cross- validation method. Lemma 1 provides a formula for further exploration of the asymptotics of the model selection criterion. Theorem 2 given an asymptotic form of the model selection criterion for the regression case, when the parameters optimization criterion has a penalty term. Theorem 2 also proves the asymptotic equivalence of Moody's model selection criterion (Moody, 1992) and the cross- validation method, when the distance measure between response y and regression function takes the form of a squared difference ... Neural networks, Model selection, Jackknife, Cross-validation.