# How do I compare cv.glmnet models with AIC?

Cross Validated Asked by Thomas on August 20, 2020

I am using the glmnet package in R, and not(!) the caret package for my binary ElasticNet regression. I have come to the point where I would like to compare models (e.g. lambda set to lambda.1se or lambda.min, and models where k-fold is set to 5 or 10). But, I have not yet achieved to compute the AICc or BIC for my models. How do I do that? I tried this and this but it did not work for me, I only get an empty list.
Code:

set.seed(123)
foldid <- sample(rep(seq(10), length.out = nrow(x.train)))
list.of.fits.df <- list()
for (i in 0:10){
fit.name <- paste0("alpha", i/10)
list.of.fits.df[[fit.name]] <- cv.glmnet(x.train, y.train, type.measure = c("auc"), alpha = i/10, family = "binomial", nfolds = 10, foldid = foldid, parallel = TRUE)
}
best.fit <- coef(list.of.fits.df[[fit.name]], s = list.of.fits.df[[fit.name]]$$lambda.1se) best.fit.min <- coef(list.of.fits.df[[fit.name]], s = list.of.fits.df[[fit.name]]$$lambda.min)

#AICc & BIC
#???


How do I find an AIC and/or BIC value from my list.of.fits[[fit.name]]$lambda.min? ## One Answer Create a function cv_aicc <- function(fit, lambda = 'lambda.1se'){ whlm <- which(fit$lambda == fit[[lambda]])
with(fit$glmnet.fit, { tLL <- nulldev - nulldev * (1 - dev.ratio)[whlm] k <- df[whlm] n <- nobs return(list('AICc' = - tLL + 2 * k + 2 * k * (k + 1) / (n - k - 1), 'BIC' = log(n) * k - tLL)) }) }  Plug in the lists now. AICc <- cv_aicc(list.of.fits.df[[fit.name]]) AICc.min <- cv_aicc(list.of.fits.df[[fit.name]], 'lambda.min')  Correct answer by Thomas on August 20, 2020 ## Add your own answers! ## Related Questions ### How do betting sites update odds during a sporting match in real-time? 1 Asked on December 13, 2020 by svexo ### How to interpret the negative variances 1 Asked on December 12, 2020 by aakash-bashyal ### How to interpret balances after ILR transform for compositional data? 0 Asked on December 12, 2020 by samme-galanakis ### Specifying specific priors for a correlation matrix via Stan 1 Asked on December 11, 2020 by sue-doh-nimh ### Example of mean independent variables but dependent still 0 Asked on December 11, 2020 by luchonacho ### When are observations not weakly exchangeable? 1 Asked on December 11, 2020 by rumtscho ### How big should my subsample be? 1 Asked on December 11, 2020 by kaecvtionr ### Spirtes’ example of d-separation not leading to independence in a directed cyclic graph with non-linear structural equations 1 Asked on December 10, 2020 by quant_dev ### Asymptotic normality for nonsmooth objective functions 1 Asked on December 10, 2020 ### Regression: is it wrong to bin a continuous variable to overcome overfitting? 1 Asked on December 10, 2020 by st4co4 ### How do you compare standard deviations? 2 Asked on December 10, 2020 by yaynikkiprograms ### How to interpret the beta estimates of a generalized linear model with a square root power link? 0 Asked on December 10, 2020 by statboy_41 ### Can k-fold CV help reduce sampling bias? 0 Asked on December 9, 2020 by aite97 ### Why is the standard deviation of the average of averages smaller than the standard deviation of the total? 0 Asked on December 9, 2020 by pinocchio ### Calculating bias of ML estimate of AR(1) coefficient 1 Asked on December 9, 2020 by andrew-kirk ### Using residuals from linear regression for normality testing for ANOVA 0 Asked on December 9, 2020 by s-ramagokula-krishnan ### How does scaled conjugate gradient work in neural network training? Comparison with gradient descent 0 Asked on December 9, 2020 by johanna ### For B-spline what does$sum_{i=0,n}N_{i,k}(t)=1\$ mean?

1  Asked on December 9, 2020

### Is it possible to detect overfitting automatically/programmatically after model creation?

0  Asked on December 9, 2020 by ayberk-yavuz

### R lmer model: degree of freedom and chi square values are zero

1  Asked on December 9, 2020 by roromario