AnswerBun.com

Why can't do ridge regression with one predictor?

Cross Validated Asked on January 5, 2022

I’m trying to fit a ridge regression model with a single predictor. However, when I try to do so in three different R packages I get the three following errors:

Error in colMeans(X[, -Inter]) : 
  'x' must be an array of at least two dimensions

Error in if (is.null(np) | (np[2] <= 1)) stop("x should be a matrix with 2 or more columns") : 
  argument is of length zero

Error in colMeans(x[, -Inter]) : 
  'x' must be an array of at least two dimensions

The bottom line from these errors is that x needs to have at least 2 dimensions. Why is this necessary for ridge regression? Does this mean that I can’t use ridge regression with a single predictor? Just seems weird I couldn’t use ridge regression to get regularization for something like a t-test.

Here is my code:

library(lmridge)
library(glmnet)
library(ridge)

# data 
set.seet(100)
y <- rnorm(100)
x <- rbinom(100, 1, .5)
z <- rbinom(100, 1, .5)
data <- cbind.data.frame(y, x, z)

# ridge
linearRidge(y ~ x, data = data)

# glmnet
glmnet(data$x, data$y, nlambda = 25, alpha = 0, family = 'gaussian', lambda = .5)

# lmridge
lmridge(y ~ x, data = data, scaling = "sc", K = seq(0, 1, 0.001))

One Answer

StatQuest does ridge with one predictor just fine in his video.

https://youtube.com/watch?v=Q81RR3yKn30

The method is somewhat silly to use in a regression with just one parameter, but I am surprised the common software implementation don’t allow it. Perhaps the StatQuest example could make sense in some setting.

But that’s just an issue with the software implementation. You’re still able to write your parameter vector as $hat{beta}_R = (X^TX+lambda I)^{-1}X^Ty$ and do the calculation.

($I$ is the identity matrix; $lambda$ is your ridge regression hyperparameter.)

Another popular software implementation of ridge regression is the sklearn packing in Python. Perhaps give that a whirl.

Answered by Dave on January 5, 2022

Add your own answers!

Related Questions

NxN- fold cross validation

1  Asked on February 21, 2021 by tauling

     

How to interpret a ROC curve?

5  Asked on February 20, 2021 by gnal

       

Ask a Question

Get help from others!

© 2023 AnswerBun.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP