Why can't do ridge regression with one predictor?

Cross Validated Asked on January 5, 2022

I’m trying to fit a ridge regression model with a single predictor. However, when I try to do so in three different R packages I get the three following errors:

Error in colMeans(X[, -Inter]) :
'x' must be an array of at least two dimensions

Error in if (is.null(np) | (np[2] <= 1)) stop("x should be a matrix with 2 or more columns") :
argument is of length zero

Error in colMeans(x[, -Inter]) :
'x' must be an array of at least two dimensions


The bottom line from these errors is that x needs to have at least 2 dimensions. Why is this necessary for ridge regression? Does this mean that I can’t use ridge regression with a single predictor? Just seems weird I couldn’t use ridge regression to get regularization for something like a t-test.

Here is my code:

library(lmridge)
library(glmnet)
library(ridge)

# data
set.seet(100)
y <- rnorm(100)
x <- rbinom(100, 1, .5)
z <- rbinom(100, 1, .5)
data <- cbind.data.frame(y, x, z)

# ridge
linearRidge(y ~ x, data = data)

# glmnet
glmnet(data$$x, data$$y, nlambda = 25, alpha = 0, family = 'gaussian', lambda = .5)

# lmridge
lmridge(y ~ x, data = data, scaling = "sc", K = seq(0, 1, 0.001))


StatQuest does ridge with one predictor just fine in his video.

The method is somewhat silly to use in a regression with just one parameter, but I am surprised the common software implementation don’t allow it. Perhaps the StatQuest example could make sense in some setting.

But that’s just an issue with the software implementation. You’re still able to write your parameter vector as $$hat{beta}_R = (X^TX+lambda I)^{-1}X^Ty$$ and do the calculation.

($$I$$ is the identity matrix; $$lambda$$ is your ridge regression hyperparameter.)

Another popular software implementation of ridge regression is the sklearn packing in Python. Perhaps give that a whirl.

Answered by Dave on January 5, 2022

Related Questions

Different z and Wald values in logistic regression analyses between SPSS and R

0  Asked on February 23, 2021 by afton-nelson

Creating an index using transactional data

0  Asked on February 23, 2021 by jamzy

How does a pdf change after a variable transformation with another random variable?

1  Asked on February 23, 2021 by fluctuation

A tail bound for an unknown distribution via sampling

1  Asked on February 22, 2021 by bashir

Negative Sampling in Word2Vec – Embedding Vector / Amount of Samples

0  Asked on February 22, 2021 by maxbit

NxN- fold cross validation

1  Asked on February 21, 2021 by tauling

Compare regression coefficients within the same model

2  Asked on February 21, 2021 by shenflow

How to predict an event for different time intervals and compute score?

1  Asked on February 21, 2021

What are arguments against using the (log-)likelihood as a loss function?

1  Asked on February 21, 2021 by joel

How to interpret a ROC curve?

5  Asked on February 20, 2021 by gnal

Minimum sample size per cluster in a random effect model

2  Asked on February 20, 2021 by roccer

Outlier detection for a univariate categorical variable?

3  Asked on February 20, 2021 by abdu

COVID-19: Individual level, cross-country Panel dataset. Which method to use?

0  Asked on February 20, 2021 by user307627

Partial dependence plot, GBM multinomial

1  Asked on February 19, 2021 by daniel-l

Why is it ok to remove variables with low variance from a dataset

0  Asked on February 19, 2021 by amnon

In a predefined (but realistic, and explaining) example, what $p(x)$ should be in a variational autoencoder?

1  Asked on February 19, 2021 by gerg-horvth

Examples of Bayesian and frequentist approach giving different answers

7  Asked on February 19, 2021

Choice of covariate under Independent Hypothesis Weighting (IHW)

0  Asked on February 19, 2021 by kyle-pena

What is the advantages of Wasserstein metric compared to Kullback-Leibler divergence?

5  Asked on February 19, 2021 by thomas-fauskanger

Regression Analysis with a Census or All Observations (Not a Sample)?

1  Asked on February 18, 2021 by statcurious