# Comparing a Bayesian model with a Classical model for linear regression

Cross Validated Asked by student_R123 on September 7, 2020

I have fitted following Bayesian linear regression model using rjags package in R, with the help of cars data set.
I used some weakly informative priors for parameters.

require(rjags)
dim(cars)
N=length(cars$speed) bayes_model="model { for(i in 1:N){ dist[i] ~ dnorm(mu[i],tau) mu[i] = beta[1] + beta[2]*speed[i] } for (l in 1:2) { beta[l] ~dnorm(0, 100) } tau ~ dgamma(.001,.001) sigma_tau = 1/tau }" model2 <- jags.model(textConnection(bayes_model), data = list(dist=cars$$dist,N=N,speed=cars$$speed), n.chains=2) params <- c('beta','sigma_tau') samps.1 <- coda.samples(model2, params, n.iter = 2000) burn.in=1000 summary.model.1=summary(window(samps.1, start = burn.in)) Stat.model.1=as.data.frame(summary.model.1$statistics)


This is how the results looks like .

 > Stat.model.1
Mean           SD     Naive SE Time-series SE
beta[1]   9.937366e-03   0.09806290  0.002191658    0.002238168
beta[2]   1.650041e-01   0.09903592  0.002213404    0.002330977
sigma_tau 2.341437e+03 522.81381343 11.684631408   11.700676273


When I fit a classical linear regression model , following results are obtained .

  summary(lm(dist~speed ,data=cars))

Call:
lm(formula = dist ~ speed, data = cars)

Residuals:
Min      1Q  Median      3Q     Max
-29.069  -9.525  -2.272   9.215  43.201

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -17.5791     6.7584  -2.601   0.0123 *
speed         3.9324     0.4155   9.464 1.49e-12 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 15.38 on 48 degrees of freedom
Multiple R-squared:  0.6511,    Adjusted R-squared:  0.6438
F-statistic: 89.57 on 1 and 48 DF,  p-value: 1.49e-12


It can be observed that the results based on Bayesian and classical method is not similar. What may be the reason for this ? Is there any problem with my prior distributions ?

Are there any diagnostic plots that should look into ? Also please feel free to let me know if I have missed any important steps in this analysis.

I am relatively new to Bayesian and I am working with different sort of examples to learn how to assign correct prior distributions.

Try:

bayes_model="model {

for(i in 1:N){
dist[i] ~ dnorm(mu[i],tau)
mu[i] = beta[1] + beta[2]*speed[i]
}

for (l in 1:2) { beta[l] ~dnorm(0, 0.001) }
tau ~ dgamma(.01,.01)
sigma_tau = 1/tau
}"


You'll get something like:

> Stat.model.1
Mean         SD    Naive SE Time-series SE
beta[1]   -16.88098  6.2008860 0.138586750     0.53168398
beta[2]     3.89631  0.3812761 0.008521332     0.03262728
sigma_tau 244.69856 53.4120568 1.193733180     1.25568735


Why? Because in JAGS the parameters of the normal distribution are mean $$mu$$ and precision $$tau$$, and $$tau=sigma^{-2}$$, so

beta[l] ~ dnorm(0, 100)
is a very strong, not a weakly informative prior, it forces beta[l] to be as close to zero as possible.

Correct answer by Sergio on September 7, 2020

## Related Questions

### How to compare gender proportions in a population?

1  Asked on January 4, 2021 by new

### Pseudo R2 and prob>chi2

1  Asked on January 3, 2021 by nsamwa

### Saddle-free Newton method for SGD – while Newton attracts saddles, is it worth to actively replel them?

1  Asked on January 3, 2021 by jarek-duda

### Relative Error is not normally distributed

1  Asked on January 3, 2021

### Tensor product between an ispline and a bspline for fitting data that should be monotonic in one dimension

0  Asked on January 3, 2021

### Interpretation of TSA::arimax output model is presented in R

1  Asked on January 2, 2021 by wasif

### Training samples with no labels: To include or not to include?

1  Asked on January 2, 2021 by aishwarya-a-r

### Custom Loss Function – Inducing sparsity

1  Asked on January 2, 2021 by mark-f

### Belief propagation on Polytree

0  Asked on January 2, 2021 by jonasc

### Q: Dividing maximum value by minimum value and reporting the difference “in times”

0  Asked on January 2, 2021

### Hypothesis test for difference of mean when two groups have different size population

1  Asked on January 1, 2021 by ambleu

### Combining Error Terms into a General Error Term

1  Asked on January 1, 2021

### Should I delete or average repeating training inputs from a Gaussian Process?

1  Asked on December 31, 2020 by mvharen

### Does data point ordering matter in LASSO regression?

0  Asked on December 31, 2020 by rik

### Bayesian inference on mean of statistic from population

1  Asked on December 31, 2020 by helmut

### How to plot $x^{1700}(1-x)^{300}$?

3  Asked on December 30, 2020

### Relaxed Lasso Logistic Regression: Estimating second penalty parameter

2  Asked on December 30, 2020 by joanne-cheung

### Chi squared test questions

0  Asked on December 30, 2020 by woodpigeon

### QQ plot comparison of z-normalized datasets

1  Asked on December 30, 2020 by prinzvonk