What's the purpose of layers without biases?

Question

I noticed that the TensorFlow library includes a use_bias parameter for the Dense layer, which is set to True by default, but allows you to disable it. At first glance, it seems unfavorable to turn off the biases, as this may negatively affect data fitting and prediction.
What is the purpose of layers without biases?

kiarash_kiani · Answer

Bias is one of the hyperparameters in neural networks, which let you shift activation function. Disabling bias means setting bias to be zero.
Even though, in many cases, bias is a big help for successful learning, in some cases, you may want to add an extra constraint to your neural network in finding the objective function. For example, in the paper below, a zero-bias layer as the last layer helps increase the output's interpretability.
https://ieeexplore.ieee.org/abstract/document/9173537

What's the purpose of layers without biases?

One Answer

Add your own answers!

Ask a Question