TransWikia.com

Capacity of neural network with one hidden neuron?

Cross Validated Asked on November 24, 2021

8 x 1 x 8

Lets say there are 8 classes. The network has one hidden layer with only one neuron in it. Can the neuron learn to map the identity function between the 8 inputs and 8 outputs? Will gradient descent learn the weights of the identity? What is the difference between this network and a softmax classifier?

One Answer

If you didn't have the 1-hidden layer, then the mapping between the input and the output would be a single-layer neural network. No, your network wouldn't learn to map the identity function between the 8 inputs and 8 outputs unless you work with Autoencoders. This network is indeed the softmax classifier except that we have 8 output classes. It is no longer a network of complex non-linear computations except that the non-linearity stems from the single neuron that we have.

Answered by dangiankit on November 24, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP