Say I have a neural net that outputs a vector of length 4 such as:

```
[0, 1, 2, 3]
```

Now say that the *only* way to calculate the loss is to convert this output to a one-hot vector matrix and pass that into the loss function:

```
[[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]]
```

This is a hypothetical question (obviously the answer to this question wouldn’t be to the aforementioned scenario, but to another more realistic, relevant one).

So, once we have calculated the loss using the one-hot vector matrix, is it still possible to back propogate and train the network even though there were two different representations used. A more general question would be, if I convert representations from the output of the neural net to the loss function ( output of neural net => some representation conversion => loss function), is it still possible to back propogate and optimize?

Cross Validated Asked on December 26, 2020

1 AnswersIt depends on how you do this.

If you're using

`PyTorch`

and you do all of your operation using`torch.Tensor`

objects, then the answer is "yes, you can backprop this loss correctly because that's the whole point of using`torch.Tensor`

objects." So, this code will work:`loss = torch.eye(x.size).dot(x).sum()`

where`x`

is your 4-vector. Just replace`sum`

with whatever differentiable function(s) you need. (Rounding or other non-differentiable operations are not suitable.)If you dump your

`torch.Tensor`

objects to`numpy`

, this is not possible because a`numpy.array`

object does not record gradients, nor does it have a`backward`

method.

Correct answer by Sycorax on December 26, 2020

0 Asked on November 30, 2020 by mathias-schinnerup-hejberg

1 Asked on November 30, 2020 by shiladitya-basu

1 Asked on November 29, 2020 by statian

conditional probability gaussian process multivariate normal

2 Asked on November 29, 2020

1 Asked on November 29, 2020 by javier-tg

machine learning maximum likelihood normal distribution optimization

0 Asked on November 29, 2020

0 Asked on November 29, 2020 by theundecided

1 Asked on November 28, 2020 by john

1 Asked on November 28, 2020 by xeon123

0 Asked on November 28, 2020 by ystein-dunker

accuracy interpretation medicine ratio sensitivity specificity

2 Asked on November 28, 2020 by durin

0 Asked on November 27, 2020 by tranquil-coder

generalized linear model maximum likelihood sufficient statistics

1 Asked on November 27, 2020 by souled_outt

1 Asked on November 27, 2020 by user3285148

0 Asked on November 26, 2020 by duke-yue

0 Asked on November 26, 2020 by etang

1 Asked on November 26, 2020 by chris-beeley

0 Asked on November 26, 2020 by jld

distributions extreme value normal distribution order statistics probability

1 Asked on November 25, 2020 by asra-khalid

1 Asked on November 25, 2020 by dkent

Get help from others!

Recent Answers

- kjetil b halvorsen on How to test consistency of responses?
- Justin Markwell on Unity app crashes when using unmodified custom Android manifest (didn’t find class “UnityPlayerActivity”)
- Philipp on How do i draw a ray in unity
- eric_kernfeld on How to test consistency of responses?
- DMGregory on MouseLook Script “Pops” back to the last value when the script is enabled after being disabled or destroyed

Recent Questions

- MouseLook Script “Pops” back to the last value when the script is enabled after being disabled or destroyed
- Unity app crashes when using unmodified custom Android manifest (didn’t find class “UnityPlayerActivity”)
- How do i draw a ray in unity
- How to test consistency of responses?
- How can I understand these variograms?

© 2022 AnswerBun.com. All rights reserved.