TransWikia.com

Would the support vectors in SVM algorithm change with scaling of the functional margin?

Cross Validated Asked by Sud K on December 23, 2021

Would the Support Vectors in the SVM algorithm change every time that I change the functional margin ?
The optimization objective in the SVM algorithm is this –
enter image description here

The rest of the SVM optimization algorithm is then derived by setting Gamma hat i.e. Gamma (hat) to 1. Does the Support Vectors derived depend on Gamma, w or b in any manner ?

One Answer

The original answer is here courtesy Arun Iyer.

The entire algorithm is then derived by setting i.e. Gamma(hat) to 1

That is not exactly true, although that’s certainly one way to think about it. The algorithm proceeds by replacing w and b with new variables w=γ^w′ and b=γ^b′. You will notice that when you do these substitutions, γ^

will disappear.

So, in this new formulation you are solving for w’ and b’ instead of w and b. Now, given this information, you can make two observations:

You can obtain the solution for the original formulation for any γ^

from the new formulation by scaling w’ and b’ with γ^ to get w and b. So, we can reason about the solution to the original problem and find that the solution to the original problem must be γ^=1∥w′∥,w=w′∥w′∥,b=b′∥w′∥

.

Now, given all this information, can you reason about the support vectors since you know the forms of w’ and b’ from the KKT conditions?

The answer is that Support Vectors will change but only in magnitude.

Answered by Sud K on December 23, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP