How to bin Colour values (categorical- high cardinality) to primary colours?

I have a colour variable with values for different e-commerce products like – olive, teal, blue, green etc.

I want to bin these colours to the primary colour values.

eg –

olive ---> green
teal ----> blue
blue and green will remain the same.

I also have images available of these products, therefore, I am wondering if there is any solution using some clustering technique or using a pre-trained model for classification in some way.

colors = {"blue": {"blue", "teal"},
          "green": {"green", "olive"},
         }