Why is my cat only losing guard hairs?

He is 6yo, domestic short hair. He had "allergies" last summer so the vet told me. He was missing hair on his legs, mostly the front ones. I kept him inside and all his hair grew back. This summer he started scratching again despite Zyrtec and so I kept him inside again, but this time his hairs have not grown back. Additionally, he has lost his guard hairs all over his belly and legs, his paws are fine and his fur is there.

I haven’t changed his food or anything else about his routine, except that he’s not allowed out anymore, which he hates. I did add a complete oil to his food about a month ago. It’s not fleas or mites. It’s just weird to me that it’s only the guard hairs.

Can anyone help?