A/B Testing - Repeated Sampling

I am doing an internship in a marketing firm where they do a lot of A/B testing.

My question/concern was that it seems that given a population, they only "split" it into an A and B group a single time and then carry out their measurements. Ideally, to reduce variance and have confidence in the results, shouldn’t they do something like bootstrapping, or repeated sampling of A and B groups?

Thanks.