What prevents a Fallen from getting an arbitrary number of thralls?

There is no mechanical ruling on how many thralls demon may have. Though on p. 155 followers are described as

Followers are servants, worshippers or even loyal hirelings working for you. They’re people willing to devote a substantial amount of their time (perhaps even all of it) to taking care of various tasks for you. A demon’s followers are often (but not always) sources of faith and worship.

So, if your thralls invest a large amount of their time into doing your bidding - Storyteller may demand you to take them as followers (and thus limit you to 5 actively helping you thralls)

Also, p.250-251 say about thralls

While the systems and mechanics of offered Faith are simple, it is not a simple thing to groom someone as a source. Entering into a bargain is a momentous and dangerous act, and demons must work hard to find mortals who have the potential for being worth the trouble and effort. This is something that can only be done through roleplaying, not through rules. Responsibility for choosing a worthy mortal and offering him a pact rests with the character (and her player).
...
While some of their Faith can be offered to their dark master, it’s a smaller amount than the demon might prefer. This decision depends strongly on the nature of the relationship between demon and mortal, and on the mortal’s own spirituality. It doesn’t really matter if the mortal loves and worships the demon or fears and hates her. What’s important is how close the bond is between them. If the mortal rarely thinks of the demon or of the gifts she’s given him, then he might not be as strong a source as a mortal with the same Faith rating, who thinks of the demon constantly and relies heavily on her infernal gifts.

Also, sidebar on p. 252 says:

Faith potential
Mortals don’t have a Faith rating. Instead, they have a Faith potential, a measure of how much Faith a demon can use to reconfigure their souls.
The Storyteller decides what Faith potential a mortal possesses. Most humans have a Faith potential of 2. Someone such as a devout priest with strong religious beliefs (no matter how twisted) would have a potential of 3. A Faith potential of 4 is the province of the fanatically religious, such as a fundamentalist preacher or Buddhist monk. Only a handful of people possess the determination, belief and devotion necessary for a potential of 5 — this is the hallmark of the pope, Mother Teresa or the Dalai Lama. Mortals with a potential of 1 are spiritually weak, with no religious beliefs and little ability to believe in the demon’s power. Such mortals make poor thralls, but may have other abilities that compensate. People with zero potential exist, but they are rare. Such spiritually dead, deeply skeptical mortals cannot enter into a pact with a demon.

On p. 253 - there are rules on how that faith potential can be used, one of which states that only up to half of that potential can be offered as faith for demon.

And now to summarize it. While there it no mechanical restrictions on how many thralls you may have, there are roleplaying ones. First - you need to find a proper thrall - someone, who'd fit you, someone who'd want what you can give (for example, someone wanting to be rich, won't be easily swayed by a demon, who is broke himself), then you'd need to convince them to make a pact with you and then you need to maintain your bond with them. All those actions are time consuming. The more thralls you have - the more time you end up 'managing' them and not doing anything to advance your agenda. Actually, you may end up spending more time advancing agendas of your thralls to maintain bond with them.

For example - imagine that you are a rich demon, who met a bum named Chester and decided to make him a thrall purely for Faith gathering. Chester is excitable, easy to impress and convince, he doesn't have any special spiritual features and have faith potential of 2, like most people. And he eagerly agrees to sign any pacts with you for a thousand dollars. You pay him, reserve one point of faith as your daily 'offering' and spend remaining point, that you can't take, to give Chester peak health - you don't want your faith farm to die from some cold, right?
For some time - let's say a week or month - you recieve faith from Chester, just as planned, and Chester somehow manages to not get killed by those envious of him, to not die from overdose/overdrinking, etc - but then it stops. You dedicate some time to finding Chester and understand that money you've given him are gone, all spent and he no longer remembers you often enough to maintain a strong connection. You solve the problem by giving him more money and make it a part of your schedule - to visit him every week and give him money. For some time it works - but then flow of Faith stops again and after dedicating even more of your time to investigating it - you find out that Chester just got used to reciving money every week. To counter it, you decide to dedicate more time to Chester, not only meeting him to give him money, but stopping to talk to him, making sure that he keeps being impressed with you, or start tormenting him with reminders that everything he has now, including his own health, is given by you, making sure that he is constantly afraid of you taking it away. Tormenting is, probably, easier, while indoctrinating him may cause his spirituality to grow and his faith potential one day getting to rating of 3 or even 4. In any case - you spend your time with him. And then imagine having more than one such thralls? Soon all your schedule would be full of meetings and calls to and from thralls.