Why do we care about indistinguishability?

Adding to J. Murray's answer.

Quantum mechanics adds a layer of confusion over this, in my opinion. The concept of indistinguishability can be exemplified even in the classical world.

In many instances, the answer lies in your question: "do you care?" Meaning: "is the question you asking (or, in other words, the calculation you are setting up) dependent on particles identity?". Which boils down to asking how you define the state of your system: are you happy with macrostates or do you need microstates?

I don't see how not being able to tell the particles apart actually changes anything.

It does change if the experiment you are "modelling" is able to distinguish between particles. Imagine you are writing the configuration of $N$ (classical) particles as "the vector of number of particles that sit at a given position". This is a macrostate, similar to the example of "how many electrons per energy shell". You will say there are $n_1$ particles at position $x_1$, $n_2$ particles at position $x_2$ and so on. You might like to write this state, borrowing Dirac notation from quantum mechanics, as the vector $|n_1,,n_2,,ldotsrangle$. Implicitly, you are stating that the particles are indistinguishable, as your state representation does not feature particles identity. Here, indistinguishability is a property of your model, independently of whether your particles are indistinguishable or not. Presumably, it is because the experiment you are planning does not involve tracking individual identities.

Obviously, however, you can construct this state vector as a sort of "projection" of the vector specifying the position of each individual particle, that we might instead write $(x_{i_1}, x_{i_2}, ldots, x_{i_N})$, where the index $i_j$ stands for the position where particle $j$ is sitting. These are your microstates. In the example here, microstates with the the same number of particles per position index contribute to the same macrostate. If you choose this representation of the states (because you are very keen on keeping track of all the identities of your particles), then you are limited by whether your particles actually are indistinguishable $-$ the electrons-vs-nucleons example given above shows this.

Let's say you have a system of two distinguishable, non-interating particles (say, a proton and a neutron), each of which can inhabit one of two available energy levels, $E=0$ and $E=epsilon$. The macrostates of this system can be characterized by the total energy, which means there are three possible macrostates corresponding to total energy $mathcal E_0=0,mathcal E_1=epsilon$, and $mathcal E_2 = 2epsilon$.

How many microstates correspond to each macrostate?

1. $Omega(mathcal E_0)=1$ because there's only one way for both particles to have zero energy
2. $Omega(mathcal E_1)=2$ because the proton could have energy $epsilon$ and the neutron could have energy $0$, or the proton could have energy $0$ and the neutron could have energy $epsilon$
3. $Omega(mathcal E_2)=1$ because there's only one way for both particles to have energy $epsilon$

---

Now repeat the same analysis for two indistinguishable electrons, and there's a big difference - $Omega(mathcal E_1)=1$, not $2$. The state with electron A having energy $0$ and electron B having energy $epsilon$ is the same state as the one with electron A having energy $epsilon$ and electron B having energy $0$. In fact, it doesn't even make sense to label the electrons as A and B.

This is the essence of indistinguishability - for a system of indistinguishable particles, we identify states which are related to each other via permutation as being the same state. For a system of distinguishable particles, this is not so.

For what it's worth, your factor of $frac{N!}{prod n_i!}$ is indeed the correct "overcounting factor" which we would need to include. If we assume that the particles are distinguishable, then this factor would be completely absent, but this leads to problem with non-extensive entropy (see the Gibbs Paradox). If we assume that the energy levels are sparsely populated (so the likelihood of any $n_i$ being greater than 1 is low), then we can approximate it by $N!$; this yields what we would usually call classical statistical mechanics.