Noether's theorems and General covariance

Given the action
$$ A = int_{M} d^{4}x mathcal{L}(phi, nabla phi) $$
where $mathcal{L}$ is a lagrangian density, or if you prefer $mathcal{L} = sqrt{-g} mathcal{tilde{L}}$ and $mathcal{tilde{L}}$ a function of spacetime.

From Noether second theorem we know that diffeomorphism invariance produces the following identity:

$$ nabla_{mu} T^{mu nu } = nabla_{mu} bigg( frac{2}{sqrt{|g|}}frac{delta mathcal L}{delta g_{munu}} bigg) = 0$$

Where we have an infinite dimensional Lie Algebra $ big( Gamma (M) , [ , ] big)$ where $Gamma ( M ) $ are the smooth sections of the Tangent bundle $TM$.
So, smooth vector field generates the lie group of diffeomorphism, hence diffeomorphism invariance is arena for the Second Noether Theorem.

In principle I know that if we have a family $xi_{(i)}^{mu}$ of killing vector fields($mathscr{L}_{xi_{(i)}^{mu}} g_{ab} = 0$), we can construct from $T^{mu nu}$ a conserved current and a corresponging conserved quantity integrating the current over a Cauchy Hypersurface.

This I know. But my question is: If in principle we start from a diffeomorphism invariant action, what stops me from defining arbitrary $xi_{(i)}^{mu}$ such that with the usual Lie Bracket $[ , ]$ form a finite dimensional Lie subalgebra of $ big( Gamma (M) , [ , ] big)$. The action would still be invariant w.r.t. the action of the Lie Group generated by this Lie Subalgebra, hence I can apply Noether First Theorem obtaining free a conserved current and an integral conserved charge. All this without the assumption that $mathscr{L}_{xi_{(i)}^{mu}} g_{ab} = 0$.

I mean Invariance from an infinite dimensional Lie Algebra implies invariance from a finite dimensional Lie Subalgebra (es. $SO(3)$). So why we need a priori killing vector field in this picture? Where Killing vector field enters the picture if my construction doesn’t need them and is arbitrary on the choise of the Lie Subalgebra generators?