How many ways are there to select exactly one heart in a hand of 5?

I think it's easiest to solve these types of problems thinking about conditional probability. Think of it this way:

Fix a card, out of the 5 possible cards where the heart will appear. Say this is the first card you draw. The probability of drawing a heart on the first draw is simply:

$P(Heart) = 13/52. $

Now, you need to chose the second card given that you drew a heart on the first draw. There are only 51 cards left after the first draw and 39 of them are not hearts, so you have:

$P(H^c|H) = 13/52 times 39/51$

where $H=heart$ and $H^c=$ is the complement or "no hearts."

Now, you need to chose the third card given that you drew a heart on the first draw and a non-heart on the second draw. 38 non-hearts remain out of the remaining 50 cards left at this point so you have:

$P(H^c|H, H^c) = 13/52 times 39/51 times 38/50$

With similarly reasoning with the fourth and fifth draws you obtain:

$P(H^c|H, H^c, H^c) = 13/52 times 39/51 times 38/50 times 37/49$; and

$P(H^c|H, H^c, H^c, H^c) = 13/52 times 39/51 times 38/50 times 37/49 times 36/48$

Because by properties of probability the probability that you select only a single heart on the first draw is:

$P(H) = P(H) times P(H^c|H) times ... times P(H^c| H, H^c, H^c, H^c)$

Now the probability above is for a heart on the first draw and no others. But you are trying to find a single heart in any card: the first, second, third, fourth, or fifth. So there are $5 choose 1$ $= 5$ possible places where you can draw the heart, so you need to multiply the above result by the 5 position positions where the heart will appear which yields:

${13over{52}} times {39over{51}} times {38over{50}}times {37over{49}}times {36over{48}}times 5 approx 0.4114196$.

Counting unordered outcomes. You seem to be mixing up ordered and unordered outcomes. Card hands are ordinarily viewed as unordered because, in most card games, the order of dealing has nothing to do with the value of the hand.

So you are correct to choose 1 Heart from among 13 Hearts and four non-Hearts from the rest of the deck: ${13choose 1}{39choose 4}$ ways in which to choose the desired 'hand' with exactly one Heart. Note that HDDCS is considered the same hand as DCDHS. (Because the order of dealing doesn't matter, you might consider listing the symbols for the four suites in alphabetical order CDDHS for tidiness.)

If you want the probability of getting exactly one Heart, then you need to divide by the total number of (unordered) hands, which is ${52 choose 5}.$ Computation in R:

choose(13,1)*choose(39,4)/choose(52,5)
[1] 0.4114196
dhyper(1, 13,39, 5)
[1] 0.4114196

The random variable $X$ that counts the exact number of Hearts in a fairly dealt 5-card hand has a hypergeometric distribution. In R (where dhyper is a hypergeometric PDF), the probabilities of $P(X = k)$, for $k = 0,1,2, dots, 5$ can be tabled to four-place accuracy as follows. (Ignore row numbers in brackets [ ].)

k = 0:5;  pdf= round(dhyper(k, 13,39, 5), 4)
cbind(k, pdf)

     k    pdf
[1,] 0 0.2215
[2,] 1 0.4114
[3,] 2 0.2743
[4,] 3 0.0815
[5,] 4 0.0107
[6,] 5 0.0005

Simulation. We can approximate this distribution by simulating a million fairly dealt hands and tallying the resulting numbers of hearts in each hand. (For simplicity, we use the notation 1 for Heart, and 2 through 4 for the other three suites. Without additional parameters, the R function sample samples at random without replacement.)

Notice that observed proportions of hands with zero through five Hearts agree with the exact hypergeometric probabilities to two or three places.

set.seed(2020)
deck = rep(1:4, each=13)
nr.h = replicate(10^6, sum(sample(deck, 5)==1))
table(nr.h)/10^6
nr.h
       0        1        2        3        4        5 
0.221315 0.412048 0.274044 0.081414 0.010698 0.000481 

In the figure below, histogram bars show simulated proportions and centers of small red circles show exact hypergeometric probabilities. The resolution of the figure is about two decimal places, so agreement appears as near-perfect.

hdr = "Simulated Numbers of Hearts in 5-Card Hands"
hist(nr.h, prob=T, br = (0:6)-.5, col="skyblue2", main=hdr)
 points(k, pdf, col="red")