Differentiate a numerically defined function

This should work:

Clear[f];
f[a_?NumericQ, b_?NumericQ] := NIntegrate[Sqrt[(Cos[t] - a)^2 + b^2], {t, 0, Pi}]

and then add //N at the end of the definition of g[a,b]

g[a_, b_] := Derivative[1, 0][f][a, b]//N
g[1,1]
(*1.80525*)

Just ignore the error message. Use

g[a_, b_] := Quiet@Derivative[1, 0][f][a, b]
(* 1.80525 *)

This answer can be verified by

(f[1.005, 1] - f[.995, 1])/.01
(* 1.80525 *)

If the function can be evaluated at complex arguments, one possibility is to use the complex step derivative approximation of Squire and Trapp.

For this function, it proceeds like so:

f[a_?NumericQ, b_?NumericQ] := NIntegrate[Sqrt[(a - Cos[t])^2 + b^2], {t, 0, Pi}]

(* complex step approximation *)
With[{x = 1, y = 1, h = $MachineEpsilon}, Im[f[x + I h, y]]/h]
   1.80525

(* analytic derivative *)
With[{a = 1, b = 1},
     NIntegrate[(a - Cos[t])/Sqrt[(a - Cos[t])^2 + b^2], {t, 0, Pi}]]
   1.8052526175436538

Here's a variation of bbgodfrey's answer that doesn't require quieting:

g[a_,b_]:=Activate @ Block[{NIntegrate=Inactive[NIntegrate]},Derivative[1,0][f][a,b]]

Check:

g[1, 1]

1.80525

The trick is that Mathematica knows how to take derivatives of inactive integrals, so temporarily inactivating NIntegrate avoids having NIntegrate trying to integrate a symbolic integrand.