Centerfires and rimfires are in 2 different leagues IMO.
of the rimfires, 22lr, 22mag, 17hmr, I think I'd pick the 17hmr. but that would mostly be for plinking and squirrels, not coyotes.
The .204ruger for coyotes would be good, w/40gr is actually higher BC's than the .223's
I think the .223rem is still the best option overall I think, at least when we're talking less than 300yds, and in an AR15 a sweet yote setup!
22-250 and 243win are great options too.
you really won't go wrong with any of the 204, 223, 22-250, 243 if you ask me.
223rem ammo is by far the cheapest.
the 204 is cool, something new/different, fast, higher bc's than the 223.
the 22-250 imo is the longer range 223rem, although when you want to shoot past 300yds, I'd probably want a 243win anyway when talking yotes, when talking prairie dogs past 300 I'd probably want the 204, less kick, less powder, etc...
good luck!
oh and to answer your question/post...
17 vs 204, get the 204ruger!
get the 17hmr if you want that too, but they're 2 different setups, I'd get a 17hmr instead of a .22lr but if you already have a .22lr would i get a 17hmr? maybe. but if you want to shoot varmints/coyotes get the 204 ruger not the 17hmr.