I'll agree with the above advice regarding shooting distances. The -06 is a fine rifle, with the right bullets for elk, and will kill at the ranges you mention. The real question (as above, and regardless of whether you use your -06 or buy a 300Wby or whatever) is whether or not you have access to practice at 400 and 500yds, regularly. If you do, you need to be sending rounds downrange with boring regularity, and you need to be able to make 1-1.5MOA groups at those distances from field positions (not from a lead sled or a bench rest or even a big pile of sandbags). If you can maintain 1.5MOA at 500yds from prone over a daypack, off of shooting sticks (kneeling or standing), and using a post to steady yourself (like you might use a tree trunk in the field), you're set. If you cannot practice at those ranges, you have absolutely zero business attempting a shot at those ranges on game. You won't even know if your drop charts are realistic (and there's a good chance they're not, if you haven't shot out your range card prior to the hunt). Maybe you know all this already, and if so, no offense intended.
If it were me, I'd think about something like the Weathermark as it will be plenty durable, and you won't be worried about dinging the stock. It will weigh about 9.5-10lbs ready to shoot, so it won't be light. But it will be a good compromise between weight and recoil. I'm not a fan of brakes because I already have some hearing loss, and don't want to wear earplugs in the field, but perhaps you don't have that aversion. If so, a ULW with a brake might be a good option.