So there are a couple if things which sort of stand out to me. First, you need to change your cleaning regimen. Search this forum for the terms "BTE regimen" to see what I and many others do. I suspect when you go through that process, you'll find your barrel was dirtier than you thought. After initial cleaning, you should only have to clean every 30-50rds at first, and that should extend after a few cleanings, as well. With my 30-06, I'm to the point I just clean it at least once a year, regardless of number of shots, primarily to get any excess carbon out of the throat.
Now, aside from that, it sounds like you have a bedding issue with this rifle, related to barrel heat. Check the barrel to see if its floating when cold. Then shoot a couple of rounds, and check it again. Use a dollar bill or a strip of copy paper to check it. I suspect there are some small imperfections in the barrel channel which make contact after the barrel heats up, or perhaps even when it's cold. Identify that as the issue, and then pull the action and gently sand the inside of the barrel channel of the stock, just to knock down the high spots. It would be smart to look at the action and stock bedding under the action for shiny spots where it appears the be rubbing, as well, while you have it out. If so, invest in having the rifle glass bedded and floated by a gunsmith. I'm betting this will identify and solve your issue.