Just about the only issue I see with not cleaning often enough is the risk in some highly overbore chambered guns, of the additional carbon fouling in the barrel, sitting in contact with the steel, changing its metal properties slightly. Maybe its insignificant, but I don't know enough about metallurgy to say yes or no, and err to the side of caution. Now, for me, cleaning "often enough" means every 50-100rds or annually, whichever comes first. I shoot a lot less right now than I used to, so I'm doing a lot of annual cleanings lately.