I don't have experience with this, but if the doc says 12 weeks, then I would give it 12 weeks. He knows better than either of us what the actual recovery time is. I'm sure you feel great, but get a few miles down a winter road and see how you feel. Also, you don't want anything to pull loose.
Besides, how much of a sacrifice is it to miss a winter of biking through snow, sleet, and slush? :-)