You are a hardy soul! The problems I have with circulation in my hands (and feet) pretty much rule this out for me if I want to be able to manipulate the brake levers and shift gears (and fit into my cycling shoes). With running, I don't mind the cold so much if I'm working reasonably hard. But on the bike the downhills would be torture. A lot like taking the lifts on a cold, windy day when I go downhill skiing. I shiver uncontrollably the whole way up.
It sure helps having better gear. Neoprene booties mean that my feet slowly get cold but so far not a problem. For the hands I wear a pair of thin gloves inside so big down mittens -- the only problem is shifting the rear derailleur to a higher gear, have to have my hand on the hood and reach down, rather than on the drop and reach forward, but then it works fine. Brakes no problem, just grab them.
Legs stay warm because they're working. Jacket has wind block on the front built in.
Face gets cold, especially on downhills, but it doesn't hurt the way hands or toes hurt when they get cold. So it's just playing the game of mind over matter. :-)
In the winter I tend to ride only on the flats because much less chance of ice on the road, and easier to deal with if there is. So max speed is a lot lower.
Surely better than going to the gym. For me, at least.
The gym feels social, even if I don't see anyone I know or say a word. And then, for me, there's the indoor track, which keeps my training honest in the winter. I find it hard to do the same activity day after day. I need to mix it up, even if its just different varieties of running (trail, orienteering, road, track).