We get angry drivers going the wrong way down our street when the bridge causes traffic to backup down Pearl, so I tend to notice it when it happens. I have no idea what could reasonably be done about it but it seems worth trying to sell some parking spaces ("walk to fenway, save yourself 45 minutes of sitting in your car going nowhere") but I'm not sure how many of the people stuck on Pearl/Granite are actually going that way, rather than just being victims.
I do wonder where people are going when they enter that morass. The issue is that Pearl seems empty until you get past Putnam and then there is no escape once you pass Chestnut. I can't really blame people too much for going the wrong way since the traffic is so, so bad.
There really is not solution, though, because there isn't any space. Well, riding a bike is a perfectly good solution. For transit, the DCR could create a busway through the middle of the rotary (under the bridge) which would let the buses bypass that bit, at least. The City could help out by designating the left lane of Granite Street for left turns only except buses, and then carve out a bus lane leading towards the rotary. It wouldn't be perfect, but better than nothing.