The interesting question is what fraction of rides need to be tougher in order for the remainder of them to still seem easy/pleasant as the years continue. My speculation is that a not very large portion of them need to be pushing the limits, and probably one should keep it that way to avoid injuries which with growing age seem to take longer and longer to heal (and in the process just increase the rate of average physical decline). But I don't have any real feel whether that means 1 or 2 days of hard workout per week or per month! For me, the usual motivating factor is getting out the door a little later than I should have for a morning meeting, and having to make up ten or twelve minutes over the usual commuting time. In any case, I think there is some real merit in having a significant part of one's daily exercise taken up by activities which didn't form too major a component of competitive life at a more youthful age, so that taking a more leisurely day doesn't make quite such a vivid (and depressing) mental contrast with time performances from four or five decades earlier!
I honestly can't think of a day in the last couple of years (since I stopped orienteering/running/golf) when I have been depressed about my declining abilities. It's just the way life is, you age.
It helps that my main form of exercise now, biking, is not something I ever did competitively. But I can look back at years when I biked more (usually due to running ailments) and it's fair to say I had a competitive attitude, against myself, against previous rides on the same loop.
That has changed. I still put out hard efforts at times, but much less often, and I've gotten much better at being happy to go at moderate or easy paces or whatever the situation calls. It's refreshing.
This will all end at some point. But until then I'm certainly not complaining.