My swoopers-in-residence have retreated from the backyard but still very aggressive in the golf course side. I'd managed to get from the car to the house 5 times without attack but it was balanced by several swoops and connections with my shoulder while in the front area. Strangely the passing golfers avoid attack- yet further away I don't.
The clever golfers probably feed the maggies! There are lots on our course here but I've never been swooped and don't know of anyone who has. We have one that sits on a fellows leg to be fed! They are smart birds, can remember something like 20 human faces.
That's more than I'm capable of.