Who is being selfish?

Here's a conversation I had with my ex-boyfriend only about a million times... He'd want me to do something with him - like go somewhere - and I'd say no thank you. Then he'd accuse me of being selfish and only thinking about myself because I didn't want to do what he wanted to do. Then I'd say are you sure it's me who's being selfish since you are the one who is upset because I'm not doing what YOU want me to do! No, he'd reply, he wasn't being selfish... and then he would stubbornly maintain that if I didn't do what he wanted me to do, I was being selfish. Don't you just love it!