I feel like that man is just lying to him in a way. I believe Al is overworked but he said it himself, "Play is better than rest." He DEFINITELY wants to just do plays and have sex with him. So he manipulated him into believing he was a nuisance/selfish for wanting Al to be his only partner.