Join the ever growing crowd. I think most of us readers have felt that way since he initially appeared and purposely ticked off Rin.
Which I think he did in the hopes of separating them and ruining Jun and Rin's friendship. He wants Jun isolated from friends and family. That is something an abuser wants and does. And his asking Jun to never abandon him is another trick by abusers. It is to sow a seed of guilt in the victims when later on, the victims try to leave. The abuser can get all weepy and say "But you promised NEVER to ABANDON me and here you are, abandoning me," so that they (the abusers) can make the victims look and feel like the bad guys for trying to get away and save themselves. Thus getting the victims to stay with the abusers. A very, VERY nasty, but unfortunately successful in most cases, tactic.
Watch. I bet dollars to donuts that the creep will start trying to undermine any of Jun's confidence and love for his parents and friends, and thus make Jun completely dependent on him, both mentally and emotionally, and possibly physically as well.
I don't trust that guy (•ˋ _ ˊ•)