Well, that's the thing, it would have been just a promise in the spur of the moment, but he kept contact with her. He was dating her before the bimbo arrived. So it wasn't just a spur of the moment when he was legitimately going out of his way for years after that gaining her trust, only to break it off just like that. Like serious case of don't know what you miss until your gone kind of shit.
how do you know that he kept going in contact with her when it wasn't even shown yet? What if this was his engagement with her was an agreement between the two of their families, so even if he doesn't want it, he'll act polite towards her. Besides, he doesn't remember Su Luxia as the one who was with him when he was a child.
How do you promise to protect her and then leave her for a bimbo who leads you to your death. Asshole!