So you preferred if he was a criminal...?

I think it's part betrayal because the only thing he thought he knew about Jay turned out to be wrong when Shin actually chose him over everything else (telling him he would run away with him if Jay needed him to and all) ; and part the fact that if Jay is actually a government agent it means - at least that's what Shin thinks - that Jay used him and deceived him. Mainly, Shin is just too angry to think things through right now