Nope. The other priests realized it's not good to be around Gabriel, however, they didn't officially lowered his rank, nor did they kick him out. And they're not gonna do anything about him, cuz they can't kick a high ranking (and by public beloved) priest without any solid reason. Keep in mind, that only main characters and the church know about Gabriel's faction's wrongdoings. And the other 'good' priests aren't gonna expose the truth simply cuz of the backlash the church would get from the public.
Right now, Gabriel is still a very powerful and high ranking priest who even has his own group of priests following him cuz they believe in his ideology.
So that's why they can't reject him, he's still officially a high ranking priest of this empire's church and is beloved by all.
If the Church is distancing themselves from the priest, wouldn't they technically be able to reject his visit based on the fact that he's lost his backing as a representative of the Church?