Like I know the girls are concubines but I didn't think they were having sex yet.

Sorry! I accidentally clicked thumbs down but I agree with you,i think they didn't do it yet.

But didnt the maid ask the first concubine if everything was alright with the last visit from the Emperor to her chamber (which is usually a synonim for sex, since why else would he directly visit her chamber?)

exactly
it is implied