My boyfriend finds it annoying that I go to gym at 5:30am and go to sleep early at night

During the week really like to go to the gym and lift early in the morning before work. It gives me energy and sets my mind right for the day ahead. Because of this, I usually go to bed at night around 9pm. My boyfriend is the opposite, he goes to the gym around 8pm and then goes to sleep around midnight or 1am. He always makes comments about how crazy and ridiculous it is to wake up that early, but I don’t give him a tough time about his schedule. I can tell he also gets annoyed at night when he wants to FaceTime and talk on the phone and I’m basically ready to go to sleep then.

We don’t live together (yet) but I’m starting to wondering if in the future this is going to become an issue when my 5am alarm goes off. I can’t decide if this is “different strokes for different folks” type of situation that’s no big deal, or if it has potential to turn into an actual element of friction in a relationship living together. I know only a conversation with him can truly answer this, but I’m wondering if anyone else has had similar issues with their partner and how they went about working through it? Did it impact your relationship?

EDIT: you probably could have assumed this but we are in our mid to late twenties