Healthy eating in social situations

Hi everyone! I love eating healthy and generally have a pretty well-set rule of eating extremely well Mon-Fri (a ton of veggies, very few carbs, fruit, and protein) and then being more flexible over the weekend.
Personally, I don't have a hard time keeping up with this, I love healthy food to be honest. However, it's a huge challenge socially! My boss constantly orders chocolates, cake, and pizza to the office and I'm the annoying one who constantly has to refuse. I go out to lunch with friends on a Thursday and they want to share pizza and dessert and I have to refuse, despite the constant pressures. People make me feel as if I'm rejecting them socially or something. I feel super anti-social, strict, and kind of annoying but I have to have some rules! Anyone have any insights on how to deal with this? Thanks :)