Is flying a German flag considered sign of looking down upon others / being a racist?

We've been living in Germany for a few years now and are moving to a new location within Germany. The neighbour across from our new home has a German flag flying.

Our landlady (who has lived at that place for 25 years) said that he might look down upon us since we're brown. I also checked with my German friends and they kinda confirmed it as well.

The thing is this neighbour approached us to introduce himself and has tried to have a conversation with us. So far he has come across as a normal, approachable person.

The point of my question is to check with y'all whether the flag is clouding my opinion about him, or should I be wary of him?

I'd rather not judge someone based on external appearance hence asking y'all.