Uh was I in the right?

So yesterday we had 6 call offs, there is some major flu (i think 4 different strands going around) being up my state. Well we had 3 servers scheduled now since most were off. Two were openers and male. One at 11:30 and female. This id wanna say late 20 y/o guest comes in at 11:05 and it's just him. He says that he "would like a table of 1, but he doesn't want a server, he wants a waitress." I tell him that that's not really how it works. And that we have two choices as of now and that they're both men. Well he insists that he'll wait until a female comes in. I told him that its really not how it's going to go. Got the manager and we basically told him that "Olive Garden doesn't have waiters and waitresses, but that we have servers. All trained the same." Well he insisted that he wants a female to serve him and that he can either be sat now or he can leave.

Yet after the fact, people were saying that I was in the wrong that he has the right to have whoever he wants yet all we had at the time was 2 guys, and it would have been 30 minutes otherwise.

Also is that not creepy? Or am I over thinking?