Special Event NPC Die Players?

Just encountered a Die NPC at Rattay which would not leave after one Game. I played like 6 Games with him, made 600 Coins before he left. I then pickpocketed him, because i noticed that some of his die were different.

He had a Lu and Fer Die on him. Did I just gamble with the Devil? Or is this just random?