Do Games With Ray-Tracing Support Run Worse, Even If Ray-Tracing Is Disabled?

I'm curious if a game built with support for ray tracing still runs worse with it disabled, like do the built in tools that make ray tracing work take a dent on performance even if ray tracing itself isn't on? If Game with RTX and game without RTX (lets say its the exact same game but one has RTX support and one doesn't) were pinned side by side, would they run the same? Or would the one with RTX support even if disabled still run worse?

https://preview.redd.it/x38mpz5jon6e1.png?width=2560&format=png&auto=webp&s=b575a379ab28c6af2d550c7af4f9b210f2f0036f