Is SUV too big for a learner to drive?

Hi I'm a learner driver, who is very anxious when driving by myself, especially in roads that I have never been to. I literally just drive in my comfort zone, only commute to and from work, which is about 5 min drive, and going grocery shopping, which is also 5 mins away from my house.

I bought my SUV but i feel like it might be too big to drive at this stage. I started to think I should have get a small hatchback such as mazda 3, toyota yaris, suzuki swift and so on.

is it actually more difficult to drive a SUV as a learner driver??