My Cursor Won't Stop Moving

When playing Sims 4 on a Xbox console, my cursor continues to move. It will scroll up all the way, and then it was scroll down all the way. Which makes playing very difficult. I like to build a lot and I um have trouble in the building and gameplay because of the movement of the mouse cursor. I've been trying to figure out why this is happening, because my controller was working for Sims, 4 back 2 days ago, normally without having this problem, and then it just started up. I wondered if it was a glitch or if it is my controller.But I wanted to make sure that it is a glitch before I decide to buy a new controller.