Can't select modes when connected to an Xbox controller

I am playing on steam using an Xbox controller. The problem is that when entered the adventure mode, I can't switch between rip, adventure scenes, and cloudy day modes using the controller. Moving the left stick on the controller has no response. Other buttons are functioning perfectly fine. And the left stick works without any problem during the game. It just has no response in this particular scene.

Anybody has the same problem? I don't know if it's the problem with my controller or the game.