Moving the controller instead of the stick seems silly. It might solve your own issues with not being able to control the stick accurately, but it's not going to put the ball exactly where you aimed it - there's still some shot dispersion and variable wind direction to introduce a degree of variability out of your control.
I'm with Putt on this one, in that randomness seems to be almost (but not entirely) independent of your inputs. It's not clear how your swing determines shot dispersion. That might be because it truly is random, or it's because the game isn't giving us the right feedback to understand how the swing input determined the ball trajectory.
I think the latter is partly true. Something is being lost in the abstraction used to give swing feedback, leading to uncertainty about why two 'identical' shots lead to different outcomes - e.g., Putt's videos from 17th at Pebble.
For what it's worth, I think people are too quick to blame randomness for their own lack of understanding of the basic aiming mechanics and, consequently, think others are cheating to not be subjected to the same 'random' outcomes. Randomness is not the difference between the top and bottom of the leaderboards. It has a noticeable effect on almost every shot, but it's not that much.