Controller intermittent freezing xbox X

Mid-game my controller intermittently becomes inoperable.  Im able to pass but not skate. It's really hard to find success when your player doesn't move and you have to hope you can successfully pass it. Once the pass is completed I can usually skate with the new receiver. Sometimes I have to pass it a couple times then I can skate again. Im on XBOX SERIES X and have tried more than one controller.. same issue