Upload speed is normally a lot lower than your download speed. It's also dependent on your location. The further from the server you connect to, the lower your speeds are going to be. Regardless, 35 for an upload speed is fine.
I just tested my connection through the Xbox network settings. Before launching the game and after. In both tests I was about 800 down and 20 up. That download speed very far exceeds what is needed for smooth online play. The upload speed is sufficient as well and I rarely ever have any issues. Upload is not as important as download. We get very few reports of unresponsive play so that further points to either an issue with your network equipment or ISP. Having a sluggish feeling is near impossible for us to test for though. What feels sluggish to you might feel normal to someone else. If you have visible stuttering or you can show hitting a button on your controller and seeing a delayed response on screen, that would indicate an issue we could troubleshoot.