Speed isn't the issue. However, 20 mb is on the lower end. Even on a high speed connection, the stability of the connection is most important. That's just not something that wireless offers as it is highly susceptible interference from other signals, walls, floors, other devices, etc. What you'll need to do to test if it's due to the wireless or not is switch to a wire to see if your connection improves. If that is not possible, the best I can advise is to go through the troubleshooting steps in this article.
You can also contact your ISP to see if they can help troubleshoot your equipment.