Are you using WiFi or an ethernet cable?
Does your game provide ping / latency stats? (may be more informative than from a speed test site, as it will be measured to the actual game servers - for instance, my ping is about 22ms, but I often see pings of 40 - 50ms in-game when I get matched on an EU rather than UK server)