you on fast path, do a traceroute or mtr instead of pinging, that should show where latency is jumping up.
here is my trace route
Tracing route to google.co.uk [173.194.34.159]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms SkyRouter.Home [192.168.0.1]
2 * 3408 ms 260 ms 97e70003.skybroadband.com [888888888]
3 32 ms 31 ms 31 ms 0278098c.bb.sky.com [99999999]
4 35 ms 30 ms 31 ms 89.200.131.173
5 * 29 ms 28 ms 72.14.215.13
6 27 ms 27 ms 27 ms 209.85.252.188
7 27 ms 27 ms 28 ms 209.85.253.115
8 29 ms 51 ms 29 ms lhr14s21-in-f31.1e100.net [9999999999]