Probably getting a bit off topic, but your right I hadn't considered that.
As you suggest, running the ping directly on the router whilst speed test on a different box gave completely different results - well during the upload test at least.
During download (on the server)
64 bytes from 212.58.237.253: icmp_seq=11 ttl=56 time=10.737 ms
64 bytes from 212.58.237.253: icmp_seq=12 ttl=56 time=12.192 ms
64 bytes from 212.58.237.253: icmp_seq=13 ttl=56 time=11.716 ms
64 bytes from 212.58.237.253: icmp_seq=14 ttl=56 time=12.056 ms
64 bytes from 212.58.237.253: icmp_seq=15 ttl=56 time=11.068 ms
64 bytes from 212.58.237.253: icmp_seq=16 ttl=56 time=12.035 ms
64 bytes from 212.58.237.253: icmp_seq=17 ttl=56 time=10.264 ms
64 bytes from 212.58.237.253: icmp_seq=18 ttl=56 time=11.630 ms
64 bytes from 212.58.237.253: icmp_seq=19 ttl=56 time=10.353 ms
64 bytes from 212.58.237.253: icmp_seq=20 ttl=56 time=11.775 ms
64 bytes from 212.58.237.253: icmp_seq=21 ttl=56 time=12.453 ms
64 bytes from 212.58.237.253: icmp_seq=22 ttl=56 time=10.611 ms
64 bytes from 212.58.237.253: icmp_seq=23 ttl=56 time=11.899 ms
64 bytes from 212.58.237.253: icmp_seq=24 ttl=56 time=11.505 ms
64 bytes from 212.58.237.253: icmp_seq=25 ttl=56 time=11.717 ms
And during the upload test
64 bytes from 212.58.237.253: icmp_seq=27 ttl=56 time=4.327 ms
64 bytes from 212.58.237.253: icmp_seq=28 ttl=56 time=4.957 ms
64 bytes from 212.58.237.253: icmp_seq=29 ttl=56 time=5.109 ms
64 bytes from 212.58.237.253: icmp_seq=30 ttl=56 time=4.601 ms
64 bytes from 212.58.237.253: icmp_seq=31 ttl=56 time=4.532 ms
64 bytes from 212.58.237.253: icmp_seq=32 ttl=56 time=4.536 ms
64 bytes from 212.58.237.253: icmp_seq=33 ttl=56 time=4.662 ms
64 bytes from 212.58.237.253: icmp_seq=34 ttl=56 time=4.943 ms
64 bytes from 212.58.237.253: icmp_seq=35 ttl=56 time=5.382 ms
64 bytes from 212.58.237.253: icmp_seq=36 ttl=56 time=4.569 ms
64 bytes from 212.58.237.253: icmp_seq=37 ttl=56 time=4.559 ms
64 bytes from 212.58.237.253: icmp_seq=38 ttl=56 time=4.347 ms
64 bytes from 212.58.237.253: icmp_seq=39 ttl=56 time=4.731 ms
Much better
--- uk.www.bbc.co.uk.pri.bbc.co.uk ping statistics ---
46 packets transmitted, 46 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 4.327/7.059/12.453/3.103 ms
Giganet 950/950Mb (CityFiber), Vigor 2927 with Three 4G Backup