Which speed test should i trust

I have a package of 20Mb/s (microwave to premises.) BT Wholesale tester says I have a download of 40Mb/s, thinkbroadband 15 and ookla 22.
Now a gut feel is that Ookla is nearest the mark because I can download 4K Ultra HD films from Amazon and that needs a pretty consistent speed around 20. The BT one is bonkers as my router is only rated at up to 30 Mb/s and the thinkbroadband needle is more like a high speed windscreen wiper. All agree on an upload of just over 4Mb/s.
The most difficult thing to prove to your ISP is that the speed has dropped but how can users do this given the range of speeds. Not surprisingly my ISP Boundless Communications, prefers Ookla.

Generally speaking I'd go with TBB. Of the three it's the only one that does both single and multiple thread tests. BTw I would ignore. My results aren't as bad as yours but it's never been very accurate.

However I'll also point out that the you are testing the transfer speed between you and the remote server. Unless the servers are located in the same location (which they aren't) your data is going to be travelling across different parts of the network. It is entirely reasonable for their results to differ. In the same way that if you measured your average speed driving to Exeter it would be unusual for it to be the same as your average speed to Edinburgh. Those averages could change at various times and sometimes it's higher for Exeter, sometimes for Edinburgh.

It's best I think to pick a collection of testers and run them all regularly. That way you're looking for changes in normal behaviour rather than just looking at an almost arbitrary headline speed.

From experience and knowledge of how they work, I would say the TBB is probably the better.

What I would suggest is also having Windows Task Manager running on your PC and you can then see what is happening on each test ad with some basic maths, calculate the approximate speeds for each and see if the tester is reasonably accurate.

The Ookla test is probably reporting more biased towards the maximum throughput achieved during the test whereas the TBB test will be reporting the average throughput. So, Ookla is showing what the line is capable of but TBB is showing what it is really doing.

It's possible that wherever the microwave is connecting to has shared backhaul with other users and therefore if that backhaul is contended then it could produce variable results depending on what other users are doing at the time. It could be they are service 20 users off a shared 100Mb/s backhaul and therefore it only takes 5 to be fully active to saturate the backhaul service (obviously made up figures as an example without knowing the design of the network).

The BT Wholesale test is meant as a tester for people using their wholesale network so not relevant.

Looking at tests for boundless my guess is that this is one of your tests
http://tbb.st/146227037110452064767 and given the shape of the graph no surprise that Ookla says 20 Mbps based on what is known. We record the charts to make post evaluation like this possible.

An odd test was http://tbb.st/146297021121088524891 but that might have been flash playing around. I'd suggest also trying http://labs.thinkbroadband.com/speedtest to see how that performs

One of the better tests was http://tbb.st/146082864502677429716 but still a long way from what I'd call a good test (e.g. http://www.thinkbroadband.com/speedtest/results.html... ) and that is good because the throughput is stable over time. If you are using Wi-Fi on top of the microwave (fixed wireless) solution then your tests make sense, since wireless tech can often be a bit more bursty and variable in how it behaves.

On the needle behaviour be wary of interpreting too much from the needle, we could smooth the needle out mathematically for a more pleasant UI experience and we know other testers that do (some of our partner testers do do this, but underlying data is not affected), but this can give a false impression of stable speeds sometimes.

How much needle variation is considered "good"? When i do tests my graph will flatline on x1 and x6 which i consider excellant as it means there is no variation in my speeds, but the one you posted that you consider a good test has both the x1 and x6 tests with speed variations during the test

I always trust the test on this site as if i have routerstats open during a test, i can see if i get a burst of CRC errors this is reflected in the graphs. Something that isnt on most tests

Edited by bobble_bob (Thu 12-May-16 16:49:52)

http://www.thinkbroadband.com/speedtest/results.html... the httpx6 is what I'd call stable, and the tbb download (Single thread) just shows a ramp up at the start which is not unreasonable for any connection (RWIN).

Upload is generally always a bit more noisy.

Need to see the link to your tests to really say more

To be honest you cannot take any as Gospel and they all praise the merits of their own site. A little like losing weight - use the same scales and you have some idea where you are. Ookla if you want to see the peak, or openspeedtest or TBB who both aim at realistic user speed. Openspeedtest also avoids the pitfalls of Flash, but does not suit all browsers or circumstances despite their claim.

http://www.thinkbroadband.com/speedtest/results.html...

Thats one i just did. Couldnt get that much straighter with a ruler

The upload does tend to jump around abit more normally, as you described

Edited by bobble_bob (Thu 12-May-16 19:58:22)

http://www.thinkbroadband.com/speedtest/button/14630...

Above is a test taken about 5 mins ago on TBB.
All other tests are about the same including BTW.