"Which" can your trust them?

"Which" are campaigning to "Fix bad broadband" and have sent me a link to their speed checker to "find out my real speed so that they can build a picture of the problems people actually face across the country.

So I run the test. No advice, such as to reboot my PC, close other web browser tabs, advise if I am using Wifi and make sure that nothing else is using my broadband connection.

I am suspicios of the latency which comes up as 16-20m/s compared with more like 5m/s that this web site records. The recorded up and down speeds of 500+ mbps are credible, but then I find that I can't select my broadband supplier, Gigaclear, from the drop down list to submit the results.

So can Which build up a sensible picture of the status of broadband across the country if they exclude results from one of the fastest, but small, suppliers? I am inclined to think this is not possible.

57ms latency here O_o, they hosted in russia or something?

Also the download and upload speed is not quite right,

Download speed 79mbps
Upload speed 60.2mbps

Worst speedtest site ever?

Note I expect speedtest sites to report TCP throughput, so on my 79987 sync a 79mbit result should be impossible. The upload completely whacked and the test was done direct not over a VPN.

Interestingly they at least allow you to check localised results (take note tbb).

How your neighbours compare with you

Provider Distance (km) Advertised speed (Mbps) Download speed (Mbps)
Virgin Cable
0.2 200.0 89.0
Virgin Cable
0.2 200.0 80.2
Virgin Cable
0.5 200.0 45.1
Virgin Cable
0.6 200.0 9.2
Virgin Cable
0.9 200.0 45.9
Virgin Cable
1.0 200.0 83.6
Vodafone
1.2 1000.0 15.3
Virgin Cable
1.5 200.0 87.1

2nd test results

Response time (latency)
36 ms
Download speed
81.5 Mbps
Upload speed
29.4 Mbps

Looks pretty grim for VM in my area considering the tester seems to over report speed.

Edited by Chrysalis (Thu 06-Apr-17 22:11:02)

In reply to a post by Michael_Chare:

"Which" are campaigning to "Fix bad broadband" and have sent me a link to their speed checker to "find out my real speed so that they can build a picture of the problems people actually face across the country.

So I run the test. No advice, such as to reboot my PC, close other web browser tabs, advise if I am using Wifi and make sure that nothing else is using my broadband connection.

I am suspicios of the latency which comes up as 16-20m/s compared with more like 5m/s that this web site records. The recorded up and down speeds of 500+ mbps are credible, but then I find that I can't select my broadband supplier, Gigaclear, from the drop down list to submit the results.

So can Which build up a sensible picture of the status of broadband across the country if they exclude results from one of the fastest, but small, suppliers? I am inclined to think this is not possible.

in a word unreliable
SS A tracert of the server ip @ server-54-192-129-22.ams50.r.cloudfront.net gives a steady ICMP result of 31-32ms so could be non mainland uk

Edited by tommy45 (Thu 06-Apr-17 22:35:37)

We don't do immediate local results for good reason, but https://labs.thinkbroadband.com/local has the maps and postcode level analysis of the speeds The search is updated monthly and currently maps are generally done on a weekly basis.

The issues with their tester have been raised publicly and we were told their supplier tests and reviews the testers reliability, so whether anything will change is an unknown.

Obviously Which? is a trusted brand and would never want to publish anything that would tarnish its reputation as a trusted public champion, but we will be monitoring any results published. We would have offered a white label solution of our tester and can do custom UI work (as we have done for various sites) if we had been approached. In addition we are often advising media outlets on broadband, and a good many result in no actual coverage for ourselves but do ensure that what is talked about does have a solid base.

82ms here
61.2 down
7 up

I have 55Mbps lol.

My adsl 2+ connection comes up with latency 222ms(!), down 15.8Mbps (about right), upload 2.3Mbps (real speed around 0.7Mbps).
And Pulse8 not an option as my provider.

WORST? I think worse than that ...

It gives my speed in excess of 80Mbps first time and 88.1 Mbps the second with latency well over 70ms. As I am sync'd at 79 I might expect to see 75 ish, but not 80. One of the neighbours, according to their data, gets 99.6 Mbps on a 76 Mbps link! Also the upload is reported at times as being over 10 - well that would be a miracle as mine is capped at 10Mbps.

Then, when I fill in the provider, advertised speed and postcode, it will not let me submit! So, they are excluding good results!

add the per isp and you golden I want to see VM results for my area on the TBB map

The problem with 'Which?' is that the demographics of their survey responders and 'readership' (or should that be sponsorship?) leans heavily towards serial complainers, accountants, Graundiad readers, those who feel they should 'campaign' on behalf of others and 'activists' - whatever they are. i.e. not representative of the 99th percentile of users of whatever their latest bandwagon is rumbling along complaining about. It therefore doesn't surprise me that their broadband campaign (more precisely, marketing push to boost their subscribers and raise their profile into the publics consciousness once again) would be skewed by 'weighted' test results - or inability to loadshare the server requirements dynamically.

Once the other changes we keep saying are coming soon then I'll have more freedom to provide custom filters for this sort of thing, I won't share my to-do list as in the past ideas have appeared elsewhere after mentioning them, i.e. others throw cash and people at things whereas we may be slower but try to do it right

I dont even see a submit option

Where we run testers for campaigns on behalf of people we do record data for comparison, and with 8 years of data can generally see trends and oddities, the fact we can publish a quarters results within 48 hours of the end of a quarter (compared to the months for say Ofcom) does not mean we don't check, but simply much of it is automated and/or presented in a manner that makes for quick verification.

Mine: 199ms - 8.8Mbps - 2.3Mbps ..I actually have around 7.7/1.0

TBB Speedtest: http://www.thinkbroadband.com/speedtest/results.html...

Neighbours:

Virgin Cable
0.4 200.0 110.2

Virgin Cable
0.4 150.0 39.4

Virgin Cable
0.7 150.0 164.8

Virgin Cable
0.7 150.0 80.4

Vodafone
0.7 150.0 20.3

Vodafone
0.7 150.0 13.0

Sky
0.9 200.0 3.0

Vodafone
0.9 200.0 0.8

so another flaw is they allow sky customers to think they on a 200mbps package.

In reply to a post by MrSaffron:

Obviously Which? is a trusted brand and would never want to publish anything that would tarnish its reputation as a trusted public champion, but we will be monitoring any results published.

I can't go into details here, but quite a few years ago I was working for a company on a joint project with the commercial publishers of Which? to develop a product offering a much better deal in a particular market sector, to be launched under their branding.

All was going well until they suddenly stopped responding to emails. We couldn't make it out. Then we discovered that Which? had decided to go with a competitor, offering a much less good product (lower quality, higher price). Oh, yes and a better margin for them.

Of course they plugged their inferior product as the best deal, but I think customers saw through it and they only did it for a while.

I have never trusted them about anything ever since - and that was admittedly a LONG time ago.

I can't comment on that as I know virtually nothing about what Sky offer

Fastest mainstream product Sky offer is up to 76 Mbps, a handful of users in York on the UFO FTTH trial, but most of those users are with TalkTalk. We see and can identify those users with our thinkbroadband tester but volume is not large enough to make it into our monthly speed test round up at this time.

Ah, right! ...so definitely no-one that close to me is on such a service

Which must notice they're gathering some rather odd results!

Yep that test seems to be an utter load of .....

My a Virgin 200Mb connection has just clock a ping of 128ms.... 239Mb Download and 14.4Mb Upload.

Running a test on Speedtest & uSwitch they both show correct speeds.

Not exactly a good start to fixing bad broadband is it !

The underlying "technology" appears to be from http://www.speedchecker.xyz/

Clearly a lot of unknowns in there .

Have a look at these figures ... the Submit button at the bottom does not work and when te cursor is over it there is a circle with 45degree angled line through.

http://tinypic.com/r/2zjcdv6/9

Considering I have a 80/10 service - yes, it is a historic issue. With max attainable of about 79/24, latency from TBB BQM is around 15ms, so I have no idea where the server is for this!

80/10?

Response time (latency)
115 ms
Download speed
6.4 Mbps
Upload speed
1.1

Not entirely sure I trust the upload - my zen line has never uploaded that fast before.

(poor latency might be due the daughter streaming youtube!)

Virgin Cable
0.5 200.0 29.8
Virgin Cable
0.6 200.0 178.3
Virgin Cable
0.6 200.0 4.0
Virgin Cable
0.6 200.0 2.2
Sky
0.9 1000.0 22.6
Virgin Cable
1.4 200.0 5.6
Virgin Cable
1.5 200.0 128.6
Virgin Cable
1.5 200.0 116.4

I do have doubts about asking the public technical things. Looking at the above seems someone has sky fttp, but only gets 22mb!! At least it confirms one thing for me - Virgin are still very poor locally.

Edited by kebabselector (Fri 07-Apr-17 09:57:35)

Ran the test twice on my Sky 38/10 line (with a router reported sync speed of 22399kbps Down 4996kbps Up)

and got the following results:

Latency 193ms DS 22.0Mbps US 68.0Mbps
Latency 176ms DS 23.0Mbps US 83.6Mbps

If only my uploads were this fast! Latency looks horrendous.

By comparison a TBB test reports

Latency 58ms Avg x1 DS 20.85Mbps Burst x1 21.11Mbps
Avg x6 DS 20.53Mbps Burst x6 22.65Mbps
Avg US 4.09Mbps Burst US 4.21Mbps

The Which speed results are certainly suspect - it gives my ADSL max line an upload speed of 1Mbps which is physically impossible, download of 7Mbps (a bit optimistic), and latency of 170ms which is an order of magnitude too high.

Is there anyone who has had a result that is in line with their connection speed?

https://twitter.com/WhichUK/status/849601833496715265 shows that we raised our concerns with Which? on Tuesday

I did a wired one earlier which was sensible bar the latency. IIRC it gave 271ms. However it has recently been suggested in this thread it is an order of magnitude out. 27.1ms would make sense on my line.

No,

Currently sync'd at 79Mbps so a speedtest will give around 75 or 76 ... with the Which test I see 80Mbps or more almost every time. I have just run a few more: 83.6, 82.4, 90.0, 86.0, 85.4 and then, when I kick off a major download (32MB) and email send/receive during the test, so they are running concurrently I still see 82.6 and 83.7.


As for latency - which is not really relevant in for what they are trying to (badly) demonstrate, it is way over what I would expect for anything UK or even Europe based. From where I spend time in the North of Scotland I get 32ms to TBB compared to 14ms in London which is to be expected - but results that vary wildly from 60 to 125ms cannot be relied on.

Tester is dubious. Having people complain to their ISPs based on suspect data sucks. Will tie up resources and lead to people with legitimate issues getting a poorer support experience.

For some, maybe, I had one result at 56ms which your comment would suggest 5.6 - unlikely.

In reply to a post by MrSaffron:

Is there anyone who has had a result that is in line with their connection speed?

https://twitter.com/WhichUK/status/849601833496715265 shows that we raised our concerns with Which? on Tuesday

Its all hit and miss, some times its not far off, and then you get a result way off from your connection speed, 3 results shown below 1mins gap between the 3 tests:

[Test 1]
65ms latency, 235Mbps (down), 20Mbps (up)

[Test 2]
33ms latency, 695Mbps (down), 95.1Mbps (up)

[Test 3]
20ms latency, 230Mbps (down), 25Mbps (up)

[Speedtest.net]
3ms latency, 234.16Mbps (down), 31.24Mbps (up)

Speedtest Net was done right after test 2.

Paul

My download is roughly the same, upload about 20% more

No, just did a second one and a comparison TBB one:

Result from Which?: Latency:191 Download: 8.9 Upload: 2.3

Result from TBB - TBB

Doesn't seem to want to play with my ADSL 2Meg connection and what about that latency value!

https://s27.postimg.org/cmn6p8rz7/Which_Speed_Test_7...

Try later, might do.

Very, very long ago I stopped taking Which seriously.
They just seem like one of these clickbait sites nowadays.

ahh yeah I do have that button and it works, but I dont think it actually submitted my result, it just showed what my neighbours got.

In reply to a post by MrSaffron:

Obviously Which? is a trusted brand and would never want to publish anything that would tarnish its reputation as a trusted public champion,

Is this the same WHICH, that publicly campaigns against the horrors of recurring transaction.... Yet uses them for their subscriptions. Never mind that they charge different prices for the same products.

Some consumer champion.... So long as they can play the game and get away with it.

Edited by deleted (Fri 07-Apr-17 21:55:21)

Which? may think they are a trusted brand but the number of people I know who have cancelled their subscription recently suggests otherwise.

Could it be that the test data used is compressible, so that the higher than possible speeds are really measuring the success of any compression used?

Doubt its that, you would expect tests to be repeatable if that were the case

And what would be the point of that? I would hope that the specified a pure data speedtest with no compression.

Which
169 ms download 23.5. Upload 9.5,,,
think BB
27ms download 20.4 .Upload 8.8...

One after the other when no on else is using network.

See how your neighbours compare and get tips on tackling slow broadband

5 mins later still waiting for the page to load Only did it to see if they were trying to sell a subscription to get the info....

Here's my proof that tester is tosh .... Just run, via wifi ....

Response time (latency)
64 ms
Download speed
87.1 Mbps
Upload speed
19.0

...The upload briefly showed 22 meg

Yours ....
DOWNLOAD
72.9 Mbps
tbbx1 70.4 Mbps

bursting to 73.9 Mbps

and 16.9 up.

Our friends at Speedtest.net ....

7 ms
DOWNLOAD
73.17 Mbps
UPLOAD
18.31 Mbps

Router stats .....

Data rate:
20.00 kbps / 80.00 kbps
Maximum data rate:
19999 / 103350
Noise margin:
23.5 / 12.9
Line attenuation:
9.9

Tried it the other day and got overoptimistic results.

Just tried again and got 500% higher latency, 10% higher download and 35% higher upload that the test I've just done using the tester on here.

The upload speed they quoted was 25Mbs or 25% greater that the maximum for my 80/20 service!

All tests done using desktop PC with 1Gbs network card connected directly to BT Smart Hub.

Seems a bit overoptimistic. 53/14 with 68ms latency, upload is only 11.5 sync.