BT Wholesale speed checker - cheating or a legitimate test?

Hi

Found an interesting thing regarding the BT Wholesale speed test at http://speedtest.btwholesale.com whilst trouble shooting slow speeds at peak times on my FTTC connection sync'ed at 80/20.

The BT speedtest is using multiple Http requests to test the download speed. Whilst the Thinkbroadband test was showing a single thread download test at around 11Mbps (mirroring what I was seeing downloading files), the BT Wholesale test was reporting around 50Mbps, almost exactly inline with Thinkbroadbands Http x6 test.

If I download a file it arrives at the speed of a single request, so in reality where I might be seeing 11Mbps download speeds (should be 74Mbps), testing on the BT Wholesale speed test site would show 50 or 60Mbps and so well above the threshold for reporting a problem. So is this a fair test for BT Wholesale to perform?

Edited by deleted (Mon 20-Jul-15 22:45:33)

I see the Maidenhead node at Speedtest.net which is hosted by Xilo is a multi-threaded test, also.

In reply to a post by LC100:

Hi

Found an interesting thing regarding the BT Wholesale speed test at http://speedtest.btwholesale.com whilst trouble shooting slow speeds at peak times on my FTTC connection sync'ed at 80/20.

The BT speedtest is using multiple Http requests to test the download speed. Whilst the Thinkbroadband test was showing a single thread download test at around 11Mbps (mirroring what I was seeing downloading files), the BT Wholesale test was reporting around 50Mbps, almost exactly inline with Thinkbroadbands Http x6 test.

If I download a file it arrives at the speed of a single request, so in reality where I might be seeing 11Mbps download speeds (should be 74Mbps), testing on the BT Wholesale speed test site would show 50 or 60Mbps and so well above the threshold for reporting a problem. So is this a fair test for BT Wholesale to perform?

It's less accurate than other speed testers, it tends to give higher results than what is possible in reality sometimes,

Hi

From memory the BT Wholesale checker used to download a large image and time that, much more real-world. The trouble is if the BT Wholesale speed test is reporting higher speeds than the real situation by running a flash style download with multiple requests, this is likely seeing many people with very slow speeds being unable to progress the issue with BT because the checker says everything is okay.

It seems to be me the BT speed tester is using around 6 requests given it pretty much matched the Thinkbroadand x 6 test, so whilst my real-world speeds were around 10Mbps down from 74Mbps at non-peak times, the BT Wholesale checker was showing me a speed around 6 times higher at 50-60Mbps. This completely misreports the true situation.

For the BT Wholesale checker to acknowledge a problem for me it needs to record a download speed of 40Mbps or less, which in reality means I can only report a problem via this tool when I'm actually browsing and downloading at around 6-7Mbps, down from 74Mbps, which is a very large difference. Is this deliberate, optimise the test as much as possible even if doesn't reflect the real-world situation to avoid having to fix anything, or just an oversight that happens to be in BTs favour?

I can understand commercial speed test sites that want to drive as much traffic through themselves as possible for the ad revenue going for the most optimistic way of speed testing, as I bet most of us would avoid a speed test site that seemed to show a much lower speed as being wrong, but the BT Wholesale checker in my mind should be an accurate reflection of true browsing speeds as otherwise how can we raise a problem regarding congestion and slow speeds when BTs own tool shows no problem? It's BT marking their own exam paper!

What's stopping you downloading a file using multiple requests?

The question is what it is a test of.

If it is a test of the maximum simultaneous throughput of the line then YES, it is pretty much doing the job.

If it is a test of congestion then no it isn't.

What the test shows is that you can max out your line and so the bandwidth is there. What it doesn't show is that there is likely congestion going on.

Currently the rules for selling and advertising are pretty much based on sync speed not throughput. Therefore the test is correct for the current rules.

Hi

Good point, nothing except that it doesn't happen by default, and things like streaming would be a single stream by definition. Also when a single connection has dropped from around 74Mbps to just 10Mbps, that isn't a constant guaranteed 10Mbps, the congestion causes that to be very "bouncy" around that figure.

Of course if everyone started downloading using multiple requests then there would be no benefit, and things would be a lot worse for everyone.

My point is the speed test is exploiting the fact the vast majority of people aren't making simultaneous requests, so pretending to be 6 people rather than 1 for the speed test steals extra from everyone else to get a higher result.

It's a bit like having a 60 watt light bulb running very dim at 10 watt because of power shortages, yet when we complain they come around and plug in 6 60 watt light bulbs running at 10 watt each and say there you go, 60 watt of light, what's your problem

Hi

Ideally it should be a test of both scenarios, it could do two tests which would be more useful, but of course you've hit the nail on the head regarding the rules and advertising of broadband, and this test reports the highest figure which sees a lower amount of referrals to BT Wholesale.

I wonder though how much is spent and hours wasted by ISPs and their support departments because of this, and looking in the wrong places for issues because the Wholesale test is reporting around 6 times higher the speed a user is actually experiencing in practice when their is bad congestion.

Edited by deleted (Tue 21-Jul-15 08:44:18)

Sorry, I don't get your argument. If browsing and downloading can be improved by using multiple streams, just do it.

If the ISP is any good then they will understand the difference between a speed test and real world usage. Speed testers are only a general guide and they have to be interpreted along with a number of other tests to diagnose issues.

The question is whether a congested speed of 10Mbps is actually a fault. The networks are designed to run at contention levels. Unless a contract specifically states that there will be no congestion then slow downs are inevitable. And if the contract doesn't state a minimum busy time rate then the ISP is potentially doing nothing wrong.

Have you asked Xilo what there minimum busy time rate is for your product?

In reply to a post by LC100:

It's a bit like having a 60 watt light bulb running very dim at 10 watt because of power shortages, yet when we complain they come around and plug in 6 60 watt light bulbs running at 10 watt each and say there you go, 60 watt of light, what's your problem

I'm sorry but electricity doesn't work like that.

If you are only getting 10 watts of power down the line then plugging in more bulbs just splits the 10 watts.

Six bulbs would have 1 2/3 watts each, still giving 10 watts.

Hi

It was a way of providing a tangible example of the situation, besides the analogy would work if the voltage of the supply had dropped, rather than each home having a limited 10 watts of power.

I wasn't being literal, it was just an example.

Edited by deleted (Tue 21-Jul-15 09:02:58)

The new Smart meters could make electricity work like that, though

In reply to a post by LC100:

Ideally it should be a test of both scenarios,

True, as the TTB test does. The BT Wholesale test is designed to show the state of the line, which it does reasonably well (though often underestimating upload speed).

The only times I have seen large differences between the two TTB download tests has been because of problems in my LAN, not because of congestion or other Openreach/BT Wholesale/ISP issues on the broadband. Of course, that meant a difference at any time, not just peak times. I do know of other cases where the difference is down to local issues; but I have no idea how often single-threaded limitations are LAN related rather than broadband.

All ookla is multi thread