SamKnows whitebox testing - video tests

To anyone who has (or is considering) a SamKnows testing unit:

This week the video testing which was in beta test and activated from a option at the top of the dashboard has gone fully live. The opt in has been removed and the tests are automatically enabled for anyone they believe is on an ISP offering unlimited usage. As a result the about of this a the amount of usage consumed by the tests has shot up.

They have confirmed to me that their tests via the box I have is consuming at least 2.1GB per day. From the experience of a number of people there is reason to suspect it could be much higher. One user has reported usage of 7GB per day when he's been away!

I believe they still recommend you only connect it on connections that are unlimited bandwidth and not on metered connections. As long as the intelligence works and only runs when the network is idle then it shouldn't cause people an issue (although that does mean you have to have it connected up properly - ie everything routed through it which isn't good if you want gigabit Ethernet from your router).

Time for ISPs, in particular wholesalers, to start blocking or charging Samknows it then. It's a serious addition to backhaul requirements and could easily add to congestion on ISPs. But that doesn't work well with the idea of net neutrality.

https://www.samknows.com/ - 440,000 volunteers

Even if only half of them are on video testing that's at least 13PB of data they are shifting round the internet each month!

I have one. But given that most months I am well below average for my bandwidth usage then I don't see it as a big deal.

The black box has gigabit Ethernet ports on it. Simply email support, you send the white box back and they send you a black (D-LINK Archer) as a replacement...

Not at home at the mo but think I already have the black one (it was replaced about 6 months ago because the old one failed). Didn't remember it had gigabit ports but that is good news for those who need it.

Some of us have the original blackbox (a Linksys WAG160N if I remember rightly) and that definitely has 100Mbit ports...

The newer black boxes are the TP-LINK WDR3600 as I have one sat in my cupboard.

In reply to a post by jelv:

https://www.samknows.com/ - 440,000 volunteers

Even if only half of them are on video testing that's at least 13PB of data they are shifting round the internet each month!

I doubt if anything like that number are actually using the boxes. In the UK Ofcom have been struggling to get statistically valid results from the samknows data http://stakeholders.ofcom.org.uk/binaries/research/b...

I have a Black box here as well - a Linksys WRT160NL - which predates the white boxes

Logging on to the samknows site, I can't see any evidence that it is testing video - perhaps the model is too old?

Is there any sense in asking for a newer model (i.e. the new TP-Link)?

I do use dual band Wi-fi on the Home Hub 5 which I suspect my existing black box can't recognise.

Edited by gt94sss2 (Sat 21-Nov-15 02:15:37)

Probably not as the traffic is before the black box, from memory it needs to be modem > samknows > router

Updated FAQ for testers on SamKnows:

What is the Test Schedule?

Test Schedule for those on a limited broadband product / data cap in place:

Download and Upload MT: Once every 6 hours until 6pm, then hourly thereafter
Jitter: Every 2 hours
Web browsing: Every hour
DNS: Every hour
Latency/loss: Every hour
Video streaming (YouTube, Netflix, iPlayer): Once every 6 hours until 6pm, then hourly thereafter

Test Schedule for those with an unlimited broadband product/ no data cap in place:

Download and Upload MT: Once every 3 hours until 6pm, then hourly thereafter
Jitter: Every 2 hours
Web browsing: Every hour
DNS: Every hour
Latency/loss: Every hour
Video streaming (YouTube, Netflix and iPlayer): Once every 3 hours until 6pm, then hourly after.

How much data does the Test Schedule consume?

If you are on an unlimited broadband product or we are aware that any data cap has been removed, you will be put on our full testing schedule. The data consumption will vary depending on your package. Maximum daily usage approximately is:

8Mbps: 820MB
20Mbps: 2.06GB
40Mbps: 3.95GB
80Mbps: 8.17GB

Please note, as there is no data cap in place, the increased data consumption from testing will not be charged by your provider as provision is unlimited.

Those users with data caps in place are put on a reduced testing schedule. Maximum daily usage approximately is:

8Mbps: 54MB
20Mbps: 54MB
40Mbps: 108MB
80Mbps: 1.17GB

During you participation, if you do move to an ISP with a limited broadband product and exceeding your data cap is a concern, please contact the support team at [email protected] so we can ensure you are placed on a reduced testing schedule.

So for someone on 80/20 FTTC on unlimited the usage is up to 250GB per month!

It's modem > router > samknows > network

So, the modem connects to the router. Then there is supposed to be one cable from router to samknows box and all other connections are supposed to come off the samknows box. The samknows box also is wireless capable and they say it is able to monitor wireless use to ensure the network isn't in use - I have no idea how it does this though as the wireless is done direct to the router and the samknows is not involved in it.

The reason all physical connections are supposed to be from the samknows is so that it knows when the connection is in use and is then not supposed to do any tests.

Mine is cabled as per your diagram as I have separate devices but the OP had a HH5 so assuming they were having a wireless AP and the HH5 was performing the routing that is the way I would cable it unless the HH5 is in bridge mode...

In reply to a post by gerarda:

In reply to a post by jelv:

https://www.samknows.com/ - 440,000 volunteers

Even if only half of them are on video testing that's at least 13PB of data they are shifting round the internet each month!

I doubt if anything like that number are actually using the boxes. In the UK Ofcom have been struggling to get statistically valid results from the samknows data http://stakeholders.ofcom.org.uk/binaries/research/b...

While you are right that nowhere near 400,000 volunteers takes part in the UK research (Samknows runs the same monitoring inthe EU and US), I don't see anything from Ofcom about them struggling to get statistically valid results.

Conversely, I see them do a lot to make sure they are statistically valid, and that the appropriate extrapolations are made for nationwide comparisons.

Europe wide there are 20,000 boxes

In reply to a post by jelv:

Updated FAQ for testers on SamKnows:

80Mbps: 8.17GB

So for someone on 80/20 FTTC on unlimited the usage is up to 250GB per month!

Was just checking through my usage reports for the last month ... which had some days totalling less than this. When I correlated it against SamKnows results, I can see those days had some missing video/download tests.

As it happens, yesterday was a pretty perfect day, from the test activity perspective. Everything ran OK, and the wife was away so didn't add her usual usage.

My usage for the day ended up as 8.8GB peak, and 1.8GB off-peak.

Testing every 3 hours is new - it used to be every 6 hours. And the change is recent, because our monthly usage hasn't ever reached 200GB+ before. Double checking the reporting page, I see the new schedule started on October 31st.

In reply to a post by MrSaffron:

Europe wide there are 20,000 boxes

Make that 19,999.

This sort of testing should be non-intrusive and care must be taken that the testing regime does not affect what you are testing.

Quadrupling my monthly usage I don't consider meets that criteria so I've just unplugged my box and will be returning it.

In reply to a post by jelv:

... care must be taken that the testing regime does not affect what you are testing.

The figures suggest that it will affect backhaul for at least some ISPs.

I don't know if you remember the Entanet troubles when they went live on WBC. So many people were getting poor performance due to WBC node routing imbalance that huge numbers were running speed tests several times a day. Completely screwing the Enta system.

Similarly on O2/Be when there were issues, mass multiple speed testing made the situation worse.

In reply to a post by jelv:

This sort of testing should be non-intrusive and care must be taken that the testing regime does not affect what you are testing.

I agree.

I'm not going to back out "because", but I will do some checks, to see whether their detection of other tasks works successfully.

The increase in tests has coincided with us starting to use an Amazon Fire TV box, which goes through a lot of data. I'd initially put the increase in usage down to that ... but now I know differently.

But there is an indication that the tests don't run when we *are* streaming ourselves. I'll check this in more detail later...

In reply to a post by RobertoS:

The figures suggest that it will affect backhaul for at least some ISPs.

An interesting prospect. I wonder how much co-ordination has gone on between SamKnows, Ofcom and ISPs?

See this post. Although it now seems most of those are outside the EU. It's probably still a fair bit in the UK.