ISP blocklists

I had an issue with my Vodafone broadband this week, where Vodafone did a DNS based man-in-the-middle hijack of a couple of sites and put the traffic through an Internet Watch Foundation proxy server. I posted about that on this forum here.

Now, I understand why they were doing it, but what I don't understand is why other ISPs weren't affected. The sources on the Wikipedia article about IWF aren't great, but it says that 95% of UK ISPs use IWF's blacklist, and that domains end up on it only if the whole domain is for hosting illegal stuff. Is that info just outdated, now that most big sites have moved to HTTPS?

Are people with other ISPs seeing sites intermittently blocked, or is this a Vodafone speciality?

It depends on how the IWF block is implemented. All large ISPs block IWF-listed content but the actual method of the blocking is down to the individual ISP. From what you say, it looks like Vodafone's blocking system needs improvement.

And there is no IWF-run proxy server, every IWF proxy server (if that's the blocking method the ISP selected) will be operated by the individual ISPs.

Edited by Oliver341 (Sat 03-Jun-17 13:49:34)

Are other ISPs using better methods? Google doesn't find reports from others, so perhaps the difference is only in how quickly sites are added and removed?

With HTTPS, the only way I can see selective URL blocking work without blocking entire domains, is if the ISPs asked customers (or the government asked OS or browser manufacturers) to install a Big Brother root certificate that then allows them to impersonate any site anywhere. When Wikipedia was on the blocklist for a week almost a decade ago, ISPs took no responsibility on their poor implementations, and IWF got the blame.

In reply to a post by MarcuT:

Are other ISPs using better methods?

Presumably, since we don't see issues reported much anymore. There are periodic complaints about image hosting sites such as imgur which are probably caused by IWF filters, but in general the incidence is low.

I can proxy https pages but without becoming the end point, in which case there is no MITM interception so to speak.

The dns servers redirect IWF listed names to a proxy server (possibly done even on the routing level for ip addresses to get round people using 3rd party dns servers), then the proxy server will either serve proxied content (probably a block page) or simply pass on the traffic depending on the url been requested.

So when I use my own squid server it can redirect https traffic and I get no certificate issues.

So basically a proxy in the middle can intercept the url, and if it passes it on, then the original host at the other end will receive it untampered, it will then send back the reply to the proxy server which passes it back to the end user, the proxy server will only see encrypted content tho and not know what it is, but it can still pass it back to the user.

From all this a block page I would expect to generate a certificate warning, but not any traffic that comes from the actual proper provider of the domain.

I am not entirely sure at the time of this post, if the proxy server only sees the domain or the full url of the https request as I think but not sure that https requests start only with the domain name and the url is requested after the handshake, but not entirely sure.

That's the thing: in HTTPS/TLS the full URL is sent only after the handshake. By then everything in the session is sent encrypted. The handshake includes the host name in plain text, required for the server to be able to pick a matching certificate with virtual hosts or SNI, but that's it. Apart from GCHQ level backdoors, it's not possible to decrypt HTTPS sessions without the private key of the endpoint, so an ISP proxy has no way of peeking into the communication. Otherwise there'd be no added security with using end-to-end HTTPS. Once the handshake fails, the browser doesn't even get to the point of requesting the full URL, and that's where the idea of filtering an encrypted protocol ends before it even got started.

Squid works with HTTPS for proxying, but it does it in blind mode and can make choices only based on the host name and IP. It can't get into any of the HTTPS headers to treat requests selectively. Wireshark is a good tool for figuring this out, and Squid's logs as well. Testing tools for tampering HTTPS traffic include an installation step of placing a Certificate Authority certificate on the client, which for testing purposes may or may not be the same machine as the server. Squid supports faking certificates as well, but the man-in-the-middle certificate needs to be added on the client machine the same way. Once that's done, you won't see anything different in the browser, unless you dig into the certificate and look at the certificate path.

Having ISPs suggest that users ignore security warnings goes completely against every other security recommendation banks etc are making, of checking the info in the browser's address bar. Customers of those ISPs instantly become more likely phishing victims for scammers.

In reply to a post by MarcuT:

Having ISPs suggest that users ignore security warnings goes completely against every other security recommendation banks etc are making, of checking the info in the browser's address bar. Customers of those ISPs instantly become more likely phishing victims for scammers.

Safiri, which is the default on iPad, doesn't help by not showing the URL unless you tap on the displayed name.

Are there ISPs that don't use the IWF list at all? I've read conflicting posts that using it is mandatory, or only if the ISP has over X customers, or that it's entirely voluntary for all ISPs.

The Open Rights Group has a list of ISPs per number of sites being blocked, but it probably isn't fully up to date or doesn't detect all types of blocking. Plusnet for instance is shown as having 0 blocks in place, but with the image they're trying to make for themselves in their ad campaigns, that can't be true. Must depend on the definition of 'block'.

AAISP

Uno.

It is important to make the distinction between the IWF list which is about blocking abhorrent child abuse imagery and other blocking e.g. as part of court orders over copyright infringement etc

This really is a specific issue to your ISP, certainly never seen anything like this on BT, Sky or TalkTalk.

EDIT I have even followed through sites in BURP suite on Reddit etc and cannot observe anything other than expected when going HTTP to HTTPS on BT.

Edited by ukhardy07 (Thu 08-Jun-17 10:49:41)

I do not suppose you could provide some example sites for me to test? Like URLs affected etc.

To test if your ISP is affected, you can try opening anything from imgur.com over HTTPS.

With a DNS redirection to the filter, running this in Command Prompt (with the IP being your modem's internal IP, assuming it's a DNS relay server):

nslookup i.imgur.com 192.168.1.1

returns an IP from the ISP's network rather than an imgur one. When it gives me the iwffilter proxy IP, I know already that HTTPS connections to the site are going to fail.

My results at the moment:
1. https://imgur.com/BAfsJ3j - Loads but without images (which are served from i.imgur.com)
2. https://i.imgur.com/BAfsJ3j.gif - Certificate error
3. http://imgur.com/BAfsJ3j - Loads
4. http://i.imgur.com/BAfsJ3j.gif - Loads

I can find past reports from several ISPs having overload issues with their IWF proxy (for instance Wired in 2014 about TalkTalk), but HTTPS makes the whole idea of a filtering proxy a technical impossibility, and I haven't seen much about that (this!) yet.

All the above images load 1-4 on Uno.

All the above images load 1-4 on BT

In reply to a post by MarcuT:

1. https://imgur.com/BAfsJ3j - Loads but without images (which are served from i.imgur.com)
2. https://i.imgur.com/BAfsJ3j.gif - Certificate error
3. http://imgur.com/BAfsJ3j - Loads
4. http://i.imgur.com/BAfsJ3j.gif - Loads

All of them load ok on Sky, including the HTTPS links with no certificate errors.

i.imgur.com resolves to 151.101.60.193 using Sky's DNS, which appears to be a Fastly CDN server.

Perhaps Fastly are applying the IWF filter to their CDN, in which case the imgur image can be served (or not served) from Fastly without any proxy being required.

I don't think there's anything wrong on the Imgur/Fastly end, but somewhere closer to home. I put together a simple diagram of what I think is happening with these filters when accessing the theoretical https://example.com/page.

The IWF list some organisations subscribed to their list, here. I don't recognise any CDN companies there.

Thanks all for testing this. If others join in, please make sure your browser is using the ISP's DNS instead of a third party one you may have configured. You may also get different results the next day: Vodafone's DNS responds with a Vodafone IP for imgur.com most of the time, but on some days gives a correct Fastly CDN one.

Edited by deleted (Fri 09-Jun-17 01:14:19)

In reply to a post by MarcuT:

The IWF list some organisations subscribed to their list, here. I don't recognise any CDN companies there.

In which case the CDN could use an API with one of the listed "Filtering Companies", looking up each URL and refusing to serve bad ones whilst not directly having possession of the complete IWF list. Then the ISP (e.g. Sky) can whitelist queries to imgur on the assumption that the bad urls are filtered at the CDN, avoiding any potential issues with proxy congestion or SSL certificate mismatches.

With a DNS redirection to the filter, running this in Command Prompt (with the IP being your modem's internal IP, assuming it's a DNS relay server):

nslookup i.imgur.com 192.168.1.1

returns an IP from the ISP's network rather than an imgur one. When it gives me the iwffilter proxy IP, I know already that HTTPS connections to the site are going to fail.

I did this on both BT and TalkTalk:
http://imgur.com/S1mLHHp

My results at the moment:
1. https://imgur.com/BAfsJ3j - Loads on BT & TalkTalk
2. https://i.imgur.com/BAfsJ3j.gif - Loads on BT & TalkTalk
3. http://imgur.com/BAfsJ3j - Loads on BT & TalkTalk
4. http://i.imgur.com/BAfsJ3j.gif - Loads on BT & TalkTalk

Sites loaded in the blink of an eye.

If you follow through your traffic, e.g. with BURP suite, where is it directing you?

Sounds to me like a big mess at VFs end (they are a new ISP afterall). I am tipping their proxy setup is not ideal and it's just failing with the load or something similar. With them being new they possibly have not got it fine tuned (btw all of this is total guessing at this stage).

Edited by ukhardy07 (Sat 10-Jun-17 21:03:48)

All the above images load 1-4 on EE.

In reply to a post by ukhardy07:

If you follow through your traffic, e.g. with BURP suite, where is it directing you?

Sounds to me like a big mess at VFs end (they are a new ISP afterall).

The DNS resolution on Vodafone looks like this:

C:\>nslookup i.imgur.com 192.168.1.1
Server:  vodafone.connect
Address:  192.168.1.1

Non-authoritative answer:
DNS request timed out.
    timeout was 2 seconds.
Name:    i.imgur.com
Address:  90.255.255.1

C:\>nslookup 90.255.255.1 192.168.1.1
Server:  vodafone.connect
Address:  192.168.1.1

*** vodafone.connect can't find 90.255.255.1: Non-existent domain

The timeout bit there can be ignored. Wireshark shows that it's because nslookup asks for both IPv4 and IPv6, and Vodafone's DNS doesn't bother responding to the IPv6 request.

With Burp Suite and the browser set up with Burp's root certificate, everything is loading OK. The browser/Burp connects to 90.255.255.1 thinking it's imgur, and without any errors, Burp gets a response and rewraps it with Burp's certificate. It all works because Burp doesn't check the certificate, which is valid only for contentcontrol.vodafone.co.uk, not imgur.com. That level of certificate validation is probably not unlike in Vodafone's iwffilter squid server...

Good soundbite in a runaround support thread started in 2015:

In reference to forum.vodafone.co.uk/t5/Pay-monthly/Website-security-certificate-co...:

As part of Vodafone�s commitment to ensuring your safety on the internet we work with the Internet Watch Foundation (IWF) to monitor websites and domains that contain offensive content. Unfortunately imgur.com has currently been flagged as one such website. Our current content controls means that HTTPS traffic to this website will appear to be insecure. We are working on a solution with our vendor that will ensure that HTTPS traffic will work normally in the future. Vodafone�s content control platform does not monitor or log your internet traffic.

Oliver is right that filtering should be done on the content provider or distributor end, which is what the outspoken ISPs have been saying all along. IWF's report for 2016 shows that most URLs on their list have been from image hosts, although it's hard to say if they're shady unknown hosts or big ones that can have collateral damage like here.

Vodafone have been in the broadband market here only since late 2015, but they've provided mobile data for much longer than that.
Vodafone 0 - AAISP, BT, EE, Sky, TalkTalk, Uno 1

Any Plusnet users reading? https://www.blocked.org.uk/results?url=https://i.img... says Plusnet comes back with an SSL error when accessing https://i.imgur.com. Is their blocklist as lagged as Vodafone's is?