It is important to make the distinction between the IWF list which is about blocking abhorrent child abuse imagery and other blocking e.g. as part of court orders over copyright infringement etc

This really is a specific issue to your ISP, certainly never seen anything like this on BT, Sky or TalkTalk.

EDIT I have even followed through sites in BURP suite on Reddit etc and cannot observe anything other than expected when going HTTP to HTTPS on BT.

Edited by ukhardy07 (Thu 08-Jun-17 10:49:41)

I do not suppose you could provide some example sites for me to test? Like URLs affected etc.

To test if your ISP is affected, you can try opening anything from imgur.com over HTTPS.

With a DNS redirection to the filter, running this in Command Prompt (with the IP being your modem's internal IP, assuming it's a DNS relay server):

nslookup i.imgur.com 192.168.1.1

returns an IP from the ISP's network rather than an imgur one. When it gives me the iwffilter proxy IP, I know already that HTTPS connections to the site are going to fail.

My results at the moment:
1. https://imgur.com/BAfsJ3j - Loads but without images (which are served from i.imgur.com)
2. https://i.imgur.com/BAfsJ3j.gif - Certificate error
3. http://imgur.com/BAfsJ3j - Loads
4. http://i.imgur.com/BAfsJ3j.gif - Loads

I can find past reports from several ISPs having overload issues with their IWF proxy (for instance Wired in 2014 about TalkTalk), but HTTPS makes the whole idea of a filtering proxy a technical impossibility, and I haven't seen much about that (this!) yet.

All the above images load 1-4 on Uno.

All the above images load 1-4 on BT

In reply to a post by MarcuT:

1. https://imgur.com/BAfsJ3j - Loads but without images (which are served from i.imgur.com)
2. https://i.imgur.com/BAfsJ3j.gif - Certificate error
3. http://imgur.com/BAfsJ3j - Loads
4. http://i.imgur.com/BAfsJ3j.gif - Loads

All of them load ok on Sky, including the HTTPS links with no certificate errors.

i.imgur.com resolves to 151.101.60.193 using Sky's DNS, which appears to be a Fastly CDN server.

Perhaps Fastly are applying the IWF filter to their CDN, in which case the imgur image can be served (or not served) from Fastly without any proxy being required.

I don't think there's anything wrong on the Imgur/Fastly end, but somewhere closer to home. I put together a simple diagram of what I think is happening with these filters when accessing the theoretical https://example.com/page.

The IWF list some organisations subscribed to their list, here. I don't recognise any CDN companies there.

Thanks all for testing this. If others join in, please make sure your browser is using the ISP's DNS instead of a third party one you may have configured. You may also get different results the next day: Vodafone's DNS responds with a Vodafone IP for imgur.com most of the time, but on some days gives a correct Fastly CDN one.

Edited by deleted (Fri 09-Jun-17 01:14:19)

In reply to a post by MarcuT:

The IWF list some organisations subscribed to their list, here. I don't recognise any CDN companies there.

In which case the CDN could use an API with one of the listed "Filtering Companies", looking up each URL and refusing to serve bad ones whilst not directly having possession of the complete IWF list. Then the ISP (e.g. Sky) can whitelist queries to imgur on the assumption that the bad urls are filtered at the CDN, avoiding any potential issues with proxy congestion or SSL certificate mismatches.

With a DNS redirection to the filter, running this in Command Prompt (with the IP being your modem's internal IP, assuming it's a DNS relay server):

nslookup i.imgur.com 192.168.1.1

returns an IP from the ISP's network rather than an imgur one. When it gives me the iwffilter proxy IP, I know already that HTTPS connections to the site are going to fail.

I did this on both BT and TalkTalk:
http://imgur.com/S1mLHHp

My results at the moment:
1. https://imgur.com/BAfsJ3j - Loads on BT & TalkTalk
2. https://i.imgur.com/BAfsJ3j.gif - Loads on BT & TalkTalk
3. http://imgur.com/BAfsJ3j - Loads on BT & TalkTalk
4. http://i.imgur.com/BAfsJ3j.gif - Loads on BT & TalkTalk

Sites loaded in the blink of an eye.

If you follow through your traffic, e.g. with BURP suite, where is it directing you?

Sounds to me like a big mess at VFs end (they are a new ISP afterall). I am tipping their proxy setup is not ideal and it's just failing with the load or something similar. With them being new they possibly have not got it fine tuned (btw all of this is total guessing at this stage).

Edited by ukhardy07 (Sat 10-Jun-17 21:03:48)