Suddenly our internet only works when using a VPN - Why?

We have been having serious issues with our fibre internet, over the last couple of days. Our ISP (Hey! Broadband) has suggested changing the WiFi router, which they have arranged for this coming week.

Specifically, one of our WiFi bands (the 5GHz) is not working at all, while the 2.4GHz Wifi works on just a few web sites, but most do not. For those that do not work, I can usually ping them no problem, but http connections just fail.

The most curious observation, and the one I am really interested in understanding, is that the internet works fine if we enable a VPN, in the browser....

I believe we had a similar situation some time ago, but I can’t for the life of me remember what the solution was, on that occasion.

Thoughts anyone?

Switch off the 5Ghz network on the router and try that as if sharing the same ssid and password can sometimes screwup devices.

Are you able to change the DNS settings within the router ? That’s what the symptom suggests to me, change them for the free Google settings and try that

Thanks. Already set up as 1.1.1.1 and 8.8.8.8. Nothing has been changed at our end. Neighbours on Hey fibre are not experiencing the same issues, so not the local fibre network.

is this happening to all devices or certain devices on the network ?

Fair enough, that’s me all out of ideas then.

Please come back and post when you get the fix, it may well help someone else .

Sounds like an ISP routing issue, you can reach your VPN Provider, but not other networks. A traceroute/tracert would be interesting, but otherwise this is one for the ISP to fix.

Just to be clear, are you using an HB provided router or your own?

It is happening on all devices.

We are using the standard (original) TP Link router, from Hey!

I ran the diagnostics in the router and it said it was unable to ping the default Gateway (which is set to 172.16.16.16). This is odd because web traffic which does get through is routed successfully via this IP address. But traffic which tries to go a different route (e.g. Google.com) does not get through. (I'm using Ping Plotter to observe this)

It's all very weird.

That is very weird. This is what I would be doing to troubleshoot:

Plug in a desktop/laptop via ethernet.
Can you ping well known IPs like 1.1.1.1 and others?
If you set 1.1.1.1 as your resolver in nslookup, can you resolve names against it?
If that works, can you point curl at common websites?

If none of those work at all, something is up with the routing and HB support are required.

Forgive me for asking the obvious, but have you tried switching it off, then back on?

Is this happening specifically on your device or all of the devices in your home network?

If this is happening specifically with your computer/laptop then a windows system restore point to an earlier date before the incident had occurred should solve your problem.

Also from the time you have noticed this issue, has there been a recent router firmware update that coincides with your issue? That way you can diagnose if the problem is with the router, devices or the network itself.

Have you installed any VPN software on your device(s) that may have triggered this issue. Also any specific third party antivirus software/firewall program? Sometimes they can also cause network issues by blocking ports.

Thanks,

HB Tier 2 support (via Chinese whispers, through Customer Support/Tier 1) is supposedly working on it, but so far their only official response has been: "Can you try re-booting the Wifi router and see if that resolves it?"

Ping 1.1.1.1 fails. But ping 8.8.8.8 is fine.

Given I have had the same problem in the past, surely this should just be a case of looking back at the previous ticket - but that seems to be too logical for HB Tier 2.....

OK, if something so fundamental as a ping fails there is no point troubleshooting further. It's a network routing issue that only HB can address.

FWIW, Support told me recently a major platform upgrade is happening this Thursday in the early hours so you may find this is resolved then.

All devices on the home network.

Happen once in the past, resolved by HB (don't know how).

No changes to the HB WiFi router recently - still using the original they supplied.

Just started out of the blue, on Friday afternoon.

Neighbours (also on HB) unaffected.

Yes, we got an e-mail about that. I did wonder if our issue was somehow related to any preparatory work, which they may be doing...

In reply to a post by Toothy_Chris:

It is happening on all devices.

We are using the standard (original) TP Link router, from Hey!

I ran the diagnostics in the router and it said it was unable to ping the default Gateway (which is set to 172.16.16.16). This is odd because web traffic which does get through is routed successfully via this IP address. But traffic which tries to go a different route (e.g. Google.com) does not get through. (I'm using Ping Plotter to observe this)

It's all very weird.

That doesn't mean much, not all ISPs have their gateway respond to ping. I think its bad form personally, but its not inherently a problem.

Have you tried using the default DNS out of the box, with no other DNS set in the router and/or at the device level? A lot of ISPs are intercepting DNS, I do not agree with it, it often coincides with marketing data being sold, but ISPs are becoming more strict on this to satisfy certain requirements.

I did enquire with Vodafone the purpose to which someone fairly senior confirmed DNS blocks are used as a method to identify access to illegal child material / access to copywrite - there are other methods but DNS is useful.

Virginmedia informed me it is enabled when you have the virus scanning / parental controls / for certain network performance monitoring.

The issue is when you set your own DNS, if the interception does not work, ie it reverts you forcibly through their DNS, often times sites stop working. It's a mixed bag, in general I can get past interception using VPN / encrypted DNS at a device level, but I do have to have all malicious website monitoring turned off and no parental controls. I noticed virginmedia even somehow intercepted modem mode, with a custom DNS set.

Give it a go - if things work, check for anything enabled that may be filtering. Eg on my ISP setting it to 18+ does not stop filtering, as it still flags if a site may be malicious - so the DNS is still intercepted. You need to be sure that using your own DNS will work, and often that means the ISP needs to turn off any potential interception, which requires confirmation you are ok with the risk of browsing "unprotected" and do not require any filtering, parental controls, ongoing monitoring of device performance / site usage etc.

In my case I have just accepted it will be on, as I got a hub 5 for the speed upgrade and my own kit is not able to get the same sync...

Thanks all for your various comments.

Here is an update:

By mid-afternoon our internet finally returned to normal. The HB Customer Services lady then called me to double-check it was OK at my end. I confirmed it was, but when I asked for a detailed explanation of what had been done to resolve the issue, I was given a rather generic “We reset your Wifi router and optimised some settings”.

I assume a router reset is much more than just a power cycle.

What was interesting is that the Default Gateway IP address was unchanged, but was no longer showing a failed ping test, as part of the router diagnostics. But it did look like our externally visible IP address had changed.

The passwords for the WiFi router also seem to have been reset to default, which supports the suggestion that they did some kind of factory reset.

Chris

So, things have got weird again, but I think this is almost certainly related to the infrastructure updates, which apparently started a day early, last night.

Most internet connectivity is fine, but I cannot get to E-Bay UK because I get the following error message:

‘Pardon Our Interruption...
In compliance with applicable economic sanctions and trade restrictions, eBay is unavailable in your location. If you believe you are receiving this notice in error, please contact eBay’s Customer Service.’

And when one of my kids tried to log into his Xbox, he got a similar message.

The new external IP address I have does seem to be registered to HB, in the UK, so why do these web sites think I am in a country under sanctions??

Anyway, this has been reported to HB who escalated it straight to Tier 2 support (who I assume will be working through the night, given the infrastructure work).

In reply to a post by Toothy_Chris:

Most internet connectivity is fine, but I cannot get to E-Bay UK because I get the following error message

I'll send you a private message about this

seb

hi, wondering if you've made any progress from HB with this issue yet?

Yesterday ( Wed 26 April), my Xbox was no longer able to login with the error message saying the service not available in this region, and I also noticed that Disney+ gave a message saying that it looked like I was travelling, and I also get the same eBay message - clearly something up with the IP address allocation & geolocation, but the strange thing is that services like Netflix & BBC iPlayer seem to pick up UK content & don't have any issues.

To test the theory, I used my phone as a wifi hotspot, connected the xbox to it, and everything worked (albeit very slowly).

I've raised the issue with HB, sent them screen shots of the Xbox & Disney+ messages, and info about the network & geolocation - just waiting for them to respond, but wondered if you'd made any progress with them.

Edited by wymkeu (Thu 27-Apr-23 13:20:59)

Ok as this is widespread I'll post it here - some of the IPs in use by your provider may have been used in other countries in the past including ones which eBay may not want to do business with. This isn't entirely the provider's fault but some databases may be out of date causing the issues. It's up to the provider to fix the geolocation issues though

hi Seb,

I checked the RIPE database, and the range that HB are providing IP's from (or at least the IP that I have) are showing as owned & registered in Australia, however most of the whois IP sites are showing the addresses as in the UK. I'm assuming that services like iPlayer and Netflix are working as if in the UK, but others like Disney, Xbox & eBay aren't because they are using different databases to check geolocation.

It looks like HB are making use of a new range of IP's as part of their network changes this week that are causing issues, and (as you say) the provider needs to make sure the geo-location databases are correctly updated. We just need to hold tight & hope they are able to resolve them quickly.

In reply to a post by wymkeu:

hi Seb,

I checked the RIPE database, and the range that HB are providing IP's from (or at least the IP that I have) are showing as owned & registered in Australia, however most of the whois IP sites are showing the addresses as in the UK. I'm assuming that services like iPlayer and Netflix are working as if in the UK, but others like Disney, Xbox & eBay aren't because they are using different databases to check geolocation.

It looks like HB are making use of a new range of IP's as part of their network changes this week that are causing issues, and (as you say) the provider needs to make sure the geo-location databases are correctly updated. We just need to hold tight & hope they are able to resolve them quickly.

iPlayer et al won't use RIPE data but a geolocation provider (there are several). They may use RIPE data in parts but it's not generally considered accurate enough as RIPE is more about where the registry is based than exact location.

Chase your ISP about this.. it's their problem to fix ultimately..

In reply to a post by seb:

iPlayer et al won't use RIPE data but a geolocation provider (there are several). They may use RIPE data in parts but it's not generally considered accurate enough as RIPE is more about where the registry is based than exact location.

Chase your ISP about this.. it's their problem to fix ultimately..

Quite, they brought the ips, its part of their job to make sure they are clean for users to be able use without being blocked.

On the flip side, its poor that geolocation providers don't check that the data they hold is right.

In reply to a post by Taras:

On the flip side, its poor that geolocation providers don't check that the data they hold is right.

Geolocation providers will look at location based on multitude of information from routing to RIPE etc.. but that's a one off check. If the IP blocks are sold or rented, they wouldn't necessarily know about this for a while so it's generally up to the ISP to contact them to fix this and they all have processes for it.

One example to earlier query about why VPN works is it could be RPKI enforcement or using IPv6 for VPN which works which avoids these issues. I know the IP block I checked earlier has a Valid ROA now but maybe it wasn't before. Some providers check RPKI some don't. Either way iplayer etc is your provider's issue to resolve.


seb

Quick update - late afternoon I lost all connectivity & noticed that the router no longer had an IP address. Called HB tech support and spoke to them for a bit (nice person, but nothing useful came of the call), and was told it should all be fixed today.

Checked back a couple hours later, router had an IP address (different range from before), and everything was working again - xbox, eBay, Disney etc... with no issues.

Not sure if the original poster issue is fixed, but given I commented here, thought I'd say mine seems to be working now, so hopefully others will be fixed too.

I see things have been busy on here - sorry, I was working today....

So, when I started work this morning (I work from home) I noticed that I could not access any of my work VPN servers, in addition to the problems we were having with XBox and E-Bay.

Late this afternoon our Hey! connection dropped for a few minutes and then returned with a different external IP address. At that point everything seems to work again, apart from E-Bay. This evening (on the same IP address) even that had returned, so - touch wood - I think we are now back up and running with a full service.

Thanks again to all who got involved in the discussion.


Chris

In reply to a post by wymkeu:

Checked back a couple hours later, router had an IP address (different range from before), and everything was working again - xbox, eBay, Disney etc... with no issues.

You're on an IP block which I think has been UK based for longer

seb

In reply to a post by Toothy_Chris:

I see things have been busy on here - sorry, I was working today....

So, when I started work this morning (I work from home) I noticed that I could not access any of my work VPN servers, in addition to the problems we were having with XBox and E-Bay.

Late this afternoon our Hey! connection dropped for a few minutes and then returned with a different external IP address. At that point everything seems to work again, apart from E-Bay. This evening (on the same IP address) even that had returned, so - touch wood - I think we are now back up and running with a full service.

Thanks again to all who got involved in the discussion.

Your IP address is an ex-US IP although that's probably some timer ago.. I think Hey probably asked the IP supplier for a new IP block. Either way it's probably fixed the issues with Geo-IP providers having more up to date records.

I wrote this up as a news item as I think it's good to raise awareness about the problem
https://www.thinkbroadband.com/news/9550-can-t-acces...

Really useful blog, thank you!

I had a very similar problem with Hey!Broadband also. The installation was completed and everything was fine, but a couple of days later, I received my welcome email and my external IP changed (I'm on a package with a static IP).

On that IP, any attempt to access PayPal just resulted in a blank page with "403 - Forbidden".

Contacted HB customer support and they changed my external IP (and into a new range) and it's been fine since.

Keeping an eye on packet loss however, yesterday saw 3 bursts of packet loss into the 10-15% range, each which lasted for about an hour

So, having got things sorted out, we now seem to have a new issue (or perhaps it is all related).

In the evenings our service - particular the upload speed, which affects my daughter's streaming - falls painfully low: Just 4Mbps on a 1Gb service, plus frequent full dropouts. Download remains reasonable.

Looking at Ping Plotter, the culprit appears to be the very first device/IP address after our household WiFi router. On a fibre system like Hey, will this be their switch (or whatever the equivalent is for fibre) at the local BT exchange? The IP of this device is 31.214.156.3.

Possibly related to this?

Thanks for that,

Looking at the descriptions of almost zero upload speeds, I suspect this may be the same thing. It will be interesting to see if the 'External Service Provider Maintenance' tonight will have any effect.

So far this evening, the upload speed has deteriorated a bit, but not down to the 4Mbps we saw last night.

Hey! have - no surprise - not contacted me with any update today, after I e-mailed them last night.

In reply to a post by Toothy_Chris:

Thanks for that,

Looking at the descriptions of almost zero upload speeds, I suspect this may be the same thing. It will be interesting to see if the 'External Service Provider Maintenance' tonight will have any effect.

So far this evening, the upload speed has deteriorated a bit, but not down to the 4Mbps we saw last night.

Hey! have - no surprise - not contacted me with any update today, after I e-mailed them last night.

If there's congestion you should see an increase on BQM at those times

So, after several months without an issue (Once Hey! finally got their act together), I now find I have the opposite problem - since this morning. When I try to connect to my work VPN (GlobalProtect by Palo Alto) I cannot connect. But all other sites seem fine.

Is anyone else with Hey! having similar issues with VPN access?

Thanks all.

In reply to a post by Toothy_Chris:

So, after several months without an issue (Once Hey! finally got their act together), I now find I have the opposite problem - since this morning. When I try to connect to my work VPN (GlobalProtect by Palo Alto) I cannot connect. But all other sites seem fine.
Is anyone else with Hey! having similar issues with VPN access?
Thanks all.

When you say "all other sites seem fine" - are you talking about our site or a specific site? Or you mean you can't ocnnect to the VPN from Hey?

More....

I reported the issue via their web site and someone called me back - which is not what I was expecting at all!

They said they are aware that customer external IP addresses get changed/reset when they have some outages, and that I may (yet again) have been given an IP address which is being associated with undesirable country. Surely they must know which IPs these are, so they don't get given to someone else or re-used?

Anyway, they say they will give me a new IP address (again), soon.

I mean that I can't connect to the VPN service, via Hey!

So maybe the VPN site is down ??

The numpties at Hey! (or their system) had given me the same problematic IP address which we had been given in the past. This has now been changed at the Hey! end and I can now access the work VPN.

I have pretty much run out of patience with Hey!. Box Broadband is being installed later in the week.

I have pretty much run out of patience with Hey!. Box Broadband is being installed later in the week

Hope that works out well.