YouFibre Partial Internet Help

(Apologies in advance for the long post)

Hi, long time lurker here - hoping for some sage advice.

The problem:
There was an outage with YouFibre in my area (Guildford) last Thursday afternoon (4th). Originally customer support after running me through the usual reboot router, reboot ONT, try different cables, factory reset the router etc. booked an engineer to visit on Monday 8th.

A few hours later on Thursday, the engineer gave me a courtesy call to confirm the Monday slot and about 30 mins after that everything started working (or so I thought), so I called them back to let them know - at that time he then informed me that the problem was actually due to a network outage that was only meant to affect new installs and not existing customers.

Later Thursday evening was when I noticed the issue (I'm calling it "partial" internet). Web browsing from my phones or tablets (Samsung Android devices) worked just fine (Chrome/Edge), but when using a number of apps on those devices I was just left with an endless spinner on app startup and no functionality:

BBC IPlayer, BBC News, Strava, TADO Heating app and many more.

IF, however, on my phone for example - I turn off my WiFi and use 4G/5G mobile data, the apps connect just fine. Everything also works if I enable Nord VPN on my Phone or tablet.

The issue appears to only be with outbound traffic as far as I can see, as I can receive notifications from those apps and it was most obvious when I had to react to a 2FA prompt from Microsoft Authenticator but it couldn't send the response when not connected to 4G/5G or via the VPN.

The Engineer visited on Monday 8th as booked, I demoed the issue to him and he was stumped but offered to try a brand new Arris router to see if that helped. It didn't. He then said he'd been able to talk to the network team via their chat system and they advised me to call YouFibre back and ask to be put through to their L3 network support team and this is where things got really frustrating.

I've tried calling YouFibre support, explained the issue and was told that they have no way to directly contact L3 (which seems ludicrous to me) and although they could escalate a ticket to them, it would be up to L3 to assess the details taken down and call me back or not. I tried to be as clear as possible about the issue with examples but so far, nothing.

The connection I have is useable for most things, so its not the end of the world, but given it was working perfectly before last Thursday 2pm and I'm finding more and more things that don't work without a VPN workaround, it's becoming quite an annoyance.

Is there any advice, information or theories you good people can give so that if I get to speak to someone which more expertise I can convey that OR if I end up stuck with L1 support I can provide more detail for them to pass on?

Additional Info:
I have a static IP with YouFibre and I have asked to speak to a manager about this. Short of having one of their L3 staff actually visit my house to witness the issue first hand - there's only so much detail I can provide to L1 support to ensure L3 have all the info they can

Additional Info 2:
I had a similar but smaller issue last year where the Xbox App on my phone would not make an outbound connection (e.g. posting MS points rewards quests) unless via a VPN or 4G/5G connection which they resolved - something to do with refreshing routing tables or something, but unfortnuately I don't have the resolution email to hand any more

Edited by MarcDark (Tue 09-Jul-24 19:41:58)

I am with Youfibre and while my own connection is working fine, I had a very similar issue with Vodafone a couple of years back. Believe me, if you think Youfibre are bad, try explaining this to Vodafone L1 support, most of whom don't seem to know which side of a keyboard is to be used for typing.

My advice to you, however, is to do the same as me - you need to reproduce this properly. A vague "some of my apps don't work properly" won't help any network engineer to diagnose your issue. You need to narrow this down to "I can't talk to this IP address from my Youfibre connection, but it works fine on my cellular connection". Provide a few examples, try to spot patterns, include detailed traceroutes.

My experience with Youfibre is that if you give them this information, they'll sort it out, but without it they don't really stand a chance. This is one of those times when you need to help yourself a bit before they can help you.

And as for my Vodafone issue? As those of us that suffered from the problem had already worked out, they had a routing black hole in their network due to failed network peering with Google. We provided detailed evidence and all of the technical information to fix it but it still took a dozen of us constantly reporting it for 6 weeks before it got resolved. And the final comment from the engineer that fixed it? "Oh, yes, it was a defect at our end resulting from some incorrectly decommissioned equipment. You should have let us know sooner as it only took a few minutes to fix..."

Might be a problem with IPv6. Worth switching this off on a machine and seeing if that solves the issue.

In reply to a post by XGS_Is_On:

Might be a problem with IPv6. Worth switching this off on a machine and seeing if that solves the issue.

I second this, had to turn it off on a BT leased line for a whole site last week as it was causing mayhem with the VOIP phones and Microsoft services

In reply to a post by XGS_Is_On:

Might be a problem with IPv6. Worth switching this off on a machine and seeing if that solves the issue.

Although if you can reach Google/Youtube and Facebook, but not much else, it might be that you have working IPv6 but broken IPv4.

Try these sites:

ip4.me
ip6only.me

They will tell you your IPv4 and IPv6 address respectively, and connect only over those protocols.

Also try https://1.1.1.1 (cloudflare accessed over IPv4 by IP address, with no DNS).

Finally, pinging the Google DNS servers on v4 and v6:

ping 8.8.8.8
ping 2001:4860:4860::8888

Hi.

Been with yf for about 2 years running pfsense. I would ask yf if you can't do it on their router to try turning ipv6 off. Up until about 2 months my ipv6 worked absolutely fine. This isn't the case anymore. I have tried this both on pfsense and opnsense. Basically after 3 or 4 days my network grinds to a halt. Windows and various apps are unresponsive. Soon as I disable ipv6 everything works immediately without any reboots required. It all started for me when there was an outage of v6 one night a few months back. I have flagged it up to support but was told there is no problem with v6 but at the moment I can't use it. It's a shame as it was fine up until that point.

I'm glad I've seen your post. I have a VERY similar problem. It's not just you (and I'm glad to hear it's not just me).
Expect a PM from me shortly.
Kroma

In reply to a post by choppersrock:

Hi.

Been with yf for about 2 years running pfsense. I would ask yf if you can't do it on their router to try turning ipv6 off. Up until about 2 months my ipv6 worked absolutely fine. This isn't the case anymore. I have tried this both on pfsense and opnsense. Basically after 3 or 4 days my network grinds to a halt. Windows and various apps are unresponsive. Soon as I disable ipv6 everything works immediately without any reboots required. It all started for me when there was an outage of v6 one night a few months back. I have flagged it up to support but was told there is no problem with v6 but at the moment I can't use it. It's a shame as it was fine up until that point.

Just out of interest were there any particular services or sites affected, I have IPv6 enabled on some of my VLANs not my main however as a lot of my security systems and DNS monitoring etc, don't support IPv6,

Windows 11. Not able to use the desktop. Locking up.
Microsoft Outlook unable to sync mail at all. Teams not working.
Didn't matter wether I have my dream se, pfsense & my opnsense box configured. Works upon enablement at the router end but 3 days in the network is unusable. I have family working from home and spent a lot of time trying to figure out what was the problem as they were unable to work. Switch off v6 once everything has locked up cures it instantly. I asked for a new prefix but that was a no. So for now its off. Was not an issue until recently as I have had v6 from the start of my service. The above were some examples.

In reply to a post by choppersrock:

Windows 11. Not able to use the desktop. Locking up.
Microsoft Outlook unable to sync mail at all. Teams not working.
Didn't matter wether I have my dream se, pfsense & my opnsense box configured. Works upon enablement at the router end but 3 days in the network is unusable. I have family working from home and spent a lot of time trying to figure out what was the problem as they were unable to work. Switch off v6 once everything has locked up cures it instantly. I asked for a new prefix but that was a no. So for now its off. Was not an issue until recently as I have had v6 from the start of my service. The above were some examples.

If it makes you feel better I have witnessed the issue, not only on my network on YF, but on my old IDNet line, I have had so many calls from customer on BT and Sky in particular with exactly that issue, Microsoft do have a weird routing issue as we escalated the case with Microsoft Partner support.

We noticed that their IPv6 routing in the cases we had was going via some MSN server in Paris and back again, we used Microsoft Connectivity toolkit to help diagnose and the moment IPv6 was turned off routeing was via London, so I think there is something at either a higher level or somewhere else

In reply to a post by choppersrock:

Windows 11. Not able to use the desktop. Locking up.
Microsoft Outlook unable to sync mail at all. Teams not working.
Didn't matter wether I have my dream se, pfsense & my opnsense box configured. Works upon enablement at the router end but 3 days in the network is unusable. I have family working from home and spent a lot of time trying to figure out what was the problem as they were unable to work. Switch off v6 once everything has locked up cures it instantly. I asked for a new prefix but that was a no. So for now its off. Was not an issue until recently as I have had v6 from the start of my service. The above were some examples.

I have just been having a play with Hurricane electric tunnel broker and to be honest the issues you describe are present via that as well.

Good to know I wasn't going mad....the thing is it wasn't an issue until fairly recently when v6 went down one night as I explained. It's a shame YF support brushed it off so quickly as no fault found ....

Firstly, thanks for all your replies. Some updates:

1. I checked and I never had IPV6 configured originally, and after factory resetting the Arris, I turned off IPV6 anyway.

2. Unlike some that have posted replies, YF L2 or L3 support never got in touch with me until I updated my Trust Pilot review to reflect my experience and suddenly they did (on Thursday, a week after the issue occurred). They apologised, saying that somehow my ticket wasn't routed to the correct team and said they would investigate this breakdown in process as a separate matter.

3. @Daern posted that I needed to provide YF support with more specific info like a TraceRoute, IPs etc. this wasn't straightforward to do for me - consider me moderately technical. I did however install the PCAPDroid app on a phone where IPlayer and BBCNews wasn't working. This works by setting up a local VPN and capturing traffic data that way. As soon as PCAPDroid was running, the apps worked - which at the time kinda led me to the conclusion that it might be the router and not the network configuration at fault (perhaps some firewall settings?)

3a. I never changed any firewall settings on the Arris to begin with
3b. I factory reset the Arris, so any potential erroneous firewall or other config settings would have been wiped anyway
3c. When the engineer visited last Monday, we did infact try 3 Arris routers - my original one, and two others he had in the van - all with the same issue.
3d. In my post I listed a number of apps that didn't work, but over time I found many more:
NVidia shield stopped working
BBCIPlayer, BBC News, Wimbledon, Microsoft Authenticator (responding to a 2FA request), Strava and more so I told YF L1 support all of this but couldn't not work out a way to give them the endpoint IP connection attempt data vs when trying related sites via a browser which does work

4. The BIG one.. I got fed up with the lack of resolution and bought a cheap router on Amazon (Tuesday). I plugged it in and it worked first time with the connection and all apps working perfectly configured to my static IP - didn't even have to clone the MAC address of the Arris.

4a. L2 called me on Thursday to have a diagnostics attempt. I plugged the Arris back in and this time, unlike the "Partial Internet" of my original post here, the Arris didn't even connect despite all lights being green on both the router and the ONT. In the web dashboard, it said it was "up" but I had some weird IP and YF support couldn't see it connected on their side. We tried another Factory reset, confirmed IPV6 was off, tried changing DNS.. nothing. So I plugged my own router back in and everything burst back to life.
4b. L2 called me again on Friday, having spoken to L3 who said they'd "reset some session variables" (or something) and it still didn't work. When I plugged my own router back in this time, they said they couldn't see that my router was connected nor the MAC address of my router - yet the connection is clearly working

So what next? Well, I have a working connection so I'm okay with that. Apparently YF do want to get it resolved though so we'll see what they come back with next.

In the meantime and I hope this info might be of use to others. What I've encountered for consideration:

3 Arris routers tried on Monday - all exhibiting the "partial internet" problem

PCAPDroid (device local VPN) - No issues, device itself can be ruled out, how can a local VPN app send outbound traffic correctly through the Arris router and YF connection?

Nord VPN (subscription VPN on device app) - No issues, requires some kind of working outbound connection but it definitely works when I use it.

4G/5G connection (nothing to do with YF's network) - definitely rules out the device

Personal router tried on Tuesday - 100% working

Arris router tried again on Thurs and Fri - No connection, factory resets, weird IP shown as being allocated but no connection of any kind.

Edited by MarcDark (Sat 13-Jul-24 09:36:25)

It appears that the fault/error has been fixed. At some point between 3 and 5pm today my apps and devices started behaving properly again (coincidentally, just as I'd started making queries via social media).

I wonder if your Arris router will work properly again?

I wonder if they'll notify me? I'm still waiting for a call which was promised last Friday (it's Tuesday as I write).

K

Can others confirm they're also seeing an improvement?

In reply to a post by Mark07:

Can others confirm they're also seeing an improvement?

I have noticed new packet loss on V6 traffic but I just logged a ticket with Networks so will wait and see what they say. I suspect it's something internal on my end though.

In reply to a post by RR_The_IT_Guy:

In reply to a post by Mark07:

Can others confirm they're also seeing an improvement?

I have noticed new packet loss on V6 traffic but I just logged a ticket with Networks so will wait and see what they say. I suspect it's something internal on my end though.

Interesting it's only on v6 though, do you still see the v6 packet loss if you ping somewhere like Cloudflare / Google?

Far as I can see, v6 will take the same route as v4

The IPv6 thing is quite likely in many cases but you've ruled that out.

Note that the incoming notification for iOS I think come via Apple so it's not necessarily a 'one way' issue.. not sure if it's a TCP packet that needs return path anyway.

One thing I've seen elsewhere is routing problems affecting some IP blocks.. so if you're on 1.2.3.4 and another YG customer is on 5.6.7.8 it doesn't mean you always have the same routing, especially back from other networks.. Ideally you do need those traces both ways to work it out.

In reply to a post by Mark07:

In reply to a post by RR_The_IT_Guy:

In reply to a post by Mark07:

Can others confirm they're also seeing an improvement?

I have noticed new packet loss on V6 traffic but I just logged a ticket with Networks so will wait and see what they say. I suspect it's something internal on my end though.

Interesting it's only on v6 though, do you still see the v6 packet loss if you ping somewhere like Cloudflare / Google?

Far as I can see, v6 will take the same route as v4

So been speaking with networks, we believe we know where the issue is on my side, so are completing extra checks, the routing may be different. I have noticed that since rebooting kit packet loss has stopped but the IPv6 address has changed so not 100% sure what's going on.

Also noticed weird behaviour between Linux and Windows so there are so many factors at the moment we are trying to rule as many out as possible, one of the network guys did some MTRs from my exchange etc and couldn't see anything.
It's all very odd
What's weird is that when I rebooted the ONT I lost all IPv6 connectivity for over an hour and only had IPv4