Strange historical FTTP issue

I have FTTP 150/150 with Swish Fibre. Last Autumn I had a minor problem with short "outages". They generally lasted less than a minute, most of the time they went unnoticed but were annoying if they occurred eg while streaming or on Zoom. My router (TP-Link) didn't lose connection and there was nothing in its logs, but for a short time no data came in or out.

Being behind CGNAT I can't run a BQM, so after this had been going on for a few weeks I knocked up a quick prog on an RPi (wired connection) to ping a list of a few well known IP addresses every 30sec, and log the results. Sites like 1.1.1.1, 8.8.8.8, www.bbc.co.uk, with/without DNS lookup, the list varied a bit. The pattern was that when one site failed to respond to a ping, they all did, and they all resumed working at the same time. So the problem didn't seem to be with just one site. At worst I was seeing ~6 events/week, pretty randomly distributed around the clock.

Then just as I was getting the logging organised and thinking of other things to try, around mid-November the issue went away. The logging prog is still running, but it hasn't seen anything for nearly 2 months now. I wondered if the problem was quantum mechanical, and measuring it changed its behaviour, but probably not. More likely something flapping at my ISP which has now been fixed.

It was never a serious problem and it's gone away, so it's not important, but for my own curiosity, since we have wizards on the list who can diagnose stuff from 3000 miles away, I wondered if anyone could suggest what it might have been?

In reply to a post by Thaumaturge:

More likely something flapping at my ISP which has now been fixed.

It was never a serious problem and it's gone away, so it's not important, but for my own curiosity, since we have wizards on the list who can diagnose stuff from 3000 miles away, I wondered if anyone could suggest what it might have been?

Flapping like a pheasant. More than likely something like that. 😅

Could be flapping card, do you have any neighbours who have the service who would like to run the same test?
If they have the same issue it rules out your hardware and points to ISP network in some way.

Do you know if you have an IPv6 address? if so, you might be able to set up the ping test to a wired laptop or desktop to see if it reports, you may have to turn off the firewall for external pings.

Also if your happy to run a traceroute on the thinkbroadband tools and publish the results it might help locate anything abnormal.

It might be worth setting up a reverse ping test on F8lure It's an exe download you leave running on a PC plugged in via ethernet and it will produce some results. In this case it might be suitable.

Forgot to ask, have you tried another router or direct into the ONT to see if the fault occurs, it could simply be faulty hardware or firmware.

You have FTTP so the easiest place to start is to see if the ONT lights do anything while your service drops. If you're losing the light down the fibre (turning the PON light off or maybe lighting an LOS light) then there's a physical issue somewhere on the fibre network.

Edited by jpm (Wed 08-Jan-25 20:01:37)

In reply to a post by RR_The_IT_Guy:

Could be flapping card, do you have any neighbours who have the service who would like to run the same test?
If they have the same issue it rules out your hardware and points to ISP network in some way.

Only know of 2 who have the same service, and sadly neither is nerdy enough to be up to running a hack 'C' program on an RPi (or anything really).

I did wonder whether my GS308 switch might be the problem, but it also occurred on wifi connections which eliminates that. Might be my router throwing a wobbly, yes. It reboots automatically once a week (as recommended by TP-Link) but around mid-November (when the issue disappeared) it got power cycled a couple of times for a kitchen fitter to have the power off. Maybe that gave it the requisite kick.

In reply to a post by RR_The_IT_Guy:

Do you know if you have an IPv6 address?

Swish don't support IPv6, and sadly hell is likely to freeze over before they do. If they did, I would run a BQM. At present I don't have a realistic alternative to Swish.

In reply to a post by RR_The_IT_Guy:

Also if your happy to run a traceroute on the thinkbroadband tools and publish the results it might help locate anything abnormal. It might be worth setting up a reverse ping test on F8lure It's an exe download you leave running on a PC plugged in via ethernet and it will produce some results. In this case it might be suitable.

Thanks for the ideas. If the problem recurs will try traceroute, but as I haven't seen it for 2 months now I guess it's unlikely to show up anything at present. Don't know f8lure, but will take a look. Does it work behind CGNAT?

In reply to a post by RR_The_IT_Guy:

Forgot to ask, have you tried another router or direct into the ONT to see if the fault occurs, it could simply be faulty hardware or firmware.

I do have a spare router and will try that on any recurrence. Going directly into the ONT isn't really feasible, as I'd have to do it for ~weeks to be reasonably sure of catching it, meantime it would cripple my LAN. SHMBO wouldn't stand for it (she edits the village newsletter, for one thing).

In reply to a post by jpm:

You have FTTP so the easiest place to start is to see if the ONT lights do anything while your service drops. If you're losing the light down the fibre (turning the PON light off or maybe lighting an LOS light) then there's a physical issue somewhere on the fibre network.

As mentioned, the drop-outs were short (< 1min), & most went by unnoticed. When I did spot them, eg if streaming at the time, I never observed the ONT with unusual light patterns. But the ONT is not in my living room & I might not have got there in time (I'm nearly 80). The fact that the router never lost connection or logged anythng unusual tends to suggest the fault was further away.

I believe Swish use DHCP, so the ONT losing connection and coming back again won't show anything in the router logs as long as the ethernet connection stayed up - your router would just see received traffic disappearing but would be unlikely to log that.

To investigate further I'd set up monitoring from your network out to the first hop on your WAN side which is likely to be the gateway IP if you're using DHCP. If it responds to ping then set up a BQM to that address. If pings stop to that IP and the BQM also shows a loss then it's a fault within Swish's network. If pings stop on your side and the BQM shows no loss then it's more likely that your fibre or ONT has issues, but observing the lights would be the best way to confirm this.

Edited by jpm (Wed 08-Jan-25 21:02:13)

Yes, Swish do use DHCP. BQM-ing something in Swish and correlating that with drop-outs is an interesting idea I hadn't thought of. Presumably it would have to be something upstream from the CGNAT translation. Where is that likely to be done? I don't know much about these things.

I will have a play and see what I can get to work.