We observed 2-3 times latency issue in network infrastructure for a minute or less duration since yesterday. We narrow down the issue and identify specific problematic module on core device. We are working on it to fix the issue and due to the part of troubleshooting, it may cause latency issue on some of connections intermittent

Do they have that in English?

2-3 times latency for a minute or less duration? That cannot seriously be the same issue.

Talk about down playing a problem

It was poorly worded and seems to down play the issue somewhat.

Fingers crossed since the last period of issues that ended around 9:15am today no sign of the same pattern of bad latency and packet loss coming back again 2 hours later. Hoping it is fixed.

Latency seems to be back to normal now - hopefully it will stay that way.

FWIW: I'm connected to BRAS 46.37.48.1 too, although I've been away for the last couple of days, so wasn't using my Cerberus line interactively.

I have the Perfsonar toolkit running continuously to a remote node, only over IPv6 (perfsonar doesn't like NAT). It sends 10 packets per second in each direction. Perfsonar didn't record any increase in latency or packet loss over the past few days.

Hence it's possible the problem affected only their IPv4 transit, not their IPv6 transit.

They still should have some monitoring of their own to notice the problem before their customers do.

Thanks for the info however in my case Thinkbroadband BQM is pinging my IPv6 address, and when I also tested to confirm the latency issues by pinging other websites (e.g. bbc.oc.uk) that defaults defaults to IPv6 which saw ping times > 500ms.

I first became aware of a problem using VoIP which goes over IPv6 as it was choppy audio and I could barely understand what the other person was saying and gave up on the call and tried again with the same results. I then reached for my mobile which is using EE Wi-Fi calling which goes over IPv4 and that was exactly the same issue, hence I realised the fault seemed to be my side.

So I would say IPv6 and IPv4 was equally affected, certainly what I saw. Thankfully seems to be fixed now, perhaps it was not affecting every customer or only certain routes?

Edited by E300 (Sat 31-Oct-20 13:25:31)

I was seeing issues on both IPv4 and IPv6 with the graphs matching pretty much perfectly. Seemed to start on Thursday afternoon, got progressively worse through Friday and was very bad overnight.

Since about 09:30 this morning the graphs have settled down.

I'm still seeing a few spikes on the graph but it seems a whole lot better than it's been the last couple of Days

Had the following update to my ticket @ 14:13:

“ This is to inform you that our Senior Network Team has performed emergency maintenance to eliminate the latency issue. We have updated Cerberus service status page for customer awareness. Request you to please monitor the connection and let us know in case of any further issue.”

BQM graphs seem to have improved since the last major spike around 08:30/9:00 this morning but still not completely back to normal (yet)...latency spikes are still evident from around 11:30 and 13:30 following the same periodicity of the last 48 hours.

https://www.thinkbroadband.com/broadband/monitoring/...

My graph looks exactly the same, so it seems the latency issues are still repeating every couple of hours as before, just they have somehow mitigated the effect, so perhaps not fixed but more a work around at this stage. Makes you wonder what could be causing it.

https://www.thinkbroadband.com/broadband/monitoring/...

Had exactly the same ticket response as well.

Edited by E300 (Sat 31-Oct-20 15:16:19)

"eliminate"

Right.

My graph is still spiking every 2 hours. Spikes are less dramatic than yesterday, but clearly still there.

https://www.thinkbroadband.com/broadband/monitoring/...