BT FTTP 900 - Packet loss

Good morning all.

I've had a new fibre connection for 1 week and I've finally ditched the supplied router and connected up my RT-AC3200. All in all things are well, except for in games where I'm constantly getting packet loss warnings.

Here's my BQM graph

What could be happening?

In reply to a post by Cheule:

Good morning all.

I've had a new fibre connection for 1 week and I've finally ditched the supplied router and connected up my RT-AC3200. All in all things are well, except for in games where I'm constantly getting packet loss warnings.

Here's my BQM graph

What could be happening?

Was it working fine when you were using the BT router?
Your graph is a bit less clear, although there's a bit of a correlation between max latency jumps (yellow) and packetloss (red) which may be congestion.

For your game, are you on a physical ethernet (not wi-fi) connection? This shouldn't affect the BQM though as it's a v4 address so it's unlikely you'll have a routable block with separate IP for laptop/desktop.

I've just run a ping stream to you and it's pretty consistent even though BQM showed latency/loss.. Although it may have just stopped - your loss is very mimimal but your latency spike should be a bit more obvious but I'm seeing very little variation:
rtt min/avg/max/mdev = 5.256/5.638/6.570/0.300 ms

The one thing about the BQM is that it uses much smaller packets than normal ICMP ping - this may make routers behave a bit differently at times. This keeps traffic lower than doing default pings and should therefore be a bit faster.

Is your router showing anything?

The problem with tiny issues is they are hard to troubleshoot.. some of yours will involve literally one very long latency packet (yellow max) and 2-3 seconds of packet loss which means you can't really compare it to an 'mtr' output as easily to see where it may be coming from.

The one thing you could do is ping with say a 0.2 second interval (I'm not sure how yo do that in Windows but in Linux you can do "ping -i 0.2 x.x.x.x" where x.x.x.x is the IP.. you could monitor how it behaves to a few different destinations.. check if it's consistent.. if it is then it's more likely to be your ISP or your network. if it's not then it may be the remote side. Also doing that from your end should make sure the BQM IP is not a CGNAT endpoint or something similar. You want to make sure you're hunting down the correct issue.

i find mine shows packet loss when maxing out the connection (which isn't often) - but have no issues in games etc

In reply to a post by BuckleZ:

i find mine shows packet loss when maxing out the connection

That's normal.

Don't forget that red on the BQM doesn't necessarily mean that the packet is truly lost, just that the tbb pingbox didn't receive a reply within a specified time limit (500mSec I think). That can easily happen if the router is busy doing something more important than responding to incoming pings.

Good morning Seb, thanks for taking the time to reply. Sorry for the delay.

I'll address your questions in paragraph order.

1) When I first used the connection with the supplied SH2, the graph was just constantly 100% red.

2) All gaming equipment is on wired 1 gigabit connections. While on the SH2 it was using v6 by default, and my RT-AC3200 has v6 disabled. Would there be a usage case for me to enable v6? (While I was on it, BT would rotate my addresses which seems pointless on v6 addressing but there we are).

3) As for pings, I checked out a number of sites, all were pretty consistent. Pings to bbc.co.uk for example were a staggering 4.6ms min and 5.6 max, which is amazing seeing as I just came from Virgin where it was closer to 20.


Finally, looking at my graph this morning and it's looking 1000% better and I've not changed a thing. Sure, there's a slight bit of red ( and I mean the odd pixel ) but that's it. Far better and no pesky lost packets warnings for the past few days. Long may it continue!

Hi,

As your graphs before was all red it wasn't responding to ICMP so we can't really compare.
Your current graph (I presume you shared a live graph?) has latency variations but no packetloss (well 2 packets I can see lost in the 24 hour period which is near enough to zero).

v4 and v6 is routed completely independently so a monitor for v4 won't necessarily show a v6 issue. You may find v6 routing works around a problem, or causes a problem. It's one to test. If there's no reason to turn it off I'd leave it on but it needs checking.

Sounds like maybe there was a shorter term issue?

seb

Yes the latency does change quite markedly, before it was spiking to 80ms+ but now its' a little less than 60 I think. Hopefully a downward trend. As you say, hopefully an early hiccup

After thinking about the earlier suggestion that the router may be too busy to send ping requests, I logged into the router and monitored the dual cores while running speed tests. Each time, CPU 0 went to 100% and stayed there, CPU 1 basically did nothing.

After hitting the net it appears many are having the same problem but no clear results on how to fix it or get both cores to share the work.

In reply to a post by Cheule:

Each time, CPU 0 went to 100% and stayed there, CPU 1 basically did nothing.

My understanding (and I may be wrong) is that to do PPPOE encapsulation a router can only use 1 processor as the packets need to be strictly processed in order, that may account for why you're only seeing activity on CPU 0.

But if CPU 0 is busy on a single thread and a new request comes in (like another download or a ping) then shouldn't CPU 1 pick up that request as it is idle?

In reply to a post by ian72:

But if CPU 0 is busy on a single thread and a new request comes in (like another download or a ping) then shouldn't CPU 1 pick up that request as it is idle?

Not on PPPOE is my understanding its a major limitation, because of encapsulation it has to be processed by a single thread to be able to go down the pipe. Thats why a lot of people complain about PPPoE and why its a bottleneck, ISP/altnet who use DHCP don't have the issue.

Interesting, seems a bit pointless having 2 cores in the router then but I guess the router would support more than just PPPoE so the other core would be utilised if it was a different configuration.

In reply to a post by dect:

Not on PPPOE is my understanding its a major limitation, because of encapsulation it has to be processed by a single thread to be able to go down the pipe. Thats why a lot of people complain about PPPoE and why its a bottleneck, ISP/altnet who use DHCP don't have the issue.

I knew that was the case on the open source implementations of PPPoE for things such as OPNsense or pfSense. Is that the case for closed source implementations as well?

I don't have a PPPoE based ISP, but it should be easy to test with a Mac/Windows/Linux computer.

In reply to a post by jchamier:

I knew that was the case on the open source implementations of PPPoE for things such as OPNsense or pfSense. Is that the case for closed source implementations as well?

Its my understanding based on Opnsense/Pfsense which both run on FreeBSD, I didn't think it was a limitation caused by their code but more about encapsulation and the OP isn't running either Opnsense or Pfsense and they appear to be having the same issue.

On Opnsense and Pfsense both 'receive-side scaling' (net.inet.rss) and the 'kernel network dispatch service' (net.isr) can be enabled/configured to improve router performance by multi threading but it still doesn't help with the PPPoE encapsulation issue.

Edited by deleted (Thu 25-May-23 11:49:17)

In reply to a post by Cheule:

After thinking about the earlier suggestion that the router may be too busy to send ping requests, I logged into the router and monitored the dual cores while running speed tests. Each time, CPU 0 went to 100% and stayed there, CPU 1 basically did nothing.

After hitting the net it appears many are having the same problem but no clear results on how to fix it or get both cores to share the work.

Even high end routers sometimes drop pings or de-prioritises them - it's a feature as pings are generally not 'important'

In reply to a post by dect:

Its my understanding based on Opnsense/Pfsense which both run on FreeBSD, I didn't think it was a limitation caused by their code but more about encapsulation and the OP isn't running either Opnsense or Pfsense and they appear to be having the same issue.

Thanks

On Opnsense and Pfsense both 'receive-side scaling' (net.inet.rss) and the 'kernel network dispatch service' (net.isr) can be enabled/configured to improve router performance by multi threading but it still doesn't help with the PPPoE encapsulation issue.

I wonder which wholesale network will be the first to remove PPPoE then.

We have CityFibre to the East and West of us, and we are in their planning phase so I'm using good 'old BT just for now.

Hopefully they don't use PPPoE? Anyone know?

Sorry if this is stupid but everyone seems to be talking about your graph showing packet loss (which I believe is red dots). Am I blind because I literally don't see any?

This is what I see: https://i.imgur.com/4qEk5tF.png

The reason is that the link in the opening post back in May 23 is to a live BQM which is constantly updating, rather than a snapshot at the time as you can see by looking at the date. So the line conditions have changed and the BQM is now looking good with no packet loss.

@Cheule,

What was the solution to this? I am having similar issues, with 5% or less packet loss constantly.

I went from 48mb FTTC to 500 FTTP, TBH be honest I never tested the FTTC connection.

thanks