What is causing ping spikes on my FTTC line?

I'm seeing random ping spikes on my connection. I use SQM so I don't believe it's bufferbloat.

I used PingPlotter to look at the latency on the many hops between my router and 1.1.1.1 (my DNS server): https://i.imgur.com/hPDws79.png.

When the latency increases, I see an increase on hops beyond my router, what is this caused by? When I've had bufferbloat-induced latency I see it starting from the hop after my router, which makes sense.

I do a lot of latency-critical tasks and I see these spikes during them, so it is impacting my service.

If I can't resolve it, I would just like to know what might be causing it?

I did post about this before on another account, if this post seems familiar.

Edited by Gigabit (Mon 14-Dec-20 17:22:24)

I do a lot of latency-critical tasks and I see these spikes during them, so it is impacting my service.

..... and not at other times ? Sounds like whatever ‘latency critical tasks’ are is causing it then.

My apologies for not being clear. They happen all the time I just obviously notice them whilst doing these tasks.

I’ve been graphing for over a day, in the early hours when the network is quiet, I still see them.

I would like some explanation about why these spikes don’t show up from hop 3, i.e. after my router but much further down the chain. To me that suggests something in the core network?

Routers do not prioritise responding to ICMP pings, so if a device is passing traffic through it your pings may be delayed.

In an ideal world you need to ping a device where you know it is otherwise idle

In reply to a post by MrSaffron:

Routers do not prioritise responding to ICMP pings, so if a device is passing traffic through it your pings may be delayed.

In an ideal world you need to ping a device where you know it is otherwise idle

Okay, so does every connection do this then? I don't recall seeing it on previous ISPs.

I see it within the thinkbroadband latency monitor as well, I would have thought - perhaps naively - that this would prioritise pings?

In reply to a post by Gigabit:

I see it within the thinkbroadband latency monitor as well, I would have thought - perhaps naively - that this would prioritise pings?

The BQM merely pings your router and times the arrival (or absence of) the reply from the router. It can't alter the firmware or priority settings in it.

The difference from what you saw with previous providers could well be down to using a new router, if you are.

The post is indeed familiar.

There is no queuing mechanism that can manage the dozens or hundreds of connections a torrent client makes inbound. Anyone claiming inbound shaping is robust in those conditions is simply wrong. You have to set the bandwidth way low to mitigate microbursts as flows can and will saturate your connection before your router dropping packets, which is all it can do over UTP however pretty the queuing is, makes an impact.

The hops between your router and 1.1.1.1 are irrelevant. If you aren't experiencing any notable issues all is fine.

Not downloading potentially interesting content via dozens or hundreds of individual connections that may be unresponsive to congestion control seems a good idea as far as ways to stop it to.

Your Zoom call is one stream. You may have 250 peers on a torrent. How does your bandwidth look split 251 ways?

The intermediate hops may spike because they have better things to do than answer your traceroute. Routers have a forwarding plane which moves your traffic around to its destination and a control plane which deals with things like pings and traceroutes. On the carrier grade kit BT Wholesale use those are entirely uncoupled from each other. The forwarding plane uses ASICs and is super fast, the control plane at least in part uses commodity hardware and has way better things to do with its time than issue you an ICMP TTL Expired so may be late in responding.

If you must torrent may I recommend a QoS policy that recognises HTTPS traffic and guarantees it 50%+ of the capacity and drops everything else? That will hopefully mitigate microbursts.

Incidentally microbursts - your first hop, the BT BRAS, doesn't care what order it delivers packets to you in, and if torrent traffic is overwhelming your capacity before the QoS you and they have in place kicks in some traffic will be delayed.

Openreach don't care what they're sending you and don't do any kind of WFQ to let smaller packets through in between gaps. BTW don't either. First in, first out.

Either the QoS policy I mentioned above whitelisting known real-time traffic and heavily shaping else so that the TCP flows can't ramp up and UTP congestion control kicks in early or stop the torrents.

Ignore the routers on the way to the end destination. They may respond late if at all as they're there to route not to respond to your trace and have better things to do. They also have rate limits set on how often they'll respond to such things to avoid DoS conditions. The line cards in BRAS, your 172 hop, are usually pretty smart, deeper into the network the intelligence drops alongside the care for your traceroutes.

Regarding the jump in latency between a couple of the hops light takes time to travel so unless you discover superluminal transmission speeds there's about an 8 ms round trip to Yorkshire in the middle of the UK going up to 20 ms or so in the Highlands to London. Light doesn't get close to full speed on regular fibre, more like 2/3rds, and even at 200,000 km a second takes a little while to get from Scotland to London and back. At 200,000 km/s, a 500 km fibre run, remember these DM aren't straight lines and usually follow major roads, you're talking a 1000 km round trip, or 5 ms @ 200,000 km/s.

That's your increase as you go along the BT network: the speed of light.

PS Congratulations on bringing me back to this forum to answer this. I thought it should have been answered either here or the other place a while back and got bored reading it so thought I'd close it off.

TL;DR

Torrents are worst case scenario for inbound shaping, inbound shaping isn't great anyway, intermediate router latency is unreliable: monitor the end host via ping not traceroute.

Edited by CarlTSpeak (Tue 15-Dec-20 01:12:49)

In reply to a post by Gigabit:

In reply to a post by MrSaffron:

Routers do not prioritise responding to ICMP pings, so if a device is passing traffic through it your pings may be delayed.

In an ideal world you need to ping a device where you know it is otherwise idle

Okay, so does every connection do this then? I don't recall seeing it on previous ISPs.

I see it within the thinkbroadband latency monitor as well, I would have thought - perhaps naively - that this would prioritise pings?

The TBB latency monitor is pinging you. It lives to send and receive pings. Those devices have such minimal port configurations to ensure they can.

If this is troublesome disconnect everything from your Hub or whatever bar one machine, kill the Torrents and everything else on there, and see how it looks then.

Congestion is really, really obvious on BQMs. Your spikes sound more like microburst though you are on partial copper. That part I leave to the VDSL obsessives. If G.inp is correcting something there will be a spike but it'll be tiny.

Thanks both for your responses, very much appreciated.

When I get some time I will try as suggested and have one PC connected to the Hub and nothing else.

Regarding the spikes, haven’t had any torrents running the last couple of days - client is closed - so I don’t think that is the cause but G.INP sounds highly possible and something I hadn’t thought of before.

but G.INP sounds highly possible and something I hadn’t thought of before.

You are selectively reading Carls comment.
G.INP would be responsible for tiny spikes.

For the benefit of others in this thread who don't use kitz the original kitz thread was called something like "high ping spikes/latency when torrenting".

I haven't contributed to the kitz thread but the increase in latency when torrenting (or any heavy downloading/uploading) is due to.... you using the connection.

As mentioned by Andrew above ICMP is not treated as priority traffic.
Your downloads get priority and the pings get delayed.

You haven't posted your BQM here or on kitz.
Congestion is usually very obvious on a BQM.
It would help if you posted them.