In reply to a post by candlerb:

No, it will be completely affected. The equipment being discussed, known as LNS or BNG, is what terminates broadband connections coming into their network. The traffic between their voice servers and the Internet won't go through these.

I think you mean: No, it will be completely unaffected

Andrew, just to clarify, I wasn’t suggesting the 9000s were overloaded - there would have been obvious performance issues had that been the case. My ponit was simply that a presumably increasing load may have exposed the weakness. Good luck anyway - I’m sre the issue will be reolved.

Many thanks for your reassurance. OK, back to your regular grievances.

In reply to a post by andrewhearn:

This is correct.

We have 20 or so of the FireBrick FB6000 LNSs which have more than enough capacity for the lower speed customers we have.

Once we've replaced a, b, c, d Gormless with FireBrick FB9000s they will then be put in to service for the faster-speed customers.

This isn't because the existing LNSs are overloaded (we don't believe load is the cause of the lock-ups), but to spread the load so fewer customers are affected.

Happy to answer more question if there are any!

Thank you for the update. With regards to X.Witless which appears to be pretty much rock solid, is that different somehow to the other Firebricks and if so has that been able to narrow down the potential issue?

In reply to a post by andrewhearn:

I think you mean: No, it will be completely unaffected

Err, yes of course! Too late to edit post now though...

In reply to a post by j0hn83:

In reply to a post by bellerby:

Assuming they are all running the same software/firmware then does this imply some very low down hardware/component/pcb build issue?

...snip...
I would put my money on it being a software issue causing them to hang or an issue with a component on the board (chip/ram etc).

Given the symptoms described in the status post, I'd be inclined to suspect something wrong with the pullup/pulldowns on the CONFIG pins in the M.2 socket for an NVMe SSD.

If the CPU reads all 4 CONFIG pins as 1s, it should ignore the rest of the pins in the socket; however, if one of them is marginal, and sometimes drops to a 0, the SoC should generate an interrupt and expect the CPU to change pinmux settings to match the CONFIG pins. I can envisage a few tight timing cases where the pinmux setting change causes the CONFIG pins to change again, and the resulting timings tickle bugs that "can't happen" if your design complies with PCIe spec (presence pair are the last two pins to connect, presence must be debounced, delay between presence connecting and hotplug being asserted), or M.2 (no hotplug allowed).

For an ISP to be this open and transparent is unheard of.

Most other ISPs would just stay silent, in the hope that you go away, making you think BT or TalkTalk is the culprit. In the meantime, trying to fix things in the background.

The benefit here is that it’s all in house (or they work very closely with Firebrick) and from what has been put out, it seems they are competent and confident in what they do.

Although I’m sure it is annoying, at least you can fall over to another LNS.

My advice is to stick with a company who clearly know what they are doing. All companies have a rough patch, and this is one of them, I’m sure with your support they’ll be able to ride through it.

I’ve used their L2TP service in the past with Starlink and found it great.

Excellent point.

Just lost my SSH session to work...

https://aastatus.net/42629
Lines on the X.Witless LNS were affected, sessions are recovering.

Edited by perlen (Tue 27-Feb-24 12:35:37)

Not only that - my long standing connection to x.witless was abruptly ended just before 02:00 this morning after 2 drops in quick succession. These are clearly shown on the blip graph but have not yet been commented upon. Now on c.gormless.