what to do with the pppoe virus?

PPPoE never leaves where it enters. Introduces 1/200 transfer and some computational overhead, which is much if the router can't offload it and encapsulation cannot be done in parallel.

What's the worst is all of the Elite-5 ISPs, Zen, Aquiss, AAISP, Trunk Networks, Cerberus use PPPoE. I would expect those smaller ISPs to have considered the advantages of DHCP and switched to it, nevertheless I know only TalkTalk and Sky who don't use PPPoE. I can't understand this because I would expect these smaller companies whose customers are more technical to react.

One advantage of PPPoE is, assuming that the ISP allows multiple PPPoE sessions, one can use PPPoE passthrough to bypass NAT and optionally use DMZ. So there's no harm in keeping PPPoE optional but I don't think it's worth it. You can try setting up PPPoE on all your computers for improved performance to see to what extent it complicates things.

</rant>

So what we can do against this? Would "saynotopppoe.co.uk" that names and shames those ISPs work? I guess that'd be more efficient than a change.org listing. What do you think?

My suspicion is that the vast majority of the country don't care and only very techy people would bother to get at all worried about what encapsulation the ISP is using.

In reply to a post by ian72:

My suspicion is that the vast majority of the country don't care and only very techy people would bother to get at all worried about what encapsulation the ISP is using.

I agree that it would be lovely if PPPoE went away, but I suspect most of the <0.5% of the population who want to get rid of it also know how to work around/with it and have the resources to do so.

In reply to a post by weelan:

What's the worst is all of the Elite-5 ISPs, Zen, Aquiss, AAISP, Trunk Networks, Cerberus use PPPoE. I would expect those smaller ISPs to have considered the advantages of DHCP and switched to it, nevertheless I know only TalkTalk and Sky who don't use PPPoE.

A good first port of call is to understand why these guys use PPPoE or not. TalkTalk and Sky use their own networks in every site they provide FTTC/P so no need to aggregate together tons of connections in a BRAS. The 'Elite-5' all make extensive use of BT Wholesale. BT Wholesale in turn use PPP. These smaller companies' more technical customers may well understand this.

Just FYI TalkTalk Business, the TalkTalk wholesale operation, also uses PPP. I would assume there are valid technical reasons for this and, again, more sophisticated customers may understand some of these.

There are 4,094 VLANs maximum. Openreach hand traffic to providers via SVLANs with CVLANs inside them. Providers have to do some quite creative stuff from here rewriting VLAN tags to bundle customers back into VLANs and deliver them to the ISPs they serve. BT Wholesale handle millions of end users via PPP and L2TP. Be quite a challenge dropping all that in favour of straight Ethernet.

The counter point is that low cost hardware is available that can handle a gigabit with PPPoE, and with baby jumbo support there's no fragmentation overhead associated with it. So if you know enough to know that PPPoE can cause problems you also likely know enough to make them a non-issue.

In reply to a post by weelan:

So what we can do against this? Would "saynotopppoe.co.uk" that names and shames those ISPs work?

No, because nobody cares. You might as well campaign for "say no to IPv6" because the headers are 20 bytes larger than IPv4.

I have PPPoE. It works fine. It's just 8 bytes of header, which is insignificant both in bandwidth and processing. I enable baby jumbos so I still get a 1500 end-to-end MTU.

For business connections it's much easier to assign /32 static IPs with PPPoE than with DHCP, avoiding a /31 or /30 link network which would burn more IPs.

Another benefit of PPPoE is keepalives. I can tell within a few seconds if my PPPoE connection has gone down, even if the layer 1 connection to the ONT is still up.

The only argument I can see to drop PPPoE is to provide 'greener' broadband. PPP requires an ASIC or a beefy CPU to maintain a PPP tunnel at gigabit (and beyond) speeds which is both expensive and power hungry.

So if Openreach provided a transparent L2 bridge between customer and ISP, the only CPE required is the ONT. This is easily possible nowadays and doesn't require even VPLS let alone VLANs.

The ISP in turn can turn off their very power hungry LNS concentrator, thus removing 2 boxes from the national infrastructure and saving quite a lot of power and plastic and provide their customer with whatever they want over that L2 network.

In reply to a post by zzing123:

The only argument I can see to drop PPPoE is to provide 'greener' broadband. PPP requires an ASIC or a beefy CPU to maintain a PPP tunnel at gigabit (and beyond) speeds which is both expensive and power hungry.

No it doesn't. Inserting or removing 8 bytes in a packet doesn't require an ASIC. Plenty of routers without ASICs can handle PPPoE at gigabit.

So if Openreach provided a transparent L2 bridge between customer and ISP, the only CPE required is the ONT.

The user will still need a CPE to perform routing functions, including NAT.

Or are you saying that the ISP should be doing NAT at their side? That would require something far more complex than a simple BRAS/LNS, as it would need to be stateful. You've replaced a BRAS with a CGN device. Oh, and it has to do stateful DHCP to track every single device within the customer's own LAN. And it has to be configurable (by the user) for inbound port forwarding.

In any case, the user would still need a layer 2 device, i.e. a switch and/or wireless access point, to plug into the ONT, simply to allow multiple devices to share the link. So they end up with a different type of CPE. As far as 99% of users are concerned, it's just a box where they connect things. They have no idea whether it's a layer 2 or layer 3 box.

In reply to a post by zzing123:

The only argument I can see to drop PPPoE is to provide 'greener' broadband. PPP requires an ASIC or a beefy CPU to maintain a PPP tunnel at gigabit (and beyond) speeds which is both expensive and power hungry.

So if Openreach provided a transparent L2 bridge between customer and ISP, the only CPE required is the ONT. This is easily possible nowadays and doesn't require even VPLS let alone VLANs.

The ISP in turn can turn off their very power hungry LNS concentrator, thus removing 2 boxes from the national infrastructure and saving quite a lot of power and plastic and provide their customer with whatever they want over that L2 network.

Openreach provide a layer 2 link between end user and the CP. This requires VLANs. Stacked VLANs usually, SVLAN and CVLAN. Without any VLAN usage how did you have in mind Openreach knowing how to reach each CP? Without VLANs or some other encapsulation how did you have in mind CPs separating out customers?

There is also the small matter that an ISP needs to have a link to every Openreach OLT/L2S if there's no encapsulation happening anywhere. Most ISPs can't afford this, it's time consuming, and that's why no-one sells from every CityFibre FEX. ISPs are waiting for their national product.

If you've solutions for these I'd welcome them, as your post carried a fair bit of confidence in the statements. Thanks!

In reply to a post by candlerb:

No it doesn't. Inserting or removing 8 bytes in a packet doesn't require an ASIC. Plenty of routers without ASICs can handle PPPoE at gigabit.

Name one 1gbps ISP-supplied CPE router that can physically push 1gbps randomised traffic at full MTU that doesn't use PPP hardware acceleration or a big CPU that takes a lot of power.

In any case, you need a processor to do PPP. I'm just saying remove that entirely... Just taking BT by example: A BT Smart Hub may not be super high power at 14W. But multiply that by the 14 million customers BT have, and that's 196 MW of power being used that's unnecessary. If using FTTP, there's another 12W being used by the ONT...

In reply to a post by candlerb:

The user will still need a CPE to perform routing functions, including NAT.

Look my post above was just trying to find one benefit to dropping PPPoE - the power usage scenario. You and I'd be mad not to have routing at each site as we care about managing our data and the sanctity, privacy and security of our LANs because we have a clue what we're doing.

But for the sake of argument, if you removed the CPE and 'extend the LAN' across Openreach to an isolated VPLS-like broadcast domain controlled by the ISP, there's no reason the ISP can't provide the DHCP and NAT their end hosted centrally with no need for any plastic rubbish at home. It'd also make life a lot simpler for ISPs supporting Grandma too as they'd have a full view of the LAN for diagnosis.

Given that ISPs practically force people to use ISP-supplied routers and take pride in managing customer's Wi-Fi the entire premise of NAT, the notion of LAN security and privacy is actually completely moot anyway. But this is all value-add you and I don't need. I'd prefer to pay for a simpler, cheaper service without any of that stuff.

The ONT is a Layer 2 device - it's essentially just a 2-port switch with one RJ45 and one PON port...

In reply to a post by CarlTSpeak:

In reply to a post by zzing123:

The only argument I can see to drop PPPoE is to provide 'greener' broadband. PPP requires an ASIC or a beefy CPU to maintain a PPP tunnel at gigabit (and beyond) speeds which is both expensive and power hungry.

So if Openreach provided a transparent L2 bridge between customer and ISP, the only CPE required is the ONT. This is easily possible nowadays and doesn't require even VPLS let alone VLANs.

The ISP in turn can turn off their very power hungry LNS concentrator, thus removing 2 boxes from the national infrastructure and saving quite a lot of power and plastic and provide their customer with whatever they want over that L2 network.

Openreach provide a layer 2 link between end user and the CP. This requires VLANs. Stacked VLANs usually, SVLAN and CVLAN. Without any VLAN usage how did you have in mind Openreach knowing how to reach each CP? Without VLANs or some other encapsulation how did you have in mind CPs separating out customers?

There is also the small matter that an ISP needs to have a link to every Openreach OLT/L2S if there's no encapsulation happening anywhere. Most ISPs can't afford this, it's time consuming, and that's why no-one sells from every CityFibre FEX. ISPs are waiting for their national product.

If you've solutions for these I'd welcome them, as your post carried a fair bit of confidence in the statements. Thanks!

https://www.juniper.net/documentation/us/en/software...

VXLAN is basically like VLAN. But instead of the 802.1q tag being limited to 4096 VLANs, you have a 24-bit VNI instead and internally you can route and manage via BGP much like a VPLS.

Edited by zzing123 (Tue 25-Jan-22 16:19:02)

I'm not against PPPoE. I'm only against making them necessary. Otherwise they're very useful. Would you mind sharing your ISP's name and router, I've never thought of *baby* jumbos?

> Name one 1gbps ISP-supplied ...

I too would be interested to see a router succeed in 900mbps PPPoE. One doesn't need to find an old router to test this, many routers allow toggling offloading. I don't think any 1Gbps router without offloading can achieve that.

Any router that allows you to set the MTU of the WAN interface and pppoe interface will allow you to do this.

I know BT support this even though they don't really advertise it. My WAN interface has an MTU of 1508 which is enough to compensate for the pppoe overhead and allows the pppoe interface to work correctly with an MTU of 1500. I know this because I tested it from a VPS in London and can see that 1500 byte ICMP packets make it through correctly:

$ ping -M do -s $((1500-28)) -c3 MY_PUBLIC_IP
PING MY_PUBLIC_IP (86.183.XXX.XX) 1472(1500) bytes of data.
1480 bytes from host86-183-XXX-XX.range86-183.btcentralplus.com (86.183.XXX.XX): icmp_seq=1 ttl=57 time=8.36 ms
1480 bytes from host86-183-XXX-XX.range86-183.btcentralplus.com (86.183.XXX.XX): icmp_seq=2 ttl=57 time=7.71 ms
1480 bytes from host86-183-XXX-XX.range86-183.btcentralplus.com (86.183.XXX.XX): icmp_seq=3 ttl=57 time=7.66 ms

--- MY_PUBLIC_IP ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 7.663/7.909/8.358/0.317 ms

Thanks, MTU increase done. Now we need to solve the computational cost problem.

In reply to a post by weelan:

I'm not against PPPoE. I'm only against making them necessary. Otherwise they're very useful. Would you mind sharing your ISP's name and router, I've never thought of *baby* jumbos?

Baby jumbos are when you increase the ethernet frame size to carry a data payload of 1508 bytes instead of 1500 bytes.

I don't use an ISP-supplied router, I do this on my own Mikrotik routers - one FTTC via Draytek modem, one FTTP to ONT. It works on both.

The ISPs are Plusnet and Cerberus respectively.

In reply to a post by weelan:

Thanks, MTU increase done. Now we need to solve the computational cost problem.

The "computational cost" of PPPoE is so tiny you won't be able to measure it. Certainly not compared with all the other things your router is doing, especially NAT, where it has to modify every packet passing through, update the checksum, and manage state tables.

I'm convinced. It's not worth taking action to make pppoe only optional.

Aware of VXLAN and use it, it's nothing like VLAN, it's an overlay network.

I have genuinely never heard anyone suggest VXLAN as a way to build an ISP. The largest MX router Juniper have can handle 32,000 of them last I checked. You'd need tons of devices with a thousand+ VTEPs to cover all the exchanges and require a bunch of SD-WAN to steer a customer to the correct handover VTEP to deliver to their service provider.

This ignoring the issues around how to learn the hosts. Flood and learn would be interesting across a a few million VXLANs spread across thousands of VTEPs.

PPP gets shoved into L2TP tunnels to reduce 'tunnel count' across networks and aggregate them. I'm not sure how so many VXLANs, need one per end host, would work given the need to aggregate a thousand exchanges to one or two NNIs.

EDIT: While I am thinking about it you know that 802.1ad allows for stacking of more than 2 tags, right?

Edited by CarlTSpeak (Tue 25-Jan-22 19:26:52)

In reply to a post by zzing123:

In any case, you need a processor to do PPP. I'm just saying remove that entirely... Just taking BT by example: A BT Smart Hub may not be super high power at 14W. But multiply that by the 14 million customers BT have, and that's 196 MW of power being used that's unnecessary. If using FTTP, there's another 12W being used by the ONT...

Openreach ONTs typically use a fraction of that, somewhere between 1 and 3 watts.

Also don’t forget BT SmartHubs will draw more due to their WiFi radios rather than any PPPoE encapsulation overhead (which runs on hardware offload rather than the ARM cores in the SoC).

In reply to a post by weelan:

> Name one 1gbps ISP-supplied ...

I too would be interested to see a router succeed in 900mbps PPPoE. One doesn't need to find an old router to test this, many routers allow toggling offloading. I don't think any 1Gbps router without offloading can achieve that.

To be fair MikroTik don't use any PPPoE hardware offload, and can manage gigabit class PPPoE connections on many of their models. Most models running quad-core ARM processors from the last few years have oodles of capacity.

Interesting, however as far as I can see all MikroTik Quad-core routers have a 10Gbps port, that's why I said "1Gbps routers". Otherwise I think i9 can also do gigabit PPPoE, unsurprisingly. My point is that if we need 10 Gbps routers to do 1Gbps PPPoE without offloading, then it stinks to require PPPoE.

I don't think you can have it all ways. A router with quad-core ARM can do multi-gigabit, but you exclude it for being "too powerful" because it has a single 10G port. A vanilla BT Smarthub can do gigabit, but you exclude it because BT optimised the design.

In short, all the ISP-supplied routers for gigabit services are capable of gigabit, and most use PPPoE. They have the necessary software and/or hardware tweaks to make this work. So what are you worried about? They are built down to a price. If supporting PPPoE made the routers significantly more expensive, and overall it were cheaper to use IPoE and DHCP (including all the central office kit required and customer support), then you can be darned sure they would do that. Conversely, the fact that they don't proves that overall the PPPoE solution is cost-effective for them.

Talktalk have chosen IPoE. I *believe* that was primarily driven by their IPTV architecture, and the support for multicast. But for whatever reason, that works fine for them too.

In reply to a post by candlerb:

Talktalk have chosen IPoE. I *believe* that was primarily driven by their IPTV architecture, and the support for multicast. But for whatever reason, that works fine for them too.

They used IPoE before Youview was even a thing.

It's just Openreach's multicast service, sent to Youview boxes. It's the exact same multicast to Youview boxes used by BT and formerly Plusnet, who both use PPPoE.

PPP is used because it's easier to wholesale.

But wasn't there "TalkTalk TV" even before YouView?

There was Tiscali TV, which Talktalk acquired in late 2009 and then rebranded. Tiscali TV was multicast.
Talktalk were still requiring user credentials during that time period.

I'm not exactly sure when Talktalk switched to IPoE but it wasn't/isn't a requirement for multicast.

I was quite a fan of Tiscali TV. We weren't allowed an aerial on our building and Tiscali TV had the Freeview channels (or whatever it was called at the time) broadcast over the multicast.
It has taken over a decade for that to be replicated.

I'd have thought an overlay network within an exchange would be quite a good idea. But QinQ works although you'd need a 4096^3 to cover the nation, so you'd need a third stacked frame. How you split that across customers, exchanges and CP's is another matter of course and it seems to be a nightmare to configure...

I'm not against PPP or the fact L2TPv2 bundles tunnels - was just providing a scenario for the OP's original argument, that the fact we need any processing power CPE-side to provide a service rather than just plain ethernet.

So how are leased lines handled from beyond the ODF in an exchange? I'm pretty sure some exchanges, eg in the City, will have more than 4096 leased lines served if they're using the same CVLAN set up, but I'm not sure how. I know BTW has it's silly EtherFlow and Etherway's, rather than TTB's simpler just-give-a-bearer to an ISP. Again, not sure how CityFibre operates, but curious.

But how would you expand the leased line framework to manage what is effectively CityFibre's Ethernet Flex product (i.e. leased line over FTTP with 200mb committed and 1000 burst) for every premises?

In reply to a post by zzing123:

I'd have thought an overlay network within an exchange would be quite a good idea. But QinQ works although you'd need a 4096^3 to cover the nation, so you'd need a third stacked frame. How you split that across customers, exchanges and CP's is another matter of course and it seems to be a nightmare to configure...

2 stacked gives 16 million VLANs: more than enough for 50% take up nationwide.

So how are leased lines handled from beyond the ODF in an exchange? I'm pretty sure some exchanges, eg in the City, will have more than 4096 leased lines served if they're using the same CVLAN set up

They go around the GEA kit that we use for FTTP and straight into switches or routers rather than via an Openreach switch. That can have its own Q-in-Q or use a routed interface and advertise the subnet to the rest of the network via an IGP.

But how would you expand the leased line framework to manage what is effectively CityFibre's Ethernet Flex product (i.e. leased line over FTTP with 200mb committed and 1000 burst) for every premises?

Set those parameters on the 802.1p shaping on the OLT. Don't allow the backhaul from the OLT to congest until it hits the ISP, then it's their problem.

My router has been double jabbed, and I have recent installed a booster, will it get this PPPoE Virus?

Interesting. So a simpler, cheaper, flatter network and proves GEA is a load of unnecessary [censored] that really should be handled as part of a larger backhaul network (whether wholesale or directly) and that Ofcom should think harder about what to regulate.

You'd probably need to trunk connections into fatter connections to the switches to save space (I don't know why the OLTs only have 10GbE and not something faster or even DWDM directly), but it seems a lot simpler, cheaper and more scalable.

been looking into this, (about to get an Aquiss 900 fttp line, when Openreach get their fingers out & finish the ONT installation) it *seems* the main problem with PPPoe these days is for those of us running *bsd based routers/firewalls (Opnsense / pfSense & the like), in these the pppoe process is only single threaded so throughput is limited to what a single core can handle (getting gb takes some serious omph). switch to a Linux based devise & throughput rockets.

There are various consumer routers which *claim* to be able to handle these speeds with PPoE (I *think* Openwret is capable on very modest hardware, might have to try it, when I can)

I've always assumed that the PPPoE is the encapsulation that lets so many companies provide service over another companies infrastructure, is there *any* isp that provides service over Openreach infrastructure (not LLU) that *doesn't* use PPPoE?

TalkTalk (resi) and Sky

> isp that provides service over Openreach infrastructure (not LLU) that *doesn't* use PPPoE?

I think he meant "Openreach infrastructure (not other than BT Wholesale)". If TT or Sky wholesales are not available I don't think Sky or TalkTalk would serve that address.

Edited by weelan (Wed 26-Jan-22 19:00:13)

In reply to a post by candlerb:

The user will still need a CPE to perform routing functions, including NAT.

Well if the ISP pulled there finger out and implemented IPv6 ....

You would still likely need a switch and WiFi access points but PPPoE needs to die die die.

What are the Elite-5 group of ISPs in any event? Does this concept exist anywhere other than the OPs imagination? 🤣😎

In reply to a post by zzing123:

Interesting. So a simpler, cheaper, flatter network and proves GEA is a load of unnecessary [censored] that really should be handled as part of a larger backhaul network (whether wholesale or directly) and that Ofcom should think harder about what to regulate.

You'd probably need to trunk connections into fatter connections to the switches to save space (I don't know why the OLTs only have 10GbE and not something faster or even DWDM directly), but it seems a lot simpler, cheaper and more scalable.

The 10G ports on the OLTs are just PHY on line cards. The OLT can certainly provide higher capacity ports when they need to, though obviously that'll mean fewer cablelinks per card due to backplane restrictions.

GEA is the same - QinQ. The PPP part runs over the top. Openreach can happily insert relevant information into DHCP options or PPP headers to give the CP the information they need to know.

The PPP most of the time is from BT Wholesale, running over the layer 2 GEA service.

Great reply Carl I suspected it was wholesaling related.

deleting rest as is old thread so no need for further discussion.

Edited by Chrysalis (Sat 12-Feb-22 11:10:06)

In reply to a post by Pheasant:

To be fair MikroTik don't use any PPPoE hardware offload, and can manage gigabit class PPPoE connections on many of their models. Most models running quad-core ARM processors from the last few years have oodles of capacity.

Agreed, I have two, one uses PPPoE (BT) soon to end, and the other uses DHCP (CFL) both routers handle them fine and has room for more speed easy.

Paul

In reply to a post by Al_T:

been looking into this, (about to get an Aquiss 900 fttp line, when Openreach get their fingers out & finish the ONT installation) it *seems* the main problem with PPPoe these days is for those of us running *bsd based routers/firewalls (Opnsense / pfSense & the like), in these the pppoe process is only single threaded so throughput is limited to what a single core can handle (getting gb takes some serious omph). switch to a Linux based devise & throughput rockets.

Right, but scrapping *bsd as an option eliminates a lot of consumer choice (e.g. PFsense etc), and why the hell would I want to use more than an entire core for pointlessly encapsulating packets in PPPoE when I could just.... not...

It means any router I use is going to significantly underperform compared to what it could otherwise do, as >1core is doing pointless [censored] it shouldn't need to do.

In reply to a post by Al_T:

I've always assumed that the PPPoE is the encapsulation that lets so many companies provide service over another companies infrastructure, is there *any* isp that provides service over Openreach infrastructure (not LLU) that *doesn't* use PPPoE?

It's a factor of how Openreach have decided to do things, and as almost all ISPs in the UK have an existing infrastructure for Openreach, they like to just reuse the same kit.

I think it was Openreach's technical director who at one point promised PPPoE would be gone by the time they hit gigabit services, but it seems not.

In reply to a post by nemeth782:

It's a factor of how Openreach have decided to do things, and as almost all ISPs in the UK have an existing infrastructure for Openreach, they like to just reuse the same kit.

Sorry, but that is utter utter nonsense.

Openreach's service is Generic Ethernet Access (GEA): it just passes ethernet frames. Whether those are PPP-over-ethernet or IP-over-ethernet, they don't care.

It's the ISPs who decide what protocol to run over it. It's their choice, based on what they decide to use as a remote access server / concentrator, and what sort of CPE routers they want to deploy, and their support and scalability needs.

If your particular choice of router has poor support for PPPoE: I'm afraid that's just poor implementation in your router. Complain to your router vendor, or contribute a fix upstream. If you must, then complain to your ISP (but they will have good reasons to use PPPoE, and will supply CPE that works well with it, and are very unlikely to take your point of view into consideration).

But in any case, *please* don't complain about Openreach when it's absolutely nothing to do with them at all.

Edited by candlerb (Sun 10-Jul-22 20:43:19)

You need to take the quote=4704680 and the square brackets either side of it out of your last post

Quite. For example TalkTalk “does” IPoE for resi customers and PPPoE for TTBiz customers on their Openreach-based FTTP services.