Recommended router for FTTP 330 Mbps?

Hi,

I've had FTTP installed just recently (I just happen to be in one of the areas), and I bought an Asus RT-N66U router, which is widely regarded to be the top Gigabit/Wireless router. However, to my great disappointment, the Asus only achieves roughly 200 Mbps (tested via speedtest.net and by downloading the 1 GB test file from thinkbroadband). Using the BT-provided Home Hub 3 I can achieve around 310 Mbps. I don't want to use the HH3 as it is very restricted in its configuration, and e.g. I have no control over the BT-Wifi sharing of my connection to the general public.

It seems that although the Asus has the capacity to route around 700 Mbps from WAN to LAN (according to Small Net Builder benchmarks), this must be without PPPoE (which seems to kill the CPU).

Has anyone had any success with any reasonably-priced routers over very fast PPPoE connections?

It may be the HH3 can manage over 700 Meg i.e. still suffers from PPPoE issues if that is the cause.

Tried Asus support, they might recommend different firmware.

The N66U has a reasonably vibrant range of firmware options in tomato, and people have been talking about that on here, so worth a look.

Do you have QOS enabled on the Asus?

Are you doing the test wired or wireless?

In reply to a post by kev_445:

Do you have QOS enabled on the Asus?

Are you doing the test wired or wireless?

QoS is disabled. Testing from a Core i7 PC connected via Gigabit Ethernet (through a Gigabit switch).

In reply to a post by MrSaffron:

It may be the HH3 can manage over 700 Meg i.e. still suffers from PPPoE issues if that is the cause.

Tried Asus support, they might recommend different firmware.

The N66U has a reasonably vibrant range of firmware options in tomato, and people have been talking about that on here, so worth a look.

A suggestion I received is that the HH3 is 400 MHz dual core vs 600 MHz single core for the Asus, and that being dual core may make the HH3 more apt at processing PPPoE.

This http://linitx.com/product/13148

or maybe a PC running pfsense.

In reply to a post by rotor2k:

In reply to a post by kev_445:

Do you have QOS enabled on the Asus?

Are you doing the test wired or wireless?

QoS is disabled. Testing from a Core i7 PC connected via Gigabit Ethernet (through a Gigabit switch).

I think you could be right, it sounds like it is a CPU limitation. The best you could hope for is a firmware release which optimises the PPPOE, I would recommend contacting Asus to see if this is the case.

I've come across the issue in the past with another make of router, where it could handle 100Mbit in routed mode, however only around 10Mbit with PPPOE.

In reply to a post by BatBoy:

This http://linitx.com/product/13148

or maybe a PC running pfsense.

I'm waiting for this to become available in two weeks:
http://routerboard.com/RB951G-2HnD

Intuitively I don't think it will be able to handle it. The one you linked has a 400 Mhz processor, and the new one I linked has 600 Mhz, which is the same as the Asus RT-N66U that can only handle ~200 Mbps.

Have you actually got the 750GL pumping 300+ Mbps (with PPPoE)?

The problem with the PC is noise, power consumption and space.

In reply to a post by kev_445:

I think you could be right, it sounds like it is a CPU limitation. The best you could hope for is a firmware release which optimises the PPPOE, I would recommend contacting Asus to see if this is the case.

I've come across the issue in the past with another make of router, where it could handle 100Mbit in routed mode, however only around 10Mbit with PPPOE.

I emailed Asus this morning -- they say it takes 48h to respond, so we'll see what they come back with.

I'm quite shocked at how CPU-intensive PPPoE is -- why do the ISPs use it? The processing power at the head end must be monstrous.

No, I use a PC running pfSense

This one looks more powerful.

Re a PC for PFSense how about this dual lan mini itx board.

those benchmarks I suspect were not done on stock firmware, so I would start by disabling any firewall functions on the router as well as QoS and check nat acceleration is enabled, if that fails try something like tomato shibby firmware and see if that helps.

Gigaclear claim burst speeds up to 1000Mbps and use these;
http://www.genexis.eu/solutions/hybrid-home-gateway-... which include the ONT.

I don't know if they would be compatible with the BT fibre network. They include a SIP VOIP ATA.

I've used many MikroTik Routerboards over the past year or so including the RB450G and now a 2011UAS rack mount (overclocked to 650mhz) and neither of them will route more than about 200 mbit/s or 10,000 pps via pppoe, if you have a few firewall rules this drops even more, add queues and it drops again.

I will probably order a FTTP connection at some point this year, I know I'm going to need something like a dual core at 1ghz+. Or one of the fancy new CCR routers!

In reply to a post by cmn:

I've used many MikroTik Routerboards over the past year or so including the RB450G and now a 2011UAS rack mount (overclocked to 650mhz) and neither of them will route more than about 200 mbit/s or 10,000 pps via pppoe, if you have a few firewall rules this drops even more, add queues and it drops again.

I will probably order a FTTP connection at some point this year, I know I'm going to need something like a dual core at 1ghz+. Or one of the fancy new CCR routers!

Thanks, that's a very useful response.

It's amazing how little information is out there when you deviate from the absolute standard benchmarks of "no firewall, no NAT, just raw throughput", which is fairly meaningless in real life.

I've found a couple of interesting possibilities:

- Ubiquiti EdgeRouter Lite
http://www.ubnt.com/edgemax#edge-router-lite
Has a dual-core 500 MHz processor, and is supposedly capable of 3 Mbps throughput (the usual raw forwarding benchmark). Seems to retail for £120 - £140, although nobody in the UK seems to have it in stock. Aside: for something with an RRP of USD$99, it's pretty annoying to be gouged £140! It's a very new product, and there don't seem to be any reviews out there.

- There's an Intel Atom N2600 which is dual core 1.6 GHz and a max TDP of 3.5W -- i.e. incredibly power efficient while still being probably super powerful for a router. Even though it was released almost a year ago, there are very few options in mini-itx / nano-itx form, and even fewer with dual (or more) Intel GigE ports. I found a company called Axiomtek who makes the perfect little board (Nano831), but it's a new board, and not available anywhere (no idea even on the price).

http://www.axiomtek.com/products/ViewProduct.asp?vie...
http://www.axiomtek.com/products/ViewProduct.asp?vie...

In reply to a post by Michael_Chare:

Gigaclear claim burst speeds up to 1000Mbps and use these;
http://www.genexis.eu/solutions/hybrid-home-gateway-... which include the ONT.

I don't know if they would be compatible with the BT fibre network. They include a SIP VOIP ATA.

Man, there's a lot of marketing speak on that site, and not a lot of technical detail!

In reply to a post by gmoorc:

This one looks more powerful.

Re a PC for PFSense how about this dual lan mini itx board.

Those Jetway motherboards would be PERFECT if they had Intel GigE ports.

How about adding this and then 2 of these.

If your problem with the HH3 is just the FON ( BT WIFI sharing) why not just go to the BT site and turn the FON off?

It seems to be a very good router considering it is free to you and delivers all the speed you have got. 310Mb on a 330Mb product is all you can expect after headers etc .

The desktop versions of the Intel Gigabit Ethernet adapters lack useful optimisations from the server versions that FreeBSD (and therefore pfSense) can take advantage of. However, the more modern Intel Gigabit server designs are PCI-Express x4 (or is it x8 - I forget) cards, which are not supported by the majority of ITX boards.

I believe pfSense 2.0.2 has backported the Intel Gigabit driver from FreeBSD 8.3-RELEASE to the underlying FreeBSD 8.1-RELEASE operating system. If so, this means you can use an Intel I350 card with pfSense 2.0.2.


As I said earlier, I run pfSense on a 1U Dell server box with a dual Intel Gigabit server adapter. I'm intending to get a 32GB SSD to replace the hard disk, which will cut the power consumption significantly. Our switches are VLAN capable, allowing me to have three separate LAN interfaces.

I wouldn't recommend home users to buy rack mount boxes unless they have suitable hosting facilities. We run a home based business, which gave us justification to convert part of our garage to an air-conditioned server room with fibre links to the main house switch. A small format desktop box of some description would be a better choice for most.

Im using a JNC9KDL-2700 which is the previous version of the Jetway JNC9KDL-2550

I'm running Ubuntu server on it. If I max out my 43Mb/s connection with bittorrent, then it's using about 10% of both cores. Thats using the latest PPPoE with an MTU of 1500.

The fan never kicks in. Very pleased with it.

Have you looked this one?
Up to 800Mbps WAN throughput (ideal for FTTx services)
http://www.draytek.com/user/PdInfoDetail.php?Id=111
Don't know about pppoe speeds so best to contact [email protected] (they helping a lot) as i am happy draytek user but never used this model

The Jetway looks to be a pretty decent product - though it's a shame about the Realtek Ethernet controllers, which are not the best. Under FreeBSD (and hence pfSense), Intel controllers are usually regarded as the best, with the higher end Broadcom controllers often being not too far behind. The 32 bit PCI slot of the Jetway will act as a bandwidth limit on an external gigabit Ethernet card.

Soekris net 6501 boards have Intel Gigabit controllers, but they're relatively low end 82574 parts. These boards are also expensive, have a maximum of 2GByte RAM (a possible problem if you want to use RAM heavy packages such as Snort) and only have single core processors.


FreeBSD - and therefore pfSense - doesn't currently support RFC 4638, so is limited to a 1492 byte MTU on PPPoE. I've scoped out what needs doing to add this support, and hope to get round to implementing it and ultimately submitting patches, but I have no timescale for this work as I'm very busy at the moment.

PPPoE uses mpd for PPPoE, which does a limited amount of work in userland but the majority using the kernel's netgraph system. The use of netgraph means I can't recycle any of the work on implementing RFC 4638 in NetBSD and OpenBSD, which implement PPPoE very differently.

I'm 99% certain I will need a modest patch to ng_pppoe.c, as much of the PPPoE negotiation is done in the kernel without calling the userland callback. The necessary kernel patch is no more ambitious in scale than a patch I've previously had accepted for inclusion in FreeBSD. The userland support is pretty straightforward once the kernel support is in place.


If you want to use RFC 4638 for a 1500 byte MTU on BT Openreach FTTx, you need to select a network controller that supports jumbo frames of at least 1508 bytes. Many gigabit controllers do support jumbo frames, though the FreeBSD 8.1 manpage for the re(4) driver isn't drawn on whether jumbo support is available for the 8111EVL controllers on the Jetway board you mention.

If you want the user manual, send me a pm with your email address. I just got the manual today.

In reply to a post by David_W:

If you want to use RFC 4638 for a 1500 byte MTU on BT Openreach FTTx, you need to select a network controller that supports jumbo frames of at least 1508 bytes. Many gigabit controllers do support jumbo frames, though the FreeBSD 8.1 manpage for the re(4) driver isn't drawn on whether jumbo support is available for the 8111EVL controllers on the Jetway board you mention.

I have another jetway board with a RTL8111E and it supports jumbo frames. Although I can't tell if the FreeBSD re(4) driver will support jumbo frames on that particular device (I use NetBSD and Linux), and as you say the documentation is not clear.

In reply to a post by kitcat:

If your problem with the HH3 is just the FON ( BT WIFI sharing) why not just go to the BT site and turn the FON off?

It seems to be a very good router considering it is free to you and delivers all the speed you have got. 310Mb on a 330Mb product is all you can expect after headers etc .

It's not just that. I want to have a PPTP server so I can VPN from my iPhone and laptop when I'm at work. The Wifi on the HH3 is pretty basic, and for the few days I was using it, I noticed regular interruptions (brief, but still). And just in general, I dislike having a BT-controlled device on my home network. I'd rather have complete control of my edge router. It's just a personal preference.

In reply to a post by gmoorc:

This one looks more powerful.

Re a PC for PFSense how about this dual lan mini itx board.

I emailed Mikrotik support asking which was more powerful between the RB450G or the RB951G, and the response was "it depends", that the RB951G has a newer generation CPU, and that they are seeing better performance on the 951G with larger packets.

In reply to a post by gmoorc:

How about adding this and then 2 of these.

I've come across these, which seem ideal, but no idea on price, I haven't been able to find anyone selling them in the UK (a couple of them are "coming soon").

NANO831
http://www.axiomtek.com/products/ViewProduct.asp?vie...

CAPA830
http://www.axiomtek.com/products/ViewProduct.asp?vie...

CAPA831
http://www.axiomtek.com/products/ViewProduct.asp?vie...

In reply to a post by babis3g:

Have you looked this one?
Up to 800Mbps WAN throughput (ideal for FTTx services)
http://www.draytek.com/user/PdInfoDetail.php?Id=111
Don't know about pppoe speeds so best to contact [email protected] (they helping a lot) as i am happy draytek user but never used this model

Yes, it is on my shortlist. I had a Drayek 2820 Vn which served me well for four years, but it maxes out at around 70 Mbps on my new connection. I ended up going for the Asus RT-N66U because it was cheaper than the 2130n, had MUCH better Wifi, and was theoretically faster than the Draytek 2130n. In the raw WAN-LAN benchmarks, the Asus beats the Draytek slightly (this is without PPPoE). So now that I know the Asus is not capable of handling the throughput, I have my doubts about the Draytek too. I might end up ordering that anyway and returning it if it doesn't fit the bill.

In reply to a post by babis3g:

Have you looked this one?
Up to 800Mbps WAN throughput (ideal for FTTx services)
http://www.draytek.com/user/PdInfoDetail.php?Id=111
Don't know about pppoe speeds so best to contact [email protected] (they helping a lot) as i am happy draytek user but never used this model

I've just emailed them, let's see what they respond.

In reply to a post by MrSaffron:

It may be the HH3 can manage over 700 Meg i.e. still suffers from PPPoE issues if that is the cause.

Tried Asus support, they might recommend different firmware.

The N66U has a reasonably vibrant range of firmware options in tomato, and people have been talking about that on here, so worth a look.

Asus support were absolutely useless. They came back with canned responses for improving Wifi coverage (I specified it was a GigE connection), and in the end I just asked them to pass my request on to the firmware developers, which they said they would. I won't hold my breath.

At this stage I'm 90% sure I'm returning the Asus to try something else.

In reply to a post by David_W:

FreeBSD - and therefore pfSense - doesn't currently support RFC 4638, so is limited to a 1492 byte MTU on PPPoE. I've scoped out what needs doing to add this support, and hope to get round to implementing it and ultimately submitting patches, but I have no timescale for this work as I'm very busy at the moment.

PPPoE uses mpd for PPPoE, which does a limited amount of work in userland but the majority using the kernel's netgraph system. The use of netgraph means I can't recycle any of the work on implementing RFC 4638 in NetBSD and OpenBSD, which implement PPPoE very differently.

I'm 99% certain I will need a modest patch to ng_pppoe.c, as much of the PPPoE negotiation is done in the kernel without calling the userland callback. The necessary kernel patch is no more ambitious in scale than a patch I've previously had accepted for inclusion in FreeBSD. The userland support is pretty straightforward once the kernel support is in place.

If you want to use RFC 4638 for a 1500 byte MTU on BT Openreach FTTx, you need to select a network controller that supports jumbo frames of at least 1508 bytes. Many gigabit controllers do support jumbo frames, though the FreeBSD 8.1 manpage for the re(4) driver isn't drawn on whether jumbo support is available for the 8111EVL controllers on the Jetway board you mention.

This is a whole fascinating new area to focus on. As you said, pfSense doesn't support RFC 4638, and neither does RouterOS, which I tested last night on a VM (and performed very well, I got up to 310 Mbps). I can set the MTU of the WAN card to 1508, and the MTU of the PPPoE connection to 1500, but the PPPoE connection gets forced back to 1492, which is apparently expected behaviour as you haven't negotiated with the PPPoE head end.

If pfSense and RouterOS don't support it, the Asus router I currently have doesn't support it, and I couldn't find any evidence of Draytek supporting it, it sounds like it could be a difficult avenue to pursue.

In reply to a post by David_W:

FreeBSD - and therefore pfSense - doesn't currently support RFC 4638, so is limited to a 1492 byte MTU on PPPoE. I've scoped out what needs doing to add this support, and hope to get round to implementing it and ultimately submitting patches, but I have no timescale for this work as I'm very busy at the moment.

From your sig, do you work for Zen? It looks like AA ISP have implemented their own custom fix for 1500 MTU on routers that don't support RFC 4638. Do you know anything about this?

http://wiki.aa.org.uk/index.php/RouterOS#1500_MTU_ov...

Oh, and PS, I tried to sign up with Zen, but they weren't interested (FTTP).

Thanks everyone for this amazing thread. Please keep the thoughts coming!

In reply to a post by rotor2k:

So now that I know the Asus is not capable of handling the throughput, I have my doubts about the Draytek too. I might end up ordering that anyway and returning it if it doesn't fit the bill.

Interesting - have you pinged MrSaffron on this site as he wrote the review that claimed over 600megabit throughput download and 500 megabit upload.

Various options, there's some other Drayteks that have slightly better internals, just look around their site.

Also Cisco do various low end routers e.g. RV180 that could do the job, and they aren't that badly priced unlike most other Cisco gear.

Also Tp-link do some that should be capable as well, but whether they are better than the ASUS you already tried who knows.

really your only guaranteed way of ensuring high speed is if you have an already existing home server and just upgrade it with a good server level NIC and let the server do all the routing work. Good luck with that though as you really have to know what you are doing to get that setup right.

Check out Drayteks comparison sheet...
Draytek Compare
You will see the firewall performance figures.

The 2130 alluded to here can do 500Mb\s.
The 2850 mentioned as well can do 90Mb\s
My 2920 can do 100Mb\s

In reading I have seen these figures are conservative. I guess, as already stated, it depends on how many rules you have inplemented.

Be good to know the impact PPPoE has on those numbers. Many routers have some acceleration for normal IP traffic but it seems handle PPP in software. Wonder if these go that way?

In reply to a post by drummerjohn:

Check out Drayteks comparison sheet...
Draytek Compare
You will see the firewall performance figures.

The 2130 alluded to here can do 500Mb\s.
The 2850 mentioned as well can do 90Mb\s
My 2920 can do 100Mb\s

In reading I have seen these figures are conservative. I guess, as already stated, it depends on how many rules you have inplemented.

Those numbers are always raw throughput. As soon as you tack on PPPoE, the numbers plummet.

With that draytek list most of the products has been updated

the 2130n was originaly up to 500mb but has been updated to 800mb for sure 100% even is at their official web saying this
The 2920 has been updated to 150 mb
2850 is still up to 90mb

It should work and get these speeds with any kind of connection (static or dynamic ip,pppoe etc) but of course let us know what they saying cos i am interesting as well
as i am direct to exchange and if the cost is not much (hoping i am near by 60 meters from the FTTC cabinet) i will think about FTTP on demand

I am not working for draytek but as you... i m very happy with a 2850 and an adsl & VM connection

Like you said if you order with a decent on line shop they will return your money if can not do the speeds but the draytek should let you know before you get it

Edited by deleted (Fri 18-Jan-13 17:23:09)

In reply to a post by rotor2k:

Those numbers are always raw throughput. As soon as you tack on PPPoE, the numbers plummet.

Hmm, good reason to use Sky FTTC rather than BTwholesale FTTC.

In reply to a post by babis3g:

the 2130n was originaly up to 500mb but has been updated to 800mb for sure 100% even is at their official web saying this

I saw the draytek.com.tw site reporting the 2130 as designed for new "ultra-fast" broadband. Shame its not got the feature set of the 28xx range.

yeah the 28xx range stayed behind but then these models are for load balance really or handle 2 connection same time
To me it seems their chippest can not hundle more for what job all ready they doing
Hopefully will be an update up to 120mb like 2920 and 2820 models(got by time higher speed from the original out of factory)
I have the 2850n and a VM connection with 100mb and i am lucky if will hit 90mb very rear times but not compalins it handles 2 connections well (adsl & VM) with many useful features & options

But the 2130 seems more power full

Edited by deleted (Fri 18-Jan-13 17:33:57)

any idea how this could be done on a asuswrt or tomotausb router?

Maybe the experts can advise more about its chippest if is strong enough because with this review they opened its case (2130 but bear in mind is about 2 years old review so may some updates have taken place)

http://www.smallnetbuilder.com/wireless/wireless-rev...

Test speed
http://www.smallnetbuilder.com/wireless/wireless-rev...

I have also emailed the main offices in Taiwan and see what they saying because i am interesting about it

Edited by deleted (Mon 21-Jan-13 17:28:28)

In reply to a post by rotor2k:

In reply to a post by David_W:

FreeBSD - and therefore pfSense - doesn't currently support RFC 4638, so is limited to a 1492 byte MTU on PPPoE. I've scoped out what needs doing to add this support, and hope to get round to implementing it and ultimately submitting patches, but I have no timescale for this work as I'm very busy at the moment.

From your sig, do you work for Zen? It looks like AA ISP have implemented their own custom fix for 1500 MTU on routers that don't support RFC 4638. Do you know anything about this?

http://wiki.aa.org.uk/index.php/RouterOS#1500_MTU_ov...

Oh, and PS, I tried to sign up with Zen, but they weren't interested (FTTP).

I don't work for Zen - the details in my signature relate to my connection at our home office (several family members work from home).


The RouterOS link you give is to a kludge - apparently you can force RouterOS to configure a non-RFC compliant MTU and can tell the AAISP end you're going to do so. The correct and supported way to negotiate a 1500 byte MTU is to use RFC 4638.


I'm gradually putting into place the elements needed to add RFC 4638 to pfSense. At the moment I'm slowly setting up the necessary development environment and debugging setup - I got part of this setup working last night. It doesn't help that I'm having to work around which of my machines have jumbo capable network interfaces, though I have now got a jumbo capable interface (an Intel ET dual port server NIC) in my pfSense box.

It's likely to be months before I have it something ready to share with my current workload. If I get it working, I'll submit the kernel patch to FreeBSD, the mpd patch to mpd, and the overall patches to pfSense.


I'm not sure about Zen's policy on FTTP. At the moment, FTTP is a niche product, though it will become more mainstream as BT's commercial deployment speeds up and as FTTP on demand becomes available.

Edited by deleted (Mon 21-Jan-13 14:14:57)

Just a quick update. I'm returning the RT-N66U -- it definitely isn't capable of handling 300 Mbps of PPPoE.

For future generations, here is the summary: I have a 300+ Mbps Internet connection, supplied over fibre (Fibre to the Premises), and requiring PPPoE. I purchased an Asus RT-N66U, thinking it would have ample power to support 300 Mbps, but unfortunately it appears that PPPoE is inefficiently implemented in the Linux kernel, and as a result CPU usage skyrockets to 100%, giving a maximum download speed of around 200 Mbps.

I looked at many options, engaged in discussions on several forums and IRC channels, and these are my findings:

- Asus RT-N66U running stock firmware (I tested two versions: 260, and beta 321). Maximum download speed of around 200 Mbps.

- The RT-N66U running Shibby. It was slower (around 180 Mbps).

- My existing Draytek 2820Vn (which served me well for 4+ years on ADSL), which maxes out at around 70 Mbps. It was never intended for these sorts of speeds. I am now using it as a wireless Access Point.

- I wouldn't mind another Draytek, but the 2130n (the fastest in its price bracket) is almost certainly not capable of achieving these speeds. Draytek Taiwan is reasonably sure it would max out at around 200 Mbps with PPPoE (from a an email someone on the Draytek forum sent to Draytek).

- A vendor (thanks Nick at LinITX!) did a test for me of a RouterBoard RB2011 which has a 600 MHz processor. Even overclocked to 750 MHz (this is a feature of the Mikrotik RouterOS) it was still only capable of 280 Mbps, and that was without any firewall rules (i.e. not apt for real-world use).

- A 4+ year-old Atom motherboard I have lying around (Intel D945GCLF2 with Atom 330, a dual-core 1.6GHz) running RouterOS. It breezed through the test: 310 Mbps at 20% CPU, including NAT and firewall rules enabled.

- Currently as my router I'm running RouterOS on a VM that runs on a Xeon E3-1220 (a quad-core 3.1 GHz) -- and of course it absolutely flies. CPU averages at 0.32%, and hitting the full 300 Mbps sends the VM CPU to a whopping 7.13% (and the VM only has 2 virtual CPUs out of the 4 physical cores available, so this is roughly equivalent to 3.5% of the processor's capability). For those of you about to point how insecure this is, I am passing through a NIC to the VM (using VT-d) for the WAN connection.

This really highlights to me how different processors handle different workloads. The MIPS-type processors used in routers may be good at some things, but are terrible at others; or maybe it is just that the Linux implementations in use aren't optimised. Hopefully as the broadband market continues to produce faster products, manufacturers will produce more powerful routers.

Another fascinating result is that the Atom motherboard idles at 24W, the Xeon idles at 30W, and the RT-N66U idles at 21W. So there is clearly no correlation between the amount of power consumed (at idle), and the raw processing power available in a CPU (I know that the wireless cards in the Asus require a few Watts to power as well). These are all measured using my UK equivalent of a kill-a-watt.

My current plan is to purchase an Intel DN2800MT motherboard (with an additional NIC), which idles at less than 10W.

I'd like to thank everyone for their input, and hopefully these test results will be useful for someone in the future.

Very informative post. Thanks.

Very useful post, glad you found a good solution in the end, I just want to point out though that Routeros is a Linux implementation, packet routing and inspection is very cpu intensive so the key is a fast CPU with multiple cores.

A CCR1036-12G-4S arrived yesterday so I'll be testing that out, 36 cores @ 1.2ghz should be more than enough I imagine!

Hmmm that's a nice piece of kit. Expensive though!

There's a sort of vision of what's needed for a high end router:

A multi-core x86-64-based system, fanless, very low TDP, with the CPU probably integrated on the mainboard, and with onboard gigabit ethernet switch controller and ports, and where data bus architecture design is critical.

The conventional PCI bus is obsolete now. A standard 32bit PCI bus has a bandwidth of just 133Mb/s maximum. And where there are multiple masters on that bus - e.g. where a riser board is used to connect multiple gigabit NICs - then each PCI master draws from that maximum 133Mb/s bandwidth. Four gigabit NICs sharing a single 133Mb/s PCI bus isn't going to work.

In theory, the newer x16 PCI-Express bus has a bandwidth of 4GB/s in each direction. However, the low-end mini-ITX boards seem to be limited to x1 or x4 PCI-Express slots. And they max out at 250Mb/s per lane, each way.

If the same PCI-e x1 bus is used for arbitrating multiple PCI-e devices, then the bus traffic is soon going to hit the buffers in the same way as it did on the older PCI-based boards. It's a bit disingenuous even selling a gigabit NIC for use on a 250Mb/s PCI bus!

To build a router board based on the x86-64 to stand the test of time, would probably cost several hundred pounds. And there's not much choice in terms of fanless CPUs with reasonable power to them.

The big question is how deep does the packet inspection need to be? At what layer of the network protocol stack is the filtering going to take place? The deeper that inspection - (i.e. the higher up the stack) - the larger the data queues need to be, the faster the buffer memory must be, and the more processing power that is needed by the router board. That's why BTOpenreach, showing surprising wisdom, kept the HG612 as a fairly primitive device, with just layer 2 filtering (VLAN tags) as standard.

A ballpark figure for such a DIY router board might be £400. My feeble DIY router effort that cost about £150, using a 1200MHz dual-core Athlon64 S1g1 board, turned out to be useless. Because of the limitations of the PCI-e bus on the low-cost Fujitsu-Siemens mini-ITX board that was chosen.

It's a harder exercise than it looks!

cheers, a

Edited by deleted (Sat 26-Jan-13 22:32:33)

You raise very valid points, but I just did some quick research (and thanks for bringing it up, it's very interesting), and it looks like modern chipsets have more or less solved that problem. The motherboard I've shortlisted (the Intel DN2800MT) has the NM10 chipset, which provides 4 PCI Express (PCIe) 2.0 x1 interfaces. Each PCIe 2.0 lane provides 500 MBps (megabytes per second) full-duplex, or 4 Gbps (gigabits per second) full duplex, i.e. 4 Gbps sending and 4 Gbps receiving simultaneously. So in total the NM10 chipset provides 16 Gbps full-duplex (16 send and 16 receive simultaneously). That's a LOT of data to push around. Of the 4 interfaces, one is used by the on-board Gigabit Ethernet controller, two are used by the two mini PCIe slots, and the fourth is provided as a PCIe 2.0 x1 slot (which in my case would have a Gigabit Ethernet card in it). In my opinion Intel has made the perfect choices, for this application anyway.

In addition, two SATA 3 Gbps ports are provided, which hang off the chipset independently (i.e. they don't consume PCIe interfaces), plus a bunch of USB ports, COM and LPT ports, audio, etc.

Even though PCIe 1.1 only provides half the bandwidth that 2.0 does, it's still 2 Gbps each way per lane, which is double what is required for a Gigabit Ethernet interface; so having 4 of them on a mini ITX motherboard would still be at least double what was needed for a very powerful router.

Check out page 18 of this: http://downloadmirror.intel.com/20714/eng/DN2800MT_T...

I worked out the cost (for the smallnetbuilder forum) to be $300 USD including two Intel 6300 3x3 MIMO 802.11N cards, RAM, case, power supply, everything. So maybe £200 - £250 tops. Still a lot more than the £108 the Asus RT-N66U currently goes for, but you're getting something with at least 5x the amount of raw processing power.

Here's the full post:
http://forums.smallnetbuilder.com/showpost.php?p=588...

I completely agree that you have to be very careful with a lot of your choices, and if I do end up building one, I will make every effort to make sure it is as future-proof as possible.

In reply to a post by rotor2k:

...Using the BT-provided Home Hub 3 I can achieve around 310 Mbps...

Your connection is 75x times faster than my maximum over ADSL (No FTTP where I live). *Only* getting 200Mbps from your preferred router is the kind of problem I'd like to have

Sounds like you've made some great choices. Be sure to keep us all informed on progress!

What operating system will the router be running? Some flavour of Linux?

cheers, a

In reply to a post by asbokid:

What operating system will the router be running? Some flavour of Linux?

I really like RouterOS (by a company called Mikrotik). You can get a free version, or there's a very reasonably-priced full-featured commercial license (you can get it for USD $30). I have one running as a VM, and it uses 16 MB of RAM, which is pretty remarkable.

If you have a moment, could you make a spec from www.scan.co.uk and post the link for the items.

In reply to a post by rotor2k:

I don't want to use the HH3 as it is very restricted in its configuration, and e.g. I have no control over the BT-Wifi sharing of my connection to the general public.

What is your objection to sharing?

If you are getting 300Mb/sec it isn't likely to cause much (any?) slowdown.

The data transferred over the Fon bit doesn't come out of your allowance and is completely partitioned away from your network.

Or are you just a misanthrope who dislikes helping others?

In reply to a post by asbokid:

There's a sort of vision of what's needed for a high end router:

A multi-core x86-64-based system, fanless, very low TDP, with the CPU probably integrated on the mainboard, and with onboard gigabit ethernet switch controller and ports, and where data bus architecture design is critical.

The conventional PCI bus is obsolete now. A standard 32bit PCI bus has a bandwidth of just 133Mb/s maximum. And where there are multiple masters on that bus - e.g. where a riser board is used to connect multiple gigabit NICs - then each PCI master draws from that maximum 133Mb/s bandwidth. Four gigabit NICs sharing a single 133Mb/s PCI bus isn't going to work.

In theory, the newer x16 PCI-Express bus has a bandwidth of 4GB/s in each direction. However, the low-end mini-ITX boards seem to be limited to x1 or x4 PCI-Express slots. And they max out at 250Mb/s per lane, each way.

If the same PCI-e x1 bus is used for arbitrating multiple PCI-e devices, then the bus traffic is soon going to hit the buffers in the same way as it did on the older PCI-based boards. It's a bit disingenuous even selling a gigabit NIC for use on a 250Mb/s PCI bus!

To build a router board based on the x86-64 to stand the test of time, would probably cost several hundred pounds. And there's not much choice in terms of fanless CPUs with reasonable power to them.

The big question is how deep does the packet inspection need to be? At what layer of the network protocol stack is the filtering going to take place? The deeper that inspection - (i.e. the higher up the stack) - the larger the data queues need to be, the faster the buffer memory must be, and the more processing power that is needed by the router board. That's why BTOpenreach, showing surprising wisdom, kept the HG612 as a fairly primitive device, with just layer 2 filtering (VLAN tags) as standard.

A ballpark figure for such a DIY router board might be £400. My feeble DIY router effort that cost about £150, using a 1200MHz dual-core Athlon64 S1g1 board, turned out to be useless. Because of the limitations of the PCI-e bus on the low-cost Fujitsu-Siemens mini-ITX board that was chosen.

It's a harder exercise than it looks!

cheers, a

PCI bus is 133 MegaBYTES per second not megabits\s = 1066mbps - plenty fast enough.

Edited by deleted (Sun 27-Jan-13 15:06:02)

In reply to a post by hoopla:

Or are you just a misanthrope who dislikes helping others?

Troll.

Edited by deleted (Sun 27-Jan-13 15:05:44)

In reply to a post by Yaz:

If you have a moment, could you make a spec from www.scan.co.uk and post the link for the items.

Here's the list:

DN2800MT Motherboard £80
http://skinflint.co.uk/746927

1 GB RAM DDR3 SO-DIMM £9
http://skinflint.co.uk/?cat=ramsoddr3&xf=1454_1024#x...

Case (M350) £37
http://linitx.com/product/12488

Riser card + IO shield (to allow a 2nd NIC) £12
http://linitx.com/product/13552

AC Power Adapter £19
http://www.logicsupply.co.uk/power-supplies/ac-adapt...

Additional Intel Gigabit NIC £21
http://skinflint.co.uk/351749

Intel 6300 Ultimate-N 450 Mbps £20
http://skinflint.co.uk/616661

mSATA 32GB £36 (this is a luxury, I would say just boot from a USB stick)
http://skinflint.co.uk/?cat=hdssd&sort=p&xf=2646_mSA...


Total with one Ultimate-N card (and excluding the mSATA): £198
Add £20 for a second card to provide simultaneous dual-band support.

Please note these are all indicative, I haven't researched every last detail in terms of compatibility, but this is the rough bill of materials needed.

In reply to a post by rotor2k:

In reply to a post by hoopla:

Or are you just a misanthrope who dislikes helping others?

Troll.

No, I'm a big fan of Fon - it's a really good idea.

What are your reasons for your unwillingness to share?

To call anyone who questions your approach a "troll" says more about you than I could.

In reply to a post by drummerjohn:

PCI bus is 133 MegaBYTES per second not megabits\s = 1066mbps - plenty fast enough.

Sorry, you're right. My bad. Megabytes not megabits.

Divide by three or four though, if the NICs share the same bus, whether directly, or on a PCI riser, and the device will hit the same limits as the HH3. And according to the wikipedia entry on the PCI bus standard, the data throughput is less than the raw speed, because of all the control transactions.

Same with USB-based ethernet NICs. They should be good for 100Mbps since the USB2.0 standard states a max throughput of 480Mb/s yet that's never achieved. These one dollar NICs max out at 40Mbps.

cheers, a

In reply to a post by rotor2k:

In reply to a post by Yaz:

If you have a moment, could you make a spec from www.scan.co.uk and post the link for the items.

Here's the list:

DN2800MT Motherboard £80
http://skinflint.co.uk/746927

1 GB RAM DDR3 SO-DIMM £9
http://skinflint.co.uk/?cat=ramsoddr3&xf=1454_1024#x...

Case (M350) £37
http://linitx.com/product/12488

Riser card + IO shield (to allow a 2nd NIC) £12
http://linitx.com/product/13552

AC Power Adapter £19
http://www.logicsupply.co.uk/power-supplies/ac-adapt...

Additional Intel Gigabit NIC £21
http://skinflint.co.uk/351749

Intel 6300 Ultimate-N 450 Mbps £20
http://skinflint.co.uk/616661

mSATA 32GB £36 (this is a luxury, I would say just boot from a USB stick)
http://skinflint.co.uk/?cat=hdssd&sort=p&xf=2646_mSA...


Total with one Ultimate-N card (and excluding the mSATA): £198
Add £20 for a second card to provide simultaneous dual-band support.

Please note these are all indicative, I haven't researched every last detail in terms of compatibility, but this is the rough bill of materials needed.

Very similar to my build which I did as I was fed up with reliability problems trying to run even 40Mbit FTTC on my Buffalo router. It was fine until my ISP switched to PPPoE and then things got hairy, made QoS completely impractical. So I built this with the intention of not having to upgrade it again even if I switched to FTTP later.

https://plus.google.com/photos/108348704364791506166...

I run a version of OpenWRT compiled for Atom though:
https://forum.openwrt.org/viewtopic.php?id=35887

Interestingly it had to have power management disabled in the kernel as there is a bug with the on-board Intel Gigabit Ethernet chipset (at least with the newer Linux driver) that causes packet lag/loss, which oddly the PCIe card of identical chipset doesn't suffer from. There are many mentions of this on Google even on lower server class boards with the same chipset.

Fortunately, power management seems to make no difference at all to the power consumption of this board in Linux. Although I never managed to get it below around 16W with any OS regardless of power management setting.

Its also worth noting that you really want PM disabled on a router anyway as the act of PM switching things on, off, frequency scaling, can add latency to the routing performance. Although I doubt its particularly noticeable its always best to aim for optimal performance IMO.

Edited by alexatkin (Mon 28-Jan-13 04:00:45)

In reply to a post by alexatkin:

Very similar to my build which I did as I was fed up with reliability problems trying to run even 40Mbit FTTC on my Buffalo router. It was fine until my ISP switched to PPPoE and then things got hairy, made QoS completely impractical. So I built this with the intention of not having to upgrade it again even if I switched to FTTP later.

Very cool. I love how clean it is (no cables). I gather you didn't run into any major issues?

Edited by deleted (Mon 28-Jan-13 16:07:28)

Impressive that broadband is getting to the point where people are struggling to find hardware capable of handling the throughput

have you found an alternative? I'm returning the n66u tomorrow ;/ still need something to replace the hh3

How come you are returning the N66U?

I've detailed what I will probably be getting (an Atom motherboard) in an earlier post.

wouldn't go past 200mb/s. With hh3 I sometimes even get 300. Don't want to be limited by the hardware

Ah ok, you also have FTTP. The Atom route is guaranteed, I tested a four year old Atom 330 running RouterOS, and it did 300 Mbps easily at 20% CPU.

Hi I am in the same position - have 300 enabled and looking for HH3 alternative...
Did you consider ASUS RT-N65U?
It says it is dual processor....
TY

In reply to a post by OlegF:

Hi I am in the same position - have 300 enabled and looking for HH3 alternative...
Did you consider ASUS RT-N65U?
It says it is dual processor....
TY

That's what was said in the first post

In reply to a post by rotor2k:

I've had FTTP installed just recently (I just happen to be in one of the areas), and I bought an Asus RT-N66U router, which is widely regarded to be the top Gigabit/Wireless router. However, to my great disappointment, the Asus only achieves roughly 200 Mbps

In reply to a post by BatBoy:

In reply to a post by OlegF:

Hi I am in the same position - have 300 enabled and looking for HH3 alternative...
Did you consider ASUS RT-N65U?
It says it is dual processor....
TY

That's what was said in the first post

In reply to a post by rotor2k:

I've had FTTP installed just recently (I just happen to be in one of the areas), and I bought an Asus RT-N66U router, which is widely regarded to be the top Gigabit/Wireless router. However, to my great disappointment, the Asus only achieves roughly 200 Mbps

Ermm... read carefully the bits I've emboldened and spot the difference...

Oh, that's very entertaining, must get those glasses fixed etc. You don't happen to know the difference between the two in terms of wan-lan throughput do you?

In reply to a post by BatBoy:

You don't happen to know the difference between the two in terms of wan-lan throughput do you?

Not the faintest idea.

As my RT-N66U is quite capable of maxing out my FTTC connection without breaking sweat, this thread is of interest to me but of no particular relevance

Looking at various benchmarks, it appears the RT-N65U is an upgrade to the RT-N56U and so may well be faster than the RT-N66U, although the latter has more features and it more expensive.

I find the product naming quite confusing.

In reply to a post by BatBoy:

I find the product naming quite confusing.

So do I at times, though Asus aren't alone in that.

But the kit is nice.

Based on what I see at ASUS - N65U is dual processor and from OP message N66U is single processor.
Also OP states that CPU is the bottleneck achieving 300 on FTTP... -
HH3 is dual processor and makes 300, N66 can't do more than 200, so I am wondering if N65U will do the job?... being cheaper than N66U as well and still providing decent WiFi

Yeah, I'm wondering that as well.

Wow thanks for the info, not a biggie having a small machine do this, but I would have preferred a dedicated device

Tech specs

N65 - http://www.wikidevi.com/wiki/ASUS_RT-N65U

N66 - http://www.wikidevi.com/wiki/ASUS_RT-N66U

not convincing, the 2nd cpu is much weaker than the 1st and the cpu on the 66 is more powerful than the cpu on the 56.

I am still thinking getting the N65U from amazon and try stock firmware / moded from Padavan.
If no luck will send it back....

Next question I guess is - is there any other routers with more than 1CPU/cores?
Fastest CPU?

I have an Asus RT56U and I am also getting FTTH/P in the next 2 weeks. I also have been running Various Firewalls in a VM using a HP Microserver N36L. This has been ideal for this purpose and costs with rebate around £120. Its been very useful I also recorded all my TV through it using DVB Viewer.

I am not sure if it will suffice for your requirements, but I have a Intel CT 1Gb card and with the Broadcom onboard I can saturate the LAN with my Adaptec Raided drives. It's more powerful than an Intel draws 45 watts with all drives in place. It presently has Untangle on providing my internal network .
Just a thought you might want to consider. I haven't though as yet tried PPPoe connection. I am currently using a Draytek Vigor 100 in between before the Asus on my present connection, and then to the Untangle box This has been an interesting read and perhaps now that you have also posted your power usuage for the N66. Perhaps I will try the Microserver when i get the new connection, and just use the Asus as a access point.

Maybe you should look into this hardware.
@ post above I also have padavans firmware the Asus was rubbish on the standard firmware before. Port 21 was open all time etc. I don't know if this also would have been worth trying on the router you are returning.

A question though regarding the points you made Rotor.

I know this is a different router the 56, but from the same family of routers. In this firmware of padavans he has enabled

In reference to rt-n56u.googlecode.com/git/changes.eng.txt:

Activated complete HW_NAT offload on both directions for WAN PPPoE (~900 Mbit/s)

Padavans firmware.

You also stated how did smallnetbuilders test their routers. I have found an interesting test/review from Bjorn3d.com, but no real mention of PPPoE loading though.

Bjorn3D Review

My initial connection will be FFTP/H but I will be only taking 76 service at first so no real problem I have the hardware here to move to higher if the Asus isn't upto spec, but when I subscribe to 160 and above then perhaps I will test your assumptions.

Further research has lead me to believe that the router doesn't suffer the CPU problem as some are getting over 430mpbs via PPPoE with this router and the firmware. Also if you see this post they show a speedtest and padavan replies to inform the actual throughput available with PPPoE enabled.

To Gon�alo

N56U performance on IPoE and PPPoE connection:
WAN -> LAN 930Mbps
WAN <- LAN 890Mbps

Tested by iperf

Edited by deleted (Wed 20-Feb-13 07:47:28)

Got RT-N65U yesterday and have it running.
Tried with stock firmware first and got the same connection speed as HH3 (280 down)
Changed to padavan's firmware and getting the same 280. This what I get with direct PPPoE via laptop.
Padavan's also shows router CPU utilization and the utilization is very low - under 5% even when running speed tests.
I have nothing connected to USB, but overall pretty happy with both wifi and wired

Well i been running RouterOS 6.11 and for a intel p4 hyperthreading @ 3ghz, 1gb ram and 20gb sata 3gb/s connected to an cisco sr224 24 port switch, The CPU uses 4% at 55Mb However making 2 bridges incrased it by 19% CPU.

Edited by francisuk25 (Mon 07-Apr-14 20:19:47)

In reply to a post by rotor2k:

I bought an Asus RT-N66U router, which is widely regarded to be the top Gigabit/Wireless router. However, to my great disappointment, the Asus only achieves roughly 200 Mbps (tested via speedtest.net and by downloading the 1 GB test file from thinkbroadband). Using the BT-provided Home Hub 3 I can achieve around 310 Mbps. I don't want to use the HH3 as it is very restricted in its configuration, and e.g. I have no control over the BT-Wifi sharing of my connection to the general public.

The RT-N66U has already been succeeded by the RT-N68U, and in a few months that will be succeeded by the RT-AC87U based on Quantenna�s QSR1000 4x4 MIMO chipset, which will deliver throughput up to 1.7Gbps.

Why not wait the short while until that is released and use the Home Hub 3 to tide you over until then, even if it's not ideal?

In reply to a post by dabotsonline:

The RT-N66U has already been succeeded by the RT-N68U, and in a few months that will be succeeded by the RT-AC87U based on Quantenna�s QSR1000 4x4 MIMO chipset, which will deliver throughput up to 1.7Gbps.

Why not wait the short while until that is released and use the Home Hub 3 to tide you over until then, even if it's not ideal?

Mainly 'cause I posted in January 2013. =)

Ironically the Router upgrade will be the cheapest component upgrade in this process.
Has anyone compared recent portable wifi devices and their compatibility with the various incarnations of 2.4Ghz and 5Ghz wireless technologies?
Taking Apple products from the iPhone 4 / iPad 2 upwards as an example.
In reality, finding a portable wireless device which can support 50Mbps+ over wifi is harder than you'd think.

Edited by deleted (Wed 16-Apr-14 21:10:25)

In reply to a post by flipdee:

Taking Apple products from the iPhone 4 / iPad 2 upwards as an example.

iPad 1 and later support 5GHz band, iPhone 5 and later support 5GHz band.

iPad Air and iPad Mini with Retina Display both from late 2013 have MIMO antenna support, and I've seen an iPad Air manage a speed test of 70Mbps that an iPad 3 and 4 couldn't get to in the same location. (BT Infinity 2 user) using an Apple Airport Extreme 2012 as the router and 5GHz network.

However for most people "it varies" is the real answer due to the variability of WiFi technology anyway.

jchamier, well said.
I just find it funny that quite a number of friends have got 70Mbps FTTC (with some very suspect wireless routers - plusnet i'm looking at you) and when they run speedtest.net and don't get 70Mbps in an ipad 2 ( 25Mbps if your lucky) they are rather miffed. Obviously it's useful to have 70Mbps to share amongst several wired and wireless devices but it doesn't help when the ISP's don't even provide wireless routers capable of 78Mbps on the wireless side.

In reply to a post by flipdee:

jchamier, well said.
I just find it funny that quite a number of friends have got 70Mbps FTTC (with some very suspect wireless routers - plusnet i'm looking at you) and when they run speedtest.net and don't get 70Mbps in an ipad 2 ( 25Mbps if your lucky) they are rather miffed. Obviously it's useful to have 70Mbps to share amongst several wired and wireless devices but it doesn't help when the ISP's don't even provide wireless routers capable of 78Mbps on the wireless side.

iPad 2 probably can't keep up, it's an old design now. Using a laptop with at least a core 2 and an up to date N wifi adapter is really needed for any fibre over 50Mbps. No idea if plus net router can do the speed without ruling out the iPad as being the bottleneck.

My IOS devices get nowhere close to the max I get from a wireless PC or laptop.

On the speed test we see iPhone 5 managing 120 Mbps on Virgin Media
http://labs.thinkbroadband.com/ispa

Once the ISPA testing is out of the way we will do some comparisons between the various hardware devices

RouterOS running at 95Mbps so it gives you a idea, I cant test 300Mbps as my network cards is only 100Mbps max

http://i.cubeupload.com/NhnyDT.jpg
Running on a old dell gx520, P4 HT 3ghz, 512mb ram with is broadcom onbroad and an old 3com tx card.

Edited by francisuk25 (Thu 17-Apr-14 13:31:46)

I'd like to thank rotor2k, alexatkin and others, for all the information they have dug up in this thread. None of the routers available at a reasonable price offered the kind of performance to be of any use after the next ISP speed upgrade.

I've just finished putting together the router from rotor2k's shopping list (minus the wireless parts) and I am very impressed with the performance. MikroTik's RouterOS was a very steep learning curve but after a day of fettling I've got the basics mastered.

I am soon to have 330Mbps FTTP installed and I want a reliable router which is not a speed bottleneck. I must admit, I am not entirely clear from this thread - is the Asus RT-AC68U capable of 330Mbps, are there better off-the-shelf alternatives or do I need to go down the self build route (which worries me in terms of reliability)?

The Cisco RV325 was suggested to me as an option which may be fast enough and be reliable but I don't know if anyone has tried it in a home context (http://www.cisco.com/c/en/us/products/collateral/routers/rv325-dual-gigabit-wan-vpn-router/datasheet-c78-729726.html)

Thank you

In reply to a post by smouty:

My IOS devices get nowhere close to the max I get from a wireless PC or laptop.

iPhone 5 with the A6 chip, or iPhone 5s or iPad Air / iPad Mini Retina which all have the A7 chip should all be able to handle 100Mbps on N WiFi at 5GHz.

The iPad Air and the iPad Mini Retina have two antennas (for diversity), and usefully often show higher speeds than iPhone 5 / 5s, one of the downsides of being hand held is that hands often obstruct an antenna.

Cheers baringtech, I'm really glad you found it helpful. I can't believe this thread is still going, 18 months after I started it!

I wouldn't mind seeing some pics of your powerful new router!