20G NAT Throughput

Greets, folks.

So a set of requirements. On the WAN side it's 2 x publicly addressed links each symmetrical and give or take 10G. These need to have per-flow load balancing.

On the LAN side it's 2 x 10G LAG via a port channel, with a single LAN-side IP address.

The Mikrotik CCR2116-12G-4S+ will just about manage and I can happily configure load balancing on it. I'd welcome other thoughts from people more familiar with this kinda hardware than me. The Mikrotik is my go-to device though this one won't leave much change from a grand so I'm open to options.

EDIT: Modified subject line due to error. Requirement is 20G of throughput, 40G of total transfer: 20 in, 20 out.

Edited by XGS_Is_On (Sun 05-Jun-22 20:22:24)

Dual 10G load balanced access circuits!! That on separate providers/tails? Blimey that must be over a grand per month before VAT 😎

I suppose if the MT box struggled for throughput you could upgrade to their flagship 2216 which looks like an absolute stonking 25G/100G beast…but utter overkill for a multi 10G access device!? In any event I’m guessing if the client is running dual 10G circuits, an extra grand or two capex for the box won’t bust the friendship

On setup…definitely run using FastTrack. Memory footprint for the state table with large numbers of users?? You’d probably know more than most here in general routing setups. Can’t think of other specific MT tuning tips - still running RoS 6 on very much yesterdays ‘second rate’ Tilera spec boxes. I think there have been some changes / enhancements to how NAT is handled with RoS 7

Maybe a post up on the MikroTik forum?

Netgate will probably be able to provide you with a platform that can run TNSR and achieve those numbers

Personal not professional use so have to be a little mindful of cost.

Unfortunate there are no 4 x SFP28 switches, they come with loads of SFP+ or GbE ports that aren't required.

What about terminating into an Intel NiC with dual 10G SFP+ ports into a decent spec x86-64 box running pfSense or OPNsense? Enough grunt?

In reply to a post by Pheasant:

What about terminating into an Intel NiC with dual 10G SFP+ ports into a decent spec x86-64 box running pfSense or OPNsense? Enough grunt?

It's an option. Need another dual port card for it.

I have a plan involving a 25G spine with 10G leaves. FS are going to love my optics bill - BiDi.

Yup of course, completely forgot the LAN side.

FS lead times seemed to have pushed out on their optics. Not surprising given events. 10 km 25G BiDi are kind-of OK-ish at around £75 a pop. Mind be great if they were £8 like 1G BiDi now...

I was thinking the same but, pfSense boxes made by Netgate don't hit 20Gbps with their top spec box.
It has an 8 core Intel Xeon D-1541 and maxes out just below 19Gbps on L3 Forwarding and Firewall performance.

Thanks Dan

To be fair Intel CPU's have come along a fair bit since the Xeon D-1541 was announced in late 2015. Take for example a modern Core i9-12900K desktop processor vs server processor ...6 years of progress.

https://cputechie.com/comparisons/intel-core-i9-1290...

https://browser.geekbench.com/processor-benchmarks
Multi-Core scores:
Xeon D-1541= 4,564
Core i9-12900K = 17,278

Single Core scores:
Xeon D-1541= 579
Core i9-12900K = 1,990

Edited by Pheasant (Mon 06-Jun-22 22:45:52)

Forgot to mention: balancing like this means per connection classification then PBR, so some CPU work to do running through and processing mangle rules to allocate each flow to a link and give each flow a connection mark, with pre-routing using the earlier marks to produce a routing mark for use in static routes, alongside masquerade and IP filtering.

Lots of work for the CPUs to do outbound, thankfully not so much inbound, and inbound can take the fast track path.

LAG out the front end also consumes CPU.

I have, however, taken care of the super fast desktop PC. I'll move a CCR2004 closer and take advantage of that router's 25G ports to give a clean 25 to the PC.