Netgate SG-1000 pfSense firewall first impressions

I have used pfSense firewalls for several years. A far more powerful and flexible a firewall than that included in consumer kit. Probably much more complicated than the average home user can cope with.

At home I use a PC Engines APU 64 bit mini ITX box, connecting with PPoE to an unlocked HG612 to Plusnet.

At my mothers I was using a PC Engines 32 bit ALIX box, connected to a BT Home Hub 5B. The hub also has a BT YouView box, a Fon access point, and a Vodafone Sure Signal connected to it.

As pfSense 2.4 will not support 32 bit processors I decided to upgrade to the recently released Netgate SG-1000, buying from a UK partner rather than direct from Netgate. https://shop.amicatech.co.uk/shop/hardware/sg-1000-m... - £172.80 inclusive of VAT and shipping. Includes a year subscription to pfSense Gold, notably providing access to the pfSense book and remote configuration backup.

The SG-1000 is a tiny ARM SoC box.
LAN and WAN interfaces - the host, LAN and WAN are connected to a 1 Gb/s switch. iperf shows the LAN capable of 103 Mb/s.
Comes with a small 5v PSU, with the cable coming out of the bottom of the plug.
Has a built in serial console to micro USB bridge. Comes with a 1m micro USB - A USB cable. Drive this with Realterm or similar at 115200 baud, ANSI.

Easy to upgrade:
Backup old firewall's configuration.
Connect SG-1000 to PC via USB and power up. Allow Windows to update its drivers.
Connect to SG-1000 console.
Shut down old firewall.
Via console, option 2, amend LAN details if 192.168.1.1 does not suit.
Plug in LAN and WAN.
Connect to LAN address with a browser.
Restore configuration.
When prompted, fix interface assignments - WAN is cpsw0, LAN is cpsw1.

Performance:

TBB Multi-streamed download and upload much the same as before.

TBB single stream download is DIRE. This might be due to BT congestion. I normally do my speed tests in the dead of night.
New box: http://www.thinkbroadband.com/speedtest/results.html...
Old box: http://www.thinkbroadband.com/speedtest/results.html...

The web interface, particularly the dashboard, consumes a lot of the weeny processor's power, often running at 100%.

So far I am rather underwhelmed (8-(

Edited by awontroba (Thu 12-Jan-17 17:17:53)

Try also using

http://labs.thinkbroadband.com/speedtest
and
https://labs.thinkbroadband.com/speedtest

Single thread but done over HTTP, whereas on the flash version its a custom TCP protocol

Thanks. Tried the labs version, much the same result - http://www.thinkbroadband.com/speedtest/results.html...

Intriguing that merely changing a device inboard of the modem/router appears to provoke the slow single threaded performance that some complain about. When I get the chance I'll try a dead of night test.

It is not just TBB single threaded speed tests that are slow. A FTP download from UKC runs at around the same speed.


[aw1@swelter /tmp]$ ftp ftp.mirrorservice.org
Trying 212.219.56.184:21 ...
Connected to ftp.mirrorservice.org.
220-----------------------------------------------------------------------------
220-Welcome to the University of Kent's UK Mirror Service.
220-
220-More information can be found at our web site: http://www.mirrorservice.org/
220-Please send comments or questions to [email protected].
220-----------------------------------------------------------------------------
220
Name (ftp.mirrorservice.org:aw1): ftp
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd /pub/FreeBSD/releases/ISO-IMAGES/11.0
250 Directory successfully changed.
ftp> get FreeBSD-11.0-RELEASE-amd64-bootonly.iso.xz
local: FreeBSD-11.0-RELEASE-amd64-bootonly.iso.xz remote: FreeBSD-11.0-RELEASE-amd64-bootonly.iso.xz
229 Entering Extended Passive Mode (|||44600|)
150 Opening BINARY mode data connection for FreeBSD-11.0-RELEASE-amd64-bootonly.iso.xz (69712628 bytes).
100% |**************************************************************| 68078 KiB 2.31 MiB/s 00:00 ETA
226 Transfer complete.
69712628 bytes received in 00:28 (2.31 MiB/s)
ftp> quit
221 Goodbye.

It may be that RWIN scaling is broken

Just tried 3 speed tests, 2 with the SG-1000 sandwiching a test with the ALIX.

The single threaded download runs considerably slower on the SG-1000 (around 12 Mbps rather than around 58 Mbps)
Even the multi-threaded download is slower (around 51 Mbps rather than around 58 Mbps).

Not Fit For Purpose (8-(

Thu 12/01/17 22:46	12.74 Mbps	50.50 Mbps	18.55 Mbps	SG-1000
Thu 12/01/17 22:42	57.79 Mbps	57.80 Mbps	18.61 Mbps	ALIX
Thu 12/01/17 22:35	11.19 Mbps	51.55 Mbps	18.38 Mbps	SG-1000

In reply to a post by MrSaffron:

It may be that RWIN scaling is broken

Perhaps, but I cannot find any complaints about this having happened in pfSense 2.4. But there is a lot of software change. FreeBSD 10.3 to 11.0, i386/amd64 to ARM. pfsense 2.3 to 2.4 (still in beta).

My suspicion is that the box just isn't powerful enough.

Caveat emptor!

I'll:
* Take it up with the supplier (and maybe upgrade to the SG-2220).
* Whinge on the pfSense forum.
* Consider other PC-Engines hardware. Though I had trouble with the last one I had an APU model here for a while, which worked fine except for it rebooting ever now and then. Never ran for more than 2 days. An older APU model at home has worked well for a couple of years

In reply to a post by awontroba:

Caveat emptor!

I'll:
* Take it up with the supplier (and maybe upgrade to the SG-2220).
* Whinge on the pfSense forum.
* Consider other PC-Engines hardware. Though I had trouble with the last one I had an APU model here for a while, which worked fine except for it rebooting ever now and then. Never ran for more than 2 days. An older APU model at home has worked well for a couple of years

In reply to a post by awontroba:

I have used pfSense firewalls for several years. A far more powerful and flexible a firewall than that included in consumer kit. Probably much more complicated than the average home user can cope with.

It seems to me that the kit the average home user buys may be a better idea than geek kit . I don't recall any major firewall failures in mainstream products being reported anywhere, particularly when allied to the software ones in mainstream IS systems.

We do keep hearing recommendations to change the default passwords of course, but even the failure to do that doesn't seem to have caused widespread mayhem.

Hi

I run pfSense but on a home built system running a Intel Pentium N3700 and 4 Intel network embedded ports. Still on pfSense 2.3, but get 74/18 on single and multi-threaded speed tests, which is my line maximum.

I've seen the device you have posted about but rather surprised at the cost, given it's not much more than a raspberry pie in hardware terms, that seems very expensive. I had read it should be capable of more throughput, although that might be maximum theoretical speeds or in the lab tests, bit I did read they stated it had 1Gbs Network ports for a reason. Perhaps it is just a case version 2.4 isn't optimised for it yet?

My setup runs at about 10 watt idle, that includes 3 watts on the IPMI (Intelligent Platform Management Interface) chip, but I suspect SG-1000 is much lower which is nice, but that might be at the cost of being under-powered in performance as well.

Hopefully some new software might improve things.

Regards

Phil

Hi

It seems to me that the kit the average home user buys may be a better idea than geek kit wink. I don't recall any major firewall failures in mainstream products being reported anywhere, particularly when allied to the software ones in mainstream IS systems.

Indeed, it isn't for the average home user, in the same way a lot of business kit isn't. But for those who like building their own bits of kit and tinkering plus wanting a reliable separate router that will go the distance, which is much more configurable and better supported than consumer hot plastic boxes, it is a great option.

My pfSense box is way more reliable and configurable than any consumer router I've owned, more future proof and better supported. I built it myself so I know it has quality memory and components, and it's fanless and in a small case like any consumer device, plus gets nowhere near as hot! I had the fun of building it and learning about it, and get a reliable router at the end of it.

Regards

Phil

You can build a kit car as well. That doesn't mean it is worthwhile other than to achieve a sense of personal fulfilment .

Building a PC from readily available components made sense at the time I used to, both for use internally in my mainly tailored business software VAR business and for customers wanting to save a bit. Whether it still does I don't know. That was decades ago.

This sort of faffing about with pfSense and Raspberry PI type stuff sounds like a great hobby. So is building a radio. I have no problem with such activities, but this idea that there is any real advantage in practical terms over mainstream boxes for private and even SME businesses strikes me as rather fanciful.

The "average" user, as in "I have used pfSense firewalls for several years. A far more powerful and flexible a firewall than that included in consumer kit. Probably much more complicated than the average home user can cope with", is more like 99.9% or more of the population than implied by the statement .

How much more powerful does a firewall need to be than the 100% protection offered by a mainstream box and decent software firewall on the computer kit itself? It's a chimera. A bit of fun for hobbyists. That's fine by me, but is all it is.

Building a standard PC is almost certainly not worth it. But, if you are a serious gamer or have other specific requirements then selecting the components can result in a much better device. Prebuilt devices will almost certainly have some components that are not "perfect" for a specific goal. I have built my last few PCs but I haven't replaced my current PC in 4 years (I used to replace every 2 years). So, for some people it is definitely a benefit but for the average user it is far easier just to buy off the shelf.

Or, if you want a middle ground then some PC suppliers (like Novatech) do bespoke build services so you can pretty much select all the items you want and they will build it for you.

I run pfsense on a vm on my home server under vmware and I found that if I reduce the CPU to what I considered a sensible level then that affected speedtests badly. So I leave the CPU uncapped.

I chose pfsense as I have two FTTC connections and pfsense gave me a free way to use load-balancing across the two WAN's rather than pay for expensive hardwired kit. There is also almost-unlimited upgradeability and a very helpful support community.

Hi

I will quote myself, as I did say, which you've just repeated:

But for those who like building their own bits of kit and tinkering plus wanting a reliable separate router that will go the distance, which is much more configurable and better supported than consumer hot plastic boxes, it is a great option.

It is a great hobby, I and hundreds of thousands of other people are using pfSense, in the same way hundreds of thousands of people flash their cheap and cheerful routers and use Tomato firmware, or download and modify open source software and tinker with that. Linux itself, that runs on virtually all consumer routers, in a large part has been built by people tinkering as a hobby. Everyone benefits from people tinkering, it should be encouraged, it's what has given rise to main stream boxes by and large at cheap prices.

No router or firewall offers 100% protection, and the number of security flaws and holes they keep finding in consumer routers proves that.

This post wasn't about trying to get more people to use pfSense, but seems to have turned into, by you, a post knocking us for using it. It isn't for you, fair enough you've made that clear, but don't try and tell us we are fanciful please. I use a lot of features to my advantage in pfSense I don't find in consumer routers and even some industrial ones, it isn't just about a firewall. On top of all the extra features, yes I do get personal fulfillment I built it myself, nothing wrong in that, that's being human, wanting a sense of pride or satisfaction in a job well done, it's not fanciful to want that, it's a human condition.

Regards

Phil

Edited by deleted (Fri 13-Jan-17 10:37:39)

The Subject is "firewall". The content is mainly about problems.

The SG-1000 is indeed not that much of a computer. A SoC similar to a Beaglebone - I use one at home under FreeBSD as a local DNS (BIND) server. The SG-1000 has an on board 1 Gb/s switch serving the host, LAN and WAN. pfSense forum discussions seem to indicate a maximum throughput per port of a third of that, but currently you can't get much more than 100 Mb/s. I got slightly more than that through the LAN with iperf.

Part of the cost must be part of the "free" (normally $99) subscription to pfSense Gold for a year. TANSTAFL. I bought from their UK distributor, so import duty, VAT, startling shipping cost and profit all take their toll. Netgate boxes are all rather expensive, but as pfSense is under Netgate's control you would expect them to work well together.

When idling (like now) with the web interface displaying something static the SG-1000 runs at 98% idle.
With the web interface displaying the dashboard with a few widgets, idle time ranges between 0-60% idle, generally at the lower end of the range.
With the web interface displaying something static, vmstat with 5 second snapshots over a TBB speed test shows that the machine has spare processor capacity throughout the test (see bottom of this).

Possibly, poor interrupt latency having a noticeable but sort of acceptable effect on multi-stream downloads and uploads, and this dire effect on single threaded downloads such as TBB single stream and FTP.

I must of course try different cables and ports on the home hub and LAN switch, but as two ALIX/APU boxes worked entirely well with these this is a clutching at straws move.

I don't know about the Pi, never had my hands on one.

I prefer to avoid as much hardware work as I can (fat finger plus the the classic programmer with a screwdriver syndrome), so usually buy my boxes ready built.

The LinITX PC Engines based ALIX (i386) and APU(amd64) boards are well constructed, and speeds were in line with my expectations. The ALIX and one of the APUs were fine, but the APU I recently got for here proved to be flakey - rebooting itself with no indications as to why after a few hours to a couple of days. Returned for testing, found fault free, sent back to me without a PSU, replacement sent when I nagged. Much elapsed time with this as I am only here some of the time. Eventually LinITX agreed to a refund, and having suffered a lemon decided to try other hardware. Perhaps I should have chosen a mini-ITX box with amd64 multiple cores and intel NICs.

5 second interval vmstat output

# vmstat -w 5
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr mm0 md0   in    sy    cs us sy id
1 2 0 448M   73M   756   0   0   1   768   15   0   0  212  5787   309 18 17 65
<idle>
0 2 0 448M   73M     0   0   0   0     1   12   1   0   71    83   147  0  1 99
0 2 0 448M   73M     0   0   0   0     0   12   0   0   36   121   115  0  1 99
<single stream download>
2 2 0 448M   73M     0   0   0   0     0   12   0   0 1048   294  1699  1  7 92
0 2 0 448M   73M     0   0   0   0     0   12   0   0 2970    81  4823  0 16 84
0 4 0 460M   72M  1032   0   0   2  1124   12   0   0 3688  1579  5495  7 45 48
0 2 0 448M   73M   688   0   0   2   709   12   0   0 3263   793  4993  5 29 66
<multi stream download>
2 3 0 450M   73M   319   0   0   1   354   12   1   0 6109   501  8473  2 52 45
1 2 0 450M   72M   691   0   0   0   774   12   0   0 9257  1123 12590  6 94  0
<upload>
3 2 0 448M   73M   723   0   0   1   747   12   0   0 4502   830  6399  5 42 53
0 2 0 448M   73M   301   0   0   0   324   12   0   0 3425   596  5180  2 30 69
0 2 0 448M   73M     0   0   0   0     0   12   2   0 2498   382  3641  1 14 84
0 2 0 448M   73M     0   0   0   0     1   12   3   0  232    81   305  0  1 98
1 2 0 448M   73M     0   0   0   0     0   12   0   0   27    69    92  0  1 99

In reply to a post by RobertoS:

The Subject is "firewall". The content is mainly about problems.

Generally the only problems I have had with pfSense firewalls, or indeed FreeBSD/ipfw firewalls (my first was an AT&T Death Star desktop I'd picked up at a computer fair in the mid 90s) have been of my own making. Usually misunderstanding and misconfiguration.

I turned to pfSense boxes when I wanted to both repurpose a HP Microserver and continue my replace FreeBSD systems I maintain for myself with FreeBSD based "appliances", which usually require far less time and effort. I seem to be having a run of bad luck recently with my choices.

Computers have fascinated me since I first programmed one at university in 1969, and turned into both my profession and a hobby. Now I am retired, and want to reduce the time spent on that hobby for the benefit of other pastimes, such as drinking beer in muddy fields listening to loud music.

In reply to a post by awontroba:

Computers have fascinated me since I first programmed one at university in 1969, and turned into both my profession and a hobby.

Me too since I took a job as a trainee programmer in 1966 instead of going into the standard management graduate training scheme at the same company.

Now I am retired, and want to reduce the time spent on that hobby for the benefit of other pastimes, such as drinking beer in muddy fields listening to loud music.

Sounds like a plan. However your complementary pastime might within a couple of years involve deep research into warmth clothing and whether Horlicks keeps well in a vacuum flask.

The UK SG-1000 supplier (Amica) happily agreed to a return and upgrade to a SG-2220. This is eye wateringly expensive (but comes with 2 paid support incidents for a year) and seems to be working very well. Best speed I have seen in the last few months.

Thu 19/01/17 16:56	59.08 Mbps	58.93 Mbps	18.59 Mbps

Looks like I can stop fiddling with firewalls (8-)

For Christmas my wife bought me one of these and it handles my 200Mbps FTTH connection with pfSense very well. I get actual speeds of up to 215 Mbps However, ClamAV is way to much for it so i've turned that off.

I'd looked at the SG-10000 but wasn't convinced - happy I stuck with the APU2.

Previously I was running pfSense on a VM but needed to separate it from the rest of my stuff I still run Astaro in a VM on my slow backup ADSL connection.

I had one of those too, but it kept on rebooting. Possibly a lemon. Otherwise it was fine. Good to know that it can keep up with FTTH. I have an older model at home which has no problems and am happy with. I wittered about ALIX/APU earlier in this thread.

ClamAV does need a lot of memory, and can use a lot of power. I run it, SpamAssassin and sendmail on my main FreeBSD box. Out of interest, how were you passing mail through ClamAV on your firewall? Or were you using it to check HTTP traffic? (I am fixated on email).

There are surprisingly few complaints about the SG-1000 single-stream performance on the pfSense forum. Could be due to low sales, a problem only affecting a few machines, people not noticing...

If you have a backup ADSL connection, why not go dual WAN? For failover it works well. Good reasons for not doing so include inertia, incompatibility and not wanting all your eggs in one basket.

Memory wasn't a problem, it was processor power that was the killer for ClamAV - only used it for HTTP traffic.

My backup ADSL connection also has my mail server on it (static IP) and Astaro handles all that - Spam and AV scanning of incoming mail, and it does it extremely well.

I did think about dual WAN, but rather keep it all separately. Fingers crossed, touch wood etc. the FTTH has only gone done once for any length of time (~90 minutes) in the last 12 months since it was installed.