Speed testing gigabit ethernet network

I've recently used iPerf3 to test my cat5e ethernet network speed using a Windows 10 desktop PC and a docked Windows 10 MS Surface Pro 4, so both machines are connected over ethernet. The overall reported speed in either direction between both PCs is around 650Mbps which seems a bit low to me so I repeated the test with Task Manager visible on both machines, and noticed that the Surface Pro CPU usage spikes quite a bit at 100% during the test, whether using it as the iPerf client or server.

In terms of speed, when copying files between both machines I'm getting a reported speed of 50MB/s to 70MB/s (the same when copying to my NAS), but do these speeds seem reasonable for a cat5e ethernet network running through a Cisco managed switch?
I'm wondering if the CPU usage spikes on the Surface Pro are affecting the results?

I get 950 megabits per second from my Win10 PC to my Synology NAS through a managed TP Link switch.

If you have third party antivirus check that is not affecting.

In reply to a post by jchamier:

I get 950 megabits per second from my Win10 PC to my Synology NAS through a managed TP Link switch.

If you have third party antivirus check that is not affecting.

Good idea. I'll also connect both PCs directly to the switch via patch cable so that the test isn't running through the cat5e cabling throughout the house.

Assumimg your NAS is up to it then sustained 110MBps is what you ahould be getting using SMB on the network. At least thats what my Samba servers at work where doing a decade ago on NetBurst based Xeons with 6GB of RAM with 4Gbps FC attached disks with ext3. It was quite extensively tuned however.
These days it is all 10Gbps or better networking at work which takes also takes tuning to max out. That said at home I can get similar performance to what I was getting a decade ago with an Atom CPU and minimal tuning, though its a home built NAS running CentOS which given the day job is easier than some web based NAS system to administer and tune. Will be interesting to try multipath TCP SMB in due course.

I get around 950 megabits per sec. with iPerf3 testing my cat5e between an HP windows 10 desktop and another desktop with ubuntu-mate 18.04

This has 15 metres of cat5e between one machine and the switch and 20 metres of cat5e from that same switch to the other machine.

P.S. This is Just as guide or comparison maybe....

I got the same result previously about 2 years ago using my Old Vodafone router as the switch so to speak also...

P.P.S. I'm sure it shouldn't use anymore than minimal CPU... It should mostly be done by the networks devices...

Edited by mrmarktigger (Tue 28-Jan-20 21:19:59)

Its ethernet over cat5e or better cable and under 100m. The length of the cable is utterly irrelevant to the discussion at hand. That is it negociates a speed of 1Gbps or it doesn't and it just works at that speed.

Now you could have a suspect link, but just check for errors in the interface counters. These should be zero or very close to zero. If they are not then you most likely have a bad cable send it for recycling. Though it could be a bad switch port or even a duff switch, but the latter two options is very rare.

The most likely reason for not getting the full speed is a cheap ass network card. Realtek chipsets are not great (thiugh better than they used to be), others are even worse. Best is Intel then Broadcom. Basically while it might be able to negociate a 1Gbps link with the otherside many chipsets can't actually push 1Gbps of traffic or the chipset itself and/or the drivers are as buggy as hell.

This can be really apparent at 10Gbps, you can get a server to just pump out UDP traffic and then look at what the port throughput on the switch is (expensive enterprise grade ones) and well it might max out at 7Gbps or even less. One forgot about it at the 1Gbps level because it has not been an issue with the server grade kit I am used to working with for over a decade.

You�ve reliably missed the end to end part of the equation. The sending machine has to be able to send at 900 Mbps+ which it may not be able if the HDD is encrypted and using CPU to decrypt thus breaking any DMA to the NIC.

I have 950 Mbps from my Realtek NIC. I have more problems with Intel drivers for WiFi than any other vendor. They are awful. The Broadcom NIC�s on enterprise servers are great.

I am assuming that iPerf3 doesn't use the HDD at all as I am guessing it just sends a stream of data without reading or writing it from the hard drive. It wouldn't be a good test of network performance if it relied on the speed of a hard drive.

In reply to a post by ian72:

I am assuming that iPerf3 doesn't use the HDD at all as I am guessing it just sends a stream of data without reading or writing it from the hard drive. It wouldn't be a good test of network performance if it relied on the speed of a hard drive.

Correct, iPerf of any version does not use the HDD for anything other than loading the program into RAM.

In reply to a post by jchamier:

You�ve reliably missed the end to end part of the equation. The sending machine has to be able to send at 900 Mbps+ which it may not be able if the HDD is encrypted and using CPU to decrypt thus breaking any DMA to the NIC.

I have 950 Mbps from my Realtek NIC. I have more problems with Intel drivers for WiFi than any other vendor. They are awful. The Broadcom NIC�s on enterprise servers are great.

I would point out that I was talking about wired Ethernet not WiFi which comes from a completely different section of the company. Intel wired Ethernet NIC's are widely regarded as excellent, if not the gold standard.

In reply to a post by jabuzzard:

Correct, iPerf of any version does not use the HDD for anything other than loading the program into RAM.

Then the whole performance of the PC, and the impact of non-vendor security software needs to be checked. The likes of Kaspersky, Norton, Bitdefender, and others. Often an uninstall and reboot leaves pieces of the product around.

In reply to a post by jabuzzard:

I would point out that I was talking about wired Ethernet not WiFi which comes from a completely different section of the company. Intel wired Ethernet NIC's are widely regarded as excellent, if not the gold standard.

On enterprise hardware I agree. On domestic grade hardware I've seen issues.

Just back from a week's hols so haven't kept up with this thread, but thanks all for your feedback and info.

I've installed iPerf3 on my NAS and run it as a server then retried the test from my desktop PC. Even with lots of background processes running with the CPU at near 100%, I get just under 950Mbps reported back by iPerf3, so compared with the desktop PC I'm guessing that the Surface Dock has pretty [censored] Ethernet hardware, given the result of around 600Mbps for the test between the desktop PC and docked Surface Pro.

But when I did the same test from the docked Surface Pro and the NAS iPerf3 server, I get 927Mbps, not quite as good as the desktop PC's result but much better than the reported speed between the desktop PC and docked Surface Pro, so the Surface Dock ethernet hardware is respectable enough.

Finally for the sake of completeness, I re-ran the tests between the desktop PC and docked Surface Pro: with the desktop as a server the docked SP reports 458Mbps; with the SP a server, the reported speed is 618 Mbps.

Some info on the network topology: the desktop PC is connected to a Cisco SG300-20 switch which is what everything else in the house is connected to. The Surface Dock is connected to another SG300-20 switch in the office which is connected to the aforementioned main switch so I'm wondering if that might be a factor. Obviously, to check for that possibility I'd need to connect the Surface Dock to the main switch and retry the test, which I can't do until I get a long enough patch cable, which I'll do this week.

Also, although you may know this already - the basic command on the iperf3 e.g. iperf3 -c 192.168.1.26
does an upload from the client to the server.

You can also do a download to the client from the server using the -R switch (reverse)

e.g. iperf3 -c 192.168.1.26 -R

As you may have noticed there is no disk usage while performing the test.

CPU usage will depend on the devices, although my Raspberry Pi 4 only uses 15% CPU while testing.

I get 910 mbps between my desktop and the Raspberry Pi 4 through 2 Netgear GS108 switches.

Edited by mrmarktigger (Mon 03-Feb-20 21:16:53)

It might be worth considering booting from a Linux live CD, that would rule out any issues with antivirus/bloatware on windows.

Good thinking. These days NIC drivers for Linux are now good, in the past I've had issues on server motherboards!

In reply to a post by jabuzzard:

It might be worth considering booting from a Linux live CD, that would rule out any issues with antivirus/bloatware on windows.

That's a great idea, did that and created an Ubuntu 19.10 bootable USB stick, had to download the iperf3 and dependent packages though, installed and ran tests and got 946Mb/s

I'll keep that USB stick handy

Really? What server motherboards are these? With a mixture of Broadcom and Intel and a smattering of Qlogic I have never had a problem with networking on Linux on a server, in >20 years. I have however recalled why Realtek chipsets are at least historically been considered a bit rubbish. The issue is higher CPU utilization than offerings from Intel and 3com for the same throughput, which where considered the gold standard in the timeframe I was thinking about. 3com another acquisition squandered by HP.

There was an issue with some Broadcom server chipsets not doing jumbo packets too, but we ditched the last of our servers with this limitation two years ago now. You would need something quite old for it to be relevant today. Dell rack mounting Precision workstations. They where a bit naff all round. For example the only way to determine which drive in a RAID array had failed was to note the serial numbers of the working drives, power down the server then start poping your hotswap drives to work out which drive was dead. I kid you not. I believe later models in the line fixed this particular deficiency.

I was copying large video files from an older Dell Optiplex 745 yesterday (to a QNAP TS453b NAS), and had diabolical throughput, i.e. <10% of Gb connection when viewed in task manager. On another PC, I could max out Gb connection to the same NAS. Hmm.

So, I checked the driver, installed the latest from the Dell website (dated 2014), still [censored]. Then in Device Manager I selected "search for updated driver from web", and low and behold the throughput jumped up to near maximum as shown in task manager. The installed driver was still ~6 years old, but it made a massive difference. CPU utilisation was very low.

Network interface is Broadcom 57xx Gigabit integrated controller, OS is Win7 32-bit on that PC.