Software to test my LAN hard drive connection speed

I'd like to know the connection speeds of a couple of hard drives both connected to my Mac and on my LAN. Is there an app - gotta be for Mac - I can test the speed. I guess it will up and download a file to the drives and time it. As I tend to use large files - 25 to 30 Mb - something which reflects that, or lets me choose a file to use, would be wonderful.

Such a thing exist?

Just download the TBB test files from https://www.thinkbroadband.com/download say 1GB and 512 MB, then do a manual copy on your network and time it. Not 100% accurate but will probably be close enough.

Does a Mac have teh equivalent of Windows Task Manager? In Windows you can see the data rates on a network card. And in Win10 File Manager shows the rates too.

Turns out I can do it using rsync in the Terminal on a Mac. The command is

rsync -a --progress --stats --human-readable path_to_source path_to_dest

https://apple.stackexchange.com/questions/32785/is-t...

I copied a 25Mb file:

1 file to consider
MyFile,dng
25.28M 100% 42.46MB/s 0:00:00 (xfer#1, to-check=0/1)

Number of files: 1
Number of files transferred: 1
Total file size: 25.28M bytes
Total transferred file size: 25.28M bytes
Literal data: 25.28M bytes
Matched data: 0 bytes
File list size: 73
File list generation time: 0.004 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 25.28M
Total bytes received: 42

sent 25.28M bytes received 42 bytes 16.86M bytes/sec
total size is 25.28M speedup is 1.00

That's from a Mac to a NAS on a LAN. Fast or slow?

Mb or MB file?

I would always use a much larger file, say 250MB as any glitches/start-up are ironed out.

16.86 MB/sec >>> 135 Mb/sec. possibly a little slow, drives can often cope with up to 800 Mb/s and the LAN ,maybe 1 Gb

Ideally you want to test a larger file.
When transferring files to my network storage I can max out a gigabit ethernet connection at around 110-120MB/s

Thanks
Dan

In reply to a post by MHC:

Mb or MB file?

MB.

I see your point but I rarely transfer files much larger than 25 MB so I thought that a fair test.

I'll do another test when there's less traffic on the LAN, see if that increases the speed.

Try the GB file and see how long ...

The reason you want to use a larger file is to see if you can sustain the transfer speeds. Transferring a small file won't tell you this because it will copy really quickly unless something is wrong. Ideally whatever speed you're getting you should be able to sustain. If the transfer speed is dipping a lot and going up and down instead of levelling out and holding that speed consistently then that's not a good sign:

https://i.ibb.co/qRHBTQ6/vHB7aDb.png

iPerf is good for pure network testing but you do actually want to know your drive performance.

When I do these kind of transfer tests, I try to use a RAMdisk as source or destination, then the drive or network connection will be the limiting factors (rather than between 2 drives on 2 different computers for example where there are multiple factors).
Especially with increasing use of SSDs as copying to/from a spinning disk won't achieve the SSD throughput once any disk cache is exhausted.

RAMdisks have fallen out of fashion which is a shame because with many systems having 8/16GB they could easily spare 1GB, it's trivially easy on Linux and (for tmpfs) is automatically shunted to disk if free RAM running low.
I use one as my temporary download folder then archive anything I want to actually keep.

I did some tests.

I take folks point about hard drives, file sizes and so on. What I want to know is how much faster a fast drive on my desk - directly connected to my Mac - will be compared with a NAS in the basement.

The results may surprise you!

Or maybe not but I didn't get the result I was expecting.

I have a NAS on my LAN, connecting over 1G Ethernet and a RAID 5 box connected to my Mac via an eSATA to USB3 connector (I think). I'm copying from my Mac to the respective drive. I'm using the Think BroadBand test zip files, 1GB and 512 MB.

The NAS gives me 74.06M bytes/sec for the 1GB and 71.59M bytes/sec for the 512 MB

The RAID 5 was 42.11M bytes/sec for the 1GB and 42.95M bytes/sec for the 512 MB.

I was thinking of making more use of the RAID as I expected it to be quicker therefore better when working with large files on my Mac. Maybe I'll just keep it as a backup. In this case RAID is a backup!

In reply to a post by Woolwich:

The NAS gives me 74.06M bytes/sec for the 1GB and 71.59M bytes/sec for the 512 MB

The RAID 5 was 42.11M bytes/sec for the 1GB and 42.95M bytes/sec for the 512 MB.

I was thinking of making more use of the RAID as I expected it to be quicker therefore better when working with large files on my Mac. Maybe I'll just keep it as a backup. In this case RAID is a backup!

It's well known that DAS (Direct Attached Storage) is usually faster than NAS/SAN traffic. eSATA/USB3 can do 200MB/s + (pure USB3 max theoretical is 625MB/s)

Its not the speed of the traffic over the network slowing the NAS data rate, but the speed in which it writes to the drives. Stick a 500mb/s SSD in the NAS and it will probably be faster - but spinning disks aren't that great from a transfer speed point of view.

The location is not really relevant - 1m or 90m will not show any difference.

The NAS is potentially limited by the drive speed/capability. 74.06 and 71.59 and much the same and if you repeat teh tests they could switch round or get closer.

The RAID5 could well be down to the drive firmware having a processing limit and pushing te data to each drive as well as calculating parity and inserting that. And parity could be another 35% meaning the drives are storing at around 56 MBps.

In reply to a post by haydnwalker:

It's well known that DAS (Direct Attached Storage) is usually faster than NAS/SAN traffic. eSATA/USB3 can do 200MB/s + (pure USB3 max theoretical is 625MB/s)

Its not the speed of the traffic over the network slowing the NAS data rate, but the speed in which it writes to the drives. Stick a 500mb/s SSD in the NAS and it will probably be faster - but spinning disks aren't that great from a transfer speed point of view.

So I was right to be surprised, the RAID should have been faster. The NAS is a Synology running SHR with 7200 drives. The RAID also has 7200 HGST drives.

Simple mirroring of pairs of drives RAID 1 or simple stripe of mirrors (RAID 10)
both provider faster and easier restoration than any parity-based schemes.

With increasing capacity per £ may become favoured in the long term over the risk and complexity burden of other methods.
With any parity-based scheme the time taken to rebuild a disk and parity can exceed the time before the next disk failure; it's not just theoretical and it does happen that people lose a whole set that way with a disks of similar age living in same environment for years.

There is also a distinct advantage in not being dependent on quirks of any particular RAID drive controller and being able recover some usable data from any single drive of a mirror.

I like the idea of Synology devices but the affordable ones don't by any means pack a powerful CPU which needs to be factored into choosing for scenarios suited to capabilities.
Marketing of NASes would like to offer you about an ever increasing range of add-ons or apps while avoiding that level of detail though, and that's true of QNAP and most others unless you build something yourself.

Edited by prlzx (Mon 12-Jul-21 21:16:28)

In reply to a post by prlzx:

Simple mirroring of pairs of drives RAID 1 or simple stripe of mirrors (RAID 10)
both provider faster and easier restoration than any parity-based schemes.

I don't disagree with what you say (other than the price of hard drives!).

The options are proprietary NAS or roll yer own from 'free' software. I gave that a go with a Centos based server distro and although it had a GUI it was a pain to learn, use when command line was needed and get any sensible instructions/support. I never did work out how to mount external hard drives so the idea or adding RAID was a non starter. All fine if you do it daily, your average punter who wants a big centralised hard drive just wants it to work.

In reply to a post by Woolwich:

I did some tests.
...snip...
The NAS gives me 74.06M bytes/sec for the 1GB and 71.59M bytes/sec for the 512 MB

The RAID 5 was 42.11M bytes/sec for the 1GB and 42.95M bytes/sec for the 512 MB.

I did some more tests, this time copying the files from the NAS and RAID to my Mac. So kinda testing read speeds I guess.

The results surprised me - again!

From the NAS to the Mac, 82.61M bytes/sec and 79.55M bytes/sec for the 512 MB and 1GB files respectively.

From the RAID to the Mac, 119.32M bytes/sec and 126.34M bytes/sec.

So the RAID is much better reading than writing. And that's RAID 5, I was half thinking of reformatting it as RAID 10 (or 1+0?). That would be a bit quicker yet?

So far that's Big Files, cos that's what you lot say to test with. In Real Life I read and write to hundreds of 4 or 5K files (kilobytes, not 4K TV!). Anyway, it seems if I want to read fast I should use the RAID, write fast, use the NAS...

The NAS figures are not much different to before - so almost certainly teh drive is limiting.

The RAID - it has to be te controller or firmware that slowed down the write and the read speeds are maxing out the Gigabit Ethernet. 125MB * 8 = 1000Mb or 1 Gb

This is a good overview:

https://blog.storagecraft.com/raid-performance/

Distilled into one salient remark, “In terms of RAID, reading is extremely easy and writing is rather complex. Read performance is effectively stable across all types. Writing, however, is not.”

Much like my comment on Monday.

In reply to a post by MHC:

The RAID - it has to be te controller or firmware that slowed down the write and the read speeds are maxing out the Gigabit Ethernet. 125MB * 8 = 1000Mb or 1 Gb

Yes but no. The RAID is the one connected via USB3.1 to an eSATA port (using convertor cable).

As said above

In reply to a post by haydnwalker:

It's well known that DAS (Direct Attached Storage) is usually faster than NAS/SAN traffic. eSATA/USB3 can do 200MB/s + (pure USB3 max theoretical is 625MB/s)

So not bad really.

In reply to a post by Woolwich:

In reply to a post by MHC:

The RAID - it has to be te controller or firmware that slowed down the write and the read speeds are maxing out the Gigabit Ethernet. 125MB * 8 = 1000Mb or 1 Gb

Yes but no. The RAID is the one connected via USB3.1 to an eSATA port (using convertor cable).

As said above

In reply to a post by haydnwalker:

It's well known that DAS (Direct Attached Storage) is usually faster than NAS/SAN traffic. eSATA/USB3 can do 200MB/s + (pure USB3 max theoretical is 625MB/s)

So not bad really.

Its probably the usb to esata adapter that is limiting you.
If you get on that supports UASP it will increase the speed massively

Thanks
Dan

In reply to a post by danielhyde:

Its probably the usb to esata adapter that is limiting you.
If you get on that supports UASP it will increase the speed massively

Using this FWIW. The only one I could find at the time.
https://www.startech.com/en-gb/hdd/usb3c2esat3

In reply to a post by Woolwich:

In reply to a post by danielhyde:

Its probably the usb to esata adapter that is limiting you.
If you get on that supports UASP it will increase the speed massively

Using this FWIW. The only one I could find at the time.
https://www.startech.com/en-gb/hdd/usb3c2esat3

Yeah it doesn't say UASP in the specs.
I'm pretty sure that is what is holding back your RAID box.

Thanks
Dan

I had one of those and I think it was limited to 1.5 Mbps. However eSATA itself has limits and if the RAID box has any other interfaces they should be investigated.

Is it a drobo by chance ?

Modern NAS is very performant / bang for buck. Some of the medium to high end Synology units (yes I'm a fan of Synology, Drobo not so much) offer incredible performance.

A few sticks of NVMe SSD cache and a slew of 7200rpm NAS-specific buffered drives will easily enable them to fill several bonded GbE connections.

In reply to a post by jchamier:

Is it a drobo by chance ?

No chance! Heard too many horror stories.

Its not this drive but clearly made by the same firm (this one an own label brand, mine is 'Hydra'). Same idea but different, I have an LCD screen and USB2, Firewire 800 and eSATA connections. Its a few years. My one doesn't do JBOB, just spanning or RAIDs.

In reply to a post by Pheasant:

Modern NAS is very performant / bang for buck. Some of the medium to high end Synology units (yes I'm a fan of Synology, Drobo not so much) offer incredible performance.

A few sticks of NVMe SSD cache and a slew of 7200rpm NAS-specific buffered drives will easily enable them to fill several bonded GbE connections.

Yes, my other RAID is a Synology! Easy to set up, no endless asking 'experts' and learning how to do something via the command line.

I sometimes wonder about adding some NVMe but I'm probably held back more by my Mac that anything else. I'm rendering photos and the graphics chip (not card)! is likely a bottleneck. But I'm still after a bit of speed when backing up from one drive to the other and moving large files around the LAN.

Using a Thunderbolt to Firewire 800 would get you up to 100MB/s.
I think it would be faster than the ESATA to USB adapter you are using.

Thanks
Dan

In reply to a post by danielhyde:

Using a Thunderbolt to Firewire 800 would get you up to 100MB/s.
I think it would be faster than the ESATA to USB adapter you are using.

Interesting. I think I didn't try that because Apple only make a Thunderbolt 2 to Firewire adaptor and then you need the Thunderbolt 2 to 3 adaptor. And you can imagine how much that all comes to at Apple...

Agreed on Thunderbolt. Quite amazing what you can run over it simultaneously. Use a docking station on MacBooks - one cable to the dock does power, external screen and 10GbE.

What model / age of Mac are you using?

Off the top of my head the Thunderbolt 3 to Thunderbolt 2 is about £45 and the Thunderbolt 2 to Firewire 800 is about £25

Thanks
Dan

In reply to a post by Pheasant:

What model / age of Mac are you using?

MacBook Pro (13-inch, 2018, Four Thunderbolt 3 Ports)
2.7 GHz Quad-Core Intel Core i7
16 GB 2133 MHz LPDDR3
Intel Iris Plus Graphics 655 1536 MB

In reply to a post by danielhyde:

Off the top of my head the Thunderbolt 3 to Thunderbolt 2 is about £45 and the Thunderbolt 2 to Firewire 800 is about £25

I think the eSATA adaptor was about half that! Also, I wasn't sure how much use I was going to make of the RAID box so probably wouldn't have paid seventy odd quid.

Somewhere there's going to be a bottleneck no matter how fast my cables are. I think we're saying that's my RAID 5 write speed for copying and I bet my Mac for rendering large photo files.

Yeah, also when spending that much you could spend a little more and get a new raid box that has USB 3 or C.
A decent RAID 5 should be able to easily hit 70-100MB/s

Thanks
Dan