USB3 speed

What sort of speeds should I get from USB3? Currently I have a Crucial X6 SSD plugged into a USB3.1 Gen 1 port and CrystalDiskMark gives me about 480MBps on sequential reads and writes. But real world transfers I am seeing 250MBps write and about the same for read, highly dependent on OS. Fastest I have found is Debian 12 at 250MBps. Of course in Windows there is the Momentum cache from Crucial which gives 5000MBps r/w on CrystalDiskMark which is effectively writing to RAM. But I don't use Windows much.

I would suggest that the USB is probably working fine, the issue is more likely the disk you are writing to or from plus its operating system overheads. You can of course create a file system in RAM using ramdisk on Linux. Tutorial here https://www.linuxbabe.com/command-line/create-ramdis... - although I had to use 'top' rather than 'htop'

That is a good tutorial for a RAM disk but what I need is a large cache like the Crucial Momentum feature in Windows (2Gb-4Gb) but for Linux as I am backing up 2Gb files to an external SSD, the Crucial X6 1Tb.

Sorry, I was only suggesting that as a means of determining whether your USB3 was working OK.

In reply to a post by DFScale:

Sorry, I was only suggesting that as a means of determining whether your USB3 was working OK.

Sorry you lost me on that.

A caching program, like Momentum, is never going to speed up the overall transfer time. If there is a bottleneck between the computer bus and the disk than the data transfer rate is still limited by that bottleneck.

A caching program will give the illusion of faster transfer rates - but you don't see, behind the scenes, that it hasn't finished writing the data to the disk. So you can't swith the computer off until all the data has been flushed.

Look at an analogy. Suppose your bath took five minutes to empty because of a restricted waste pipe width. You could put another tank in between the bath and the outflow with a much bigger pipe joining them. Then the bath would appear to empty in - say - 1 minute. But the water is in the intermediate tank still draining out at the restricted flow rate. It still takes 5 minutes to transfer all the water; all you have done is to introduced another overhead and another delay because of that.

Now with a bath that doesn't matter - it is, to all intents and purposes empty. But in your case the backup isn't complete - the data is still in RAM waiting to be flushed to the disk. Turn the computer off - or have a poiwer cut - and that data is lost and your backup is corrupt. You can't beat the laws of physics.

What does it matter how long the backup takes? Just leave it to do its own thing whilst you get on with whatever else you were doing. We are long past the age of DOS when you had to wait for one program to finish before running another one.

And don't worry about benchmarks; they are designed to sell you something not to measure real-world performance.

I went through a similar exercise recently:

When USB 4 (throughput) isn't quite claimed...in Mac-land

In reply to a post by TinyMongomery:

A caching program, like Momentum, is never going to speed up the overall transfer time. If there is a bottleneck between the computer bus and the disk than the data transfer rate is still limited by that bottleneck.

A caching program will give the illusion of faster transfer rates - but you don't see, behind the scenes, that it hasn't finished writing the data to the disk. So you can't swith the computer off until all the data has been flushed.

Look at an analogy. Suppose your bath took five minutes to empty because of a restricted waste pipe width. You could put another tank in between the bath and the outflow with a much bigger pipe joining them. Then the bath would appear to empty in - say - 1 minute. But the water is in the intermediate tank still draining out at the restricted flow rate. It still takes 5 minutes to transfer all the water; all you have done is to introduced another overhead and another delay because of that.

Now with a bath that doesn't matter - it is, to all intents and purposes empty. But in your case the backup isn't complete - the data is still in RAM waiting to be flushed to the disk. Turn the computer off - or have a poiwer cut - and that data is lost and your backup is corrupt. You can't beat the laws of physics.

What does it matter how long the backup takes? Just leave it to do its own thing whilst you get on with whatever else you were doing. We are long past the age of DOS when you had to wait for one program to finish before running another one.

And don't worry about benchmarks; they are designed to sell you something not to measure real-world performance.

Yes I see now suppose I will have to suffer 2 hour backups for my machine.

In reply to a post by Pheasant:

I went through a similar exercise recently:

When USB 4 (throughput) isn't quite claimed...in Mac-land

Interesting read and yes I am seeing similar low speeds in real world but tests are not too bad.

In reply to a post by Banger:

what I need is a large cache like the Crucial Momentum feature in Windows (2Gb-4Gb) but for Linux as I am backing up 2Gb files to an external SSD, the Crucial X6 1Tb.

Is it specifically 2GB files that you are backing up or a hard drive with 2GB files? May be rsync [there might be a pun in there] will do what you want?

rsync will back up a file system to a clone of that filesystem, only backing up anything which has changed since the last backup. I believe it can run in the background too, which could address your concerns about USB speed if your workflow will accommodate this.