One of the things we still get, and perhaps more of them, are posts of slow ZFS write behavior.
A recent post was a user that wanted to have 3 x NVMe drives attached to USB type A, and another NVMe drive also attached to USB type A. But, things were slow.
Many other file systems perform the writes more or less immediately, which spreads the write out over a longer period of time compared to ZFS.
For ZFS, it will collect data for writing into RAM, and wait a bit, (for asynchronous writing), to allow more data or updates to be added. When a timeout is done, ZFS will create a directory entry update, (standard metadata), create an updated free block list, (for the updated directory entry, and possibly data), and updated critical metadata. Then, schedule this to be written out. This behavior is known as a write transaction group, (aka TXG).
The timeout for ZFS to perform asynchronous writes is about 5 seconds, adjustable. This means that ZFS won’t start writing, immediately, (though heavy incoming writes and updates may force sooner).
Upshot of this, is that some file systems seem faster for some tasks, (and may very well be actually faster). The theoretical write to a slower USB attached storage device from a different file system would start more or less immediately. And if it takes 5 seconds, ZFS might not even have started writing, yet. Then when ZFS tries another write, the first may not be finished.
In the end, ZFS does things different enough that certain hardware, like medium and slower speed USB storage devices can actually appear even slower. Thus, not recommended for use with ZFS for data pools. (But may work okay for boot pools.) To be clear, those USB storage devices may work fine, just slower…
I am not trying to stop people from using USB attached storage. Just pointing out WHY things can take longer.
My own local backup solution uses temporarily attached USB drives. This works fine, since I have no constraint on how long the backup needs to run.
