Truenas Scale with USB connected hard drives like Sabarent 5/10 drive unit

My Truenas is becoming more and more unstable. I never quite know when it will be working and I get frequent unreadable segment errors on my boot drives which I’ve never figured out how to get rid of. Part of the issue is that I have to use USB boot drives as my box will not boot from its SSD drives and if I use the hard drives for boot it will severely restrict my storage (Truenas is running on a QNAP box).

I’m getting fed up with this and thought I would perhaps try something different but along the way I found a Sabarent disk encloseure that allows attaching up to 5 disks via USBC at 10gbps which would be plenty for my needs. Has anyone tried this? And would it be possible to move the array out of the QNAP box and into this box without needing to reformat etc? i.e. how sensitive is the array to disk order etc.

ZFS does not care about drive order.
But a multi-drive USB enclosure is almost certainly a bad idea. USB is not a stable connection, and anything involving SATA port multipliers is incompatible with ZFS.

Suitable enclosure are SAS enclosures, connected to a LSI HBA in the NAS.

1 Like

Booting from USB(s) can be a bad idea, they’ll probably burn out quickly. Would recommend trying to figure out why it won’t boot from an SSD first, or use a USB<->SATA adapter to avoid burning out your boot drive.

I can’t say I’d recommend using USB/Thunderbolt attached storage, you might get it to work but SATA multipliers are not officially supported and there have historically been a lot of problems with them :). I’ve had luck in the past when I was first starting out (albeit with a PCIe SATA multiplier), but YMMV.

See: Multiply your problems with SATA Port Multipliers and cheap SATA controllers | TrueNAS Community & Why you should avoid USB attached drives for data pool disks | TrueNAS Community

1 Like

@tre4B

If you value your data and USB boot is your only option, use a SSD with higher P/E cycles in an external USB enclosure, rather than cheap inexpensive USB thumb drives.

Personally, I replaced the QNAP Apacer 512MB eUSB DOM with a 16GB SLC eUSB DOM

1 Like
  1. There is a BIG difference between a USB FLASH drive and a USB SSD drive. If you use a flash drive for boot you are asking for trouble. Switch to a USB SSD boot drive ASAP. I use an SSK USB SSD. (I also have swap and an apps pool on the same drive.)

  2. Make sure that you run a scrub on all your pools on a regular basis. This will pick up disk errors at an early stage.

  3. If you are using a USB SSD drive, it will likely support SMART - run SMART short / long tests on a regular basis.

  4. I have no idea what technical issues you may face using an external disk rack connected by USB - but I suspect a fair amount. Some potential issues I have heard about:

    • Disk serials not readable - significant issue with ZFS.
    • SMART not working
    • Disk write sequence not guaranteed - may create corruption on a journalled file system like ZFS.
    • Lack of parallelism / bandwidth constraint.
1 Like

Thank you all for your replies

The boot drives are both M2 drives in a usb container. I did that as I thought it would be more likely to be stable. Something in the QNAP PCle stops the drives it holds from being seen as boot drives. I tried everything I could find before resulting to the USB SSDs

It sounds like adding an enclosure of this type will be have if it even works. I might have to bring forward building a suitable server of some sort. I did run SMART for a while but thought that might have been the cause of the errors so turned it off again. When I get the errors its not only in the boot disks other disks see it as well.

I am currently running scale on a $169 N100 MiniPC booting from a mirror on the internal NVMe and an internal SATA drive. My data is on a 5-bay Sabrent unit that was $250 at the time. On the N100 it has been rock solid with light usage. I did need to update the memory to 32G ($60) and switch from K3 to Jails to solve memory issues, but now it works great. Months of uptime, only rebooting for updates or power failures.

However, at one point I tried to update to a GMKtec Mini PC i5-12450H and the USB drives would disconnect every few days and the ZFS pool would get corrupted so I switched back to the N100. So it can work, but not everyone builds these things correctly. I suspect an Intel NUC would have worked, but I am not wasting any more money because after jails came out the N100 works just fine for what I need.

1 Like

Dirt cheap mini-PC.
Dubious external enclosure (to put it politely…).

And then waste a precious NVME slot and an even more precious SATA port from the mini-PC, which doesn’t have enough of either, to mirror the boot drive. Which is totally useless in a home setting.

Why, oh WHY? :man_facepalming:

@etorix, Is this really the type of community you want to be building on this forum? Just assume everyone else is an idiot and let them know because they probably don’t.

In this case I was answering the OPs question. Can these enclosures work? Yeah, but not always. So I focused on that part. I needed bulk storage for backups and spinning drives in a usb 3.2 enclosure works surprisingly well when you include the slog on the NVMe.

For the this pc I made a custom Truenas installer script so the boot volume is mirrors from a pair of small partitions on the internal drives, but I also have partitons for zfs cache & log and a fast SSD mirror for applications that need fast local storage.

This hardware is actually much better suited for bcachefs with the ssd & nvme caching the slower drives on usb, but I was hoping scale would be less work than maintaining sometime myself.

1 Like