Hi all
I note that Claude helped me with phrasing this
I’m planning a new pool and would like a sanity check on where to put the hot spare.
**Hardware / software**
- TrueNAS Community Edition (SCALE) 25.10.7
- Head unit: ASRock Rack B650D4U, AMD EPYC 4245P (6C/12T), ConnectX-4 Lx
- RAM: 96 GB DDR5-5600 ECC (2× 48 GB Kingston UDIMM, 2 of 4 slots used, so room to go to 192 GB if it would help)
- HBA: Broadcom/LSI 9500-8e (SAS3808, external ports only, mpt3sas)
- JBOD: Supermicro SC847E26-RJBOD1, 45 bays (24 front + 21 rear), dual SAS2X36 expanders per backplane (6 Gb/s). Because the drives are SATA (single-ported), only the primary expanders are usable. Currently front and rear are cascaded on one x4 link; I plan to recable so each backplane’s primary expander gets its own HBA port (2× x4 6 Gb/s).
- Disks: Seagate Exos 28 TB ST28000NM000C (SATA, factory-recertified), building out in batches rather than all at once (starting with ~16)
- L2ARC: Samsung 990 PRO 2 TB NVMe (persistent L2ARC on). ARC hit rate is ~98% and the L2ARC hit rate only ~5%, so I may drop it or switch it to metadata-only on the new pool.
- Boot pool and app pool are on SATA SSDs on the onboard AMD chipset SATA controller
**Use case**
Mostly large media files (TV shows and movies) for Plex/Emby, plus torrent seeding (read-heavy, low write load). Planning recordsize=1M, lz4, atime=off on the media datasets. Not much small-file or block storage on this pool.
**Layouts I’m considering (all RAIDZ2)**
1. 5× 9-wide = 45 disks, JBOD completely full, ~980 TB nominal. Hot spare would sit in the head unit, attached to the onboard chipset SATA controller (the HBA has no internal ports).
2. 3× 9-wide + 2× 8-wide = 43 disks, ~924 TB nominal, leaving 2 JBOD bays for a hot spare and a free bay for in-place replacements. I know this needs a forced add (mismatched replication level) and the GUI may complain about mixed widths.
3. 5× 8-wide = 40 disks, ~840 TB, leaving 5 bays for spares/free slots. Cleanest, but gives up the most capacity.
I ruled out dRAID because I’m growing in steps and dRAID vdevs can’t be widened later.
**Questions**
1. Is there any downside to having the hot spare in the head unit while all data vdevs are in the JBOD? In my case it would be on a completely different controller (onboard AMD AHCI rather than the 9500-8e + SAS expander). Are there issues with ZED behaviour, enclosure/slot mapping, or anything the middleware doesn’t like?
2. With option 1 the JBOD is full, so there’s no free bay for a `zpool replace` with the failing disk still attached. Is relying on “spare kicks in → pull dead disk → insert new disk in the same bay → replace” acceptable practice?
3. Would you go with the mixed 9/8 layout (option 2) at all, or is the mixed width more hassle than it’s worth?
Thanks!