Hi all,
I have had a trusted Nas for 8 years, which is running zfs in an OmniOS on an old version of Esxi (6.7). That beast has served me well, but I find myself having way less time than before to properly care for it, which means that my Esxi is old, OmniOS hasn’t been updated in a while, and this beast gulps 200W (2 E5-2670) for a dubious usage.
It was happily running along until a few weeks ago (when building contractors shut down power improperly), where I started seeing issues, which I have been trying to debug since then. My zpool is now degraded.
ZPool status
NAME STATE READ WRITE CKSUM
tank DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
c1t5000C500DBABD8A5d0 DEGRADED 0 0 0 too many errors
c1t5000C500DBE6B88Ed0 DEGRADED 0 0 0 too many errors
mirror-1 DEGRADED 0 0 0
c1t5000CCA254EEC0ACd0 ONLINE 0 0 0
c1t5000CCA260D5C520d0 DEGRADED 0 0 0 too many errors
mirror-2 ONLINE 0 0 0
c1t5000C500DC0BAAE1d0 ONLINE 0 0 0
c1t5000C500DC0F253Cd0 ONLINE 0 0 0
logs
c6t0d0 FAULTED 0 0 0 too many errors
cache
c6t1d0 FAULTED 0 0 0 too many errors
I thought I could just buy a few hard-drives and replace these, as I have done in the past. The mirror-0 drives are two Exos 16TB, the mirror-1 degraded one is a 8TB WD. I bought four MG09 16TB, and things unraveled.
OmniOS started showing resilvering in progress, but then I get lots of error logs for “incomplete write”, and things get stuck (presumably with my trusted LSI 9011-8i timing-out during reset). I presume that’s because the old drives were all 512N, whereas the new ones are not.
Not only does that freeze omni-os, but I’m even getting various PSOD in Esxi, with esoteric messages that made me wonder whether the HBAs being frozen are the cause, or if my CPUs have started misbehaving as well.
So, my current plan is to replace the beast with a smaller non-virtualized NAS, presumably using TrueNAS scale. I now come to my questions (although any suggestion regarding the above is most welcome):
-
I’m thinking of a fractal node 804. Any suggestion for a mATX MB/CPU combo that won’t break the bank or the power meter? Should I look at a standard i3-12500 + cheap MB and a good HBA? Should I instead look for low power ryzen + x570? Embedded Xeon/Ryzen MB? None of the cheap options seem to have 8 Sata. Again, no need for power here, while I would like to have another powerful machine, fact is that I don’t have time to install anything requiring significant power on it.
-
I’m unsure how to best get things going to a state where I can transfer the pool:
- I can remove the 4k drives, keep the pool in a degraded state, and zfs send it to a new truenas system.
- I can remove the 4k drives, export the pool in a degraded state, move the drives and zfs import in a new truenas system (but then I guess I will still have issues with sector side, and will need to change ashift?)
- I can try to upgrade OmniOS (I’m on a very old 151016 IIRC), but I’m not sure this will really help my troubles, and I’m taking the risk of further instability.