I’m planning to build a high-capacity TrueNAS system using a Supermicro SuperStorage 6049P-E1CR36L server I recently acquired, and I’d love to get some expert advice from the community before diving in.
Centralized storage and backup for large video files
I’m hoping for some guidance on:
Do I need to flash the firmware on the HBA card into IT mode for this setup to work.
The best way to configure my drives and pool layout (RAIDZ2 vs RAIDZ3, mirrors, etc.) for a balance between performance, redundancy, and future expandability. I was hopping to do three VDEV 8 drives pools on RAIDZ2, and the combine the three into one RAIDZ2? . or do two VDEV of 12 drives and combine the two into one?.
Thank you so much for your help on this in advance, from a really troubles (NAS troubled) person in London
That is a big system. Before anyone can give you good guidance on layout, what kind of data are you storing?
#TB need fast access #TB need for archival #TB need for any VM #TB need for other things
Risk profile of each data type (can never lose, ephemeral, etc)
First I would recommend a mirror on your boot drive and depending on data need, some SSD’s might be in order for fast access or caching.
Thank you for your input, this will be to store raw OCN, this is build for size not for performance. This will act as near line storage for some Online machine. If I can extract some performance out of it, great, but the main aim is to storage data securely.
Currently I am building 24 bays out of 36, once I have that stable, I may expand into more storage, or fast storage.
My main concern is the HBD IT mode flash. This will be the 3rd copy, other two in LTO, but loosing the raid woudl be a total pain. So I am open to suggestion in terms of setup
You’d 100% want to confirm the firmware on the hba before making a pool, sas2flash or sas3flash will provide the current firmware results; if you can post them here, it’d be a big help.
Depending on timelines it is always a good idea to perform some burn-ins, stress test the cpu(s), run some memtest, and if you got an extra week to wait then a pass of badblocks and smart long tests on the drives will reduce infant mortality.
Good points you made there. I have a 2nd card in order, will arrive tomorrow, and I will try to flash the firmware on that. Will report my findings here. I may have the week to test. So it would be good to know which tests are you referring too, if they are self-contain on Truenas or do I need additional software to do this. I would much appreciated if you could point me tosuch a tests.
Gracias
Manu
For memtest you’d just download it to a usb & boot off of it. For cpu benchmark, I like cinebench personally, but you’d have to boot to windows
For hard drives it is a good idea to burn them in, a nice smartctl -t short to make sure they aren’t dead on arrival, then if you have a week to spare before deployment, running badblocks, then afterwards a smartctl -t long would be golden.
Only problem with badblocks is that tmux is no longer native to TrueNAS - generally it is a good idea to read a few guides on how to use tmux, then run badblocks on each individual disk. Be careful though, badblocks is a DESTRUCTIVE test, be VERY cautious running it on any system with data that you want to keep. Do not run it on any disk that is already part of a pool. Always quadruple check which disk you’re about to run it on.
With all those warnings out of the way, I still love badblocks, I can’t think of anything else that would confirm each & every sector of a hard drive is working prior to desployment. Nothing worse than finding out a disk was bad 8 months after the store warranty expired & having to pay shipping for manufacturer RMA.
This is a very old guide & has a lot of out of date tunables that don’t work on Scale, and mentions Tmux (which again, isn’t on scale, consider booting into any other linux distro for testing). But the bones of what to do are there.
So in short, expect nothing to be built-in to Truenas in regards to testing
I also cannot stress just how long it’ll take for badblocks to fully run. It’ll also be an excellent way of testing that you have proper cooling on your HDDs because they will be going full tilt for days.
I did it for my IBM 1015 HBA years ago.
I took me like 3 minutes from start to end.
The only problem can happen is if you have a power cut during this 3 minutes.
SO if you have access to a UPS, connect the PC to that for the time, you do the reflashing.
Capacity storage = raidz#
3*8-wide Z2 is a reasonable option. (But see below…)
2*12-wide Z3 is a reasonable option.
2*12-wide Z2 feels a bit wide, but as a third copy it might be acceptable.
Since the end game is 36 drives, think how you’d like to fill these bays.
3*12? 4*9? 5*7 + 1 spare? 4*8 + up to 4 spares?
