Hello everyone from TrueNAS community,
I have been reading up on TrueNAS Scale over the past weeks and while I feel like understand most of the general ideas and concepts, I’d like to have some inputs when building the NAS. This is mostly related to the vdev layout and disk size.
The idea now follows very closely to the hardware highlighted in one of the qotom 1u servers that was reviewed last year on servethehome (details below). Briefly, the server runs Proxmox VE and will host pfsense as well as TrueNAS Scale. The server will eventually be used as a router: I have a 10Gbps FTTH connection that goes into the WAN port, and the LAN port(s) is connected to a few devices (TV, a NUC12 enthusiast running jellyfin, etc) as well as an AP that will take care of the wireless connections.
The main use-case for the NAS in my home network will be:
- Storing media (currently sits at 5Tb) for Jellyfin Server (that runs on a separate computer);
- Lightroom catalog (a little over 1Tb);
- Time machine backup for two macs (< 1Tb in total);
- eventually an additional space of about 1Tb when I figure out how to host my own cloud storage.
adds up to 8Tb, so I imagine I should start with a storage pool of at least 16Tb?
The hardware currently running proxmox is the Qotom 1U with the following:
C3908 (Denverton Intel Atom, 16 core, 2.0Ghz);
2x 32Gb ECC RAM;
2x 512Gb NVMe SSD (currently used for proxmox ve in a ZFS-mirror configuration);
4x SFP+ (1WAN, 1LAN, 2 undefined);
4x 2.5Gbe (currently undefined but will be assigned for LAN eventually).
1x SFF8088 port that I intend to hook up to the following:
1x QNAP TL-R400S (a 1U form-factor that will allow me to add up to 4 disks)
I have a few question that I can’t seem to arrive at a reasonable “end point”. Have been going deep down the rabbit hole, reading, etc but there seems to be so much possibilities/options! 
Question 1: What configuration would be ideal for my use case? I have been reading a lot that starting with 3 disks for RAIDZ1 is a “bad” idea due to failures, long resilver time, etc. I am currently planning to start with a simple mirrored storage pool for data redundancy, and then use a separate small SSD for the boot pool (in the case that having the boot pool on the same shared drive holding the proxmox VE is a bad idea). So that means I should get 2x 16Tb drives to achieve this (plus the separate small SSD). Would this be the right way forward?
Question 2: For future expansions, would it be correct to say that TrueNAS expects me to add a pair of drives at minimum e.g. 2x 16Tb or higher. This way, TrueNAS will allow me to “expand” the pool?
Potentially irrelevant question 3: Proxmox VE allows for high-availability through implementing nodes as a cluster with a minimum of 3. Currently, due to the single SFF8088 port on the Qotom unit as well as bandwidth capacity, I am restricted to the 4-drive configuration. Would I be able to add QNAP TL-R400S with additional drives on separate nodes and add them in the same storage pool? This is assuming the above setup is node1. And on node2 and/or node3, i replicate the same hardware pairs (qotom1u + TL-R400S + x number of drives) and create a vdev. Is this a ridiculous way of having more drives? It is highly likely that I do not fully understood how proxmox cluster nodes works in this case.
Thank you in advance for your time in reading and answering to my post! Any help/insights/advice welcomed! 