I am a member of a not-for-profit video ediing studio build a NAS server. I am a software engineer, and have very limited experience with building servers. So, I will share the main requirements, and would be grateful for any advise you have to share.
Primary use: The server will serve as a TrueNAS-based storage solution for five concurrent 4K video editing stations, plus additional general workstations. The workstations are a mix of Windows PCs and Apple laptops.
So the most important advice I need is: how much memory (I’m assuming it has to be ECC), how powerful the CPU needs to be, and also networking (especially the network switch, would a 10 Gb be enough, or would 25 Gb or even higher be recommended?)
You should propably plan with a 2 Pool layout. One fast ssd or nvme based pool where active projects are stored and can be edited from (if that’s the plan and not editing on local disks) and a bulk hdd pool for storing finished projects longterm.
A lot of the answers to your questions really come down to budget. I’d say 10Gb for your clients would be a minimum but obviously more is better. Then from a server side you would ideally want 40Gb or more. Based on these speeds then you will most likely want some form of flash storage but things start to get expensive now. For 250TB of storage then I’d say you’d want a minimum of 256GB RAM but again more is better.
I say edit the video locally on a good NVMe and then Truenas with HDD only storage. For that 10Gb/s would probably be enough. CPU and RAM is then also not that important.
Live, simultanious 4K editing on the NAS means serious IOPS/bandwith demands → $$$
Thanks everyone for your responses so far. Our budget is around 10K EUR.
We’ve been consulting with a vendor and they have recommended the below. What are your thoughts on this? Does it generally make sense? Is there anything else to take into consideration?
Memory and CPU:
refurbished Apollo 4200 server with an Intel Gold 5218 CPU (16 Core, 2.30GHz) It should offer sufficient PCIe lanes, though only 6 memory channels.
256GB of ECC RAM (8x HPE 32GB DDR4-2666)
The server has two 3.84TB SAS RI SSDs. What role could these play in ZFS/TrueNAS (boot pool, SLOG, metadata, or L2ARC)? Should I consider NVMe cache?
Storage: The server will initially have 24x HPE 14TB SAS drives, with the possibility of adding a second 24-bay expansion shelf within the next two years (We have an offer for refurbished drives with a 3-year warranty).
Planning to use 2 double network cards on the server for 4x connections to the 10GbE switch in LACP mode.
The chassis has hardware RAID controllers included, but I understand we should use a SAS HBA in IT mode, for example LSI 9305-16e. In a few years, we wish to expand to another shelf connected through another HBA.
Your cpiu doesn’t need to be a high end cpu but it should be reasonable. While i like both AMD and Intel, Intel is the better supported platform. Server grade hardware is needed and 256GB of ECC RAM.
An UPS is required.
250TB is a lot of storage but if you only consider very large capacity drives, ensure some high redundance.
As other’s have said, some fast SSD media as well.
I highly recommend that the video editing be done on the workstations, not the NAS directly. Just copy the files over first.
And by all means, have a backup plan. The NAS could fail and your data could be gone.
Whle I understand you are a non-profit but maybe you could contact the TrueNAS customer sales and ask about a system and mention your non-profit. It can’t hurt.
What program?
Adobe or Davinci?
Will you work with proxies or original media?
Your workflow will probably not profit much from RAM.
ECC is not a must. Your clients also don’t have ECC. There is nothing about ZFS that requires ECC more than any other FS.
The faster, the better, right?
Here is an unpopular opinion. If you don’t work in the same projects at the same time, and if you don’t use proxies, I would get fast internal and external SSDs and only a primitive 1GBit/s TrueNAS (backup up with backblaze) as your archive destination.
A good SSD will easily outperform even a 25GBit/s SMB share. Also the macOS implementation of SMB is IMHO a PITA and you will have a very hard time getting over 10GBit/s on any Apple device. For both, hardware and software reasons.
We use Davinci, and work both with proxies and raw footage.
So far, we have been getting by using external disks and RAID arrayas and it has become a nightmare. So there’s no question in my mind that we need a NAS.
Maybe Blackmagic Cloud could be interesting then for you.
“Project Server” offers Project Libraries for 5$ per month. Each Library can have unlimited projects. You could combine that with TrueNAS as original media storage backend.
Or you use Blackmagic Cloud Storage (roughly 30$ per TB per month).
You can only upload proxies to the Cloud too.
Disclaimer: I have not used it in production, only read up on the topic. Like with an IT project, I would do some extensive testing first.
