This resource was originally created by user: [WI_Hedgehog] (https://www.truenas.com/community/members/wi_hedgehog.116993/) on the TrueNAS Community Forums Archive. Please DM this account or comment in this thread to claim it.
TrueNAS is not as simple as videos make it look, though it is fairly easy to get a system running if you read the documentation and don’t color outside the lines.
Show : tl;dr: Hacking TrueNAS
If you want to save yourself lots of money and loads of time, the top of the page has Documentation and Resources tabs which contain extremely helpful information like the TrueNAS manuals. There’s also a Search tool at the upper-right that works extremely well. If you understand the system and requirements before jumping in you’ll save money, time, and frustration.
As general advice, if you run TrueNAS as a NAS and don’t add things to it it tends to run fine and without issue, though I suggest using an Uninterruptible Power Supply and NUT.
Precautions: (dreaming big is fine, but let’s nail down a few things that go badly for new-to-TrueNAS users)
- Adding Virtual Machines, Jails, Containers, TrueCharts, scripts, and especially running TrueNAS in a virtual environment (Proxmox) is where people get themselves into trouble and start losing data. It appears most initial problems occur due to network bridging not being configured properly, with drive access perhaps being the second biggest issue. (keep reading)
- If you saw a video with TrueNAS 1) in a VM, 2) being used as a gaming server, or 3) Plex/Jellyfin server: You’re probably in for more work than you expect. First build TrueNAS as a NAS on server-grade hardware, and when that’s tested and stable incrementally expand it, or better yet use a different platform completely as TrueNAS doesn’t work spectacularly for any of those purposes (though it does work well as NAS, not surprisingly). (keep reading)
- TrueNAS CORE (BSD) is faster and more stable than SCALE (Linux), so consider running CORE. There’s almost no interaction with the operating system anyway.
- DO NOT use gaming hardware for a NAS; TrueNAS is a Datacenter-grade platform and will not run well on gaming hardware just like games don’t run well on server hardware.
- If you’re trying to repurpose an old desktop system use Ubuntu Server (possibly with ZFS though that’s often not the best choice for older systems, use LVM instead), not TrueNAS. Videos showing TrueNAS working for a day of filming before tearing down the system for another build is entertaining, but this isn’t good for long-term stability (or even short-term stability). TrueNAS uses hardware in ways desktops were not designed for and they’ll eventually break and your data will be lost forever. Use Ubuntu, --or Gentoo if you want to wander into the deep end of the pool. Repurposing a desktop can get real expensive real quick and you’re better off buying used server equipment–you’ll save money and time.
(Note that Ubuntu Server and TrueNAS have a text-only terminal attached, unlike Windows or Ubuntu which are graphical and use a mouse.) - Is TrueNAS “a lot cheaper and as easy to use as a Synology NAS?” Well, cheaper…that depends on Total Cost of Ownership, including the time you invest in learning any system. It’s hard to beat Synology if you want an easy-to-use home system that you just plug in and click around a bit, though Synology gets expensive when storing a lot of data.
- TrueNAS as a VM Under Proxmox or VMWare: Don’t do this, Proxmox can operate as a SMB Server for Windows or NSF Server for Linux. VMWare sharing methods: (link1, link2)
- Minecraft: This isn’t a good idea, Minecraft is best run on an old desktop where it will run far faster and smoother–and without the cost of server hardware TrueNAS requres. The ZFS filesystem TrueNAS uses can be used there but you don’t want to do it as it will wreck performance.
- Steam Games and Deduplication: If you have the money, it’s doable. Otherwise…
- Plex/Jellyfin: This isn’t a great idea for many reasons, though it can be done. Transcoding is somewhat pointless as even cheap phones can display 4K data streams, so a Raspberry Pi [alternative] with Kodi is enough to stream video and will use far less power and be much quieter. Both work great under Proxmox (which is designed to run containers), however be prepared for a lot of hardware and configuration headaches you’ll most likely face if trying to do this with TrueNAS.
If the above suggestions don’t meet your needs because you have valuable data and truckloads of it, and you’re convinced you need an industrial storage solution on server-grade equipment to handle it (without breaking the bank), then TrueNAS is probably what you want.
(If you’re reading this on a phone, turn it sideways and read in Landscape Mode, longer lines will make more sense)
Fundamentals
Basic Understanding
-
Search Before Posting (get immediate answers!)
-
Slideshow explaining ZFS, VDev, zpool, ZIL and L2ARC for noobs
-
RAID-Z1 is Dangerous (well, -Z0 is worse)
-
Universal Serial Bus of Doom (why USB drives aren’t used in a NAS)
-
ECC Memory: Do I have to??? (explained in a build thread)
-
Ryzen “may-be ECC” vs Xenon 4-bit ECC (normal Intel ECC is 1-bit)
-
RAID Controllers won’t work; Use a Host Bus Adapter (HBA) (realllly important, even though…)
-
System Manuals (each version’s manual is under Documentation at the top of the page)
Hardware
Range of Builds:
-
Under 30W Systems (The most economical systems / Die sparsamsten Systeme)
-
Building a Power Efficient Home Server (23W D-I-Y full-size server, YouTube - Wolfgang’s Channel)
-
General Overview (UNRAID, though applies to TrueNAS as well)
-
Super Powerful Micro Server (Supermicro Superserver E300-9D-8CN8TP about $2,200 USD 04/2023)
-
LattePanda Sigma - Hackable x86 Windows/Linux Single Board Computer/Server (Dave’s Garage video, $580 05/2023)
Required Hardware:
-
Hardware Guide (READ THIS BEFORE BUYING ANYTHING!!!)
Storage: -
LSI HBA Card Insight (and ‘how to frugal’)
-
Fake LSI Cards (eBay is full of them)
-
Westerm Digital Red / SMR Drive Incompatibility with ZFS (ZFS eats them for lunch)
-
Non-Running Drive when Shucking Western Digital Hard Drives from USB Enclosures
-
5,400 RPM Drive Speed (a good thing)
-
Fibre Channel (playin’ with the big boys) QLogic QLE2564, QLogic QLE2694-SR.
Power: -
Power Supply Unit (PSU) Selection and Sizing (also important)
- Detailed Look Inside Power Supplies
-
Current Ripple and Life Calculation (triple your ripple and say goodbye to your power supply) (explained, filter considerations)
-
Realtek Network Adapters ARE Bad (why to avoid cheap Ethernet)
-
Intel Atom vs CORE
-
Note: The Intel Quark SoC, lower-power chips than Atom, were discontinued in 2019.
Low Power Mainboard: -
‘On the frugal’: Mainboard: Supermicro X9SCM-F, CPU: Intel Xeon E3-1230
-
More Power: Supermicro A2SDi-2C-HLN4F / A2SDi-TP8F, X10SDV-TLN4F (note)
Insight:
Example Systems
- ESXi and Plex on a Node 304
- Pheran’s 80TB build (with photos)
- beard-strokers
Building
-
SFF-8644 External HD MiniSAS Connectors: Not necessarily difficult to plug them in wrong.
Configuration
- Leave the Boot Drive Alone (do not partition it for data storage)
- Setting the Time Correctly (NPT drift)
- Structuring Datasets
- Useful Commands
- Useful Settings
- Supermicro’s Fans Fix via IPMI Sensor Thresholds
- Removing SMB1, NetBIOS, and WINS
- rsync fails when copying Windows datasets
Maintenance
-
Drive Life Expectancy (about 50,000 hours for Enterprise drives)
-
Hard Disk Drive Study (Backblaze)
-
System Update Procedure - !!! READ THIS FIRST !!!
DO NOT add the following “stuff” to your system until you actually need to as it will unnecessarily complicate things and possibly break other stuff:
Ease of Use
-
S.M.A.R.T. Reporting Script (multi_report.sh for CORE and SCALE)
-
ZFS transfers with netcat on trusted networks
bc
the basic calculator is in CORE and the script will work there out of the box; SCALE however does not includebc
in the base system (yet…this may change) but can be added if you want to use the script there.
Advanced
Storage
LSI/Avago/Broadcom Host Bus Adapters:
-
LSIutil (fixing your LSI card)
-
Crossflashing LSI 9211/9300/9311 HBA (and variants)
-
Erasing the LSI 9207-8i BIOS (not the firmware) for Faster Boot
Data Deduplication: -
How to Kill Performance (and raid your wallet) With Deduplication (why you shouldn’t use de-dup)
Bottleneck Reduction:
-
What are IOPS ??? (un-bogging drive-bound systems)
-
ZIL & SLOG (additional info) (sync write backup & using a separate backup device)
-
Fusion Pools (ZFS allocation classes, ZFS special vdevs, and metadata vdevs)
-
Nope: Boot, SLOG and Fusion Pool cannot be on the same mirror (how to break everything)
-
Hierarchical Storage Management (also known as Tiered storage)
-
Oracle HSM is done via the Sun StorageTek SAM-FS software
-
Open Source: Online Hierarchical Storage Manager (under development, whitepaper)
Boot Resiliency:
-
Highly Available Boot Pool (3 SSDs for redundancy)
Large-Scale RAID: -
dRAID (CLI currently in SCALE, GUI in upcoming SCALE Cobia)
iSCSI / Block Storage
- How to Configure Block Storage (not file shares)
- Why iSCSI often requires more resources for the same result
- Why Use Mirrors for Block Storage
Network
SAMBA Performance:
-
SMB Multichannel
Bandwidth: -
10 Gbps Ethernet Primer (a diet for your wallet)
-
40 Gbps Mellanox InfiniBand (about, notes, also note the switches are different than Ethernet and a bit spendy)
Switch, Firewall:
-
Ultimate CHEAP 2.5 & 10GbE Switch Guide (generally under $200 in 1st quarter 2023)
-
Even Cheaper!!! (generally under $125 in 2nd quarter 2023, see notes)
-
CHEAP powerhouse 2.5GbE Mini PC Router/Firewall ($300 in 1st quarter 2023)
-
MikroTik CRS518 (16) 25GbE & (2) 100GbE port Low Power Switch ($600 in 4th quarter 2022)
Special-Use Cases (Fast Networking by Breaking Convention): -
Understanding LACP (Link Aggregation)
-
Remote Direct Memory Access (RDMA)
-
How RDMA Works and Bypassing RDMA Security (USENIX Security 2021 video)
UPS
- Network UPS Tools (NUT support)
- Energenie UPS
Dangerous Territory
-
Nextcloud-iocage-FreeNAS (maybe TrueNAS)
-
Plex (I think…I didn’t try it, your mileage may vary)
-
Virtualize TrueNAS (iXsystems)
Applications
TrueNAS is an excellent NAS appliance, and there are sound reasons to stretch that further, though doing so is not without inherent risk. While the linked threads contain some “bold exaggerations” they also contain valuable information on what to expect and how to keep your system out of trouble while capitolizing on the resources.
- TrueCharts: Things may not go as expected (so plan ahead)
- Community App Repository (similar situation to TrueCharts, plan ahead)
CREDIT: This list was originally based on @Chris Moore’s Links to useful threads and @Davvo’s list (in his signature).
DISCLAIMER: What you do is up to you, opinions and comments herein non-withstanding. Comments are based on enterprise-level hardware reliability statistics and critical data retention, which to different people means different things and holds different values. You could conceivably run TrueNAS in a VM without ECC memory on generic consumer hardware for years without problems, buuut that’s not really “best practice.” Sometimes though, “you gotta do what you gotta do and throw caution to the wind.” I’ll just mention you might want to keep on top of your backups… [image]