Time to build a couple new servers, priced out a Supermicro and had someone mention adding Optane for ZIL. Looking around it seems like Optane may be end of life, so does that leave me with nvme? The board appears to have a pair of sockets, I need to look up the PCIe generation and lanes to see what should be good.
But how do I size these if their only purpose is as cache drives? I’m going to add 64GB of RAM, so do I match that or do I find the fastest drives I can in smaller sizes (SLC drives) like 8GB or 16GB?
I haven’t messed with this is a long time, and the last time I did the cache drives I had were slower than RAM and didn’t add any value.
These new servers are going to be all SATA SSD, but only 6 drives (right now) plus 2 system drives, only need around 3-4TB of storage. Probably going to be running Scale by the time I am able to buy them.
Obligatory “Have you talked to someone about TrueNAS Enterprise yet?” We have a number of all-SSD units for varying workloads and sizes, give us a shout.
The first question to ask is “do you actually need a log device?”
If you’re running applications that rely on synchronous writes (VM hosting over iSCSI/NFS, database servers) then yes - but for protocols like SMB which typically use asynchronous writes, adding a SLOG often isn’t necessary. If you do, then read on.
Optane is indeed EOL, so you’re limited in finding a new-old-stock or a used-pull if you want to go that route. NVMe is just a protocol; in general your SLOG device needs to be the fastest, low-latency device with power-loss-protection for in-flight writes you can get your hands on.
Sizing you’ll generally not use more than 16G of the device, but the greater capacity adds to both endurance (more space for wear-leveling) and performance (more NAND channels) so you’ll find that the fastest drives might be larger.
I guess that is part of why I posted, do I need a log device?
VMs over NFS, though I’m starting to play with moving them to SMB because there is a small performance boost. Currently running them with NFS sync off, not a great thing to do, but the speeds are incredibly slow if I leave sync on. I’m guessing by that statement, I could improve a lot of things with proper cache drives and sounds like I can save a bit of money by going smaller like 32GB of SLC drives.
I have talked to ixsystems and generally what I want is out of the allowed price range. I’d really like an M series with duplicate controllers, etc. I’ll have to reach back out with these specs. and see what they can offer.
The question of sync being on and off is about the safety of the data - if you’re running with sync off, there’s potentially data that could be lost in case of a sudden outage.
Yes - a good log device makes the difference between fast and slow sync writes; however, I wouldn’t recommend the smaller SLC devices. Small SLC units are typically designed as an industrial boot device - high endurance and temperature tolerance is usually what they’re designed for, but performance doesn’t necessarily meet the needs of a log device where “basically every write to your array” goes to it. Unfortunately there’s no direct replacement for Optane - but a couple NAND manufacturers are working on options I believe.
Feel free to drop me a DM directly if you’d like, happy to help get you in touch with the right people.
If they go for it, I did mention that I want another big and fast server too, this might fit into an R series or maybe an M series. I priced a rough Supermicro at around $28k so I have a feeling this would be something you can make fit nicely into one of your products. I doubt they will go for it, but always hoping at this point.
It would be nice to have something that can handle having our students edit videos directly from a share. Running 200mbps files and normally only 2 per project. It was easy when we were using 35mbps files. But what would support over 40 streams now only handles about 4 to 8 to be safe. It might handle more, but I don’t want it to be a habit because I know it will let us down with only a few users. Only 8 spinning drives so only so much you can try to push through the old thing and stay safe. 20+2 NVME and dual 25gbps connections would be really nice!
We have a number of all-SSD units for varying workloads and sizes, give us a shout.