N00b TRUENAS-MINI Setup: cache & pools

TrueNAS General
1

Hello all.

First post from a TrueNAS/ZFS/NAS N00b.

I have a new setup that I’m trying to configure:
TRUENAS-MINI-3.0-X+
8 Core, 2x10GSFP+, 64GB RAM
TrueNAS-13.0-U6.1
5x WD Red 14TB HDD
2x Samsung 870 EVO 2TB SSD

Per my conversation with a support person while configuring the hardware before purchasing, I bought the SSDs with the intended use as read cache and write cache.
While searching for instruction on how to set up read and write cache, I came across an older post which suggests these 2TB SSDs may be inappropriate for the intended purpose. Is that correct? Should I try to return them? What is more appropriate?

For the record, I see no results in documentation when I search for “read cache” or “write cache”:
Search | TrueNAS Documentation Hub
?query=write+cache

I also searched in forums but found many posts that aren’t quite what I’m looking for. Support says the question has been asked hundreds of times. That a question should have to be asked so often tells me documentation, organization , and search tool quality are insufficient. If it is asked so often, perhaps all support people should be able to give better guidance for customers configuring new hardware so as to head off such issues.

Additionally , I’d like to set up pools, etc. I see doc on how to do it:
Storage Configuration | TrueNAS Documentation Hub
This doc tells we where to click on the interface but not why; what strategy to use, what the consequences of allocation choices are. I’m trying to implement a ZFS NAS to have more reliable data stewardship. I’d rather not faff about and fubar myself by making uninformed choices with consequential outcomes.

Does anyone have any suggestions for resources that could help me get my head into this?

Thanks in advance for your time and attention.

Warmly,

Juniper

2

If this is for home use & you don’t have dozens of folks connecting to the NAS then I’d either return the SSD drives or set them up in a mirror for apps & VMs.

I’m surprised support was, uhhh, un-supportful. But generally the advice is ‘if you don’t 100% understand & can explain why your usecase needs it; you don’t need it & it likely would be a detriment instead of a benefit’.

I could be mistaken but something like a SLOG would only be of benefit of you’re dealing with intense sync writes. L2ARC would likely not be of use unless you check reporting & see that in ZFS you’re getting a lot of misses from regular ARC. And then metadata special device would likely be clutch for spinning rust, but introduces the risk of ‘if it fails then your entire pool’s worth of data is gone forever’; in which case two consumer SSD drives may not be what you want.

This is a simplified explanation on why you may not want caches. To beat a dead horse; for why you’d want them would need to go beyond ‘more performance more better’ & instead look at your specific use.

//edit: corrected ‘nvme’ to ‘ssd’ as I had thought the 870 evo was nvme originally; the point, however, remains the same.

3 Likes
3

Live and learn, when you noted

with a support person

Was this an iXtreme support person from TrueNAS? Or someone else?

The way TrueNAS is, first and foremost = more RAM before considering L2ARC (except as @Fleshmauler noted, specific use cases where other options would provide a benefit, but they are very specific and usually start at 64GB/128GB min of system ram)

4

Welcome to the forums!

I just tried the search - it’s actually the :search fiuntion that doesn’t work, because there is information there CORE Hardware Guide | TrueNAS Documentation Hub under Storage Solutions > Hybrid Storage & Flash Cache (SLOG/ZIL/L2ARC). There you will find an expanded version of @Fleshmauler’s comments - however, they may be a bit opaque to you as ZFS doesn’t use “cache(s)” in the same way that does e.g.windows, nor does it use the word cache in the function name.

Go here Introduction to ZFS | TrueNAS Community
and download that primer by mod @ericloewe and find his introduction to the “ZFS cache” topics on page 11.

This by @Arwen is additive:

ArwenTrueNAS MVP

May 21

Welcome to TrueNAS forums.

There is no such thing as a ZLOG, it’s SLOG or Log device. It is always helpful to use the proper terminology. ZFS does not have a write cache, the SLOG is not a write cache. All pools have a ZIL for synchronous writes, which is inside the data vDevs.

Further, if you have to ask about SLOG, Special Metadata vDev or L2ARC / Cache devices, you probably don’t need them. They can always be added after.

In general, L2ARC / Cache device(s) would only be 5 to 10 times your RAM size. So with 256GB of RAM, using 2 x 1.92TeraByte SSDs for L2ARC / Cache is a bit too much. This is because the pointers to the data in the L2ARC / Cache are in RAM. Too much L2ARC / Cache and too much RAM is used.

I highly suggest reading about ZFS and it’s pool configuration options. I don’t have a list of Resources or docs to read, but you can find some in the Resources section of this forum. (See left hand side, Resources’ Categories.)

Formerly of Rivendell,
Now far east, near Mordor, Middle Earth

With respect to “what” on storage, start in the docs with the simple at Storage Configuration | TrueNAS Documentation Hubexam where a mirrored storage pool is created.

But quite likely you want a more potentially robust (more redundancy) - go back to Introduction to ZFS, pages 5 and 6 and on for an overview that includes RAIDZ1, RAIDZ2 and RAIDZ3.

By then you may want to come back here with more questions …

When you come back, best if you describe your intended use for your data store and your expectations of its management, particularly backups and recovery.

1 Like
5

Than you for your response, Fleshmauler.

My use is a small studio (mostly just me, perhaps a rare other simultaneous user or two ) with archiving plus video/audio/still editing . The machine doing the editing work has 8TB of SSD; if that has any bearing on the issue.

Sounds like neither L2ARC nor a SLOG device would be merited in my instance. I won’t use the two SSDs for those purposes.

6

Thank you for your response, MBILC .

That was TrueNAS support.
I bought the maximum amount of RAM that the sales config allowed ( 64GB ).

1 Like
7

I guess it could be an idea to use the two SSDs for things that are more ‘in-progress’ while you have groupwork going and the hdds for purely archival work. At least that way you’ll get closer to being able to use the 10gig nics. A poor man’s layered storage if you will. I could see it at the least helping scrub speeds. Though, I doubt it’d be of much use when you don’t have multiple folks working at the same time considering the plentiful storage space available on your workstation.

You could even get snapshots from the 2 ssds onto to the hdds; so that way if something goes critically wrong you’ll have the latest day/week/whatever you setup backed-up on the slower storage.

But yeah, otherwise, unless you make-up a need for apps/vms/other nonsense, these SSDs are likely pointless.

Sucks man, I’d have expected more in terms of educating why you would/wouldn’t want it to at least upsell.

8

The Samsung 870 is a great consumer SSD, and it would be fine for L2ARC - although potentially a little large - but @Fleshmauler’s suggestion to use them as a separate high-speed pool is likely the ideal suggestion for your use case of editing video/images over a network.

The short version of the recommendation for your pool needs would likely be a 5-wide RAIDZ2 - a parallel to a “RAID6” in traditional RAID, this gives you the equivalent of three disks of usable space, with the potential to tolerance two drive failures. While it doesn’t give ideal random-access speeds compared to simple mirrors, you’re largely working with large files (or large chunks of a file) at a time, from only a single user, so the workflow will be a lot closer to the bulk, sequential access that hard drives excel at.

2 Likes
9

Thank you for your welcoming response, Redcoat.

Seems like it might be a good idea to get :search to work more intuitively.
Thank you for the links. I know very little of windows but a little bit about UNIX-like systems. I’ll read these documents you suggest.

The TrueNAS hardware config page ( Configure & Buy TrueNAS Mini - TrueNAS - Welcome to the Open Storage Era ) specifically uses the terms, “Read Cache” and “Write Cache” . It sounds like there could be some improvement in the consistency of communication practices.

Thanks again so much for you helpful response, your time, and your attention!

2 Likes
10

Thank you for this useful additional input, Honeybadger.

The idea of fast SSD pool is interesting, though the small size makes the extra workflow complexity seem less benefit vs the organizational cost.

11

Yes - but my experience is that this issue is industry wide and has been so since the dawn of personal computing. I have always believed it is the product of developer-types writing the early docs - which are subsequently translated for human consumption by marketers.

I looked at this early piece from the TrueNAS Archive Resources again yesterday. Much of it is still relevant, certainly as foundation material. Michael Lucas’s series of books on ZFS in the IT Mastery Series are a worthy read, too, depending on how far you want to go.

Good luck!

1 Like
12

Thank you for the further resources, Redcoat.

13

Oh in general; look-up snapshots & consider implementing them. Considering you work with media & accidents can happen, it might save bad mistakes. Ain’t no recycling bin on TrueNas - you delete it, consider it gone.

Also setting up correct permissions for when folks do connect to your NAS so people who only need to copy from archive only have ‘read’ access will further reduce these mistakes.