VDEV online(?) but not working

TrueNAS General
1

Hi, I’m new to TrueNAS with the goal to run a low powered Server/NAS having trouble with cache. Setting up the system seemed straight forward since currently I only have one HDD and one nvme for storage. The nvme is intended to run as cache, so that the hdd should be able to stay spun down most of the time. However that does not seem to work. When copying data to the storage pool, speed seems to be limited by the hdd.
On the management website under Storage, it says that the VDEV is offline. When I click on the “manage devices” button in the ‘topology’-section, no drives show up. However, zpool status shows that the cache is online. zpool iostat shows that the cache is in use
Help is appreciated

sudo zpool status
  pool: Tschechow
 state: ONLINE
config:

	NAME                                    STATE     READ WRITE CKSUM
	Tschechow                               ONLINE       0     0     0
	  66***66e-****-****-affb-92b4df54****  ONLINE       0     0     0
	cache
	  60***437-****-****-860a-b4fd3ddf****  ONLINE       0     0     0

errors: No known data errors

  pool: boot-pool
 state: ONLINE
config:

	NAME        STATE     READ WRITE CKSUM
	boot-pool   ONLINE       0     0     0
	  sda3      ONLINE       0     0     0

zpool iostat -v
                                          capacity     operations     bandwidth 
pool                                    alloc   free   read  write   read  write
--------------------------------------  -----  -----  -----  -----  -----  -----
Tschechow                               3.12T  4.14T      0     45      9  37.0M
  66***66e-****-****-affb-92b4df54****  3.12T  4.14T      0     45      9  37.0M
cache                                       -      -      -      -      -      -
  60***437-****-****-860a-b4fd3ddf****  2.96T   785G      0    286    176  35.1M
--------------------------------------  -----  -----  -----  -----  -----  -----
boot-pool                               2.89G   229G      0      2  3.72K  32.3K
  sda3                                  2.89G   229G      0      2  3.72K  32.3K
--------------------------------------  -----  -----  -----  -----  -----  -----

Bildschirmfoto_20250929_211904
Bildschirmfoto_20250929_2119042577×323 49.7 KB

Bildschirmfoto_20250929_210616
Bildschirmfoto_20250929_2106161278×488 23.3 KB

Running TrueNAS Community Edition 25.04.2.4

2

That isn’t how cache works with ZFS. You’d be better off removing that device.

2 Likes
3

I see (and googled zfs cache). Do I understand correctly that VDEV is L2ARC and therefore generally does not cache sequential writes?

4

That VDEV is L2ARC and does not cache any writes AFAIK.

1 Like
5

Yes it is. ZFS is all about data integrity: All writes go to disk.
L2ARC may be thought of as a read cache—read only.
SLOG is NOT a write cache, it is a log which holds copy of incoming sync writes (and sync writes only), up to the value of two transaction groups (txg). You can not “cache” an arbitrary amount of incoming writes somewhere, waiting to commit to permanent storage when it is convenient.

1 Like
6

Thank you for helping to clear up my misunderstanding of cache. :grinning:
Bit of a pity I kinda based my network upgrade to 10G on that wrong idea. :see_no_evil:
How bad would it be two combine 2 nvme and 1 ssd of roughly the same size (~3.8TB) in a raidz1? Can I simply switch out the ssd for an nvme when I have the funds for it and benefit from the speed boost?

7

Yes - its easy enough to swap out a disk the only caveat is that the disk must bethe same size or larger. Not always certain these days.

Your 2*NVMe and 1 * SATA SSD would I imagine have its performance somewhat impacted - but it should work.

8

I hope you have at least 32 GB RAM. Since the primary read cache (ARC) and the sole write cache of ZFS are in RAM, the faster the network, the bigger the txgs, the more RAM you need.

9

Yes I do have 32GB Ram and plan expanding to 48 when installing Apps (Nextclound&PaperlessNg).