FANGTOOTH observations / issues

TrueNAS General
1

I have been using ZFS well pretty much since it came out

for the last almost 10 years… been running NAPPIT / ominos all in one on a ESXI host feeding NFS shares back to ESXI for vm and storage and SMB for NAS duty…

I wanted to try out SCALE in this role and here are first some issues with FANGTOOTH then some observations

1> where did all the performance charts go… ZFS only has an ARC chart… pretty sure there use to be more… like ZIL and ARC stats . not just ARC size… in general I feel that Truenas really does not present enough of its most core function data (ie ZFS) to the user

2> in this all-in-one role, I assign truenas 2 virtual NICS… one for LAN traffic, and one for a software defined internal to ESXI host virtual switch / network for NFS traffic just to ESXI for vm storage and between VMs for fast data … iperf3 reports 15gb/s on this link BUT this second Nic was a bear to set up… took multiple attempts and at least an hour… didnt write down the steps and should have but it was a lot more painful than populating the setup

3> in this all-in-one role or just for stability and performance in general… I wish there were more buttons for stopping un needed services and such from middleware that are not needed… for example… if I just wanted to use ZFS and NAS features… I should be able to have clean interface and not a bunch of deamons and process running for services that are not needed

4> back to Truenas not providing what I consider must have info for a ZFS appliance…

  • when scrubs are running … where the hell is the stats… nowhere to be found without command line. I want to see the typical stats that zpool status would show during a scrub at a minimum
  • disk smart info… non existent without command line… just seeing a pass is not what I want to see if I am looking for disk health… or trending relocated sectors etc…
  • arcstat and zilstat… you have them in the command line… why dont you have these critical stats presented in the gui…
  • Disk charts… labels are verbose and almost useless… I prefer unlike most… to have stats, locations, drives data presented by serial number as the serial number is what I tag the disk slide with… also … no way to indentify a drive in an array from command line for replacement to light up its activity led… gents… have you never heard of dd? in DISKS each drive should have a ID/DD button… press it… array sends dd commands to the drive… lights blink… ah … we found the right drive… clearly these devs have never looked at a wall of drives looking for the one to replace using truenas…

5> nvme is all but useless in the gui… no nvme smart… not even temps… almost everything needs to be done in the command line with nvmetool

In general ZFS has always been a very command line driven storage solution … and most people not even todays DEVs understand how to use it properly…

that said. … nobody had done a great job of a GUI…

Take the solaris storage appliance demo / trainer for a spin… that one is pretty good… especially with some bolt ons from their dev labs… you could actually log and graph latency changes at the drives if loud noise or minor vibrations were present…

Nappit does a ok job of giving more stats… but the '80s called and want their GUI back… its got a dated flavor and kinda klunky … 1 step better than command line… BUT unlike truenas … I dont have to use the command line hardly at all to see the critical ZFS and disk stats that I need.

Truenas devs hate it when we use command line… but their product design demands it for anyone other than a housewife running a NAS for their kids homework storage …

1 Like
2

Were you using the UI or the console tool?

I too have multiple interfaces, one of which is a bonded pair of 10Gbps SFP+ ports for my LAN. I had no issues setting up either the primary or secondary (DMZ on 1Gbps ethernet) interfaces. BUT … it is different than FreeBSD or Solaris, I found the tutorial here Interface Configurations | TrueNAS Documentation Hub more useful than the UI reference. I probably could have eventually figured it out without the tutorial, but the tutorial gave me the clues I needed to get it done right the first time.

3

Many of the services (Apps/Docker, Instances/Incus, Shares) do not start at boot until you first configure them. If you configure Apps to play with, you can unconfigure and that service will stop and not restart.

Most of the middleware that is running is TrueNAS. It is the layer that provides the services that enable the browser (and now API) based user interfaces, provide logging, provide monitoring, provide notifications, and much more.

4

In the Storage screen the ZFS Health brick for the zpool in question will display estimated time to completion. I agree that having all of the underlying information would be nice for power users like us, but for the general purpose users (who greatly outnumber us) they need to keep the UI simple.

I am hopeful that one day there will be a global switch to enable power user mode :slight_smile:

5

Paul

thanks for taking the time to reply… and I know who you are so… thanks for the engagement…

I was doing as much as possible during provision from the gui… obviously some steps were likely done initially from the initial iso boot interview and setup …

adding the second ‘virtual’ Nic for the internal network … the vm was provisioned with 2 nics during ISO install… and the setup I believe I was trying to do from they 25.01 release gui… not the beta…

as for ‘power user’ mode…
the power of ZFS rests in 2 things… the unbelievable flexibility of provisioning options, monitoring… and lastly absolute data integrity . or at least that last one was the SUN/Oracle focus until openzfs came along and it became a race of feature additions and keep-up with the joneses (oracle)… oracle ZFS is still the gold standard… but since they locked it up… openzfs is the only game left in town … the ominos guys are slow… but methodical … but always behind in zfs feature implementation …and even drivers… but they focus on reliability

anywho… back to ZFS monitoring … it should not be a power feature or very hard to show during a scrub… the basic stats as that is the first giveaway of an issue… and I can see faults as they are piling up if there is an issue long before the scrub completes… maybe if there are enough issues on a bit pool right at the start… I might have to cancel the scrub thrashing the drives… and take steps to remediate using the last remaining life of those drives… also … plenty of gui real estate your not using … should not have to go to the reports page… if I pull up a dataset… you could have some live stats right there…

also you didnt mention were the zfs graphs are reports went in fangtooth… are you brining these ZFS charts back… just having an arc size chart is well… useless… as the arc typically will just grow to its defined size and stay there… but again… tons of ZFS stats that should be historically available by chart to tune and monitor

also … no NFS charts any more… or other NFS or SMB stats…

while I am at it… this is a BIG one…

you guys ever going to get around to replacing or at least giving the option of using KSMBD to replace samba… as KSMBD performance is WAY better being run in kernel space just like the original SOLARIS ZFS intended …

6

If you are new to TrueNAS, there’s been some changes.

Netdata reporting was moved to an App for security reasons: Netdata | TrueNAS Apps Market
Try it out and provide your feedback.

The WebUI only uses the TrueNAS API which was REST and rather static. Netdata bypasses that API.

Fangtooth enables an improved websocket API. That API is better for relaying real-time info and will enable improved integrated UI in the future. api.truenas.com Power users are welcome to use this API for super cool things (please share).

Future versions of TrueNAS can improve the WebUI with that. Any feedback can be made on Feature Requests. category.

7

@duecedriver
You make quite a few good points which I do agree with. I too would like to see the stats and in a better (what I would like) layout.

My little script, which is not part of TrueNAS, does perform NVMe tests, and a few other things. Look for Multi-Report in the resources. But it does not address a great deal of what you are talking about.

As @Captain_Morgan has said, there is an API which those advanced users can utilize to create a new personal interface/GUI if you desire and a lot more. I’m not a programmer so that is out of my reach for now. Also there has been a lot of hardening of TrueNAS for security. It has bit me as well but I’d rather have a more secure system than one a hacker could easily infiltrate.

If you are able and willing to create some new GUI interface or add-ons, please do share. There are a lot of us out here who love data that allows us to make smarter decisions. That wall of drives part you mentioned, I cannot imagine but it does not sound like fun.