Dataset or not?

Robert_Heselwood · April 13, 2024, 4:15pm

I have been using TrueNAS for some time now and I usualy create datasets for file storage etc.
I have been running xeoma cctv software on a linux machine for a few years now. I created datasets for the name of the program, ie xeoma, then 4 other child? datasets, 1 for each of my 4 cameras. This has worked fine and the files produced by xeoma on a local computer are mounted onto TrueNAS.
More recently, like 2 weeks ago we suffered a power cut and the local computer was shutdown immediately along with any running software.
When I started the said computer back up and also xeoma, it did not mount any files onto the NS drive. I have contaed xeoma support, but I think thhey are struggling to find an answer??
Anyway after a few days, I managed to get videos mounted onto the NAS drive under the respective camera name, ut it would not mount any pictures under the xeoma/Pictures/camera name dataset.
Fast forward to today, xeoma is now mounting pictures, but no videos now, only on the local computer.

Just as an experiment, I deleted the Pictures dataset, and I suspect that is why they are now being mounted to my NAS drive.
Should I also delete the camera name data sets , because in xeoma I have the path to store or mount the video, ie /mnt/CCTV/xeoma/ Camera name, which I give it…

Apologies for the loong post but just wanted to give some background information.

Thanks
If xeoma produces vieos and pictures

joeschmuck · April 13, 2024, 4:31pm

This advice has nothing to do specifically with the application you are trying to get working properly but when you had the power failure, did you run a SCRUB?

First: What is the output of zpool -v status ?
Second: Run a Scrub
Third: Once the Scrub has completed, do step 1 again.

Please post the output for the zpool commands.

The goal here is to ensure you have no data corruption and if you do, fix it before continuing forward.

Good luck.

winnielinnie · April 13, 2024, 4:33pm

When dealing with software and technology, you have to be precise with your terminology.

I’m having difficultly even understanding your setup to begin with.

What do you mean by “mount”? Are these videos being recorded to a separate computer, and then “shared” via SMB/NFS to the NAS server somehow?

Where does TrueNAS factor into this? Is it the direct/primary storage for these videos?

Do you mean that you have TrueNAS expose SMB shares that your local computer accesses? The software runs on the local computer and uses these SMB shares as network drives?

And why would you just destroy a dataset that supposedly has your files to “test” something? That’s not a good habit to develop, even if you’re currently “troubleshooting”. You would want to resolve the issue without resorting to destructions, to emulate the real-world use of your NAS.

Robert_Heselwood · April 13, 2024, 4:52pm

@winnielinnie
Ok, xeoma is running on a separate local computer and in the /ect/fstab file I have:
192.168.0.42:/mnt/CCTV/xeoma /mnt/CCTV/xeoma nfs credentials=/home/rob/.credentials

The ip addresss is TrueNAS, and nfs is the share type.
The files produced by xeoma are recorded to the local computer, then mounted onto my NAS drive
I want to mount the recorded file onto my NAS drive so that I can view them from any other device in my house, without having to go to the local computer all the time.
You could say that TrueNAs is the secondary storage??.

Regarding deleting the said dadasets, should I now add them back.

Thanks

Robert_Heselwood · April 13, 2024, 4:56pm

@joeschmuck
this is the output:

root@freenas:~ # zpool -v status
unrecognized command ‘-v’
usage: zpool command args …
where ‘command’ is one of the following:

    version

    create [-fnd] [-o property=value] ...
        [-O file-system-property=value] ...
        [-m mountpoint] [-R root] <pool> <vdev> ...
    destroy [-f] <pool>

    add [-fgLnP] [-o property=value] <pool> <vdev> ...
    remove [-npsw] <pool> <device> ...

    labelclear [-f] <vdev>

    checkpoint [-d [-w]] <pool> ...

    list [-gHLpPv] [-o property[,...]] [-T d|u] [pool] ...
        [interval [count]]
    iostat [[[-c [script1,script2,...][-lq]]|[-rw]] [-T d | u] [-ghHLpPvy]
        [[pool ...]|[pool vdev ...]|[vdev ...]] [[-n] interval [count]]
    status [-c [script1,script2,...]] [-igLpPstvxD]  [-T d|u] [pool] ...
        [interval [count]]

    online [-e] <pool> <device> ...
    offline [-f] [-t] <pool> <device> ...
    clear [-nF] <pool> [device]
    reopen [-n] <pool>

    attach [-fsw] [-o property=value] <pool> <device> <new-device>
    detach <pool> <device>
    replace [-fsw] [-o property=value] <pool> <device> [new-device]
    split [-gLnPl] [-R altroot] [-o mntopts]
        [-o property=value] <pool> <newpool> [<device> ...]

    initialize [-c | -s | -u] [-w] <pool> [<device> ...]
    resilver <pool> ...
    scrub [-s | -p] [-w] <pool> ...
    trim [-dw] [-r <rate>] [-c | -s] <pool> [<device> ...]

    import [-d dir] [-D]
    import [-o mntopts] [-o property=value] ...
        [-d dir | -c cachefile] [-D] [-l] [-f] [-m] [-N] [-R root] [-F [-n]] -a
    import [-o mntopts] [-o property=value] ...
        [-d dir | -c cachefile] [-D] [-l] [-f] [-m] [-N] [-R root] [-F [-n]]
        [--rewind-to-checkpoint] <pool | id> [newpool]
    export [-af] <pool> ...
    upgrade
    upgrade -v
    upgrade [-V version] <-a | pool ...>
    reguid <pool>

    history [-il] [<pool>] ...
    events [-vHf [pool] | -c]

    get [-Hp] [-o "all" | field[,...]]

What should I use?
Thanks

winnielinnie · April 13, 2024, 5:06pm

NFS? Credentials? Did you mean to use cifs (“SMB”)?

If you’re using NFS, then you would need to also export (from the host) and mount (at the client) the paths to each discrete filesystem (“child dataset”) as well.

Otherwise, such child datasets (nested under CCTV/xeoma) will be seen and treated as plain folders by your client. Hence, anything written to them will be written inside a folder within the CCTV/xeoma dataset. But because you also have child datasets with the same names as the subfolders, they essentially “overlap” and hide the contents of xeoma’s subfolders.

chuck32 · April 13, 2024, 5:29pm

There was a typo in the command, it’s

zpool status -v

Robert_Heselwood · April 13, 2024, 5:35pm

Warning: the supported mechanisms for making configuration changes
are the TrueNAS WebUI and API exclusively. ALL OTHERS ARE
NOT SUPPORTED AND WILL RESULT IN UNDEFINED BEHAVIOR AND MAY
RESULT IN SYSTEM FAILURE.

root@freenas:~ # zpool status -v
pool: Backup_Data
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using ‘zpool upgrade’. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 01:48:10 with 0 errors on Sun Mar 3 01:48:11 2024
config:

    NAME                                            STATE     READ WRITE CKSUM
    Backup_Data                                     ONLINE       0     0 0
      mirror-0                                      ONLINE       0     0 0
        gptid/ce563f3f-528e-11e6-9216-0cc47a0c1eef  ONLINE       0     0 0
        gptid/cf115aaa-528e-11e6-9216-0cc47a0c1eef  ONLINE       0     0 0

errors: No known data errors

pool: CCTV
state: ONLINE
scan: scrub repaired 0B in 00:01:00 with 0 errors on Mon Apr 8 09:55:52 2024
config:

    NAME                                          STATE     READ WRITE CKSUM
    CCTV                                          ONLINE       0     0     0
      gptid/8a645d77-1f1b-11ee-9297-0cc47a0c1eef  ONLINE       0     0     0

errors: No known data errors

pool: WinShare
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using ‘zpool upgrade’. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 01:19:09 with 0 errors on Sun Mar 10 01:19:09 2024
config:

    NAME                                            STATE     READ WRITE CKSUM
    WinShare                                        ONLINE       0     0 0
      raidz1-0                                      ONLINE       0     0 0
        gptid/434ebf5b-4792-11e6-a4b6-0cc47a0c1eef  ONLINE       0     0 0
        gptid/441279f9-4792-11e6-a4b6-0cc47a0c1eef  ONLINE       0     0 0
        gptid/44db1bfb-4792-11e6-a4b6-0cc47a0c1eef  ONLINE       0     0 0
        gptid/459c45ee-4792-11e6-a4b6-0cc47a0c1eef  ONLINE       0     0 0
        gptid/464e7612-4792-11e6-a4b6-0cc47a0c1eef  ONLINE       0     0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using ‘zpool upgrade’. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 00:02:02 with 0 errors on Wed Apr 10 03:47:02 2024
config:

    NAME          STATE     READ WRITE CKSUM

chuck32 · April 13, 2024, 5:42pm

@Robert_Heselwood good, now run a scrub on your pools.

Robert_Heselwood · April 13, 2024, 5:50pm

@joeschmuck ,
I’ve added the CCTV pool to be scrubbed but it won’t be done until midnight, unles it can be done now??

I did a scrub on 8th April right after the power came back on, and it was ok.

Now I am trying to add a dataset back , which I deleted but I get an error saying:
Path /mnt/CCTV/xeoma/Pictures already exists

chuck32 · April 13, 2024, 5:58pm

Depending on your hardware / pool usage this will take some time there’s no way around it.

Edit: didn’t check all dates. Cctv seems to have been scrubbed after the power loss (8 the April) so you should be good there.

Scrub the other pools though.

Robert_Heselwood · April 13, 2024, 6:03pm

@winnielinnie ,
oops, it was a typo,
this is the correct line in the local computer /etc/fstab file:
192.168.0.42:/mnt/CCTV/xeoma /mnt/CCTV/xeoma nfs user=root,rw,nfsver=3

Davvo · April 13, 2024, 6:08pm

zpool scrub CCTV

joeschmuck · April 13, 2024, 6:09pm

To manually run a scrub, two ways:

TrueNAS GUI, list your pools (under storage), and there is a Gear icon to the right of the pool name. Click and locate Scrub, click. Wait.
I prefer using the CLI, open an SSH window or the GUI Shell and type scrub poolname where poolname = your pools name.

Not sure you really need to run another scrub, but that is how to do it.

winnielinnie · April 13, 2024, 6:41pm

I believe the root of the problem (no pun intended) is @Robert_Heselwood has folders and datasets confused. See this reply from earlier:

This is a common misconception about folders vs datasets, and subfolders vs child datasets.

As far as NFS is concerned, you must export every single nested child dataset (“filesystem”) you want accessed by the client. (For the reason I explained above.)

Robert_Heselwood · April 13, 2024, 7:22pm

@winnielinnie
ok thanks, I wasn’t aware of that ’ every single nested child dataset with nfs.
To be honet, it did work fine before the power cut, it is just since that 1 time only videos were mounted to my NAS drive, then or now only Pictures. But amazingly after a reboot of the local computer, the videos are now mounted but not the pictures!!!
I tried to re add the Pictures dataset earlier but it said that:
Path /mnt/CCTV/xeoma/Pictures already exists
But it doesn’t as I deleted it.

Thanks

winnielinnie · April 13, 2024, 7:28pm

Do you actually need separate datasets (“filesystems”) for each camera?

If not, you can make this simple and get rid of the child datasets, and just use subfolders. This shouldn’t be an issue with NFS.

(Just don’tr accidentally destroy anything. You need to have a clear understanding where your files really are, and what you’re doing.)