Configuration file download

claude · March 5, 2025, 4:29pm

Is there a way to periodically and automatically download the configuration file?

NugentS · March 5, 2025, 4:31pm

Yes

Several ways.

multi-report is one (from @joeschmuck). There are others

awalkerix · March 5, 2025, 4:34pm

Note that you also need to make sure you’re backing up the “secret seed”. Uploading a config without the encryption key will result in many areas of nas being broken.

claude · March 5, 2025, 4:50pm

I looked at the script file (multi-report) and it seems to do a lot of things other than downloading the config file. It doesn’t seem easy to use to me.
Isn’t there an easier way?

Redcoat · March 5, 2025, 5:08pm

I strongly suggest that you install and deploy it “just” with the defaults and try it out before you judge Multi-Report unworthy.

Don’t be overwhelmed by the copious (excellent) documentation!

claude · March 5, 2025, 8:33pm

Wow, I just said it seemed too complicated for my limited skills.
But you’re right, I’ll try Multi-Report and give my opinion afterwards.
PS : I am French and my English is also limited, maybe I did not understand the meaning of “unworthy”.

dan · March 5, 2025, 9:15pm

It does, and they’re all things you should be doing.

Redcoat · March 5, 2025, 9:30pm

Thanks, this Brit had no intention to offend. In the context of my sentence, I meant it to be read as “not deserving of your trial”.
I hope that you enjoy it, find it useful and informative.

essinghigh · March 5, 2025, 10:33pm

Thought I’d take the opportunity to try and figure out how the WebUI calls the backup job seeing as I could not find any reference to it at all online, this differs to Multi-Report which creates a backup zip from /data/freenas-v1.db and /data/pwenc_secret, same end result though and multi-report has a lot of bells and whistles you should use it for!

python3 configuration_backup.py --output-dir /mnt/tank/dataset/whatever

root@truenas[.../truenas-scripts/configuration-backup]# python3 configuration_backup.py --output-dir /mnt/data/SMB/Backups 
Requesting download via midclt ...
Download response received:
ID: 64896
Path: /_download/64896?auth_token=<snip>
Retrieving UI port ...
UI port: 81
Download URL: http://localhost:81/_download/64896?auth_token=<snip>
Downloading file ...
Download successful. File saved to: /mnt/data/SMB/Backups/truenas-25.04-BETA.1-20250305222506.tar

Have set up a cronjob for this and it seems to run fine.
0 3 * * * /usr/bin/python3 /mnt/data/bin/truenas-scripts/configuration-backup/configuration_backup.py --output-dir /mnt/data/SMB/Backups > /dev/null

claude · March 6, 2025, 10:13am

It works very well. The cronjob launched directly too. I will see the schedule tomorrow morning.
Thanks.
I will test Multi-Report quickly.

dan · March 6, 2025, 1:47pm

I wonder if it really is, though. That’s the method that pretty much every database backup script I’ve seen has used, back to Cyberjock’s scripts 10+ years ago on the old forum. But none of them do anything to lock the database or otherwise ensure that it’s in a consistent state. The database isn’t modified all that often, so it probably isn’t very likely to be an issue, but…

claude · March 6, 2025, 2:46pm

Hello dan I understand what you’re saying. Does Multi-Report lock the database?
But at 3 o’clock at night there is less risk of database modification, but…

essinghigh · March 6, 2025, 3:40pm

I can’t say for sure that the midclt call does this either. From a very brief read it looks like it doesn’t:

github.com/truenas/middleware

src/middlewared/middlewared/plugins/config.py

fa9e4103e


      
          @private
          def save_tar_file(self, options, job):
              with tempfile.NamedTemporaryFile(delete=True) as ntf:
                  with tarfile.open(ntf.name, 'w') as tar:
                      files = {'freenas-v1.db': FREENAS_DATABASE}
                      if options['secretseed']:
                          files['pwenc_secret'] = CONFIG_FILES['pwenc_secret']
                      if options['root_authorized_keys'] and os.path.exists(CONFIG_FILES['admin_authorized_keys']):
                          files['admin_authorized_keys'] = CONFIG_FILES['admin_authorized_keys']
                      if options['root_authorized_keys'] and os.path.exists(CONFIG_FILES['truenas_admin_authorized_keys']):
                          files['truenas_admin_authorized_keys'] = CONFIG_FILES['truenas_admin_authorized_keys']
                      if options['root_authorized_keys'] and os.path.exists(CONFIG_FILES['root_authorized_keys']):
                          files['root_authorized_keys'] = CONFIG_FILES['root_authorized_keys']
                      for arcname, path in files.items():
                          tar.add(path, arcname=arcname)
          
                  with open(ntf.name, 'rb') as f:
                      shutil.copyfileobj(f, job.pipes.output.w)

But entirely possible I could be wrong.

claude · March 15, 2025, 7:53am

@essinghigh
Since I upgraded to 25.04 RC1 the script no longer works. Tested in the shell.

essinghigh · March 15, 2025, 10:15am

This has stumped me for most of this morning, absolutely no idea what has changed from BETA to RC to break this - seems no matter what I do I get a 401 response, yet the WebUI works fine.
Have raised a ticket but probably outside of scope…

essinghigh · March 15, 2025, 7:24pm

Rewritten using the websocket API due to /_download now using strict origin matching. Thanks for the pointer @awalkerix.

github.com/essinghigh/truenas-scripts

configuration-backup/configuration_backup_websocket.py

main

import argparse
import os
import json
import subprocess
from truenas_api_client import Client
from datetime import datetime

def main():
    parser = argparse.ArgumentParser(description="Automated TrueNAS configuration backup utility.")
    parser.add_argument("--output-dir", required=True, help="Directory to save the backup file. Will be created if it does not exist.")
    args = parser.parse_args()

    token_cmd = ["midclt", "call", "auth.generate_token", "30", "{}", "false", "false"]
    result = subprocess.check_output(token_cmd, stderr=subprocess.STDOUT)
    token = result.decode().strip()
    print("Generated token:", token)

    ui_port_cmd = ["midclt", "call", "system.general.config"]
    result = subprocess.check_output(ui_port_cmd, stderr=subprocess.STDOUT)
    ui_port = json.loads(result.decode().strip()).get("ui_port")

This file has been truncated. show original

root@truenas[.../truenas-scripts/configuration-backup]# python3 configuration_backup_websocket.py --output-dir .
Login result: True
Download successful. File saved to: ./truenas-25.04-RC.1-20250315192338.tar

awalkerix · March 15, 2025, 7:34pm

    token_cmd = ["midclt", "call", "auth.generate_token", "30", "{}", "false", "false"]
    result = subprocess.check_output(token_cmd, stderr=subprocess.STDOUT)
    token = result.decode().strip()
    # print("Generated token:", token)

    ui_port_cmd = ["midclt", "call", "system.general.config"]
    result = subprocess.check_output(ui_port_cmd, stderr=subprocess.STDOUT)
    ui_port = json.loads(result.decode().strip()).get("ui_port")
    # print("UI port:", ui_port)

Since you’re already using our middleware client you can do something like this with your script:

    with Client() as c:
       # Connect with AF_UNIX socket to get auth token and UI port
       token = c.call('auth.generate_token', 30, {}, False, True)  # Single-use token
       ui_port = c.call('system.general.config')['ui_port']

    with Client(uri=f"wss://localhost:{ui_port}/websocket", verify_ssl=False) as c:
        result = c.call("auth.login_with_token", token)

In this part you’ll want to adjust the uri depending on TrueNAS version. Starting in 25.04:

f"wss://localhost:{ui_port}/api/current"

essinghigh · March 15, 2025, 7:51pm

Cheers, first time interacting with the middleware client and websockets, only managed to figure it out reading dan’s deploy-freenas script!
I’ve updated this based on your suggestions and looks all good, thanks!

claude · March 15, 2025, 7:52pm

It works well. Thanks to @essinghigh , enthusiast and expert.

awalkerix · March 15, 2025, 7:56pm

I’m slowly adding code stubs in the examples dir in our api_client repo. api_client/examples at master · truenas/api_client · GitHub.

Let me know if there’s anything in particular for which you need an example.