How to install Urbackup in a jail with ZFS (Instructions)

Apps and Virtualization
, ,
1

Introduction
These instructions describe how to install UrBackup inside a jail and allowing it to directly manage the assigned ZFS dataset from within a jail. This allows UrBackup take advantage of the ZFS functions such as snapshots and deduplication which greatly improves the space management for backups.

This tutorial is lengthy, as I tried to describe as many steps as I could, but it is not complicated. I divided it in the following parts in an effort to make it easier to follow:

Part I – Datasets
Part II – Jail creation
Part III – Installation
Part IV – Recover data from TrueNAS host
Part V – Upgrades.

It is worth noting that this method assigns some “jailed datasets” to be used by UrBackup. The “jailed” datasets will be managed exclusively by the jail and will become unavailable to the truenas Host. This means that the Truenas host will not be able to browse or “ls” into the datasets (from terminal) or edit its permissions (from the web ui).

This is a normal behavior, but it caught me by surprise as this is my first time using this feature. However, data saved in these “jailed” datasets are not being held hostage by the jail. I will provide Instructions at the end of the tutorial on how to recover and access the datasets from the Truenas host in case the jail is ever deleted or corrupted.

Some other important things to point out are:

  • These instructions use both the web UI and the CLI terminal.
  • I assume the reader has some understanding on how folder paths work and has basic knowledge on how to use the command line. All required commands will be provided on this tutorial but they may have to be adjusted to match the user’s environment.
  • Creation of datasets is beyond the scope of this tutorial. Please refer to Creating Datasets | TrueNAS Documentation Hub if user is interested in learning about this topic.
  • I assume you already know how to use text editors in a terminal (vi or nano for example). The use of these text editors is beyond the scope of this tutorial.
  • The jail in this tutorial will use a static IP 10.192.168.27/24. Please adjust the ip to match your system.
  • My main pool is called “Vault”. Please adjust the name to match your system.
  • My main Dataset is called “Apps”. All new datasets listed in part I of this tutorial will be created as child datasets of “Apps”. Please adjust to match your system.
  • Urbackup will run as “root” user to avoid permission problems.
  • I put a lot of time into researching and writing this set of instructions but mistakes should be expected. I welcome any comments, suggestions or corrections as the last thing I want is to spread inaccurate information.
  • While this method worked for me, I cannot guarantee that it will work in every environment. I don’t do this for a living (far from it) and I didn’t develop Urbackup, TrueNAS or FreeBSD. By following this set of instructions you assume responsibility for researching anything you don’t understand and for taking care of your own data.

This tutorial is based on information found on the following resources:

https://www.urbackup.org/administration_manual.html

Lets’s begin.

1 Like
2

Part I - Datasets

  1. Create the datasets for the persistent data for UrBackup. In total we will need 5 new datasets, similar to the last 5 items shown on the following dataset-tree. (Please refer to introduction for explanation on the first 2 items).

image

I will now explain why we need these datasets and what kind of data will be saved in them.

  1. According to the instructions for deploying urbackup in a docker container, the persistent data we care about are the urbackup “database” and the “backup” files.
    https://hub.docker.com/r/uroni/urbackup-server
    image

Based on this, we will use the dataset
/mnt/Vault/Apps/urbackup-database for… well the database. And, in part II we will use a mountpoint inside the jail to point this dataset to /usr/local/var/urbackup. Note, the mountpoint for this dataset is a bit different from the path on the screenshoot above due to the slight differences of linux vs FreeBSD.

Next, we will use the dataset
/mnt/Vault/Apps/urbackup/backup for the “backup” files. And, in part II we will mount /mnt/Vault/Apps/urbackup as a “jailed” dataset which will also mount all of its child datasets.

In addition, since we will use ZFS, we will need separate datasets for “files” backups and “image backups as explained in the UrBackup administration manual (Section 11.7.1) UrBackup - Server administration manual

image
image

For these we will use the following datasets
/mnt/Vault/Apps/urbackup/images
/mnt/Vault/Apps/urbackup/files

  1. Deduplication, checksum & record size.
    Enable deduplication with checksum SHA512 for the dataset
    /mnt/Apps/Vault/urbackup/files

This is based on the UrBackup administration manual (Section 6.3.1)
image

Similarly, enable deduplication with checksum SHA256 and record size 512K for the dataset
/mnt/Apps/Vault/urbackup/images

This is based on the UrBackup administration manual (Section 6.3.2)
image

  1. Summary

The following chart has a summary of the datasets we need for this installation.

image
image863×190 5.68 KB

3

Part II – Jail creation

  1. Create a new Jail via the TrueNAS web UI by going to “Jails” and clicking on the “Add” button.

image

  1. Click on the “Advanced Jail Creation” button
    image

  2. Modify as follows (Adjust to match your system as necessary):
    3a. Basic properties
    i. Under Name type “urbackup”
    ii. Under Jail type select “Clone Jail”
    iii. Under Release select the most recent release. In our case “13.2”
    iv. Enable the option “VNET”
    v. Under vnet_default_interface, select your LAN interface. In my case “ix1: Jails, VMs and plugins” which I have previously setup to “Disable hardware offloading”.
    vi. Under IPv4 Interface select “vnet0”
    vii. In IPv4 Address, type the ip we want for this jail. In this case “10.192.168.27”
    viii. Under IPv4 Netmask select 24
    ix. Under Ipv4 Default Router type the ip of our router. In our case “10.192.168.1”
    x. Enable the option “Auto-start”

3b. Jail Properties
i. Enable the options

  1. “allow_set_hostnames”
  2. “allow_raw_sockets”
  3. “allow_mount”
  4. “allow_mount_zfs”

3c. Network properties
i. Under host_hostname type “urbackup”

3d. Custom Properties
i. Leave everything by default.

3e. Click save.

image
image
image
image

  1. Start the jail

image

  1. Go to the TrueNAS terminal.

  2. Get into the urbackup jail by

iocage console urbackup

Notice how the terminal prompt changed from “root@truenas-hostname” to “root@urbackup”
image

This means that this terminal is for the urbackup jail and not the TrueNAS host.

  1. Create a local folder to be used as mounting point for the database dataset we created.
mkdir -p /usr/local/var/urbackup
  1. Exit the jail and go back to the TrueNAS terminal.
exit
  1. Stop the urbackup jail via the TrueNAS web UI or by the CLI using the command:
iocage stop urbackup
  1. On the Truenas web UI, map the folder in the jail (the one we created in step 8) to the database dataset we created on step 1. To do so, click on the button “Mount points” for the jail “urbackup”.

image

  1. Click on “Actions” then “Add”
    image

  2. In the “Source” indicate the location for the dataset outside the jail, in the “Destination” indicate the jail folder that we should map to the “Source folder”. DO NOT enable “Read-only” option. When done, click the “submit button”.

The end result should look like

image

  1. When that is done, “Edit” the jail settings.

image

  1. Go to “Custom properties” and edit as follows:
    14a. Custom Properties -->Enable the options:
    i “host_time”
    ii “jail_zfs”

14b. In the field called jail_zfs_dataset type ”Apps/urbackup” (ADJUST THIS TO MATCH YOUR ENVIROMENT). Note, this is the “jailed” dataset and will automatically be mounted along with all its child datasets.

14c. Click “Save”
image

  1. Start the jail
    image

  2. Go to the TrueNAS terminal.

  3. Get back into the urbackup jail by

iocage console urbackup
  1. Verify that the datasets are mounted by using the command
zfs list

The end result should be a list of the datasets we created on Part 1. The output should be Similar to:
image

  1. Verify that the filesystems of the datasets are mounted by using the command
df -h

the end result should be similar to

image

Great, we are done with the most difficult part of this process.
If for some reason your datasets or filesystems are not mounted, please go back and doublecheck your steps.

Please do not proceed to the next part until datasets are mounted properly inside the jail.

4

Part III - Installation
According to freshports.org, at the time of writing this tutorial, the official UrBackup port has no maintainer
image

In addition, according to UrBackup - Install UrBackup Server on FreeNAS “The FreeBSD port is currently broken”
image

Because of this, we will install UrBackup from the source. This process looks intimidating but it is not that complicated. Everything has been organized in a sequential order. Just copy the commands provided below and adjust them as necessary to match your environment.

  1. Go to the TrueNAS terminal.

  2. Get into the urbackup jail by

iocage console urbackup
  1. Update the packages in the jail
pkg update
  1. Install the packages needed by UrBackup: (respond “y” if prompted)
pkg install -y curl wget
  1. Download the source code for UrBackup. Adjust to the latest version which, at the time of this writing is 2.5.33
wget https://hndl.urbackup.org/Server/2.5.33/urbackup-server-2.5.33.tar.gz
  1. Extract the compressed file we just downloaded:
tar -xf urbackup-server-2.5.33.tar.gz
  1. Delete the compressed file we downloaded in step 5.
rm *.gz
  1. Navigate into the newly created folder:
cd urbackup*
  1. Then
./configure --enable-embedded-cryptopp
  1. Then compile the source code with make using 4 parallel jobs to make the process faster. Feel free to adjust the number of jobs to match the limitations of your system. On my server this process takes about 20 mins:
make -j4
  1. Install UrBackup from the compiled code:
make install
  1. To test, run the following command
urbackupsrv run -u root

  1. Open a web browser and go to the webpage http://IP-OF-JAIL:55414. (in my case http://10.192.168.27:55414). You should see the UrBackup web interface.
    If not, please go back and check your steps.

  2. The first time we open urbackup, it will give us a warning that the backup folder is not accessible
    image

  3. To fix this go to settings–>General–>server and in the field “Backup Storage path put in the following path “/mnt/Vault/Apps/urbackup/backup” (without quotes). Note, this is one of the datasets we created in part I of this tutorial. PLEASE ADJUST TO MATCH THE DATASET ON YOUR SYSTEM.
    image

  4. Scroll down the webpage and click “Save”. The warning should have disappeared.

  5. When done, close the web browser, go back to the jail terminal and stop urbackupsrv by pressing the keys ctrl+c.

  6. Once urbackupsrv stops running, delete the folder we used for decompressing and compiling the source code

cd ..
rm -R /root/urbackup*
  1. To make urbackup run at jail boot, create a file “/etc/rc.local”
touch /etc/rc.local
  1. Open the file we just created in a text editor (this tutorial uses “vi” but you can use other text editor your prefer)
vi /etc/rc.local
  1. Add the following to the text
#!/bin/sh
    /usr/local/bin/urbackupsrv run -d -g 104857600 -u root

When done, save and close the file.

  1. Make the file executable :
chmod +x /etc/rc.local
  1. Get out of the urbackup jail by using the command:
exit

this will take us back to the TrueNAS terminal.

  1. Restart the UrBackup jail via the TrueNAS web UI or via the CLI by using the command:
iocage restart urbackup
  1. Open a web browser and go to the webpage http://IP-OF-JAIL:55414 to verify urbackup is running.

If it is not, STOP and go verify your steps before moving forward.
If UrBackup is running then keep reading.

  1. Close the web browser, go back to the TrueNAS CLI and get into the UrBackup jail
iocage console urbackup
  1. Set ZFS for the image backups by issuing these commands (as advised on the UrBackup administration manual Section 11.7.1 ZFS). Please adjust datasets to match your environment.
mkdir -p /etc/urbackup
echo "Vault/Apps/urbackup/images" > /etc/urbackup/dataset
echo "/mnt/Vault/Apps/urbackup/backup" > /etc/urbackup/backupfolder
  1. Test that the ZFS is working by
urbackup_snapshot_helper test

this will produce a result similar to this
image

  1. Now set ZFS for the file backups by issuing these commands (as advised on the UrBackup administration manual Section 11.7.1 ZFS). Please adjust dataset to match your environment.
echo "Vault/Apps/urbackup/files" > /etc/urbackup/dataset_file
  1. Get out of the urbackup jail by using the command
exit
  1. Restart the UrBackup jail via the TrueNAS web UI or via the CLI by using the command:
iocage restart urbackup

This is the final step for the installation. Urbackup should be up and running. If not, please review your steps.

5

Part IV - Recover data from TrueNAS host
AS I mentioned on the introduction part, this tutorial uses “jailed datasets” which are managed exclusively by the jail.

By doing so, the dataset becomes unavailable to the truenas Host.

If we ever need to recover the data from the dataset from truenas (example, jail is deleted or corrupted),
then we will need to do the following from the Truenas CLI

  1. Go to the TrueNAS terminal.

  2. “Unjail” the urbackup dataset. Please adjust to match your environment).

zfs set jailed=off Vault/Apps/urbackup
  1. Mount the dataset and all its child datasets
zfs mount Vault/Apps/urbackup
zfs mount Vault/Apps/urbackup/backup
zfs mount Vault/Apps/urbackup/files
zfs mount Vault/Apps/urbackup/images

After that, truenas will have total control over the dataset. Feel free to browse them via the CLI, copy its content, create SMB shares or access your data in any way you desire.

6

Part V – Update UrBackup
This is the part that I admittedly have the least amount of knowledge of how to do it properly because I usually don’t install anything from source. Please feel free to provide suggestions on how to make this process better.

This part will focus on how to update the UrBackup software. Updates and upgrades to the jail system is beyond the scope of this tutorial.

The steps are very similar to the installation steps. For this example we assume we will upgrade from UrBackup 2.5.33 to a hypothetical UrBackup 2.6 which I just made up.

  1. Go to the TrueNAS web UI and create snapshots of all the datasets associated with your UrBackup Jail. This will be used in case the update goes wrong and we need to get our previous installation back.

  2. Go to the TrueNAS terminal.

  3. Get into the urbackup jail by

iocage console urbackup
  1. Download the source code for UrBackup. Adjust to the latest version which, for this example would be the hypothetical 2.6
wget https://hndl.urbackup.org/Server/2.6/urbackup-server-2.6.tar.gz
  1. Extract the compressed file we just downloaded:
tar -xf urbackup-server-2.6.tar.gz
  1. Delete the compressed file we downloaded in step 5.
rm *.gz
  1. Navigate into the newly created folder:
cd urbackup*
  1. Then
./configure --enable-embedded-cryptopp
  1. Then compile the source code with make using 4 parallel jobs to make the process faster. Feel free to adjust the number of jobs to match the limitations of your system. On my server this process takes about 20 mins:
make -j4
  1. Install UrBackup from the compiled code:
make install
  1. To test, run the following command
urbackupsrv run -u root

  1. Open a web browser and go to the webpage http://IP-OF-JAIL:55414. (in my case http://10.192.168.27:55414). You should see the UrBackup web interface.
    If not, please recover your previous installation from the snapshots created on step 1.

  2. When done, close the web browser, go back to the jail terminal and stop urbackupsrv by pressing the keys ctrl+c.

  3. Once urbackupsrv stops running, delete the folder we used for decompressing and compiling the source code

cd ..
rm -R /root/urbackup*
  1. Get out of the urbackup jail by using the command:
exit

This will take us back to the TrueNAS terminal.

  1. Restart the UrBackup jail via the TrueNAS web UI or via the CLI by using the command:
iocage restart urbackup
  1. If everything is working properly, delete the snapshots created on step 1.
7

One note about Deduplication. - DO NOT USE DEDUPLICATION
I know i recommended, under Part 1 - Datasets, to enable deduplication on some datasets. That advisement came from my interpretation of the manuals and general information floating on the web.

However, the advisement received on this forum was that deduplication is very resource intensive and usually not worth it. Please refer to Can i use a partition for a SLOG? for more information.

I also asked for additional information on the UrBackup forum but there has been several days since my original post and nobody has said anything yet. Please refer to ZFS Dataset deduplication in TrueNAS CORE if you are curious to see how that post is going.

1 Like
8

I would suggest everyone interested in deduplication to wait until fast deduplication lands.

1 Like