[SOLVED] Proxmox LXC: For the life of me, I cannot get write access to TrueNAS NFS

TrueNAS General
1

Please let me know if you know an official documentation explaining this coz I have spent 4h tonight and got nowhere.

Back history: I tried unRAID, everything was beautiful until it could not keep NFS/SMB connection up from Proxmox, it drops everything if you have NVMe cache, it must be HDD only, it took a day to transfer 800GB coz of dramas.

TrueNAS: So far the connections aren’t dropping but I cannot use them either :slight_smile:

Goal: TrueNAS NFS >> Proxmox mount >> LXC mount point(mp) >> LXC write/read
This worked before with unRAID but like I said, no NVMe cache or the connections drop.

Proxmox: mount -t nfs 10.19.0.12:/mnt/data/jellyfin/ /mnt/pve/jellyfin/
Proxmox has write access and all
Proxmox: pct set 100 -mp0 /mnt/pve/jellyfin/,mp=/data/,shared=1

“shared=1” is the only way for the “data” folder within LXC to have “root:media” ownership, otherwise, I get a “nobody:nogroup”.
That was not required with unRAID for the mount to work.
LXC gets no access

”sudo -u {radarr,qbittorrent,jellyfin,sonarr} touch test” gets access denied no matter what I try.
Everybody is within “media” group.

TrueNAS:

  1. I have created an user with matching UID
  2. I have added user to “built_users” group
  3. ACL Editor: I must have tried everything I can think off there
  4. I have tried SMB: mount -t cifs //10.19.0.12/jellyfin/ /mnt/pve/jellyfin/ -o credentials=~/smb-unraid.credentials
  5. Using Proxmox GUI is worse coz it takes away all the control and only works on its level, mapping to the LXC have the same issue.

I have watched a ton of YouTube videos and everybody does things differently and they are heavily Windows user, I run Linux everywhere, so their steps are very basic.
Folks can create shares without permission at all and it all works, mine gets all blocked haha

It should be as simple as:

  1. Proxmox: mount -a
  2. Proxmox: chmod 2775, “root:media”
  3. Proxmox: mount to LXC
  4. LXC: Do their things
  5. Everybody is happy :slight_smile:

I appreciate any help.

Please, I am novice, “wrong again” and alike I got from other post, does not help anybody.
If you cannot help, thanks anyway.

2

when you’Re on the containers webui page top right corner click on configuration and map user/group ids. Ther you can map a local truenas user to a user from inside the lxc. That way you should be able to give the user you use for the nfs connection the apropriate permissions inside the lxc

Edit:
Another option would be to add the truenas_container_unpriv_root user to the acl list of your datasets. That user gets automatically mapped to the containers root user

3

Thanks a lot for the reply.

My issue is that in running everything on Proxmox, VMs and LXC containers, while the communities seems to use TrueNAS/unRAID built-in VM/Containers so things are waaaay easier.

I finally found an awesome coverage by WunderTech of this whole TrueNAS ACL.

I got a few things wrong and made the most novice mistake I don’t do at work and should know better: Give permission to groups and not users.

Things make more sense now to an extent, I’ll give it another try tomorrow and I’m hopeful to get it all working haha

This whole TrueNAS ACL is very complex, it isn’t a bad thing. I prefer the least privilege approach.

Thanks a lot :slight_smile:

4

Do note that unless you’re:

  1. forcing kerberos authentication for NFS
    or
  2. squashing all users / groups to specific accounts (and having NFS server determine group membership)

Server-side permissions are at best considered advisory because the client can determine who it is mounting as. This is simply how SYS auth in NFS works, but many people these days seem to be setting up NFS exports without understanding the protocol and its limitations. This is OK for home use, but if it’s being done at $placeofwork, you should be carefully understand the protocol and how to use it securely.

1 Like
5

Thanks for the heads-up, I do not manage NFS at work otherwise I would have fired myself lmao

Systems and platforms you don’t manage by users but groups.

That LXC container has a “media” group and the users radarr, sonarr, jackett, qbitttorent are member of it making the permission management easier.

Weirdly, with unRAID it all worked and that isn’t necessary good.

With TrueNAS, ACL works as it should, you either get the permission right or you get the permission right. Wunder’s showed everything I got wrong: https://m.youtube.com/watch?v=FNhusHrAzA8&pp=0gcJCcUKAYcqIYzv

I’ll give it another go tonight and see how that plays out.

Thanks a lot

6

Note this is just my own $0.02. I haven’t looked at unraid or other products recently and I’m not a company spokesperson – just a developer. As a developer it’s really trivial to make it so that permissions “just work” all the time. The problem is that it’s almost always by adopting insecure defaults that are not suitable for small or medium businesses.

The world of small and medium business IT in my experience has been one where increasingly people who don’t have training are by various circumstances forced, cajoled, or somewhat naively volunteer to become the “IT guy / gal”. A lot of us are tech enthusiasts and it can be really exciting to set things up and solve problems.

It works great for these businesses because the end result is that a lot of responsibility is shifted to someone who is trying really hard in their free time to hone their skills so that they can handle the responsibilities. I won’t go into the ethics of this, but it can translate to significant savings because skilled people are expensive.

There are some parts of business / enterprise storage that are intractably complex. We could in theory hide it away and adopt a simpler UX, but that can leave a false sense of security and also miss out on an opportunity to help people become better in this situation (where better means a growing skillset and understanding of storage, security, etc).

The end result of this is that we have a product that has some complexity to it that’s not hidden away because if you’re using TrueNAS (Enterprise or CE) in your day job you need to know / understand these things.

I think the balance overall has been good - we don’t hide away complexities and avoid insecure defaults to make things “just work”.

There’s always something new to learn and become better at, and for that matter we’re also constantly trying to improve our UX as well so that it’s easier to learn and manage TrueNAS.

2 Likes
7

Thank you so much for that reply.

I’m DevSecOps and haven’t had to deal explicitly with this raw level of permission since idk, Windows Server 2003 lmao so yeah, it has been some time.

Now everything is cloud based, groups, and what not so you won’t get even close to what I’m trying to do even if the job title says otherwise, period.

At home I only run Linux (even my 3D printer runs Debian Linux) and my current “NAS” is my Linux PC lol, you cannot access its one share haha if not from my GrapheneOS phone (firewall) via SMB, it does requires the user to be added into its conf file, permissions and what not. Moving to NFS is where the fun started haha

I found TrueNAS UI clean, simple without bs, I did mention that and I do not blame my dramas in a tool when it’s clearly an user skill issue haha

That is the beauty in it with TrueNAS, you cannot fake things up, you must know what you are doing with the permission or it won’t work. That isn’t a bad thing, you aren’t getting 3rd party API access with broken permission either so.

The YT that showed me what I’m doing wrong, dude trust TrueNAS to run his business, sure there are backups as you must but you got the point. Under those circumstances, you will get things wrong once, there’s no second.

I’ll give it another try tonight until I get it to work, the NFS shares have not disconnected on me once, that is a big win right there haha

Cheers

8

Yaaaaaaaaaaaaayyy

Only took me near 2h, I mean, better than 4h yesterday to get nowhere.
I can make changes from within the LXC container as long as the user is within the group media.
I can see the files on Proxmox

oot@qbitorrent:/data2# sudo -u qbittorrent touch test1
root@qbitorrent:/data2# sudo -u radarr touch test2
root@qbitorrent:/data2# sudo -u sonarr touch test3
root@qbitorrent:/data2# sudo -u jackett touch test4
root@qbitorrent:/data2# 
exit
root@proxmox03:/mnt/pve# ls -lh htpc/
total 11K
drwxr-xr-x 2 100000 101000 2 Mar 11 19:37 test
-rw-rw-r-- 1 100999 101000 0 Mar 11 19:42 test1
-rw-rw-r-- 1 100996 101000 0 Mar 11 19:42 test2
-rw-r--r-- 1 100103 101000 0 Mar 11 19:42 test3
-rw-rw-r-- 1 100995 101000 0 Mar 11 19:42 test4
root@proxmox03:/mnt/pve#

TrueNAS:

  1. Create the group media with the same GID as to the LXC container: 1000
  2. Dataset > Permissions > root:media
  3. Shared > Advanced > Maproot/Mapall > root:media
  4. sudo mount -t nfs 10.19.0.12:/mnt/data/jellyfin/ /mnt/pve/htpc/
  5. chown -R 100000:101000 htpc/

5th is what was killing me, I must run that to remap the group to the unprivileged LXC containers.
I always forget that sh… and because I was having permission denied, I didn’t even tried to run it once I fixed the ID mappings.

The process is dead simple, if you know what you are doing haha

Copying 600GB over to see what sup before setting snapshots and what not.

Thanks all

9

600GB copied within 2h, no errors or dramas or NFS share going kill mode. THANK YOU!!!

I am sold, I will start drafting a real one with ECC memory and what not.
This 2U has reached its hardware limit anyway haha