TrueNas Scale as a Wireguard client

mago · July 21, 2024, 10:37pm

Hi there,

I’m a new TrueNAS user (Scale), running TrueNAS-SCALE-24.04.2.

This system is running on a remote location, and I want it to connect to a wireguard server, located in my home network.

I’ve tried the following:

using the wg-easy chart: that can’t be used as client.
I have found this post that seems to solve this use case. But it uses the community wireguard chart from TrueChart. I understand using TrueChart is not an option anymore ? Basically anything community based is probably not a good idea right now, as the transition from k3s to native docker will have to be manually managed, right ?
Installing and configuring wireguard manually via shell but … Can’t install apt packages so it’s seems to be a dead end too.

Is there currently any way to configure a wireguard client on TrueNAS Scale ?

Thanks in advance

sfatula · July 22, 2024, 5:04am

Wireguard is already loaded on Scale so yes, manually via shell is quite possible and I am using it myself.

mago · July 23, 2024, 8:14am

Thanks @sfatula I didn’t realized I wireguard was already installed.

Did you need to make any changes to the iptables rules ?

sfatula · July 23, 2024, 9:04am

Just make a conf file and start via post start script. I have the usual postup and postdown stuff like most wireguard configs. Nothing fancy

mago · July 23, 2024, 10:10pm

Thanks, but could you please elaborate ? What post up scripts do you have for the wg0 interface ?

Basically I’ve simply created the /etc/wireguard/wg0.conf file and ran wg-quick up wg0. I’m pretty sure the conf file is correct as it was auto generated by pivpn. Also I have a fedora machine running in the same network that is also a wireguard client of the same server, that uses a similar config file (with different keys / IPs of course)

Troubleshooting:

Output of wg-quick up wg0

root@truenas[~]# wg-quick up wg0 
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 1.1.1.1/24 dev wg0 (redacted IP of the client)
[#] ip link set mtu 1420 up dev wg0
[#] resolvconf -a wg0 -m 0 -x
[#] wg set wg0 fwmark 51820
[#] ip -6 route add ::/0 dev wg0 table 51820
[#] ip -6 rule add not fwmark 51820 table 51820
[#] ip -6 rule add table main suppress_prefixlength 0
[#] nft -f /dev/fd/63
[#] ip -4 route add 0.0.0.0/0 dev wg0 table 51820
[#] ip -4 rule add not fwmark 51820 table 51820
[#] ip -4 rule add table main suppress_prefixlength 0
[#] sysctl -q net.ipv4.conf.all.src_valid_mark=1
[#] nft -f /dev/fd/63

Output of wg:

interface: wg0
  public key: XTtTxZi94CJoZjHHmarkKiz6q2RoM4CBw3C8NS+kRCM=
  private key: (hidden)
  listening port: 59755
  fwmark: 0xca6c

peer: iiek4/zu+yuWiqd/xxQ26Rrlp15Txwkg1MdJHw8kKzo=
  preshared key: (hidden)
  endpoint: <REDACTED>
  allowed ips: 0.0.0.0/0, ::/0
  transfer: 0 B received, 8.67 KiB sent

Endpoint is reachable when VPN is down.

sudo dmesg -wT (after enabling wireguard verbose logging):

[Tue Jul 23 23:23:02 2024] wireguard: wg0: Interface created
[Tue Jul 23 23:23:02 2024] wireguard: wg0: Peer 2 created
[Tue Jul 23 23:23:03 2024] wireguard: wg0: Sending handshake initiation to peer 2 (<REDACTED>)
[Tue Jul 23 23:23:08 2024] wireguard: wg0: Handshake for peer 2 (<REDACTED>) did not complete after 5 seconds, retrying (try 2)
[Tue Jul 23 23:23:08 2024] wireguard: wg0: Sending handshake initiation to peer 2 (<REDACTED>)
[Tue Jul 23 23:23:13 2024] wireguard: wg0: Handshake for peer 2 (<REDACTED>) did not complete after 5 seconds, retrying (try 2)
[Tue Jul 23 23:23:13 2024] wireguard: wg0: Sending handshake initiation to peer 2 (<REDACTED>)
[Tue Jul 23 23:23:18 2024] wireguard: wg0: Handshake for peer 2 (<REDACTED>) did not complete after 5 seconds, retrying (try 3)
[Tue Jul 23 23:23:18 2024] wireguard: wg0: Sending handshake initiation to peer 2 (<REDACTED>)
[Tue Jul 23 23:23:23 2024] wireguard: wg0: Handshake for peer 2

tcpdump -env -i any icmp when pinging wireguard server (IPs redacted 1.1.1.1 is the client, 2.2.2.2 is the server)

23:58:27.333868 wg0   Out ifindex 13 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 62365, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.1 > 2.2.2.2: ICMP echo request, id 25149, seq 1, length 64
23:58:28.365650 wg0   Out ifindex 13 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 62386, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.1 > 2.2.2.2: ICMP echo request, id 25149, seq 2, length 64
23:58:29.389600 wg0   Out ifindex 13 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 62621, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.1 > 2.2.2.2: ICMP echo request, id 25149, seq 3, length 64
23:58:30.413599 wg0   Out ifindex 13 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 62713, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.1 > 2.2.2.2: ICMP echo request, id 25149, seq 4, length 64
23:58:31.437648 wg0   Out ifindex 13 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 62966, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.1 > 2.2.2.2: ICMP echo request, id 25149, seq 5, length 64

So only packages going out, no replies. On the server, I don’t see anything arriving. Not sure where the packages are dropped.

Others:

root@truenas[~]# ip r          
default via 192.168.178.1 dev enp2s0 
10.23.23.0/24 dev wg0 proto kernel scope link src 1.1.1.1 
172.16.0.0/16 dev kube-bridge proto kernel scope link src 172.16.0.1 
192.168.178.0/24 dev enp2s0 proto kernel scope link src 192.168.178.50 
192.168.178.0/24 dev enp1s0 proto kernel scope link src 192.168.178.46 linkdown 
root@truenas[~]# netstat -tunlp | grep 59755
udp        0      0 0.0.0.0:59755           0.0.0.0:*                           -                   
udp6       0      0 :::59755                :::*                                -

So yeah, not sure what I’m missing but I can’t seem to make it work at the moment. Any help appreciated ! Thanks

sfatula · July 24, 2024, 12:52am

I can’t help with any networking issues (they quickly get out of hand and Wireguard can be very frustrating at times), but, sure, in my wg10.conf is:

PostUp = iptables -I FORWARD 1 -i wg10 -j ACCEPT
PostUp = iptables -A FORWARD -o wg10 -j ACCEPT
PostDown = iptables -D INPUT -i wg10 -j ACCEPT
PostDown = iptables -D FORWARD -o wg10 -j ACCEPT

One can make them fancier with only allowing certain traffic through but I do not. Note I am using this for inbound traffic, but outbound works as well. Scale connects to a public facing VPS as I am behind CGNAT.

jct · July 24, 2024, 3:14am

FWIW: I happened to start from the instructions for TrueNAS Core. After migrating the remote sites to SCALE, they’re down to just this Post-Init command:

cp /root/wg0.conf /etc/wireguard/ ; /usr/bin/wg-quick up wg0

I only use these nodes point-to-point for mutual replication, so I don’t have any scripts in the conf file. I will on occasion use ssh tunnels to reach other machines at the remote sites, but that’s an exceptional case for me.

jct · July 24, 2024, 3:23am

So only packages going out, no replies. […] So yeah, not sure what I’m missing but I can’t seem to make it work at the moment. Any help appreciated !

Apologies in advance for asking the “did you plug it in” question, but… you didn’t happen to mention port-forwarding the UDP at your perimeter gateway. Using different UDP ports for your different NAT hosts. Is that part under control?

mago · July 24, 2024, 9:39am

@sfatula thanks, unfortunately that didn’t solve it.

@jct I don’t think I need to configure anything on the gateway for a wireguard client ?

Unfortunately this is critical for my setup and I think I’ll have to ditch TrueNas as a result. Thanks all for the help.

sfatula · July 24, 2024, 6:31pm

Without posting the conf file I don’t think anyone can guess.

You said you had Scale at a remote location, and, wanted to connect to your home wireguard. So, the endpoint from Scale should be home wireguard public IP which should at home be port forwarded to your wireguard server (unless that’s the router itself). OR, you could connect from home wireguard to your new endpoint Scale, in which case unless it’s the router or exposed to the internet, yes, you do need to forward the UDP port.

mago · July 24, 2024, 6:49pm

@sfatula Yes in my case the TrueNAS Scale server is a client that connects to the wireguard server in my home network.

I can confirm that my home network is correctly configured as I have other clients connecting to it successfully.

jct · July 24, 2024, 6:55pm

I don’t think I need to configure anything on the gateway for a wireguard client ?

For one Wireguard client — and only provided other conditions are met in the path. You can greatly improve those odds (and stability across unexpected transitions weeks later) by adding your own explicit port forwarding at one or both ends.

Regular NAT traversal can’t/won’t demultiplex two stateless UDP conversations using the same port number. So at the minimum, you’ll need to assign them different port numbers.

That oversight, of course, is in no way limited to TrueNAS. But good luck out there!

sfatula · July 24, 2024, 7:00pm

So you’d have to post your conf files as otherwise, no one can answer your questions. There’s an error somewhere!

Constantin · July 25, 2024, 8:40am

I have to say, after bashing my head against the wall for weeks with EdgeRouter thanks to undocumented, but necessary, configuration details to get a VPN to function, the ease and simplicity of the built-in WireGuard implementation of Mikrotik was simply astounding, even when using DDNS.

If you cannot get WireGuard to function on the NAS, give the Mikrotik routers and like solutions a look, to create VPNs that also allow you traverse networks. For me, joining multiple local networks had other benefits too, like being able to reach other equipment for SSL certificate dispersal, for example.

mago · July 25, 2024, 10:05am

I now installed debian on this server, after using the exact same wireguard configuration file it works out of the box. FYI here is the config file I’m using on the NAS box - it was auto-generated by pivpn (I’ve edited the IP addresses FYI):

[Interface]
PrivateKey = <REDACTED>
Address = 10.0.0.4/24
DNS = 172.172.172.2

[Peer]
PublicKey = <REDACTED>
PresharedKey = <REDACTED>
Endpoint = <fqdn>:51820
AllowedIPs = 0.0.0.0/0, ::0/0
PersistentKeepalive = 25

sfatula · July 25, 2024, 6:01pm

You have some soft of issue with that config on Truenas obviously. Mine simply gives this, and nothing redacted. Perhaps some of the extra fluff you are adding from pivpn is an issue, no idea, as I said earlier, networking issues are tough as we never have enough info here. This is on Dragonfish, though nothing changed from Cobia. A few things to try - use DNS=1.1.1.1 as a start and the ip not dns name of the endpoint, are you really using ipv6? 10.0.0.4 needs to be unique if that is being re-used from another config. Once you simplify and get it to work, if you can, then worry about adding stuff back. I don’t see the iptables rules in your config either, pretty sure that’s a problem on Scale, don’t think it will work without them given all the other iptables rules in Scale. Can’t think of anything else off hand.

/usr/bin/wg-quick up /mnt/tank/Scripts/Wireguard/wg10.conf
[#] ip link add wg10 type wireguard
[#] wg setconf wg10 /dev/fd/63
[#] ip -4 address add 172.26.0.3/32 dev wg10
[#] ip link set mtu 1404 up dev wg10
[#] resolvconf -a wg10 -m 0 -x
[#] ip -4 route add 172.26.0.1/32 dev wg10
[#] iptables -I FORWARD 1 -i wg10 -j ACCEPT
[#] iptables -A FORWARD -o wg10 -j ACCEPT