Offsite server - security recommendations

General Discussion
, ,
1

Hello all,

I want to move my second machine offsite. Currently it’s at least not in the same apartment as my main machine, but I’d like a bit more physical segregation.
I could place the server at my father in law.

  • Access would be via tailscale in order to avoid opening ports on his network
  • Segregation of his home network and my server would be the goal
  • as far as I know he is not using VLANs nor does he have any kind of advanced network setup going on. It’s just a fritzbox and I assume some generic switches.
  • the server should just be a destination for replication tasks / be an available rsync target for another friend

I’m under the impression that truenas assumes on living in a secure network. I have doubts to just move it to some network I’m not the admin of.

  1. Do I need a physical device that can act as a firewall?
  2. If so, what cheap device could you recommend (EU / Germany)?
  3. Is it possible / enough to disable SSH and use 2FA on the webgui?
  4. Open to other ideas :slight_smile:

If I need additional hardware, low power consumption would be nice.

I was thinking of getting a cheap router for example that would be capable of running pfsense. Connect to the local network on the WAN site and create a LAN that is not accessible from the local network. This way I could hopefully also still connect via IPMI.

Thank you!

Offsite Server

TrueNAS-SCALE-23.10.2
Supermicro X10SLL-F, i3 4130, 16 Gb ECC RAM, Seasonic Prime PX-750
in Fractal Define XL USB 3.0
Data pool: 2x8Tb mirror
Data pool: 1x8Tb stripe
boot pool: 2x128 Gb SSD
UPS: Eaton Eco 650

Main machine staying at home

TrueNAS-SCALE-23.10.1
Supermicro X10SRi-F, Xeon 2640v4, 128 Gb ECC RAM, Seasonic Focus PX-750
in Fractal Design R5
Data pool: 6x4Tb striped mirror + 1 hot spare
VM pool: 2x500Gb SSD mirror
boot pool: 2x250 Gb SSD mirror
UPS: Eaton Eco 1200
NIC: Intel X520 10g

2

Using 2FA is certainly possible, but since replication takes place over SSH, disabling that isn’t possible. Though it can (and should, IMO) be configured to disable password logins, and probably also to disable root logins.

It would be best, if possible, to configure the box to only be accessible over the Tailscale network–but I’m not sure how that would be accomplished.

3

D’uh! Yes obviously SSH needs to stay enabled… Oversight by me. I disabled passwords for SSH even on my local network so that should be okay.

That’s the million dollar question - or rather will dictate how I may spend one of the upcoming weekends :wink:

4

I would configure replication as Pull instead of Push. That way, the remote server will not need any port / tool to connect the source server. It will just go through NAT and the port that will need to be opened will be on your side.

Should you wish to get more than replication going (you wish to connect the WebUI remotely), I would do it with a site-to-site VPN. That may require a more powerful router on his side and the need to configure that before sending him the router. You can also install OpenVPN client in the remote TrueNAS.

If that is not an option, you can put a script that will call home over SSH and do TCP forwarding for taking a TCP socket on your side and forward it locally to port 443 in the server his side. Again, no open port on his side and a connection that is 100% yours instead of going through someone else.

Remember that you can not give what you do not have. If a third party can give you back access to your internal network, it is only because themselves have access to it.

3 Likes
5

Fritz!boxes can provide VPN through IPSec or WireGuard.
The question is how much @chuck32 's father-in-law is comfortable with letting him set some advanced fucntions (VPN, port sharing, remmote access…) on the box.

6

What’s the WAN’s throughput, anyway? That’s going to affect the choice of VPN endpoint, if you do set up a separate device.

7

MikroTik makes connecting two networks via Wireguard super easy, even with DDNS on both sides. That in turn makes administrating and troubleshooting easier since all network components can be checked and interrogated from your home.

Put your remote server on a private VLAN and your FIL will never even know it’s there.

Thanks to @dan, Mikrotiks can be furnished with SSLs remotely, allowing you to secure any admin webGUI sessions with the remote router as well (or you can log in via SSH).

Depends how much your FIL trusts you, eh? :slight_smile:

Anyhow, for the how to I recommend the network berg series on YouTube, specifically the videos on wire guard with the latest routeros and VLANS.

Lastly, I’ve come to accept just how much my ISP throttles outbound connections. Always replicate everything locally before physically moving the remote server to the remote site because incremental updates are more digestible.

2 Likes
8

^^ This is what I did when I was setting up a similar situation. We use MikroTik routers here at iX as well and having the Wireguard built-in has made these kinds of setups super easy.

2 Likes
9

Philosophically I’d argue for a “Push” vs “Pull”. A push allows you to set up a remote server with minimal “knowledge”. No SSH keys to your machine, etc.

If the dataset is encrypted on your home machine, it’ll be encrypted at rest as well, minimizing the attack surface even if the machine is compromised.

I’m still kind of surprised that the default SSH key exchange is at the root level of privilege, which seems awfully high, or am I misinterpreting things?

1 Like
10

Since replication involves creating and deleting snapshots, I think this is necessary.

11

I think you can work around that with zfs allow, though I do know that Linux does not have support for non-root users to mount filesystems - my question is if that would affect zfs recv, insofar as snapdir=visible would sort of imply a new filesystem being mounted on receive.

On FreeBSD, there should not be any roadblocks (if this is one at all on Linux).

12

Isn’t SSH currently set up to only allow root logins?

Anyhow, I’d prefer if root is only needed for initial replication setup and the day to day of snapshot shipments to the remote server could happen at a lower level of non-su privilege.

13

Thank you all for your input so far!

  1. openVPN was suggested, I’d prefer to stick to wireguard or tailscale, with wireguard I’m already familiar and I’m currently working my way into tailscale. Unless openVPN has some major advantages.
    While I’m open to suggestions on this end (establishing the remote connection in the first place), the question itself focuses more on the security in the other local network
  1. My main goal / question is specifying setting up the firewall part in this little image:
    remoteserver.drawio
    remoteserver.drawio1490×691 38.8 KB

We haven’t talked in detail yet, it may be that once the server lives there, he may wants to use it for backups too (SMB).

Condensed from the answers so far and in line with what I originally thought I’m looking at a router that can run the VPN and supply a VLAN / separate nework.

Since I also have a friends HDD in my server, that he uses as a rsync target for his synology I’m not sure pull would also be an option for that.

Additionally I would like to get IPMI access so a site to site VPN would be the best setup in my opinion. With the caveat that I would like to avoid to add FILs whole network to the VPN but rather the portion where my server lives.

It’s a 15 min drive there, so setting up everything can be done in place (I also need to get the server there anyway).

I take this as I should prefer the open port of wireguard over tailscale?
Tailscale would be the easiest to manage though. I think can still avoid open ports when using wireguard directly if my friend (who also needs access) and me both have wireguard working (with ddns) and the remote server just connects to our machines.

Honestly I’d rather stay away from either replacing his current router (I’d additionally need a cable modem then, too) or changing any configuration on there. It just opens the door for beeing responsible if something doesn’t work, even if it’s completely unrelated to any changes made by me.

35 Mbit/s upload on my side is the limited factor, so not a whole lot :wink:
Initial replication is already made for the replication server and in case I need the data quickly I can just come grab it physically.

I also stumbled upon mikrotik but haven’t dwelved into choosing a special model. I’ll check out the berg series on youtube, this will hopefully clear up if I’m looking for their router line up or CRS switches.

Great! I’ll check it out!

While I prefer to manage everything here, the likelihood of any machine getting stolen is probably the same. Unless your talking about being compromised another way, that’s why I’d prefer to separate the networks.

I think the only downside was, that I needed to Allow all sudo commands with no password but I’m currently using the remote admin account instead of root for the replication.

No, SSH can be also used with the admin account, I haven’t tested other accounts though.

14

For such modest requirements, I’m partial to the hEX S. It’s a neat little box to play around with, but it is rather limited. I imagine it would work great in your scenario, with room to grow.

2 Likes
15

I have had very good experiences with the RB4011. Solidly built, full routeros, good number of ports, and one SFP+. It’s a perfect router to hang a NAS from.

You’d be surprised how much slower upload speeds can be than forecast by a dslreports or like speed check once Comcast is your ISP. They clearly recognize “server” traffic and throttle it accordingly.

1 Like
16

Hex S is very interesting from a construction POV. Dual power supply capable, etc.

But the quickset bridge function for HEXs is broken. Do not expect it to actually implement a bridge like my other mikrotiks can. It has to be set manually via the app or ssh. The CPU inside also struggles with some capabilities like hanging off a SMB share.

So while I like the HEXs for its router role, there is weirdness ahead if you don’t use it in that role.

17

If you added a dual port Ethernet card to your TrueNAS you could run pfsense in a VM, with hardware passing for wan/lan ports.

In the past, I’ve used this sort of arrangement for offsite backups too.

The host (in your case FIL) receives an advanced router, tech support and local backup services in return for hosting :wink:

Anyway, with something like pfsense you can setup a site to site vpn, and then your main nas can push directly to the offsite NAS without you adding holes to your FILs network.

2 Likes
18

Something to keep an eye on, if you stick with the Tailscale approach:

I think, but I’m not 100% sure, that Tailscale sometimes fails to get a connection via STUN, and then falls back to routing your traffic through a relay node.

I use Tailscale, and I love it, but every once in a while I have a session that seems dramatically slower than the usual, and that’s my guess as to why. You might sometimes see two different speeds for your replication/rsync.

-Bill

1 Like
19

Followup:

This describes the relay node thing I referred to earlier:

1 Like