Ssh connection into a container

On a TN 26 beta 3 system (test system) I have created an LXC container running Debian. It’s set up with the network connected to a bridge, so it should be able to see everything (and in fact I can ping and ssh out from it). It seems to work, I’ve created a user there, I’ve mounted some host datasets there and can access them, I’ve installed some software there, I’ve installed and started sshd there. I can’t find a fiewall running there.

But I can’t ssh into it from anywhere I’ve tried (in my LAN). systemctl says sshd is running, and lsof shows the port is open. Trying to ssh in just gets “connection refused”. (ssh from a shell there also fails with the same message, it’s not just outside connects.)

Some step I’m missing?

first check what address sshd is actually listening on inside the container:

ss -lntp | grep ':22'

if it shows 127.0.0.1:22, set ListenAddress 0.0.0.0 (or the container IP) in sshd_config and restart ssh. if it shows 0.0.0.0:22, make sure you are connecting to the container’s actual IP, not the TrueNAS host IP. with an LXC using NAT, a LAN client usually cannot reach the container directly, so you need a bridged interface or a host port forward. the fact it can reach out only proves the outbound path works.

I checked before other ways, but I get the same confirmation that way:

root@play03:/# ss -lntp | grep ':22'
LISTEN 0      4096         0.0.0.0:22        0.0.0.0:*    users:(("sshd",pid=326,fd=3),("systemd",pid=1,fd=121))
LISTEN 0      4096            [::]:22           [::]:*    users:(("sshd",pid=326,fd=4),("systemd",pid=1,fd=122))
root@play03:/# 

As I said above, I set it up with a bridged interface, not NAT (since I knew I would need to get to the container). I can ping the container IP from another system on the LAN (not the host at all), so the IP connectivity is right at least.

Most lxc templates don’t include an ssh server. I had to manually install open ssh in my lxc on 25.10, set a password for the root account ( I only use root inside the lxc) and enable ssh for root before I could ssh into the lxc

Yes, I installed the server, and started it successfully with systemctl. It reports that it’s running.

I’m getting the connection refused before it gets to negotiating what user to log in as (remember, I’ve already created a personal user in the container; I was going to specify that user and its password when ssh connected, but it never did).

I’ve looked at /etc/ssh/sshd_config (exact spelling from memory), and I don’t see anything obviously wrong, but that’s kind of a complicated config. I didn’t change anything, anyway. I didn’t see a “do not run” config line anywhere :slight_smile: .

Yes, the verbose connect log supports that; the simple tcp connect is failing, not something complicated later:

$ ssh -v ddb@192.168.1.113
OpenSSH_8.9p1 Ubuntu-3ubuntu0.17, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /home/ddb/.ssh/config
debug1: /home/ddb/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to 192.168.1.113 [192.168.1.113] port 22.
debug1: connect to address 192.168.1.113 port 22: Connection refused
ssh: connect to host 192.168.1.113 port 22: Connection refused
ddb@ddblap8:~

I’m suspecting some kind of firewall issue on the host where the container runs. I don’t know how a firewall interacts with connecting a container via a bridge. Anybody ever done that? Does the firewall apply to everything under the bridge?

is UsePAM set to yes or no in your sshd_config?

TrueNAS doesn’t use a firewall.

Yes, I’ve found that info myself since last post.

So that’s not the problem here!

(In the longer run, that’s a huge security problem I don’t know how to solve. Any direct outside connect has to go through the router – but anybody who manages to compromise any ordinary user account in the household LAN can now connect to any port on the NAS. I don’t know how to solve it because adding a firewall is blocked along with all the other software additions one might want to make. The other way to solve it is a dedicated router just between the LAN and the NAS, and that’s something I don’t really have the resources for, and a very unusual practice. I’m kind of surprised commercial outfits will purchase a NAS box with no firewall capabilities.)

Well commertial customers or data centers usually have an external firewall and no need for one to run on storage boxes

UsePAM is yes in my sshd_config

Everybody usually has an external firewall, it’s the default in home routers, and of course routine in commercial setups.

But that only protects against connections directly from the outside. Lots of things, most notably corporate data, have to be protected to a far higher level than the rest of the systems, and have to be protected against things like legit employees (so they have accounts on systems inside the firewall) who might try to exceed their authority by accessing data on the server. I mean, even Windows laptops have a firewall by default.

But NOT the problem I’m facing now, I’ll drop this part of the discussion here.

Kind of think I see where things are going astray.

I’ve created and destroyed (or rolled back to snapshots) multiple virtual test setups, but here’s what I think is happening in general.

I think the bridging setting and ip network in container configuration is likely to be the problem.

I’ve got a setup where the container can ssh to itself, and the TN26 box can ssh to the container. The container can ssh out to anywhere. But I can’t ssh in to the container from further away than the box; not from other virtuals, not from my laptop on the LAN.

And I’m not at all sure I understand what the ip network setting in container configuration is for.

So…for anybody who has things working so they can ssh into a container from other than the host system…what do you have in container configuration for bridging (I have no choice other than “automatic”), and what network address do you have there, and why do you have whatever choices you have?