Learning NPM. Working with most apps but cant access old nextcloud instance

jackdinn · December 2, 2024, 2:53pm

So i have been trying to learn NPM.

I have got it mostly working, even got https certs.

I have several app working on subdomains as a test but i just cant figure how to access my old nextcloud instance

It was built with the auto installation script by @dan (im fairly sure).

If i point my router directly to the ip:port of the nextcloud instance it works fine as it has been for ages.

But if i set the router to point to NPM and i have a proxy host in NPM pointing to the nextcloud IP im getting Bad gateway Error code 502

root@nextcloud2:~ # cat /usr/local/www/Caddyfile
{
        # debug
        #acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
        email bob@bob.co.uk
        # default_sni bob.co.uk
}

bob.co.uk, localhost {
        root * /usr/local/www/nextcloud
        file_server
        log {
                output file /var/log/bob.co.uk.log
        }

        php_fastcgi 127.0.0.1:9000 {
                env front_controller_active true
        }

        tls {
        protocols tls1.2 tls1.3
        }

        header {
                # enable HSTS
                # Strict-Transport-Security max-age=31536000;
        }

        # client support (e.g. os x calendar / contacts)
        redir /.well-known/carddav /remote.php/dav/ 301
        redir /.well-known/caldav /remote.php/dav/ 301
        redir /.well-known/webfinger /index.php/.well-known/webfinger 301
        redir /.well-known/nodeinfo /index.php/.well-known/nodeinfo 301

        # Required for legacy
        @notlegacy {
                path *.php
                not path /index*
                not path /remote*
                not path /public*
                not path /cron*
                not path /core/ajax/update*
                not path /status*
                not path /ocs/v1*
                not path /ocs/v2*
                not path /updater/*
                not path /ocs-provider/*
                not path */richdocumentscode/proxy*
        }
        rewrite @notlegacy /index.php{uri}

        # .htaccess / data / config / ... shouldn't be accessible from outside
        @forbidden {
                path /.htaccess
                path /data/*
                path /config/*
                path /db_structure
                path /.xml
                path /README
                path /3rdparty/*
                path /lib/*
                path /templates/*
                path /occ
                path /console.php
        }

        respond @forbidden 404
}
root@nextcloud2:~ #

I followed Nginx Proxy Manager | Dan's Wiki and as i say i got a handful of apps working through NPM with HTTPS, some on truenas core and some on a separate server truenas scale, but i can not do it for the old nextcloud

I eventual will swap to scale but im still testing and tinkering so i need to be able to switch nextcloud back and forth between NPM and direct forward from router, for now.

jbettigole · December 2, 2024, 5:05pm

Did you modify your Nextcloud config to work behind a reverse proxy?

The following article explains the settings that need to be changed:
https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html

jackdinn · December 2, 2024, 5:15pm

Thx for the info, I will have a read and see if it will help as I was unaware of this, so I suspect it might well be the answer even though it looks over my head as usual ^^

I must rest my aching head for a couple of hours first as I’ve been banging my head against a wall for many hours, even turned to LLM’s until after several hours they start repeating things they already tried/looked at.

It seemed to be some confusion between the certs being both on nginx and self signed on nextcloud, but now with your info i think that maybe not.

Fingers crossed ^^

jbettigole · December 2, 2024, 5:32pm

In your Nginx config, you should have proxy_ssl_verify off; to ignore the self-signed cert on Nextcloud. I see in your screenshots that this is in the Custom Nginx Configuration. Hopefully the config generator puts this in the right place. It should be in a location block along with the proxy_pass directive.

But yes, Nextcloud is very picky about how it is accessed and will generally throw errors that the proxy doesn’t understand. I know that Bad Gateway error isn’t very helpful. Watch your Nextcloud logs for hints as to why it fails.

jackdinn · December 2, 2024, 6:04pm

O well was worth a quick try, i edited the config.php but it still says bad gateway 502. I checked that is still works when i directly forward to it in the router.

Where exactly do you mean this setting should be?

Also where are the “nextcloud logs”

in the config.php
'logfile' => '/var/log/nextcloud/nextcloud.log',

but

root@nextcloud2:~ # cat /var/log/nextcloud/nextcloud.log
cat: /var/log/nextcloud/nextcloud.log: No such file or directory
root@nextcloud2:~ #

EDIT:
ok i created the log file and it started logging

root@nextcloud2:~ # cat /var/log/nextcloud/nextcloud.log
{"reqId":"z0hCXir39YTX082D6tF0","level":1,"time":"2024-12-02T18:15:13+00:00","remoteAddr":"","user":"--","app":"memories","method":"","url":"--","message":"Indexing completed successfully","userAgent":"--","version":"30.0.2.2","data":{"app":"memories"}}
root@nextcloud2:~ #

but when i switch back to the NPM there are no logs generated when i try to access the nextcloud.

jbettigole · December 2, 2024, 7:57pm

What about the Nginx server logs on your Nextcloud host? Do they indicate if there is traffic coming from the proxy?

After reading a bit about NPM, I’m assuming you don’t have to actually mess with config files. But, poking around a few other forums, lead me to understand how that custom block gets included.

Try changing the Custom Nginx Configuration to the following:

location / {
  proxy_redirect off;
  proxy_ssl_verify off;
  proxy_ssl_server_name on;
  proxy_set_header Host $remote_addr;
  proxy_set_header Referrer $remote_addr;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Host $server_name;
}

jackdinn · December 2, 2024, 8:10pm

what the absolute, i just refreshed my domain and i am presented with this ??? it on my https domain??

O crap,

I switched back to router pointing to my nextcloud server directly and shut down the whole Nginx server.

What is this ??

Pheww, i though for a minute i been had but turned out it was due to the setting change that you suggested

location / {
  proxy_redirect off;
  proxy_ssl_verify off;
  proxy_ssl_server_name on;
  proxy_set_header Host $remote_addr;
  proxy_set_header Referrer $remote_addr;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Host $server_name;
}

Putting this into that advanced setting tab caused it, because when i changed them back again it was gone and we are back to bad gateway 502

I don’t know how but i think we are in the clear. I suspect it part of NPM is some weird situation they try to upsell something, i dunno but thx for the serious shock ^^

P.S.

The nginx server logs say:-

g oot@docker-fa1798522fe3:/app]# cat /var/log/nginx/error.log
2024/12/02 17:44:20 [warn] 159#159: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:4
2024/12/02 17:44:20 [warn] 159#159: the "listen ... http2" directive is deprecated, use the "http2" directive instead in /data/nginx/proxy_host/1.conf:19
2024/12/02 17:44:20 [warn] 159#159: the "listen ... http2" directive is deprecated, use the "http2" directive instead in /data/nginx/proxy_host/4.conf:19
g ;32mdocker-fa1798522fe3:/app]# cat /var/log/nginx/error.log
Broadcast message from root@truenas (Mon 2024-12-02 20:11:00 GMT):
  oot@docker-fa1798522fe3:/app]# cat /var/log/nginx/error.log

jbettigole · December 2, 2024, 10:57pm

Sorry about that. I misunderstood the way NPM does things behind the scenes.

So, I installed NPM (originally, I had been using an Nginx docker host with manual config, etc), and managed to get it working with some trial and error. After installation, I inspected the .conf files it generated and learned what should, and should not be.

Hopefully this works for you:

empty out that Custom Nginx Configuration. This doesn’t put anything in the right place for it to work
go to the Custom Locations tab and Add Location.

The location will be: /
The Scheme, Forward Host, Forward Port will match what you put on the Details tab

click the gear icon (to the right of location) and add
```
proxy_ssl_verify off;
proxy_ssl_server_name on;
```

That seems to be the approved way to add/override configuration specific to the location block. I believe the reason for failure on my previous advice is because Nginx doesn’t like duplicate blocks; which was what I had you add.

jackdinn · December 2, 2024, 11:20pm

Unfortunately no dif

info dump:-

root@nextcloud2:~ # cat /usr/local/www/Caddyfile
{
        # debug
        #acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
        auto_https disable_redirects
        email bob@bob.co.uk
        # default_sni bob.co.uk
}

http://bob.co.uk, http://localhost {
        root * /usr/local/www/nextcloud
        file_server
        log {
                output file /var/log/bob.co.uk.log
        }

        php_fastcgi 127.0.0.1:9000 {
                env front_controller_active true
        }

        tls {
        protocols tls1.2 tls1.3
        }

        header {
  GNU nano 8.2                 /usr/local/www/Caddyfile

        # client support (e.g. os x calendar / contacts)
        redir /.well-known/carddav /remote.php/dav/ 301
        redir /.well-known/caldav /remote.php/dav/ 301
        redir /.well-known/webfinger /index.php/.well-known/webfinger 301
        redir /.well-known/nodeinfo /index.php/.well-known/nodeinfo 301

        # Required for legacy
        @notlegacy {
                path *.php
                not path /index*
                not path /remote*
                not path /public*
                not path /cron*
                not path /core/ajax/update*
                not path /status*
                not path /ocs/v1*
                not path /ocs/v2*
                not path /updater/*
                not path /ocs-provider/*
                not path */richdocumentscode/proxy*
        }
        rewrite @notlegacy /index.php{uri}

        # .htaccess / data / config / ... shouldn't be accessible from outside
        @forbidden {
                path /.htaccess
                path /data/*
                path /config/*
                path /db_structure
                path /.xml
                path /README
                path /3rdparty/*
                path /lib/*
                path /templates/*
                path /occ
                path /console.php
        }

        respond @forbidden 404
}

root@nextcloud2:~ # cat /usr/local/www/nextcloud/config/config.php
<?php
$CONFIG = array (
  'passwordsalt' => 'xxxx',
  'secret' => 'vFmh+xxxx+xxxx/yr',
  'trusted_domains' =>
  array (
    0 => 'localhost',
    1 => 'bob.co.uk',
    2 => '192.168.2.150',
    3 => '192.168.1.109',
    4 => '192.168.1.105',
  ),
  'maintenance_window_start' => 5,
  'datadirectory' => '/mnt/files',
  'dbtype' => 'mysql',
  'version' => '30.0.2.2',
  'overwrite.cli.url' => 'https://bob.co.uk/',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost:/var/run/mysql/mysql.sock',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'xxxxx==',
  'installed' => true,
  'instanceid' => 'xxxxxxx',
  'logtimezone' => 'Europe/London',
  'default_phone_region' => 'GB',
  'log_type' => 'file',
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'loglevel' => '1',
  'logrotate_size' => '104847600',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'redis' =>
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
  ),
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'overwritehost' => 'bob.co.uk',
  'overwriteprotocol' => 'https',
  'htaccess.RewriteBase' => '/',
  'mail_from_address' => 'bob',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'bob.co.uk',
  'mail_smtphost' => 'bob.net.uk',
  'mail_smtpport' => '587',
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'bob@bob.co.uk',
  'mail_smtppassword' => 'xxxx',
  'maintenance' => false,
  'memories.exiftool_no_local' => true,
  'memories.vod.path' => '/usr/local/www/nextcloud/apps/memories/bin-ext/go-vod-amd64',
  'memories.vod.ffmpeg' => '/usr/local/bin/ffmpeg',
  'memories.vod.ffprobe' => '/usr/local/bin/ffprobe',
  'theme' => '',
  'enable_previews' => true,
  'enabledPreviewProviders' =>
  array (
    1 => 'OC\\Preview\\PDF',
    2 => 'OC\\Preview\\Image',
    3 => 'OC\\Preview\\Photoshop',
    4 => 'OC\\Preview\\TIFF',
    5 => 'OC\\Preview\\SVG',
    6 => 'OC\\Preview\\Movie',
    7 => 'OC\\Preview\\MKV',
    8 => 'OC\\Preview\\MP4',
    9 => 'OC\\Preview\\AVI',
    10 => 'OC\\Preview\\JPEG',
  ),
  'auth.webauthn.enabled' => false,
  'app_install_overwrite' =>
  array (
    0 => 'apporder',
    1 => 'bbb',
  ),
  'log.condition' =>
  array (
    'apps' =>
    array (
      0 => 'admin_audit',
    ),
  ),
  'logfile_audit' => '/var/log/nextcloud/nextcloud.log',
  'trusted_proxies' =>
  array (
    0 => '127.0.0.1',
    1 => '192.168.1.117',
    2 => '127.0.0.1',
  ),
  'forwarded_for_headers' =>
  array (
    0 => 'HTTP_X_FORWARDED_FOR',
  ),
  'memories.db.triggers.fcu' => true,
);


############################################################# tried looking at these logs but nothing is added as i try to connect

 root@nextcloud2:~ # cat /var/log/nextcloud/nextcloud.log
 root@nextcloud2:~ # cat /var/log/caddy/caddy.log
 root@nextcloud2:~ # tail -f /var/log/php-fpm.log
 root@docker-ebcd  # cat /var/log/nginx/error.log

jackdinn · December 3, 2024, 11:14am

I figured it, only took 3 days in the end.

The self-signed cert created in nextcloud by the original dan script had to be directly pointed at by the TTL section of caddyfile settings. Caddy did not have the paths to your existing certificate and key, so it was failing to properly serve HTTPS.

The resolution:-
/usr/local/www/Caddyfile

tls /var/db/caddy/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/bob.co.uk/bob.co.uk.crt /var/db/caddy/data/caddy/certificates/acme-v02.a
pi.letsencrypt.org-directory/bob.co.uk/bob.co.uk.key { 
    protocols tls1.2 tls1.3 
        }

jackdinn · December 3, 2024, 12:27pm

No i was incorrect, its not fixed at all, i had accidentally left the router pointing to the nextcloud directly. Its been such a slog i am making serious mistakes.

So what now? what next?

iv got some logs that i didnt have before

2024/12/03 12:35:27 [error] 198#198: *589 SSL_do_handshake() failed (SSL: error:0A000438:SSL routines::tlsv1 alert internal error:SSL alert number 80) while SSL handshaking to upstream, client: 172.70.86.13, server: bob.co.uk, request: "GET /index.php/204 HTTP/2.0", upstream: "https://192.168.1.109:443/index.php/204", host: "bob.co.uk"

jackdinn · December 3, 2024, 2:16pm

So i increased the debugging level in Caddyfile

{
    debug
}

found more detailed logs which did this time point towards the problem.

Problem Recap

The issue revolved around TLS handshake errors and proxy misconfiguration:

TLS handshake errors due to mismatched configurations between Nginx Proxy Manager (NPM) and the Caddy server hosting Nextcloud.
Proxy settings in NPM weren't aligned with Caddy's certificate or its handling of local IP (192.168.1.109).

What Resolved the Issue

The actual resolution came from:

Adding the local IP (192.168.1.109) to the Caddyfile block for the tls configuration. This ensured that Caddy could correctly serve a certificate even when accessed by IP, which was previously unconfigured:

bob.co.uk, localhost, 192.168.1.109 {

Ensuring NPM's upstream settings were correct:
    Proxying to https://192.168.1.109:443 rather than relying on incorrect scheme assumptions.
    Configuring proxy_ssl_verify off to bypass certificate verification when using a self-signed certificate.

Debugging and ensuring the Caddy TLS setup was aligned with what NPM expected for SSL communication.

Summary of Fix

The root cause was a mismatch in TLS expectations and an incomplete Caddy configuration for the local IP. The fix involved:

Adding the IP to Caddy’s domain list.
Correctly configuring NPM to proxy HTTPS requests to the Nextcloud server.
Ensuring certificates in use matched what the servers expected.

If this is incorrect or misses a detail, feel free to clarify!

jbettigole · December 3, 2024, 6:01pm

Nice!

Based on your log entry, I suspected that could’ve been the issue. I don’t know Caddy, so I wouldn’t have known how to advise on that.

I was going to suggest trying in your custom location config on NPM

   proxy_ssl_verify off;
   proxy_ssl_server_name on;
   proxy_ssl_name bob.co.uk;

By default Nginx proxy sends whatever you specified in the hostname field, which in this case was an IP address that Caddy didn’t know what to do with. If Caddy understands Server Name Introspection, it would handle the request based on the domain name given that extra config entry (proxy_ssl_name).

Glad you got it all sorted.

jackdinn · December 3, 2024, 6:31pm

Thx, but even though i have got access to the system through NPM i still have noticed a problem that is strongly related.

In Nextcloud admin settings it’s got an orange warning.

The reverse proxy header configuration is incorrect, or you are accessing Nextcloud from a trusted proxy. If not, this is a security issue and can allow an attacker
to spoof their IP address as visible to the Nextcloud. For more details see the documentation ~.

I have tried putting these headers in all sorts of places but they just do not get past to caddy

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
proxy_ssl_verify off;
proxy_ssl_server_name on;

iv put them in the custom locations, iv tried them just pasted as the list above and also in the way you suggested

location / {
here
}

iv pasted both just a list and behind the / location in here

I spent another few hours with an llm working from the link you provided above but nothing actually passes these headers into caddy and so i get that warning all the time.

Maybe i shall ask on the nextcloud forum, maybe just leave it.

I might just reinstall nextcloud on my second server on scale and see if i can figure a way to import everything and try nginx with that nextcloud installation that does not already have self signed cert’s.

Happy days

dan · December 3, 2024, 7:00pm

You are in fact accessing Nextcloud from a trusted proxy. Thus, this isn’t a problem.

jackdinn · December 3, 2024, 7:22pm

Now that is exactly what i thought the wording meant, however.

root@nextcloud2:~ # curl -v -H "Host: bob.co.uk" https://192.168.1.109 --insecure
*   Trying 192.168.1.109:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: [NONE]
*  start date: Dec  3 13:48:25 2024 GMT
*  expire date: Dec  4 01:48:25 2024 GMT
*  issuer: CN=Caddy Local Authority - ECC Intermediate
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
*   Certificate level 0: Public key type ? (256/128 Bits/secBits), signed using ecdsa-with-SHA256
*   Certificate level 1: Public key type ? (256/128 Bits/secBits), signed using ecdsa-with-SHA256
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connected to 192.168.1.109 (192.168.1.109) port 443
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://192.168.1.109/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: bob.co.uk]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.10.1]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: bob.co.uk
> User-Agent: curl/8.10.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 302 
< alt-svc: h3=":443"; ma=2592000
< content-security-policy: default-src 'self'; script-src 'self' 'nonce-0UTh6JrOEc7ppsqC16kZOzvndBqD/0PYM43gb2zVZnM='; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *; object-src 'none'; base-uri 'self';
< content-type: text/html; charset=UTF-8
< location: https://bob.co.uk/login
< referrer-policy: no-referrer
< server: Caddy
< set-cookie: oc_sessionPassphrase=eNts0OQkcr4Kk49qj3395xA7KG4UvQ6azcgIwagnJwlDRQwGqstdmz0cxOSg468%2BSxAXaT0rxzg5cZRN5oR19KLU%2Bfb0ycRLTA5TC8fD7jx2WJwHthpboT%2BLPFNsDDHC; path=/; secure; HttpOnly; SameSite=Lax
< set-cookie: __Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
< set-cookie: __Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
< set-cookie: ocfuu1fbyyca=chp71tre970mrl188fn9bdet2t; path=/; secure; HttpOnly; SameSite=Lax
< status: 302 Found
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-permitted-cross-domain-policies: none
< x-powered-by: PHP/8.3.12
< x-robots-tag: noindex, nofollow
< x-xss-protection: 1; mode=block
< content-length: 0
< date: Tue, 03 Dec 2024 19:13:49 GMT
< 
* Connection #0 to host 192.168.1.109 left intact
root@nextcloud2:~ #

should i not be able to see the headers being passed across here?

I probably have not understood something highly important and am completely wrong though. ^^

O well, its good news that the exact reading of the error message was just what it said.

Thx for pointing that out @dan

jbettigole · December 3, 2024, 8:34pm

I don’t have that exact error, but I’m seeing a couple others I need to clean up.

On another forum I found someone suggested to try removing the trusted proxies from Nextcloud config. I have no idea if this will work, but worth a try.

A lot of those headers I gave you previously seemed to already be included in the config generated by NPM. That’s why I omitted them. If you want to try them, they should be included in that custom Location / block. Until I can recreate that error you’re seeing though, I don’t know what exactly solves it.

For now, I’m going to go back through Nextcloud’s proxy docs and poke till I break it:
https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html

jackdinn · December 3, 2024, 8:59pm

thx but i wouldn’t worry about it, as Dan pointed out its supposed to be that way in my case as i do have `trusted proxy’s set up in the caddyfile so the warning is saying that

The reverse proxy header configuration is incorrect, **OR you are accessing Nextcloud from a trusted proxy.**

I need to set it with trusted proxy’s because of the self signed cert that is setup on nextcloud already. For some reason it just has to be that way.

All over my head but if Dan says just ignore it then so be it.

Cheers for your time today