Auto-solved this one, but just sharing if that can spare some hairs to somebody:
If Firefly-iii appears as a text website when accessing it behind a reverse proxy, it is probably because it misses the TRUSTED_PROXY parameter.
To do this, under Firefy-III Configuration section, add an environment variable:
TRUSTED_PROXY=* (or this could also be the IP of the proxy server)
Reload and it should work !
Tanguy