Problem installing paperless-ngx

[oxyde]

I have install those 2 services in 2 different jails, and used caddy only for reverse proxy.
According to paperless configuration guide, they must be set both together

[betov]

Damn, I just installed it on the same paprless server hehe

[oxyde]

They both use different Port, so maybe you have done right… Let me know hows go, if u can solve the problem i have i will install both on same jail of paperless

[betov]

I followed your instructions except for using caddy since if the servers are on the same server I don’t see the need to use a local domain, however when trying to consume a .odt it gives me the following error:

  File "/usr/home/paperless/paperless-ngx/src/documents/consumer.py", line 608, in run

    self._fail(

  File "/usr/home/paperless/paperless-ngx/src/documents/consumer.py", line 302, in _fail

    raise ConsumerError(f"{self.filename}: {log_message or message}") from exception

documents.consumer.ConsumerError: Desde donde posan las aguilas.odt: Error occurred while consuming document Desde donde posan las aguilas.odt: Could not parse /tmp/paperless/paperless-ngxfyru9nv3/De donde posan las aguilas.odt with tika server at http://localhost:3000/: Client error '404 Not Found' for url 'http://localhost:3000/tika/form/text'

I will do more tests in the afternoon, greetings and thanks

[betov]

Well this is something embarrassing, I put the server address wrong hehe now I have another problem:

  File "/usr/home/paperless/paperless-ngx/src/documents/consumer.py", line 302, in _fail

    raise ConsumerError(f"{self.filename}: {log_message or message}") from exception

documents.consumer.ConsumerError: Desde donde posan las aguilas.odt: Error occurred while consuming document Desde donde posan las aguilas.odt: Error while converting document to PDF: Server error '503 Service Unavailable' for url 'http://localhost:3000/forms/libreoffice/convert'

You will know why it is, I already increased the API time but it still gives an error

--api-timeout=300s

[oxyde]

is related to Gotenberg, have you check if service is running?

[betov]

If it is running, add these two flags that I read in a github post and now it gives me the rejection error:

  File "/usr/home/paperless/paperless-ngx/src/documents/consumer.py", line 302, in _fail

    raise ConsumerError(f"{self.filename}: {log_message or message}") from exception

documents.consumer.ConsumerError: From where the eagles pose.odt: Error occurred while consuming document From where the eagles pose.odt: Error while converting document to PDF: [Errno 61] Connection refused

[oxyde]

Seems i have fixed the certificate problems, on both (tika and gotenberg).
First time i use 2 jail just as api endpoint, and until now i always use crt and key, but here i need to restore the fullchain correctly.
So need, to the above istruction, must add

nano root_ca.pem
cat /usr/local/etc/ssl/certificate.crt /usr/local/etc/ssl/root_ca.pem > /usr/local/etc/ssl/fullchain.pem

and the Caddyfile must use the fullchain.pem instead of certificate.crt.

For reverse proxy of Paperless, i remove caddy and use nginx instead, because as documentation say, The officially supported method is Nginx. and i dont wanna put more possible problems

Now, trying consume a odt i got this error:

File “/usr/home/paperless/paperless-ngx/src/documents/consumer.py”, line 302, in _fail

raise ConsumerError(f"{self.filename}: {log_message or message}") from exception

documents.consumer.ConsumerError: --.docx: Error occurred while consuming document —.docx: Error while converting document to PDF: Server error ‘503 Service Unavailable’ for url ‘https://gotenberg.—.cloud/forms/libreoffice/convert’

For more information check: 503 Service Unavailable - HTTP | MDN

for the moment i can’t continue debug anymore.

edit: already tried ur workaround, nothing change (api stil fail fast then 300s)

For your problem:

• can u reach the service? navigating from browser into the address
• have u tried to use some test curl from shell?

edit2:
the problem is libreoffice itself, prob same as you:

root@Gotenberg:~/gotenberg # ./gotenberg --libreoffice-auto-start=true --libreoffice-start-timeout=60s

---

/ / / /___ ___ / / ___ _______ _
/ (_ / _ / / -) _ / _ / -) / _ '/
_/_/_/_////.__/_// _, /
/__/

A Docker-powered stateless API for PDF files.
Version: snapshot

[SYSTEM] modules: api chromium exiftool libreoffice libreoffice-api libreoffice-pdfengine logging pdfcpu pdfengines pdftk prometheus qpdf webhook
[SYSTEM] chromium: Chromium ready to start
[SYSTEM] pdfengines: exiftool libreoffice-pdfengine pdfcpu pdftk qpdf
[SYSTEM] prometheus: collecting metrics
[FATAL] starting api: launch supervisor: start process: execute LibreOffice: context done: context deadline exceeded

service can’t neither start after a timeout of 120s.

Edit3: i manage to install other pkg that probably were required, this is the update list
pkg install go git nano caddy chromium p5-Image-ExifTool libreoffice unoconv pdftk qpdf webhook prometheus pdfcpu
the only thing i can say about that problem… for sure im missing a dependency that is related to the api mentioned in the gotenberg startup (i found other missing elements on that)

[betov]

I am glad that you were able to resolve the certificate issue, I do not have that part covered since I plan to use paperless locally, but it is very good to document everything, on the other hand the problem is directly with gotenberg, I did the following tests with the services separately and all were successful except gotenberg:

Create an example.pdf and example.docx file for the following examples

-We test that the tika server is responding:

curl http://localhost:9998/tika

We test that a file can be sent to the server:

curl -T example.pdf http://localhost:9998/tika --header "Content-Type: application/pdf"

We test that LibreOffice can convert the files to pdf:

libreoffice --headless --convert-to pdf example.docx

Finally we try the gotenberg api to convert a file and that’s when the error occurs

curl --request POST \
  --url http://localhost:3000/forms/libreoffice/convert \
  --form files=@example.docx

[oxyde]

Im stuck at same point, can’t start LibreOffice api on gotenberg.
And the worst thing is i can’t see any detailed debug that can point on a solution, so can’t search valid help on Google/ Ai.
The next step for me will be try to get as info as possible from the docker installation, for try to replicate every step

Edit: i have try to ask on github, because i dont think we can manage that… im worried about some incompatibility problem

[betov]

Good news friend, I managed to get it working yay!! hahaha, it is necessary to use version 7.4.2, delete your version of gotenberg, this is done by deleting the directory where it is installed, later we install version 7.4.2 and that’s it, SUCCESS!!!

fetch https://github.com/gotenberg/gotenberg/archive/refs/tags/v7.4.2.tar.gz
tar -xzvf v7.4.2.tar.gz
cd gotenberg-7.4.2
go build -o gotenberg ./cmd/gotenberg
./gotenberg &

[oxyde]

Im on it buddy

[oxyde]

Work like a charm, nice catch really!! With small file the conversion is fast (and i dont have a high power cpu).
I have tried some file in the pc, seems going without problem!
Only 1 file rejected by Paperless, due to the size (3mb)

and i have csv bigger than that… i will investigate for sure but for the moment i think we have reached the goal!!!

[betov]

How happy I feel at this moment, I have not had problems with any files but they are not very large files either, my equipment is worse than the one you have but for personal use I think we are fine, we will have to make a complete manual for those of us left behind my friend and avoid suffering, maybe that mistake you have could be solved with the flag of

--api-timeout

Edit:

On the other hand, have you managed to create the services so that they work automatically?

I am having problems with the paperless daemon, sometimes it does not start and with the tika service I cannot get it to work

[oxyde]

After the upgrade to the 13.3-RELEASE, with my script i managed to launch automatically the server and consumer, but celery/beat never start.
For Tika and Gotenberg, actually only Tika autostart properly with a script.

cd /usr/local/etc/rc.d/
nano tika

----

#!/bin/sh
#
# PROVIDE: tika
# REQUIRE: DAEMON
# KEYWORD: shutdown

. /etc/rc.subr

name="tika"
rcvar=tika_enable
start_cmd="${name}_start"

export JAVA_HOME=/usr/local/openjdk11
export PATH=$JAVA_HOME/bin:$PATH

tika_start()
{
    echo "Starting ${name}..."
    /usr/local/openjdk11/bin/java -jar /root/tika-server-standard-2.9.2.jar &
    echo $! > /var/run/${name}.pid
}


load_rc_config $name
run_rc_command "$1"

-----

chmod +x /usr/local/etc/rc.d/tika
sysrc tika_enable=YES

Edit: here the gotenberg script. Work launched manually with /usr/local/etc/rc.d/gotenberg start but not automatic…

#!/bin/sh
#
# PROVIDE: gotenberg
# REQUIRE: DAEMON
# KEYWORD: shutdown

. /etc/rc.subr

name="gotenberg"
rcvar=gotenberg_enable
start_cmd="${name}_start"
stop_cmd="${name}_stop"

export CHROMIUM_BIN_PATH="/usr/local/bin/chrome"
export EXIFTOOL_BIN_PATH="/usr/local/bin/exiftool"
export LIBREOFFICE_BIN_PATH="/usr/local/bin/soffice"
export PDFTK_BIN_PATH="/usr/local/bin/pdftk"
export QPDF_BIN_PATH="/usr/local/bin/qpdf"
export UNOCONV_BIN_PATH="/usr/local/bin/unoconv"
#export UNOCONVERTER_BIN_PATH="/usr/local/bin/unoconv"

gotenberg_start() {
    echo "Starting ${name}..."
    cd gotenberg-7.4.2
    ./gotenberg &

    echo $! > /var/run/${name}.pid
}
gotenberg_stop() {
    echo "Stopping ${name}..."
    if [ -f /var/run/${name}.pid ]; then
        kill "$(cat /var/run/${name}.pid)" && rm -f /var/run/${name}.pid
    else
        echo "${name} is not running."
    fi
}

load_rc_config $name
run_rc_command "$1"

[betov]

Hello friend, thanks for your scripts, the problem I have is more in the services that do not start, the scripts run separately without problem but when it comes to linking them to the service is where it does not start, I am reading this documentation I hope it helps you, greetings

https://docs.freebsd.org/en/articles/rc-scripting/

[oxyde]

Yeah, is time to stop depending too much from stack overfow and chatpgt… And start understand a bit more about scripting if i have news ill post there

[oxyde]

Hi buddy, have you made some progress on the paperless auto start?
For the moment, i just managed to fix properly the auto start of Gotenberg (if u need the script rell me)

Edit: those auto script works fine for me:

## autostart
nano /usr/local/etc/rc.d/paperless

---

#!/bin/sh
# PROVIDE: paperless
# REQUIRE: DAEMON redis
# KEYWORD: shutdown

. /etc/rc.subr

name="paperless"
rcvar="${name}_enable"
pidfile="/var/run/${name}.pid"
start_cmd="${name}_start"
stop_cmd="${name}_stop"

paperless_start() {
    echo "Starting paperless service" >> /var/log/paperless_service.log
    su - paperless -c '/usr/local/sbin/paperless'  >> /var/log/paperless_service.log
}

paperless_stop() {
    echo "Stopping paperless service" >> /var/log/paperless_service.log
    kill $(cat ${pidfile})
}

load_rc_config $name
run_rc_command "$1"


----

nano /usr/local/sbin/paperless

---

#!/bin/sh
cd /usr/home/paperless/paperless-ngx/src
/usr/local/bin/python3 manage.py runserver 0.0.0.0:8000 &
/usr/local/bin/python3 manage.py document_consumer &
/usr/local/bin/python3 -m celery --app paperless worker --loglevel INFO &
/usr/local/bin/python3 -m celery --app paperless beat --loglevel INFO &


----

chmod +x /usr/local/etc/rc.d/paperless
chmod +x /usr/local/sbin/paperless

sysrc paperless_enable="YES" # nano /etc/rc.conf

i literally just add a log on your… dont know why without not working…

what about open a thread in resource with all the steps? Hoping will be helpful for others. This thread is a bit messy

[betov]

Hello friend, well I’m the same as you, I have the script for paperless and for gotenberg but tika doesn’t start I don’t know why, it’s practically the same as the one for gotenberg anyway, I’m still doing tests, I think it’s good to make the thread now with the correct steps to the others, but we will have to wait to have all the scripts to make it beautiful , greetings, I leave you the tika script so you can compare it with yours, use the information on the FreeBSD page:

#!/bin/sh

export PATH=/usr/local/openjdk11/bin:$PATH

. /etc/rc.subr

name=tika

start_cmd="tika_start"
rcvar=tika_enable

tika_start() {
    echo "Starting tika service"
    /usr/local/bin/java -jar /root/tika-server-standard-2.9.2.jar
}

stop_cmd="tika_stop"

tika_stop() {
    echo "Stopping tika service"
    pkill -f "/usr/local/bin/java -jar /root/tika-server-standard-2.9.2.jar"
}

load_rc_config $name
run_rc_command "$1"

[oxyde]

Our tika script are pretty different, try mine as is, the only script work at first launch without issue
Still dont understand why putting log debug make my scripts work perfectly, but glad that i finally end this setup…