QNAP TS-877 Truenas Journal

mooglestiltzkin · April 28, 2024, 2:08pm

ty u i appreciate and definitely will check it out. i just finish up immich then call it a day first xd

i will schedule that for tomorrow, thx again

mooglestiltzkin · April 28, 2024, 2:35pm

so after deploying immich, opened the app, browse around, toggle on transcoding (later switch it off cauz i don’t need that), then tested task for scan new faces etc, then ran a search seemed fast.

checked dozzle did not see any errors mentioned. i can only conclude it works.

if there is a better way to test it conclusively, plz let me know.

afaik i ran a search for moogles and the return result was pretty fast

for accuracy tested “boy with girl” something simple but clear

well spotted 2 mistakes. in 2 of them were both girls. unless the machine learning discovered Tanya was a dude but was reborn as a girl, then that would be pretty impressive. But i’m sure it was a mistake

one thing that impressed me about Immich was how they lock down face boxes almost perfectly

Hardware-Accelerated Machine Learning [Experimental]

This feature allows you to use a GPU to accelerate machine learning tasks, such as Smart Search and Facial Recognition, while reducing CPU load. As this is a new feature, it is still experimental and may not work on all systems.

as i thought, the reason why smart search was fast was because it’s accelerated. so yes for me there is a benefit.

I guess that also applies for facial recognition. while indexing and perhaps looking for a similar face.

But yes it’s very responsive when running the smart search. Is it worth buying graphic card just for that? don’t know but i liked the performance.

Immich is great but what is fares poorly is in importing existing folders to create virtual albums. There are third party work arounds using unofficial scripts. But most know not to run scripts over the internet from unknown origins, it’s rarely ever safe doing. That said i tried one of them and it worked. But when will they officially add that feature to Immich? no idea, but it’s what’s holding it back from being the best solution imho.

mooglestiltzkin · April 28, 2024, 2:50pm

did i speak too soon? not sure what this is, but it’s under machine learning

8/04/2024

[04/28/24 ] INFO     Initialized request thread pool with 8 threads.    
28/04/2024

[04/28/24 ] INFO     Application startup complete.                      
28/04/2024
22:37:54
[04/28/24 ] INFO     Setting 'ViT-B-32__openai' execution providers to  
28/04/2024

                             ['CUDAExecutionProvider', 'CPUExecutionProvider'], 
28/04/2024

                             in descending order of preference                  
28/04/2024

[04/28/24 ] INFO     Loading clip model 'ViT-B-32__openai' to memory    
28/04/2024

2024-04-28 xxxxxxxxxxxx [W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 15 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
28/04/2024

[04/28/24 ] INFO     Shutting down due to inactivity.                   
28/04/2024

[04/28/24 14:47:08] INFO     Shutting down                                      
28/04/2024

[04/28/24 ] INFO     Waiting for application shutdown.                  
28/04/2024

[04/28/24 14:47:08] INFO     Application shutdown complete.                     
28/04/2024

[04/28/24 14:47:08] INFO     Finished server process [48]                       
28/04/2024

[04/28/24 14:47:08] ERROR    Worker (pid:48) was sent SIGINT!                   
28/04/2024

[04/28/24 14:47:08] INFO     Booting worker with pid: 77                        
28/04/2024

[04/28/24 14:47:13] INFO     Started server process [77]                        
28/04/2024

[04/28/24 14:47:13] INFO     Waiting for application startup.                   
28/04/2024

[04/28/24 14:47:13] INFO     Created in-memory cache with unloading after 300s  
28/04/2024

                             of inactivity.                                     
28/04/2024

[04/28/24 14:47:13] INFO     Initialized request thread pool with 8 threads.    
28/04/2024

[04/28/24 14:47:13] INFO     Application startup complete.

this got flagged as an error twice
**ERROR** Worker **(** pi**d:48** **)** was sent SIGINT!

github.com/immich-app/immich

[BUG] Machine Learning container fails to start after upgrading to v1.77.0 from v1.76.1

opened 05:58PM - 06 Sep 23 UTC

closed 01:27AM - 07 Sep 23 UTC

thariq-shanavas

bug needs triage

### The bug I upgraded Immich and the machine learning container fails to sta…rt. Output of `sudo docker logs -f immich_machine_learning` `[09/06/23 10:35:59] INFO Booting worker with pid: 4585` `[09/06/23 10:36:29] CRITICAL WORKER TIMEOUT (pid:4585)` `[09/06/23 10:36:31] ERROR Worker (pid:4585) was sent SIGKILL! Perhaps out of memory?` The container tries to restart, then fails with the same timeout error. I suspect a bug from https://github.com/immich-app/immich/pull/3934 I'm running on a system with 2 GB RAM (with 1 GB ZRAM and 1GB swap), so I've enabled only face recognition among the machine learning features. The processor is an Intel Atom Z8350. It works great in v1.76.1 In my .env file, I have pinned the version to v1.76.1 until this is resolved. Thank you all so much for this amazing software! I'll be happy to post any other logs as needed. ### The OS that Immich Server is running on Debian 12 ### Version of Immich Server v1.77.0 ### Version of Immich Mobile App NA ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: [ "start.sh", "immich" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env depends_on: - redis - database - typesense restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.yml # service: hwaccel command: [ "start.sh", "microservices" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env depends_on: - redis - database - typesense restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always immich-web: container_name: immich_web image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release} env_file: - .env restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data # remove this to get debug messages - GLOG_minloglevel=1 volumes: - tsdata:/data restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3 restart: always database: container_name: immich_postgres image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release} environment: # Make sure these values get passed through from the env file - IMMICH_SERVER_URL - IMMICH_WEB_URL ports: - 2283:8080 depends_on: - immich-server - immich-web restart: always volumes: pgdata: model-cache: tsdata: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION= [Redacted] # The Immich version to use. You can pin this to a specific version like "v1.71.0" # IMMICH_VERSION=release IMMICH_VERSION=v1.76.1 # Connection secrets for postgres and typesense. You should change these to random passwords TYPESENSE_API_KEY= [Redacted] DB_PASSWORD= [Redacted] # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis #IMMICH_MACHINE_LEARNING_ENABLED=false #TYPESENSE_ENABLED=false ``` ### Reproduction steps ```bash 1. Start the container with docker-compose up -d ``` ### Additional information _No response_

mooglestiltzkin · April 28, 2024, 2:56pm

so i ran another smart search and noted down the exact activity in the logs

28/04/2024

[04/28/24 ] INFO     Setting 'ViT-B-32__openai' execution providers to  
28/04/2024

                             ['CUDAExecutionProvider', 'CPUExecutionProvider'], 
28/04/2024

                             in descending order of preference                  
28/04/2024

[04/28/24 ] INFO     Loading clip model 'ViT-B-32__openai' to memory    
28/04/2024

2024-04-28 xxxxxxxxxx [W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 15 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.

so all it output was this. no error i assume

mooglestiltzkin · April 28, 2024, 3:33pm

saw this

how to update

https://download.docker.com/linux/debian/dists/bookworm/

mooglestiltzkin · April 28, 2024, 3:41pm

well i did this

checks for updates but doesn’t update
sudo apt-get update

root@docker:~# apt list --upgradable

this updates anything it finds outdated
sudo apt-get update -y && sudo apt-get upgrade -y

hm

The following packages will be upgraded:
  docker-ce docker-ce-cli
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

then i exit restart jlmkr restart docker

jlmkr shell docker

root@docker:~# docker version
Client: Docker Engine - Community
 Version:           26.1.0
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        9714adc
 Built:             Mon Apr 22 xxxxxxx 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.0
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       c8af8eb
  Built:            Mon Apr 22 xxxxx 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cdxxxxxxxxxxx94a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
root@docker:~#

seems to be latest

do my docker containers work? ya tested all fine

*update

some good ideas for how to go about updating in jailmaker

Linux Jails (sandboxes / containers) with Jailmaker

Here’s how I keep the software in my FreeBSD jails up-to-date with TrueNAS Core.

WARNING. THIS IS A VERY COMPLICATED PROCEDURE:
pkg update -f && pkg upgrade

and if you are wondering
https://www.reddit.com/r/termux/comments/nxhpvj/pkg_vs_apt/

We strongly recommend to use a pkg utility instead of apt directly. It is a wrapper that performs a number of tasks:

Provides command shortcuts. Use “pkg in” instead of “pkg install” or “apt install”.

Automatically runs “apt update” before installing a package if necessary.

Performs some client side repository load-balancing by automatically switching mirrors on a regular basis. That is important to prevent us hitting quota limit on hosting.

is it alright to use apt?

As long as you manually run apt update before you try to install something you will be alright

mooglestiltzkin · April 28, 2024, 5:34pm

studying up cron jobs. so can schedule updates for docker and stuff

https://www.truenas.com/docs/scale/scaletutorials/systemsetting

credits to dan for the idea also to winnie

mooglestiltzkin · April 28, 2024, 5:37pm

hm, truenas > system settings > advanced

but how do u run a cron job within jailmaker shell docker to perform the pkg update -f && pkg upgrade

mooglestiltzkin · April 28, 2024, 5:39pm

Execute Command in Jail

You may want to execute a command inside a jail, for example from a shell script or a CRON job. The example below executes the env command inside the jail.

jlmkr exec myjail env

This example executes bash inside the jail with a command as additional argument.

jlmkr exec myjail bash -c 'echo test; echo $RANDOM;'

mooglestiltzkin · April 28, 2024, 5:42pm

so i assume

jlmkr exec myjail shell -c 'pkg update -f && pkg upgrade;'

is that correct? but it’s missing cron hm x-x;

mooglestiltzkin · April 28, 2024, 5:49pm

according to this, you specify a file type to run
https://www.ibm.com/docs/kk/aix/7.1?topic=c-crontab-command

To run a script called maintenance every day at midnight in August, enter the following: 0 0 * 8 * /u/harry/bin/maintenance

mooglestiltzkin · April 28, 2024, 5:50pm

to translate the numbers for cron and the formating used

mooglestiltzkin · April 28, 2024, 6:03pm

Found a solution for immich to copy pics from existing folders.

but does that create albums in immich based on those folders (which is what i want) ? also when he said copy files, is that a symlink? or an actual full sized file meaning you are now storing double the file size because you are storing 2 exact copies on the same machine? hopefully that is not the case.

mooglestiltzkin · April 28, 2024, 6:11pm

detailed explanation how immich handles the image imports/external libraries

If you are new to immich, think about this. Your only option for importing pics is either the upload, or the import external libraries.

Imagine you already have folders setup and you labeled them nicely e.g. cats, dogs… etc. So you want to add them to Immich to keep them separated by their folder structure you painstakingly did beforehand.

Immich by default can’t do that… instead, it all goes into one location. once done, then you can create albums, For which they then expect you to select the picks you want to put in that album.

So more work for you to do.

Question then, why can’t they simply create albums based on folders already setup?

Photoprism can do this already. But immich can’t That is Immich’s only sore point imho. But there are hacks around that but it ain’t pretty and potentially risk hazard if you accidentally run a malicious script some rando posted over the internet.

mooglestiltzkin · April 28, 2024, 7:17pm

credits to Davvo

mooglestiltzkin · April 28, 2024, 7:23pm

root@xxxx[~]# jlmkr exec docker bash crontab -e
bash: crontab: No such file or directory
root@xxxx[~]#

root@xxxx[~]# jlmkr shell docker
Connected to machine docker. Press ^] three times within 1s to exit session.
root@docker:~# crontab -e
bash: crontab: command not found
root@docker:~# bash crontab -e
bash: crontab: No such file or directory
root@docker:~#

mooglestiltzkin · April 28, 2024, 7:30pm

figured it out /cd derp

im tired and been using shell too much lately.

mooglestiltzkin · April 28, 2024, 7:41pm

root@xxx[~]# cd /mnt/xxxxxxxx/jailmaker/jails/docker
root@xxx[/mnt/xxxxxxxx/jailmaker/jails/docker]# select-editor

Select an editor. To change later, run ‘select-editor’.

/bin/nano <---- easiest

/usr/bin/mcedit

/usr/bin/vim.tiny

Choose 1-3 [1]: 3
root@xxxxx[/mnt/xxxxx/jailmaker/jails/docker]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab
“/tmp/crontab.0ZSNCD/crontab”:23: bad minute
errors in crontab file, can’t install.
Do you want to retry the same edit? (y/n)

i used nano but that didn’t work so i did this to switch to vim
select-editor

now i can follow instructions. but when i tried to save, it couldn’t. bit confused

Davvo · April 28, 2024, 7:44pm

Mine and @winnielinnie’s instructions were for FreeBSD jails. For Linux, refer to How to Edit Crontab in Linux? - Scaler Topics. Main difference is just vi commands.

mooglestiltzkin · April 28, 2024, 8:03pm

i’ll try work it out. ty appreciated