Managed to make it work only uploading the separate ollama using custom app (even the ollama with ROCm didn’t work)
It was also required to specify the rocm package and HSA_OVERRIDE_GFX_VERSION.
After that, ollama is working pretty quick with GPU support available.