Problem/Justification
After getting Ollama and OpenWebUI, image generation is still missing from the catalogue. This makes OpenWebUI only half-baked compared to OpenAI.
Impact
OpenWebUI could be a proper self-hosted private OpenAI replacement if we had an image generation app. This technology is gaining more importance and should be accessible to everyone.
User Story
Content creators can leverage their TrueNAS not only for storage, but also for content creation like script improvements or thumbnail generation.
Having a full AI suite could be a selling factor for a TrueNAS system for a business.
Homelab people can educate themselves and their surroundings into using AI in a local and therefore private way.
I remember a long time ago when NAS servers were used for storage.
Is it a cost reason why it’s preferable to cook your system with a hot, high-end GPU directly on the same server that is supposed to keep your data reliably secured?
Can’t have a dedicated AI machine or a repurposed gaming rig that saves its generations to the NAS over SMB on the network?
I think thats a perspective thing. Having many services on one machine is way more efficient for me, hardware- and power-wise. Thats the beauty of containers. I need the GPU for transcoding anyway, so why not also use it for diffusion as well? This way I only need to have one set of hardware and only one system powered on 24/7. As each service is only used sporadically, it works great for me.
And btw I could not imagine a better place for a hot GPU than a well ventilated server chassis.
Having seen my Nvidia GPU lock up more than once with different AI software, I wouldn’t feel comfortable if it was affecting my NAS server. In my opinion, reliable storage and data protection is the highest priority for having a NAS.
With TrueNAS, how do you deal with the fact that some of these AI solutions require a lot of swap? Some even require up to 40 GB of swap.
I fear that ComfyUI as an official App offered by iX would divert resources and troubleshooting away from what really matters. It reads like “scope creep”.
I totally get your point and I highly respect someone having dedicated servers for dedicated services. Thats how it should be done professionally.
For me its not a big deal if the server locks up or reboots if an error happens. As long as it not corrupts any data at rest, I am fine with it. And that I highly doubt.
I am aware that stable diffusion is very memory hungry, which in my opinion makes it even more suitable to be run on a ZFS-based NAS, because it should have a lot of memory. So I hope its enough for the diffusion models I plan to use and I don’t run into swap. If it is required though, I dont know. Maybe map some SSD space that the app can use as swap?
But I also get that the unusual memory usage might need iX ressources to implement cleanly - thats probably one reason why its not done yet.
But, iX has the steam headless app which makes your truenas a game server. Playing the right game (eg. citis skylines) you can also eat a lot of memory, and truenas seems to handle that fine, at least in my testing.
Use an SDXL model with a refiner and LoRAs on high resolution images on Windows with the page size set to “automatic”.
Watch what happens to your Windows paging file size. You’ll see it easily climb to 20,000 MB and higher. This is why people often are confused by “insufficient memory” errors when they restrict their paging file size to a lower value, and why it’s recommended to allow Windows to automatically grow the size as needed.
How will this seamlessly work on a TrueNAS system? I’m not sure. Perhaps you’re right in that you’ll need to dedicate a swap file on an SSD.
I didn’t run it on my NAS. (I will never do that.) I have a different Windows 11 PC for heavy loads which doesn’t hold important data.
The system remains running (fans, lights, power), but the display freezes and there’s no response from any input. If I unplug the display and plug it back in, the monitor says “No video input”. A hard reboot is required.
It doesn’t happen often, but it’s still enough of a problem that I don’t want it to interfere with my NAS being a NAS.
I’ve been using my nvidia gpu for transcoding in jellyfin for over a year and never had lockups or driver problems. For me it was basically set and forget.
I managed to run ComfyUI as a custom docker app, worked fine until I increased the image resolution beyond 1024x1024. Then the system rebooted sontaneously a few images into a batch generation. I dont know why, will investigate. I am still a novice when it comes to docker and Comfy, so that might be on me. Lower res worked without issues so far. Here a a few glimpse on the setup:
