I have installed Ollama and anything LLM but as far as i can tell my server is to slow to run those apps. My apps are working fine but when i put in hello or 2+2 i get an error about my networking error. Can anyone please help Thank you
I expect this is a case where you’d need a compatible GPU.
Would you have any suggestions. I would not even know where to start for that. The two apps can run from a GPU. Thank you
Not really; I know very little on the subject myself. Here’s a topic where I’m asking for some guidance:
Based on the video I linked there recently, I just bought an Nvidia V100 module, but I’m still waiting for a PCIe adapter card. Once that arrives, I’ll see what if any use I can put it to.
Thank you. I appreciate your help. I too will look for a GPU That I can install.
The Mini-R may not be suitable for a GPU. Check the type of PCIe slot available and what you have for a power supply. It may be fine, but it’s better to check before buying a GPU.
If i may say, i think you have 2 different problems there:
are you sure that both container can “see” each other?
without a dedicated GPU, there are some model that run pretty well but with off course limited capacity. Nothing that can be actually comparable to basically every “free” online version, but that can be good for other minor tasks (eg Gemma 2b is not multimodal but si pretty fast also on my old gen 6 Intel).
If you put a GPU, it should have a large amount of vram, otherwhise performance will still be poor (with the offloading). IMHO under 8gb of vram is not either worth spend money, unless you really don’t pay so much the GPU itself