Open WebUI Support

[meminens]

Does anyone have instructions to make ramalama work with open webui?

[ericcurtin]

It's dependant on this feature being implemented:

#598

[ericcurtin]

This is something we should prioritise, open webui has some rag features that would be worth exploring. It's just finding someone with the cycles

[hanthor]

Alpaca expects an Ollama server. But any standard API would work to be able to add connectivity to Alpaca.

https://github.com/Jeffser/Alpaca/wiki/Packaging-Alpaca#ollama-involvement

[ericcurtin]

Btw just saying, used this client today with ramalama works flawlessly:

https://github.com/Mintplex-Labs/anything-llm

I actually prefer it, one doesn't have to login, etc.

But we will probably get to open webui support eventually

[mrwsl]

Thanks for providing an alternative! What do you choose as the llm provider in anything-llm? Just Ollama?

[ericcurtin]

"Generic OpenAI"

Personally I actually prefer anything-llm after playing around with it, no need to create user accounts etc.

It also can connect to ChatGPT, Perplexity, etc. Seems to connect to any LLM service you throw at it.

[ericcurtin]

@meminens @hanthor @mrwsl

I figured it out:

ramalama serve some_model

and

podman run -it --rm --network slirp4netns:allow_host_loopback=true -e OPENAI_API_BASE_URL=http://host.containers.internal:8080 -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

get open webui working just fine for me.