endpoint not Nextcloud Ai complatible

[Bobmorton-TX]

great project, decent speed using the rk3588 NPU. 👍

I am struggling to get the server to work with nextcloud AI

server on localhost:8080

I appears nextcloud is looking for this endpoint and cannot resolve it:

http://localhost:8080/models

anyone got this working?

[Bobmorton-TX]

It was a network port configuration issue. Nextcloud can find the endpoint but now reports this error:
Modelle konnten nicht geladen werden: Ungültige Modellantwort erhalten
Models could not be loaded: Invalid model response received

[danielferr85]

I will try to look this during the week. I have an old nextcloud instance waiting to update it (my old current version doesnt have AI assistant functionality). After update it, I will try the integration with RKLLAMA.

Can you tell me the exact version of your Nextcloud instance?

[danielferr85]

Hi, the problem is that NextCloud use the OpenAI compatible API that Ollama exposes.

RKLLama also expose some OpenAI services but /v1/models is not implemented yet. I will go to include this.

Stay tune....

[danielferr85]

Fixed in new PR.. Please wait for the owner to approve it.

Remember to use the OpenAI endpoint finishing with the "/v1". For example:

http://<your_rkllama_ip>:<your_port_rkllama>/v1

[Bobmorton-TX]

thanks for adding this functionality!
my versionis -- Nextcloud Hub 25 Autumn - (32.0.2)
I am using the docker of rkllama - is there a docker version of your PR that I can test?

[danielferr85]

No! But the author is always approving quickly the PR.

Meanwhile, you can build the image with my repo....

1. Clone my rkllama repo:
   git clone https://github.com/danielferr85/rkllama

2. Go inside the folder and build the image. For example:
   docker build -t danielferr85/rkllama .

3. Then you can run your docker (or with compose) with the image name "danielferr85/rkllama" or whatever your named.

[Bobmorton-TX]

Hello Daniel!
Thanks for the fix and the quick info.
I ran your docker instruction an have a container running.

It seems that internally it changed from using .sh scripts to the command rkllama_client

I am unable to load any model unfortunately. Can you confirm this works with the docker container?

`root@2f6dc2c76b94:/opt/rkllama# rkllama_client list
Available models:
- qwen
- tinyllama
root@2f6dc2c76b94:/opt/rkllama# 
root@2f6dc2c76b94:/opt/rkllama# rkllama_client info tinyllama
Model Information: tinyllama
--------------------------------------------------
Family: llama
Parameter Size: 1.1B
Quantization Level: Q8_0
Size: 1.10 GB
Modified At: 2025-12-10T14:52:17.653902Z
License: Unknown
System Prompt: None
Template: {{ .Prompt }}
Hugging Face Info:
  Repo ID: punchnox/Tinnyllama-1.1B-rk3588-rkllm-1.1.4
  Description: 
  Tags: llama, en, dataset:cerebras/SlimPajama-627B, dataset:bigcode/starcoderdata, dataset:HuggingFaceH4/ultrachat_200k
  Downloads: 16
  Likes: 3
Advanced Model Info:
  general.architecture: llama
  general.base_model.0.name: tinyllama 1.1B
  general.base_model.0.organization: Llama
  general.base_model.0.repo_url: https://huggingface.co/punchnox/Tinnyllama-1.1B-rk3588-rkllm-1.1.4
  general.base_model.count: 1
  general.basename: tinyllama
  general.file_type: 15
  general.languages: ['en']
  general.parameter_count: 1100000000
  general.quantization_version: 2
  general.size_label: 1.1B
  general.tags: ['chat', 'text-generation']
  general.type: model
  llama.attention.head_count: 32
  llama.attention.head_count_kv: 4
  llama.attention.layer_norm_rms_epsilon: 1e-06
  llama.block_count: 32
  llama.context_length: 4096
  llama.embedding_length: 4096
  llama.feed_forward_length: 11008
  llama.rope.freq_base: 10000
  tokenizer.ggml.pre: llama
root@2f6dc2c76b94:/opt/rkllama# rkllama_client run tinyllama
Error loading model: 400 - Unexpected Error loading the model tinyllama into memory. Check the file .rkllm is not corrupted, properties in Modelfile (like Context Length allowed by the model) and resources available in the server
root@2f6dc2c76b94:/opt/rkllama# `