Chat not working "AI not responding"

[RipleyTom]

I'm using a llama-server on a mini PC with tls on with a self signed cert. I have specified the cert in llama.vscode settings.
It works for code completion(I can see the /infill requests on the llama-server logs and it shows the completion in vscode) but chat doesn't seem to work(no request visible in llama-server logs), AI not responding is all it says.

console log

console.ts:137 [Extension Host] Webview received message: {command: 'sendText', text: 'This is a test!'}command: "sendText"text: "This is a test!"[[Prototype]]: Object
console.ts:137 [Extension Host] PostMessage command called with: {command: 'updateText', text: '***This is a test!***\n\n'}command: "updateText"text: "***This is a test!***\n\n"[[Prototype]]: Object
console.ts:137 [Extension Host] Webview found, sending message
bundle.js:2 Received message from extension: {command: 'updateText', text: '***This is a test!***\n\n'}command: "updateText"text: "***This is a test!***\n\n"[[Prototype]]: Object
console.ts:137 [Extension Host] PostMessage command called with: {command: 'updateText', text: '***This is a test!***\n\nNo response from AI  \n'}command: "updateText"text: "***This is a test!***\n\nNo response from AI  \n"[[Prototype]]: Object
console.ts:137 [Extension Host] Webview found, sending message
bundle.js:2 Received message from extension: {command: 'updateText', text: '***This is a test!***\n\nNo response from AI  \n'}command: "updateText"text: "***This is a test!***\n\nNo response from AI  \n"[[Prototype]]: Object
console.ts:137 [Extension Host] PostMessage command called with: {command: 'updateCurrentState', text: 'AI not responding'}command: "updateCurrentState"text: "AI not responding"[[Prototype]]: Object
console.ts:137 [Extension Host] Webview found, sending message
bundle.js:2 Received message from extension: {command: 'updateCurrentState', text: 'AI not responding'}command: "updateCurrentState"text: "AI not responding"[[Prototype]]: Object

[RipleyTom]

For some reason an error appeared in llama-server logs with TLS mode disabled:

srv    operator(): got exception: {"error":{"code":500,"message":"Can only get item pairs from a mapping at row 47, column 78:\n        {{- '\\n<parameters>' }}\n        {%- for param_name, param_fields in tool.parameters.properties|items %}\n                                                                             ^\n            {{- '\\n<parameter>' }}\n at row 47, column 9:\n        {{- '\\n<parameters>' }}\n        {%- for param_name, param_fields in tool.parameters.properties|items %}\n        ^\n            {{- '\\n<parameter>' }}\n at row 40, column 29:\n    {{- \"<tools>\" }}\n    {%- for tool in tools %}\n                            ^\n        {%- if tool.function is defined %}\n at row 40, column 5:\n    {{- \"<tools>\" }}\n    {%- for tool in tools %}\n    ^\n        {%- if tool.function is defined %}\n at row 37, column 51:\n{%- endif %}\n{%- if tools is iterable and tools | length > 0 %}\n                                                  ^\n    {{- \"\\n\\nYou have access to the following functions:\\n\\n\" }}\n at row 37, column 1:\n{%- endif %}\n{%- if tools is iterable and tools | length > 0 %}\n^\n    {{- \"\\n\\nYou have access to the following functions:\\n\\n\" }}\n at row 1, column 86:\n{# Copyright 2025-present Unsloth. Apache 2.0 License. Unsloth Chat template fixes #}\n                                                                                     ^\n{% macro render_item_list(item_list, tag_name='required') %}\n","type":"server_error"}}
srv  log_server_r: request: POST /v1/chat/completions 192.168.1.192 500

Realizing that it probably had something to do with tools I disabled all the tools and lo and behold, chat now works.
I'll just assume I'm doing something wrong or have setup the server wrong in some way :D.

[RipleyTom]

Ok I digged further into this, dumped the query, and the issue is actually from llama.vscode, partially at least, it's failing a validation with this tool:

{"type":"function","function":{"name":"get_diff","description":"Gets the files changes since last commit","parameters":{"type":"object","required":[]},"strict":true}}

The jinja templated into the model I'm using(Qwen3-Coder-30B-A3B-Instruct-Q8_0.gguf) has:

        {{- '\n<parameters>' }}
        {%- for param_name, param_fields in tool.parameters.properties|items %}

And since the properties field is missing on the get_diff tool, the jinja template fails, produces an exception on the server and results in a 500 on the request.

[igardev]

@RipleyTom Very good! I will add properties field for get_diff tool in the next version.

As for the self signed cert - yes, it is currently implemented only with code completion. Makes sense to implement it for the other models as well.