Whisper.cpp mode not working on mac

[andrea-carlon]

Buzz 1.4.4 on macOS Apple Silicon appears to launch whisper-server with an incompatible inference path.

Buzz log shows:

whisper-server --port 3003 --inference-path /audio/transcriptions ...

The server starts successfully and is reachable:

curl http://localhost:3003

returns the Whisper.cpp Server HTML page.

However, the server homepage shows the active endpoint as:

/inference

Testing Buzz’s expected endpoint returns 404:

curl -v http://localhost:3003/audio/transcriptions

returns:

HTTP/1.1 404 Not Found
File Not Found (/audio/transcriptions)

Model file is valid. For example, ggml-tiny.bin points to a 74 MB blob. Port 3003 is open and whisper-server is listening. The issue appears to be that Buzz expects /audio/transcriptions while the bundled whisper-server exposes /inference.

[raivisdejus]

@andrea-carlon Issue on Mac can be that when the model launches for the first time it needs to be compiled to the Mac optimized format. For larger models I have seen this taking even up to 20 minutes. While this compilation is in progress the ANECompilerService will be running in the process list / activity monitor. During this time you will get 404s

For tiny model compilation should be very quick.

A thing you can try is see if the issue is resolved in the latest development version

See if the ANECompilerService is running while Buzz is starting the whisper server

Copy the command Buzz runs to start the whisper server and run it manually in the terminal. That may give some useful information.