Running on-device Apple Intelligence locally through an API (with Open WebUI or others)

Claude and I have created an API that exposes the Apple Intelligence foundation on-device model to use with the OpenAI API standard on a specified port. You can use the on-device model with open-webui. It's quite fast actually. My project is located here: https://github.com/scouzi1966/maclocal-api .

For example to use with open-webui:

1. Follow build instuctions with requirements. For example "swift build -c release"
2. Start the API . For example ./.build/release/MacLocalAPI --port 9999
3. Create an API endpoint in open-webui. For example http://localhost:9999/v1
4. a model called 'foundation' should be selectable

This requires MacOS 26 Beta (mine is on 5) and an M series Mac. Perhaps xCode is required to build.

Read about the model here:

https://machinelearning.apple.com/papers/apple_intelligence_foundation_language_models_tech_report_2025.pdf