Your Mac Already Has a Built-In LLM. This CLI Tool Unlocks It.

📖 11 min read

Quick Verdict: apfel is a free, open-source Swift CLI that pipes your Mac’s built-in Apple Intelligence LLM through your terminal. One brew install and you get a UNIX tool, an OpenAI-compatible local server on port 11434, and an interactive chat — all 100% on-device, no API keys, no model downloads. It is genuinely useful for shell scripting, code review, and quick AI features in apps that already speak OpenAI’s API. The catch: it needs macOS 26 Tahoe on Apple Silicon with Apple Intelligence enabled, the underlying model is small (~3B parameters with a 4,096-token context), and you cannot swap in a bigger model. For the niche it targets, it is hard to beat. Rating: 8/10

Discovered: April 2026

What It Is

apfel is a small Swift binary, released under MIT and currently sitting around version 1.6.x, that wraps Apple’s Foundation Models framework — the same on-device language model that powers Siri, Writing Tools, and the rest of Apple Intelligence — and exposes it as three practical interfaces:

A UNIX command-line tool. Pipe prompts in, get answers out. Supports stdin, file attachments, JSON output, streaming, system prompts, proper exit codes, and token counting.
An OpenAI-compatible HTTP server. Run apfel --serve and you get a local endpoint at http://127.0.0.1:11434/v1/chat/completions that any OpenAI SDK, LangChain, or curl request can talk to. The default model name is apple-foundationmodel.
An interactive chat REPL. Multi-turn conversations with automatic context trimming and optional MCP tool servers attached.

Under the hood it is just a thin Swift 6.3 wrapper around LanguageModelSession (Apple’s SystemLanguageModel), with Hummingbird handling the HTTP layer. The maintainer (Arthur-Ficial) has done the work Apple didn’t bother to ship: a real CLI, real exit codes, JSON output, and a server that speaks the de facto local-LLM protocol pioneered by Ollama. The binary is published to Homebrew core, so installation is a single line.

Why It Matters

Starting with macOS 26 Tahoe, every Apple Silicon Mac (M1 and newer) ships with a roughly 3-billion-parameter language model baked into the OS as part of Apple Intelligence. Inference runs on the Neural Engine and GPU, with no network round-trip, no telemetry, and no API key. By default that model is locked inside Apple apps: it summarizes things for you in Mail, rewrites paragraphs in Notes, and answers Siri’s questions. There is no terminal command, no HTTP endpoint, no way for a third-party app to even know it exists unless it links against Foundation Models and writes Swift code.

That gap is exactly what apfel fills. The practical consequences are larger than they look:

Cost. The model is free forever, with no per-token billing, no subscription, and no rate limits. Anyone running thousands of small AI calls per day for shell scripting, commit hooks, or local automation stops paying OpenAI or Anthropic for them.
Privacy. Nothing leaves the device. There is no cloud, no proxy, no telemetry in the binary. For regulated work — legal drafts, medical notes, source code under NDA — that is a real advantage, not a marketing claim.
Latency. On an M-series chip the first token typically streams in well under a second, with no internet round-trip. For interactive shell use that matters more than benchmark scores.
Composability. Because it speaks OpenAI’s API on localhost, you can swap it into existing tools — Continue, Cursor with a custom base URL, LangChain, LlamaIndex, anything that points at a local endpoint — without code changes.

In other words, apfel turns a feature that Apple gated behind Swift app development into something a sysadmin or weekend scripter can use in five minutes.

How to Install It

The short version is one command, assuming you meet the requirements:

brew install apfel

That’s it. The binary lands in /opt/homebrew/bin/apfel (Apple Silicon Homebrew) or /usr/local/bin/apfel (Intel Homebrew, though Intel is not supported — see below). No model download, no ollama pull, no vllm startup script. The model is already on disk as part of macOS.

Requirements, spelled out:

Apple Silicon. M1, M2, M3, M4 — any generation. Intel Macs are not supported; the model only runs on the Apple Neural Engine.
macOS 26 Tahoe. This is the cut-off. macOS 15 Sequoia does not include the Foundation Models framework for third-party use; macOS 14 and earlier definitely do not.
Apple Intelligence enabled. System Settings → Apple Intelligence → toggle on. If your device or region is not on Apple’s supported list, the Foundation Models calls will return a “model unavailable” error at runtime.
Supported device language/region. Apple’s on-device model currently ships with broad language coverage (English, German, Spanish, French, Italian, Japanese, Korean, Portuguese, Simplified Chinese) but the system must be set to one of those locales for inference to work reliably.

If you’d rather build from source — useful if you want the latest unreleased features or you are on Nix/Mint/mise — the repo also documents a clean build path with just the macOS 26.4 SDK and Swift 6.3 (no full Xcode install required):

git clone https://github.com/Arthur-Ficial/apfel.git
cd apfel
make install

To check the install worked, run something small and confirm you get a real answer back, not an error about the model being unavailable:

apfel "What is the capital of Austria?"
# The capital of Austria is Vienna.

Real Test: What It Actually Does

Once installed, the most common mode is one-shot CLI calls. The interface is deliberately UNIX-y — quote a prompt, pipe text in, or attach files with -f:

# Single prompt
apfel "Summarize this commit message in plain English"

# Streaming
apfel --stream "Write a haiku about a Mac running an LLM"

# File attachment
apfel -f README.md "Summarize this project"

# Multiple files
apfel -f old.swift -f new.swift "What changed between these two versions?"

# Pipe a diff in, attach the project's conventions file
git diff HEAD~1 | apfel -f CONVENTIONS.md "Review this diff against our conventions"

# JSON output for scripting
apfel -o json "Translate to German: hello" | jq .content

# Pre-flight token check before a large prompt
apfel --count-tokens -f README.md "Summarize this"

# System prompt
apfel -s "You are a pirate" "What is recursion?"

# Quiet mode, exit code, stdout capture
result=$(apfel -q "Capital of France? One word.")

For longer workflows, the OpenAI-compatible server is the killer feature. Start it in the foreground for testing or as a background service for production-like use:

apfel --serve
# Server running on http://127.0.0.1:11434

brew services start apfel    # background, like Ollama
brew services stop apfel

Then any OpenAI client works unchanged — you just point it at http://127.0.0.1:11434/v1:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="unused")

resp = client.chat.completions.create(
    model="apple-foundationmodel",
    messages=[{"role": "user", "content": "What is 1+1?"}],
)
print(resp.choices[0].message.content)

or with plain curl:

curl http://localhost:11434/v1/chat/completions 
  -H "Content-Type: application/json" 
  -d '{"model":"apple-foundationmodel","messages":[{"role":"user","content":"Hello"}]}'

For exploratory sessions, apfel --chat opens an interactive REPL with multi-turn context, optional system prompts, and Ctrl-C to quit. Context is trimmed automatically using one of five strategies the project documents, so you do not have to manage the 4,096-token window manually.

One of the more interesting additions is Model Context Protocol (MCP) support. You can attach any MCP-compatible tool server — local Python file, remote Streamable HTTP endpoint, even multiple at once — and apfel discovers the tools, calls them when appropriate, and feeds results back into the conversation:

apfel --mcp ./mcp/calculator/server.py "What is 15 times 27?"
# tool: multiply({"a": 15, "b": 27}) = 405
# 15 times 27 is 405.

MCP works in all three modes (CLI, server, chat), which means you can give the on-device model access to your filesystem, a database, a weather API, or anything else you can wrap in an MCP server — without writing glue code. A small calculator MCP server ships in the repo as proof-of-concept.

The repo also bundles a set of apfel demos shell scripts that you can write out anywhere with a single command. They cover common patterns: natural-language-to-shell-command (cmd), complex-pipe-chain-from-English (oneliner), a Mac narrator that describes what’s happening on the system, a “what’s this directory?” orienter, a port-finder, and so on. They are not magic — they are just well-written wrappers around the CLI — but they show what the interface is good for.

Performance and Speed

The underlying model is roughly 3 billion parameters, quantized to about 3.5 bits per weight with mixed 2/4-bit precision for certain layers. That is small by 2026 standards — Llama 3 70B, Qwen 3, and the larger Mistral variants are an order of magnitude bigger — but it is also what allows the model to fit in a few gigabytes of RAM and run entirely on the Neural Engine. You do not need a 64 GB Mac Pro; an M1 with 8 GB unified memory handles it fine.

Real-world throughput on an M2 or M3 is typically in the range of 30–60 tokens per second for short prompts, with first-token latency well under a second once the model is warm. Cold starts (the very first call after boot, or the first call after the model has been swapped out of memory) are noticeably slower — sometimes 2–4 seconds — but subsequent calls are fast. Streaming mode (apfel --stream) makes this feel snappy even for longer outputs, because the user sees tokens as they generate.

Quality is the obvious trade-off. Compared to a frontier cloud model, apfel’s responses are competent but visibly smaller — less nuanced prose, more conservative answers, occasional confusion on multi-step reasoning or large code refactors. For the things it is actually designed for — short summaries, code review notes, commit messages, simple translations, JSON extraction, natural-language-to-shell-command — it is good enough that you stop noticing. For long-form creative writing or hard analytical work, you will want a bigger model.

Two practical notes on speed: first, the --count-tokens flag is worth using before any prompt that approaches the 4,096-token context window, because the model will silently truncate otherwise. Second, JSON output (-o json) is slightly faster than pretty-printing and is the right default for any scripting pipeline.

Limitations and Gotchas

Apple Silicon only. Intel Macs, Hackintoshes, and macOS VMs on non-Apple hardware will not work. The Foundation Models framework requires the Neural Engine.
macOS 26 Tahoe required. If you are on macOS 15 Sequoia, the framework is not available to third-party apps and apfel will refuse to start. Plan an OS upgrade before adopting it.
Apple Intelligence must be enabled. If the user has toggled Apple Intelligence off in System Settings, every call will fail with a “model unavailable” error. This is the most common cause of “I installed it and it does nothing.”
Region and language. The model only fires reliably on systems set to a supported language/region. If your Mac is set to a locale Apple does not support for Apple Intelligence, you will need to change it — which has knock-on effects on date formats, sorting, and a few other system behaviors.
4,096-token context window. That is small. Anything that involves pasting a long file, a long conversation, or a long document plus a system prompt will hit the ceiling. apfel’s chat mode handles trimming automatically; the one-shot CLI does not — you must use --count-tokens yourself or risk silent truncation.
You cannot swap in a bigger model. Unlike Ollama or LM Studio, you are not loading arbitrary GGUF files. apfel exposes the system Foundation Model and nothing else. WWDC 2026 did open the Foundation Models framework to other backends, and there is now an MLX-based MLXLanguageModel conformance, but apfel as shipped uses Apple’s on-device model only.
Quality ceiling. For tasks that genuinely need a large frontier model — long-form reasoning, complex multi-file refactors, nuanced prose — a 3B on-device model will visibly struggle. Treat it as a fast, free, private default for the easy 80% of calls, not a replacement for GPT-5-class reasoning.
Permissive mode is not a license to ignore safety. The --permissive flag exists to reduce guardrail false positives on long creative prompts; it does not unlock anything Apple has explicitly blocked. If the Foundation Model refuses a request, --permissive will not change that.
Shell quoting in zsh. Because zsh performs history expansion on !, single-quote your prompts when they contain exclamation marks: apfel 'Hello, Mac!' rather than the double-quoted form.
Server port collision. The default port (11434) is the same as Ollama’s. If you happen to run both, you will get an “address in use” error. Either stop one of them, or front both behind a reverse proxy.

Who Should Use This

apfel is a strong fit for a fairly specific reader. If you are a developer or sysadmin on a recent Mac who wants a free, private, zero-config local LLM for shell scripts, commit hooks, code review notes, JSON extraction, or any other “ask an LLM something small, thousands of times a day” workflow, this is one of the best options available in mid-2026. The OpenAI-compatible server mode makes it especially attractive if you already have tooling that talks to OpenAI — you can point it at localhost and stop paying per token for the easy calls.

It is also genuinely useful as a default model inside larger pipelines: route simple prompts to apfel (free, fast, private) and only escalate to a cloud model when the local one returns low confidence or refuses the task. That kind of cascade is easy to set up because the API surface is identical to OpenAI’s.

It is not the right tool if you need a 70B-class reasoner, if you want to fine-tune or train your own model, if you are still on macOS 15 or earlier, or if your work happens on non-Apple hardware. For those cases, Ollama, LM Studio, or a hosted API are the right answer. apfel does not try to compete with them — it competes with “not having a local LLM at all.”

Final Verdict

apfel is a small, well-designed piece of glue that turns a feature Apple shipped locked behind Swift app development into a UNIX tool. The interface is clean, the install is one command, the OpenAI-compatible server is a genuine productivity multiplier, and the on-device guarantee is real. The 3-billion-parameter ceiling and the macOS-26-on-Apple-Silicon requirement are real constraints, but they are honest ones — the tool does not pretend to be something it is not.

For developers who already live on a Mac and want a free, private local LLM that “just works” without a model download or an account, apfel is the easiest recommendation I have made this year. Eight out of ten, with the two missing points going to the small context window and the lack of model flexibility. If Apple ships a larger Foundation Model next year, or if the project adds first-class support for swapping in MLX models via the new MLXLanguageModel conformance, this easily becomes a nine.