How to Fine-Tune a Local LLM on an Apple Silicon Mac (MLX + LoRA, the Actual Commands)

Everyone assumes fine-tuning means renting a GPU, wrangling CUDA, and watching a cloud bill tick upward. For a lot of practical jobs, it doesn't. If you have an Apple Silicon Mac — even a 16GB Mac Mini — you can fine-tune a 7B model on your own machine, offline, in a single evening, using Apple's MLX.

This is the whole path, command by command: install it, format your data, run the training, and use the result. Every flag is explained so you're not copy-pasting magic. The example here trains a model to write in one fixed structure — but the exact same recipe works for a classifier, a formatter, a tone-matcher, or any "always respond like this" behavior.

One honest caveat up front, because it will save you a wasted weekend: fine-tuning teaches behavior, not facts. If you want the model to know things — your docs, your data — that's retrieval (RAG), not fine-tuning. Reach for the recipe below when you want to change how a model responds, not what it knows. More on that at the end.

When fine-tuning is actually the right move

A quick gut-check before you spend the evening:

You want the model to…	Right tool
Always answer in a fixed format / structure	Fine-tune ✅
Match a specific tone or writing style	Fine-tune ✅
Classify into your own categories reliably	Fine-tune ✅
Know facts from your documents	RAG / put it in the prompt
Use up-to-the-minute information	RAG / tools
Do something a good system prompt already nails	Just prompt it

If a careful system prompt already gets you 90% there, try that first — it's free and instant. Fine-tuning earns its keep when prompting plateaus and you need the behavior to be reliable across inputs you haven't seen.

Step 1 — Install MLX

One package pulls in everything (the framework, the LoRA trainer, an inference server):

pip install mlx-lm

That's it. No CUDA, no drivers. MLX runs on the Mac's unified memory and Metal GPU directly. This gives you four command-line tools you'll use below: mlx_lm.lora (train), mlx_lm.generate (one-shot inference), mlx_lm.server (OpenAI-compatible endpoint), and mlx_lm.fuse (bake the adapter into the model).

Step 2 — Pick a base model that fits in memory

The trick that makes this work on 16GB is starting from a 4-bit quantized base. The mlx-community org on Hugging Face has pre-quantized versions of most popular models, ready to go. For this walkthrough:

mlx-community/Qwen2.5-7B-Instruct-4bit

A 7B model at 4-bit is roughly 4GB of weights — it loads and trains comfortably on a 16GB machine with room to spare. You don't download it manually; the training command below pulls it on first run and caches it. If you have 8GB, drop to a 3B model (mlx-community/Qwen2.5-3B-Instruct-4bit); if you have 32GB+, a 14B is within reach.

Step 3 — Format your training data

MLX expects a folder with train.jsonl and valid.jsonl inside it. Each line is one training example: a JSON object with a messages array, exactly like a chat API call.

my_data/
├── train.jsonl
└── valid.jsonl

Each line looks like this (one line per example — pretty-printed here for readability):

{
  "messages": [
    { "role": "system", "content": "You write product blurbs in exactly three sentences: what it is, who it's for, one concrete detail. No marketing fluff." },
    { "role": "user", "content": "Write a blurb for a stainless steel insulated water bottle, 750ml, keeps drinks cold 24h." },
    { "role": "assistant", "content": "A 750ml stainless steel water bottle built to keep drinks cold for a full 24 hours. Made for commuters and gym-goers who refill once and forget about it. The double-wall vacuum seal means no condensation ring on your desk." }
  ]
}

Three things that matter more than they look:

The assistant message is what the model learns to imitate. Every example should be a gold-standard version of the behavior you want. Garbage-in is very real here — 30 excellent examples beat 300 mediocre ones.
You need fewer examples than you think. The run this post is based on used 30 training examples with 11 held out for validation and it measurably changed the model's behavior. Start small, see if it moves, then grow.
Keep your validation set truly separate. Don't just reword training examples into valid.jsonl — hold out different cases. (Why this matters so much is the entire subject of the companion post linked at the bottom.)

Step 4 — Run the training

Here's the actual command, with the exact settings from the run this is based on:

mlx_lm.lora \
  --model mlx-community/Qwen2.5-7B-Instruct-4bit \
  --train \
  --data my_data \
  --fine-tune-type lora \
  --num-layers 8 \
  --batch-size 1 \
  --iters 120 \
  --learning-rate 1e-4 \
  --max-seq-length 2048 \
  --mask-prompt \
  --adapter-path my_adapter \
  --save-every 100

What each flag is doing:

--fine-tune-type lora — train a small LoRA adapter instead of all the weights. This is the whole reason it fits in memory: you're learning a few million extra parameters that sit on top of the frozen base, not retraining 7 billion of them. The adapter comes out as a tiny file (a few megabytes).
--num-layers 8 — how many of the model's layers get an adapter attached. More layers = more capacity to learn, more memory. 8 is a modest, safe default for a focused behavior.
--iters 120 — how many training steps. With a small dataset this is enough to shift behavior without overcooking it.
--learning-rate 1e-4 — a standard LoRA learning rate. Too high and the model forgets how to speak; too low and nothing changes. 1e-4 is a good starting point.
--batch-size 1 — one example per step. Keeps memory low; fine for a dataset this size.
--max-seq-length 2048 — the longest example (prompt + answer) the trainer will handle, in tokens. Raise it if your examples are long, but it costs memory.
--mask-prompt — only train on the assistant's reply, not the prompt. You want the model to learn to produce the answer, not to memorize the questions. Leave this on.
--adapter-path my_adapter — where the trained adapter gets written.
--save-every 100 — checkpoint the adapter every 100 steps.

Two things I deliberately didn't pass, because MLX's defaults already match what the run used: the LoRA rank (8), scale (20), and dropout (0). If you ever want to change those, you pass a small YAML file with -c config.yaml containing a lora_parameters: block — but you rarely need to at the start.

While it runs, you'll see the training and validation loss print every few steps. You want validation loss trending down and then flattening. When it finishes, my_adapter/ holds your adapter.

Step 5 — Use the fine-tuned model

Quickest way to eyeball it — one-shot generation, base model plus your adapter:

mlx_lm.generate \
  --model mlx-community/Qwen2.5-7B-Instruct-4bit \
  --adapter-path my_adapter \
  --temp 0 \
  --prompt "Write a blurb for a mechanical keyboard with hot-swappable switches."

Note --temp 0. When you're checking whether the fine-tune worked, pin the temperature to zero so the output is deterministic — otherwise you're judging the dice, not the model. (This is a bigger deal than it sounds; it's a whole section in the companion post.)

To use it like a normal API — for an app, or to run a batch eval — start the OpenAI-compatible server:

mlx_lm.server \
  --model mlx-community/Qwen2.5-7B-Instruct-4bit \
  --adapter-path my_adapter \
  --port 8081

Now http://localhost:8081/v1/chat/completions speaks the OpenAI format. Any code or tool that talks to OpenAI can point at it by changing the base URL — no cloud, no key, no data leaving the machine.

Step 6 (optional) — Fuse the adapter for distribution

The adapter needs the base model alongside it. If you'd rather ship one self-contained model — to hand to a teammate, or to convert to another format — fuse them:

mlx_lm.fuse \
  --model mlx-community/Qwen2.5-7B-Instruct-4bit \
  --adapter-path my_adapter \
  --save-path my_fused_model

You now have a standalone model directory that behaves like the fine-tuned version with no adapter flag needed. For everyday local use you can skip this — --adapter-path at inference time is simpler.

Traps worth knowing before you start

Don't fine-tune facts into the model. This is the mistake that wastes weekends. If the model needs to know specific numbers, documents, or anything that changes, put that in the prompt (via retrieval) and fine-tune only the shape of the response. A model fine-tuned to "know" facts will state them confidently even when they've changed — you've baked in a hallucination.
Small, clean data beats big, messy data. A few dozen hand-checked examples move behavior. Thousands of scraped, inconsistent ones teach the model to be inconsistent.
Validate on unseen cases, not reworded training ones. A model can score perfectly on your validation set and still have learned nothing but your test answers. Hold out genuinely different examples.
Pin temperature to 0 when evaluating. A "did it improve?" comparison at temperature 0.7 is mostly measuring randomness.
Start with a system prompt. Seriously — if prompting alone gets the behavior, you just saved yourself the whole exercise. Fine-tune when prompting plateaus.

The recipe, in three lines

pip install mlx-lm, grab a 4-bit base from mlx-community, and put your examples in train.jsonl / valid.jsonl as chat messages.
Train a LoRA adapter with mlx_lm.lora (30 good examples is a real starting point), then run it with mlx_lm.generate or mlx_lm.server --adapter-path.
Fine-tune behavior and format, never facts — and keep your validation set genuinely unseen.

That last point is the one that separates a real result from a lucky-looking one. Once you have an adapter that seems to work, the next question is whether it actually works or just memorized your test set — which is exactly what I dug into in How to Prove a Fine-Tune Actually Reduced Hallucination. Train it here; trust it there.

2026.07.07