How to Set Up Ollama with MLX Acceleration on Your Apple Silicon Mac

If your local AI models feel slower than they should, you’re not imagining it. Usually the inference engine underneath is the real bottleneck, not your Mac’s hardware. This guide walks you through installing Ollama 0.19 or later, pulling an MLX-tagged model, and confirming that Apple’s MLX framework, not a generic CPU fallback, is actually doing the work.

By the end, you’ll have a local AI chat setup running entirely on your Mac. No cloud account, no subscription, and if your hardware qualifies, a real speed boost from Apple Silicon.

What Is Ollama with MLX?

Ollama is a free, open-source tool that downloads and runs open AI language models directly on your computer. Think of it as a lightweight, Terminal-friendly alternative to ChatGPT or Claude that never sends your prompts anywhere. You pull a model with one command and chat with it using another.

Starting with version 0.19, Ollama added support for MLX, Apple’s own machine-learning framework built for Apple Silicon’s unified memory architecture. Before this, Ollama ran on llama.cpp (via a Metal backend) on Macs, the same engine it used on Windows and Linux. MLX gives it a second inference engine tuned specifically for M-series chips. Ollama’s own benchmarks show faster response times on models tagged with an -mlx suffix. This is currently a preview feature: not every model has an MLX-tagged build yet, and Ollama recommends at least 32GB of unified memory for the best experience. If you’ve been running Ollama for a while and haven’t updated, it’s worth doing now. The switch replaces your old inference path without changing how you use the tool day to day.

Before You Begin

Make sure you have:

• An Apple Silicon Mac (M1, M2, M3, M4, or M5); MLX does not run on Intel Macs
• macOS Sonoma 14 or later installed (macOS 26.2 or later if you’re on an M5 chip and want Neural Accelerator gains)
• At least 10-20GB of free disk space for a first model (larger models need more)
• Ideally more than 32GB of unified memory, since Ollama’s MLX preview targets this range for the best experience
• Basic comfort using the Terminal app
• An internet connection for the initial download (no account or sign-in required for local use)

Requirement	Details
Device	Mac with Apple Silicon (M1 or later)
OS version	macOS 14 (Sonoma) or later, check in Apple menu > System Settings > General > About
Ollama version	0.19 or later, for MLX preview support
Memory	32GB+ unified memory recommended for MLX acceleration
Disk space	10-20GB minimum, more for larger models

You do not need an Apple ID, Ollama account, or any subscription to run models locally. Ollama’s paid Pro/Max plans only apply to its optional cloud model hosting, which this guide doesn’t cover.

Step-by-Step Guide

Step 1: Confirm You Have an Apple Silicon Mac

MLX acceleration only works on Apple Silicon. Open Terminal (find it via Spotlight with Cmd + Space, then type “Terminal”) and run:

uname -m

You should see arm64 in the output. If you see x86_64 instead, you have an Intel Mac, and MLX acceleration won’t be available. Ollama will still run using the standard backend, just without the MLX speed boost.

Step 2: Check Your macOS Version

MLX performance improvements, especially on M5 chips, depend on having a current macOS release. Go to Apple menu > System Settings > General > Software Update and confirm you’re on macOS 14 (Sonoma) or later. If you have an M5 Mac, aim for macOS 26.2 or later to get the full benefit of its Neural Accelerators.

If an update is available, install it now. Running an outdated macOS version is one of the most common reasons MLX doesn’t perform as expected.

Step 3: Install Ollama on macOS

You have three ways to install Ollama on a Mac. Pick whichever fits your workflow.

Option A: Install with Homebrew

If you already use Homebrew, this is the fastest path and makes future updates a one-liner:

brew install ollama

Option B: Install with the official install script

This downloads and installs the app directly from Ollama’s servers:

curl -fsSL https://ollama.com/install.sh | sh

Option C: Download the DMG manually

Visit ollama.com/download, download the macOS .dmg file, open it, and drag the Ollama app into your Applications folder. Launch it once from Applications to start the background service.

Tip: Homebrew is generally the easier long-term choice if you already use brew upgrade; it keeps Ollama current alongside your other command-line tools. The installer script or DMG works just as well if you’d rather skip Homebrew entirely.

Step 4: Confirm Your Ollama Version

Whichever method you used, verify you’re on version 0.19 or later. That’s the version where MLX support showed up. In Terminal, run:

ollama --version

You should see something like ollama version 0.19.x or higher. If MLX matters to you, this number is the single most important thing to check before anything else.

If your version is older than 0.19, update it:

brew upgrade ollama

Or, if you installed via script or DMG, just re-run the install script from Step 3. It’ll overwrite the older version.

Step 5: Check the Ollama Menu Bar App

On macOS, Ollama runs as a background process. Look for a small icon in your menu bar, near the clock at the top right of your screen. Click it to see a simple menu with options to view running models, open settings, or quit. This confirms the local server is active. Ollama uses this background process to serve requests from Terminal or any app you connect to it.

Step 6: Browse Available Models

Before pulling a model, take a look at what’s available. Open ollama.com/library in your browser, no sign-in needed. Search for models with an -mlx tag; these are the versions built to use Ollama’s MLX engine.

Tip: Not every model has an MLX-tagged version yet. If you don’t see one for a model you want, check back later. Or run the standard (non-MLX) tag while you wait; Ollama will just use its default backend instead.

Step 7: Pull Your First MLX Model

Back in Terminal, download a model tagged for MLX. For this example, we’ll use gemma4:12b-mlx. It’s a mid-sized model, a reasonable fit for Macs with 32GB or more of unified memory:

ollama pull gemma4:12b-mlx

This downloads the model’s weights to your Mac. Expect anywhere from a few gigabytes to well over ten, depending on the model size. Watch the progress bar in Terminal until it completes.

Warning: If you’re on a Mac with 16GB of RAM or less, choose a smaller model (an 8B parameter model or smaller) instead of a 12B+ model. Larger models can exhaust your memory and make your whole Mac sluggish.

Step 8: Run and Chat With the Model

Once the download finishes, start a chat session:

ollama run gemma4:12b-mlx

This drops you into an interactive prompt right in Terminal. Type a question and press Return to see the model respond.

Type a message like this to test it:

What’s a good weeknight dinner idea using chicken and rice?

You should get a response within a few seconds, depending on your Mac’s specs. Type /bye to exit the chat when you’re done.

Step 9: Verify MLX Is Actually Being Used

This is the step people skip, and it’s the one that actually matters. Ollama can quietly fall back to its older backend if MLX can’t initialize, and nothing on screen will scream about it. Check it these three ways.

Check response speed and system behavior. Open Activity Monitor (Spotlight search: “Activity Monitor”) while a model is running, and switch to the Memory tab. With MLX active on a supported Mac, you’ll typically see high, efficient memory usage tied to the ollama process. CPU usage stays calm instead of thrashing. MLX is built to lean on the GPU cores and unified memory rather than pure CPU cycles.

Check the terminal output for backend errors. If MLX fails to load, Ollama usually prints a message referencing the MLX runner or falls back with a warning instead of a hard crash. A clean run with no such warning, on Apple Silicon, with Ollama 0.19+, using an -mlx tagged model, is a strong sign MLX is active.

Compare against a non-MLX tag. As a sanity check, pull and run the standard (non-MLX) version of the same model, if one exists. Compare response speed:

ollama run gemma4:12b

If the -mlx version is noticeably faster on the same prompt, that’s your confirmation MLX acceleration is doing real work.

Check the server log directly (most reliable). Open a second Terminal tab and run tail -f ~/.ollama/logs/server.log, then start a chat session in your original tab. Watch for a line mentioning mlx or --mlx-engine as the model loads; that’s Ollama confirming which runner it picked. If MLX never shows up in the log, it fell back to the standard backend even if everything else looked normal.

Step 10: Install and Verify on Windows (Brief Overview)

MLX is an Apple-only framework, so none of this applies on Windows. But if you’re helping a family member set up Ollama on a Windows PC alongside your Mac, here’s the quick version.

Open PowerShell and run:

irm https://ollama.com/install.ps1 | iex

Or download OllamaSetup.exe from ollama.com/download and run the installer. Windows installs always use Ollama’s standard backend. There’s no MLX equivalent, since MLX is built specifically for Apple Silicon’s unified memory architecture.

Verify the install the same way you would on Mac:

ollama --version

Configuration: Common Settings and Patterns

• Model storage location: Downloaded models live in ~/.ollama/models on macOS by default. If you’re low on disk space, this is the folder to check before deleting anything manually. Use ollama rm <model:tag> instead.
• Removing a model: Free up space with:

ollama rm gemma4:12b-mlx

• Listing installed models: See everything you’ve downloaded so far:

ollama list

• Switching models mid-session isn’t needed: just run ollama run <different-model> and Ollama loads the new one, unloading the previous model from memory automatically.
• Local API access: Ollama exposes a local server at http://localhost:11434 that other apps can connect to. This is useful later if you want to hook Ollama into a code editor or AI coding assistant, but it’s outside the scope of getting your first model running.

Tips and Troubleshooting

Model download fails or “not found” error

Why it happens: The model name or tag doesn’t exactly match what’s published. A common mistake is leaving off the -mlx suffix or using an outdated tag name.

Fix: Double-check the exact tag on ollama.com/library, then pull it again with the exact spelling:

ollama pull gemma4:12b-mlx

Mac becomes slow or unresponsive while running a model

Why it happens: The model is too large for your available unified memory. This is especially common on Macs with 8GB or 16GB of RAM trying to run a 12B+ parameter model.

Fix: Close memory-heavy apps (browsers with many tabs, video editors) before running the model, or switch to a smaller model size. Check memory pressure in Activity Monitor > Memory tab to confirm this is the bottleneck.

Error mentioning “mlx runner failed” or a missing MLX library

Why it happens: The MLX backend isn’t initializing. This can happen after a corrupted install or if you’re mistakenly on an Intel Mac.

Fix: Confirm you’re on Apple Silicon with uname -m (it should read arm64), then reinstall Ollama cleanly:

brew reinstall ollama

MLX doesn’t seem to speed anything up

Why it happens: A few things can cause this: an Ollama version older than 0.19, a model without the -mlx tag, or less than the recommended 32GB of unified memory for the MLX preview.

Fix: Re-run ollama --version to confirm 0.19+, confirm the model tag ends in -mlx, and check your Mac’s total memory in Apple menu > About This Mac.

M5 Mac isn’t showing expected performance gains

Why it happens: Apple’s Neural Accelerator improvements for M5 chips require macOS 26.2 or later.

Fix: Go to System Settings > General > Software Update, install the latest macOS, and re-test the model.

Where to Go Next

Once your setup is running, the next step is picking the right model size for your Mac’s memory. An 8GB Mac and a 128GB Mac Studio have very different sweet spots. If you’re up for more setup, try connecting Ollama’s local API to a coding assistant or editor for AI-assisted development work.

Wrapping Up

You’ve now got Ollama 0.19+ installed, an MLX-tagged model downloaded and running, and a way to confirm MLX is actually accelerating your responses instead of quietly falling back to the older backend. If you’ve got 32GB or more of unified memory on an Apple Silicon Mac, this upgrade is worth doing right away. The speed difference on -mlx tagged models is noticeable, not marginal.

Step	Action	Applies To
1-2	Confirm Apple Silicon and macOS version	macOS
3-4	Install/update Ollama to 0.19+	macOS
5-6	Check menu bar app and browse model library	macOS
7-8	Pull and run an MLX-tagged model	macOS
9	Verify MLX acceleration is active	macOS
10	Install standard Ollama (no MLX)	Windows

Resources

• Ollama official site
• Ollama MLX announcement
• Ollama MLX performance follow-up
• Ollama model library
• Ollama download page
• Homebrew Ollama formula
• Apple MLX open source project
• Ollama documentation
• MLX performance details