Local AI tooling

Local AI

A practical path for running open-source models on Apple Silicon.

A measured path to local models

Local AI is a practical toolkit and notebook for running open-source language models on Apple Silicon with different local inference servers. It keeps the common workflow separate from runtime-specific instructions and machine-specific settings, so a working setup can be measured rather than guessed.

I use it as a reference when choosing a model and runtime, smoke-testing a local server, and qualifying settings for a particular Mac and workload.

Start here

Install it where you work.

This is a practical reference repository rather than a packaged runtime. Start with docs/getting-started.md, then follow the guide for MLX, llama.cpp, oMLX, or MTPLX on a compatible Apple Silicon Mac.

git clone https://github.com/joacod/local-ai.git
cd local-ai

Use it

Choose, qualify, then run

Follow the model → runtime → smoke test → profile path. Confirm that the server starts, reports health and a model, and completes a small chat request before tuning settings:

less docs/getting-started.md

less mlx/README.md

less llama-cpp/README.md

less docs/tuning.md

What it keeps honest

Choose the runtime that supports the artifact and workload; no runtime is universally best.
Treat hardware profiles as measured or clearly labeled starting points, not universal defaults.
Change one tuning variable at a time and keep the model, workload, and machine context explicit.

Current scope

This is not a model-quality leaderboard, a best-model list, or a cross-runtime performance competition. Measurements answer which settings run a model through a server on a particular Mac for a particular workload.