How to Run AI Locally for Free

You can run an open AI model on your own computer for free: no account, no API key, no rate limit, and your prompts never leave the machine. Three tools cover almost everyone: Ollama (terminal and local API), LM Studio (desktop app) and llama.cpp (the engine underneath much of this, for people who want full control). Every number and command below comes from each vendor's own site, checked on 2026-10-21.

What you get

• Ollama: open source under the MIT license, for macOS, Windows and Linux. You pull a model by name from its library and chat with it in the terminal or call it from code on http://localhost:11434. Ollama recommends 8 GB of available VRAM, or unified memory on a Mac; without enough VRAM it falls back to system RAM, which is slower.
• LM Studio: a desktop app with a chat window, model search and download (from Hugging Face) and a local server. The Free plan costs $0 and covers running local models; the app's terms license it for personal and internal business use. It is not open source, and some features are paid (Bionic+ at $20 a month and Pro at $100 a month add its hosted agent features). LM Studio "can operate entirely offline" once you have model files.
• llama.cpp: open source under the MIT license; a command-line runner and an OpenAI-compatible server for GGUF model files from Hugging Face.

All three run the model on your hardware, so the cost is your electricity and disk space.

Which model fits your computer

A model's download size is roughly the memory it needs to run, plus extra for the conversation: longer context windows need more memory. Local downloads are usually quantized: each weight is stored in fewer bits, for example about 4 instead of 16 in the q4_K_M tags, which is why a 12-billion-parameter Gemma 4 download is 8 GB and not 24 GB (its bf16 tag). How a 30 GB model fits on a laptop shows the trade-off in five minutes.

Download sizes in the Ollama library (ollama.com/library, checked 2026-10-21). Qwen3 rows are the default tags as listed; Gemma 4 rows are the 4-bit (q4_K_M) tags. Name the 4-bit Gemma 4 tag, because the short one (gemma4:e2b) is listed at 4.6 to 7.5 GB depending on the build:

The last column is our rule of thumb (the file plus room for the system and the conversation), not a vendor figure. Each model has its own license on its model page; check it before you use one in a product. Ollama tags ending in cloud (such as gemma4:cloud) run on Ollama's servers, not on your computer.

Steps

Ollama

1. Install it. On macOS or Linux: curl -fsSL https://ollama.com/install.sh | sh. On Windows, in PowerShell: irm https://ollama.com/install.ps1 | iex. Or download the app from ollama.com/download.
2. Run a model: ollama run gemma4:e2b (Ollama's own first example; the download is about 7.2 GB). The first run downloads it, then a chat starts in the terminal; type /bye to leave.
3. Pick a size from the table above, for example ollama run qwen3:4b on an 8 GB laptop.
4. On Linux, if the server is not running, start it with ollama serve.

LM Studio

1. Check the requirements: on a Mac, Apple Silicon (M1 to M4) and macOS 14 or newer (Intel Macs are not supported; 16 GB of RAM is recommended, 8 GB Macs should stay with small models). On Windows, an x64 CPU with AVX2 or an ARM Snapdragon X Elite, with 16 GB of RAM and 4 GB of dedicated VRAM recommended. On Linux, Ubuntu 20.04 or newer, x64 or ARM64.
2. Download the installer for your system from lmstudio.ai.
3. Search for a model inside the app and download it, then load it and chat.

llama.cpp

1. Install it. On macOS or Linux: curl -LsSf https://llama.app/install.sh | sh. On Windows, in PowerShell: irm https://llama.app/install.ps1 | iex. Prebuilt binaries are also on the GitHub releases page.
2. Chat with a small model straight from Hugging Face: llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF.
3. Serve it as an OpenAI-compatible API: llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF.

Use it from your code

• Ollama listens on http://localhost:11434, for example curl http://localhost:11434/api/chat.
• LM Studio: open the Developer tab and switch on Start server, or run lms server start in a terminal. It offers OpenAI-compatible endpoints, among others.
• llama.cpp: the server listens on 127.0.0.1:9931 by default and has OpenAI-compatible chat completions, responses and embeddings routes.

Because these speak the OpenAI API, most SDKs and tools work by changing the base URL to your local address.

Limits and the catch

• Smaller models know less. What fits in 8 to 16 GB is a few billion parameters; it chats, summarises and writes code snippets well, but it misses facts and multi-step reasoning that the large cloud models handle. Why the free model feels dumber explains why.
• Speed depends on your hardware. A recent Mac or a PC with a GPU is fast enough to chat; on CPU only, expect slow answers, and Ollama warns that falling back to system RAM is slower.
• Disk and memory. Models are gigabytes each, and while one runs it holds that memory.
• You maintain it. Updates, model choice and licenses are your job.

When a free cloud tier is better

Use a cloud free tier when you need a bigger model than your computer can hold, when your laptop is slow, or when you want to try many models quickly:

• Free AI APIs with no credit card compares the free API tiers.
• Free OpenRouter models and a free Gemini API key give large models for free, within rate limits.
• Free GPUs for AI gives you Kaggle and Hugging Face GPUs for models too big for your machine.

Keep local when the data must not leave your computer, when you are offline, or when rate limits get in the way.

If it doesn't work

• The model is very slow or the computer freezes: pick a smaller model or tag from the table, close other apps, and keep the context short.
• ollama says it cannot connect: start the server (ollama serve on Linux, or open the Ollama app on macOS and Windows).
• LM Studio does not start on an Intel Mac: Intel Macs are not supported; use Ollama or llama.cpp instead.
• LM Studio on Windows fails on an older PC: x64 CPUs need AVX2 support.