Skip to content
AI Digested
TutorialIntermediate3 min read

How to run AI on your own computer, privately and for free

Run a ChatGPT-style AI model on your own laptop or PC with free tools like LM Studio, Ollama or Jan. Nothing leaves your computer. Step-by-step for beginners.

First published · AI Digested

When you use ChatGPT or Gemini, your messages are processed on the company's computers. But you can also run AI entirely on your own computer. It works offline, it's free, and nothing you type goes anywhere.

This is possible thanks to open-weight models: AI models whose makers let anyone download them.

What you'll need

  • A laptop or desktop from the last few years.
  • Memory (RAM): 8GB can run small models slowly. 16GB is a comfortable start. 32GB or more lets you run larger, smarter models.
  • Helpful but optional: an Apple Silicon Mac (M1 or later), or a Windows PC with a recent Nvidia or AMD graphics card.
  • Disk space: a few gigabytes per model.

Step 1: Install a free app

The easiest options for beginners:

  • LM Studio (lmstudio.ai): a polished app for Windows, Mac and Linux with a built-in model browser. Recommended for most people.
  • Jan (jan.ai): a simple, open-source chat app.
  • Ollama (ollama.com): popular and lightweight, with a simple app and a command-line tool. Developers love it.

Download the installer from the official website and install it like any other program.

Step 2: Download a model

Open the app and find its model browser (in LM Studio, the search or Discover tab).

Model names look intimidating, like "Llama-3.1-8B-Instruct-Q4". Here's how to read them:

  • The family: Llama (Meta), Gemma (Google), Mistral, Qwen (Alibaba), DeepSeek, gpt-oss (OpenAI).
  • The size, in billions of parameters: "8B" means 8 billion. Bigger is smarter but needs more memory.
  • "Instruct" or "Chat" means it's been trained to follow instructions. That's what you want.
  • "Q4", "Q5" and so on describe compression. Q4 is a good balance of quality and size.

A good first choice: a model of roughly 4 to 8 billion parameters in Q4 format. The app usually shows whether a model will fit your computer.

Step 3: Start chatting

Load the model, wait a few seconds, and type into the chat box exactly as you would with ChatGPT. The first reply may take a moment while the model warms up.

Try unplugging from the internet and asking another question. It still works.

Step 4: Try your own documents

LM Studio and Jan let you attach documents to a conversation, so you can summarise or ask questions about private files without uploading them anywhere. This is one of the best reasons to run AI locally.

What to expect

Good at: drafting and rewriting text, summarising, brainstorming, explaining concepts, simple coding help, and anything private.

Weaker at: up-to-date facts, complex reasoning and very long documents, compared with the biggest cloud models. Smaller models also hallucinate more, so check facts carefully.

Troubleshooting

  • Very slow? Choose a smaller model, or a more compressed version (for example Q4 instead of Q8).
  • Won't load / out of memory? The model is too big for your RAM. Pick a smaller one and close other apps.
  • Strange or repetitive answers? Make sure you downloaded an "Instruct" or "Chat" version.

Why this matters

Local AI means you can use AI with confidential work, medical or financial information without trusting a company with it. It also means AI keeps working with no internet connection, no subscription and no usage limits.

Keep going

Common questions

Is it legal to download these models?

Yes. Open-weight models are published by their makers, such as Meta, Google, Mistral, Alibaba and DeepSeek, under licences that allow personal use. Some licences restrict certain commercial uses, so check before building a business on one.

Will it slow my computer down?

While the AI is generating an answer, it uses a lot of memory and processing power. When you close the app, everything returns to normal.

Can a local model search the internet?

Not by default, which is part of the privacy benefit. Its knowledge stops at its training date. Some apps can connect to search if you choose.

How much disk space do models need?

Typically between 2GB and 20GB per model, depending on size.

Spotted a mistake or something out of date? Tell us and we will fix it. Read how we research and write our articles.