Local AI models, what are they and what do you need?

Short answer: a local model is an AI model that runs on your own computer or server instead of at a provider in the cloud. What you type stays in your own environment. For that you need a model with open weights and enough memory to load it.

Open and closed models

A model is a huge collection of numbers, the weights, fixed during training.

With a closed model, the maker keeps the weights to itself. You use the model through that maker's app or API.
With an open-weight model, you can download the weights and run the model yourself.

Examples of makers that publish open weights: Meta (Llama, downloadable after accepting the licence), Mistral AI (some of its models, several under the Apache 2.0 licence), Moonshot AI (some of its Kimi models) and xAI, which released the weights of its first Grok model in 2024.

Why run locally?

Your data stays in. Ollama, a widely used program for running models locally, states that it does not see your prompts or data when you run locally.
No dependence on a single provider and its prices or terms.

On the other hand, the most capable models are often closed, or so large they do not fit on an ordinary computer. You are also responsible for installation, security and updates yourself.

What hardware do you need?

The rule of thumb: the model has to fit in memory. Ollama explains that a model is loaded into ordinary system memory (RAM) when it runs on the processor, or into the graphics card's memory (VRAM) when it runs on the GPU. A GPU (graphics card) is much faster, as long as the whole model fits. If it does not, Ollama spreads the model over several cards, or over GPU and processor.

Exactly how much memory you need depends on the model, the size of the conversation (the context) and how many people use it at the same time. The largest open models need server hardware: Moonshot's Kimi K3, for example, has 2.8 trillion parameters according to Moonshot.

What is Ollama?

Ollama is open-source software for macOS, Windows and Linux that lets you download and run open models. Many apps connect to it, such as chat programs and developer tools.

Can you use open models for business?

That depends on the licence. Apache 2.0 is permissive. Other models have their own licence with conditions, as with Llama and Kimi. Always read the licence before you use a model in your business.

As of October 2026. Models and licences change fast; check the current situation at the source.

Sources

Accessed 21 October 2026.