LM Studio is a desktop app for running large language models on your own hardware. No API keys, no per-token billing, and nothing you type leaves your machine. You install it, search for a model, hit download, and start chatting. That is the entire pitch, and it holds up.
It has become my default way to test local models, mainly because the friction is so low that a new model goes from curiosity to conversation in about five minutes. Here is what it does, how to pick a model that actually fits your machine, and where it stops being the right tool.
What LM Studio Does Under the Hood
The app is a friendly front end for llama.cpp, the inference engine behind a lot of local AI software. On Apple Silicon it can also use MLX, Apple’s own framework, which tends to be faster on M-series chips. You never touch either of those directly. LM Studio handles the jobs that would otherwise eat an afternoon: finding models, choosing the right file, and fitting it into memory.
Models arrive as GGUF files, mostly from Hugging Face. Rather than browsing repositories and comparing cryptic filenames, you type something like “Qwen2.5 7B Instruct” into the search box and get a list of quantised versions with download sizes attached. That one piece of interface saves more time than any feature in the app.
Choosing a Model That Fits Your Machine
Size is where most people go wrong. They grab the biggest model on the list, watch it thrash, and decide local AI is too slow. The rule of thumb is simple: a model needs its file size plus a couple of gigabytes of headroom for the context window. These are the Q4_K_M numbers, which is the quantisation most people should start with.
- Llama 3.2 3B – about 2GB. Runs on almost anything, including an 8GB laptop.
- Mistral 7B Instruct or Llama 3.1 8B – roughly 4.5 to 5GB. The practical sweet spot for a 16GB machine.
- Qwen2.5 14B – around 9GB. Comfortable on 32GB, tight on 16GB.
- Qwen2.5 32B – about 20GB. Wants 32GB of RAM, more if you also want long context.
- Llama 3.3 70B – near 40GB at Q4. A 64GB Mac or a 24GB GPU can do it, and it will feel slow.
Larger quantisations such as Q8 sit closer to the original model but roughly double the file size. Q4_K_M trades a little accuracy for a big win in speed and memory, and on everyday writing and summarising tasks you will rarely notice the difference.
Loading it and reading the numbers
Once a model is downloaded, you load it and LM Studio reports how many layers sit on the GPU, how much memory is in use, and how fast tokens arrive. Anything above 20 tokens per second reads comfortably for chat. Drop below 8 and you will feel every reply land.
The Two Settings Worth Touching
GPU offload
If a model is partly on the GPU and partly on the CPU, you pay a tax on every single token. LM Studio shows a slider for how many layers go to the GPU, so push it as high as memory allows. On a machine with a 12GB card and a 14B model, you might only fit 60 to 70 percent of the layers. That is fine. What matters is knowing why something feels slow, and it is almost always this slider.
Context length
Every doubling of context costs memory. Setting 32,000 tokens when your conversation never passes 4,000 just wastes RAM that could hold model layers instead. Start at 4,096, raise it when a task genuinely needs it, and drop it back afterwards.
Features That Go Beyond a Chat Box
The chat window is the obvious part. The useful parts sit underneath it.
Presets let you save a system prompt, temperature, and context setting as a single unit. I keep one for rewriting at low temperature and another for brainstorming at a looser setting. Switching takes one click instead of retyping instructions every session.
Document chat lets you attach PDFs and text files to a conversation and ask questions about them. It is closer to simple retrieval than a full pipeline, but for a 20-page contract or a stack of meeting notes it works well enough to save a copy-paste marathon.
Tool use and MCP support arrived in recent releases, which means models can call functions or connect to external services. It is early days, and small models handle it clumsily, but the direction is clear.
Turning It Into a Local API
Flip to the Developer tab, start the server, and LM Studio exposes an OpenAI-compatible endpoint at localhost:1234/v1. Any tool that speaks the OpenAI format can point at it instead, which means your Python scripts, coding assistants, and automation workflows suddenly run against a model sitting on your desk.
One of the best pairings here is Open WebUI, the self-hosted interface that rivals ChatGPT. Run LM Studio as the backend and Open WebUI as the front end, and you get chat history, multiple users, and a browser interface without sending a byte to a third party.
LM Studio vs Ollama
These two get compared constantly, and the honest answer is that they overlap less than people assume. Ollama is a command line tool built for scripting and servers. One command pulls a model and starts an API. LM Studio is a graphical app built for exploring and evaluating models, with sliders and readouts that show exactly what your hardware is doing.
If you are wiring models into an application, Ollama is usually the cleaner fit. If you want to see how a model behaves before committing to it, LM Studio gets you there faster. Plenty of people keep both installed.
When the cloud is still the better call
Local models are not free wins. A 7B model on your laptop will lose to a frontier model on anything demanding real reasoning, long context, or precise instruction following. For those jobs, hosted options stay ahead, and some of them cost nothing at all. Google AI Studio offers a free Gemini workspace that handles plenty of the work people try to force onto small local models. Use local for privacy, offline work, and volume. Use hosted for difficulty.
Where It Pays Off in Practice
The real value appears in workflows where data cannot leave, or where volume would get expensive. Summarising internal documents. Drafting from templates. Cleaning up messy transcripts. Turning rambling notes into structured records.
It also works as the engine for agent experiments. Frameworks like AutoGen can run entirely against your local endpoint, which means you can let a team of agents loop through a task fifty times without watching a billing meter climb.
Creative teams get something else from it. Being able to generate a hundred headline variations or scene descriptions without counting the cost changes how you brainstorm, and it is a large part of why scaling creative output with AI has turned into a real working strategy rather than a slogan.
Habits That Make Local Models Feel Fast
A few small routines do more for the experience than any hardware upgrade.
- Keep one small model loaded all day for quick edits and formatting, so you are not reloading a 20GB file every hour.
- Stick with Q4_K_M unless you have actually measured a quality problem at that level.
- Turn flash attention on and keep context trimmed to what the task needs.
- Match the model to the job. Loading a 70B model to fix a spelling mistake wastes electricity and patience.
- Run the same prompt against two models before assuming the bigger one is better. It often is not.
The gap between what runs on a laptop and what runs in a data centre keeps narrowing, and the tools around local inference are improving faster than most people expect. Downloading LM Studio and loading a single 8B model is a small enough experiment that the cost of being wrong is half an hour. The cost of being right is a private, offline assistant that answers instantly and never sends your documents anywhere.