Stable Diffusion shipped as open weights in August 2022, and the model itself was never the interesting part. The interesting part is the scaffolding people built around it: node editors, one-click installers, upscalers, ControlNet preprocessors, prompt generators, and a small industry of LoRA trainers. Most of it is free. A surprising amount of it is maintained by one person answering issues at 2am.
That abundance is the problem. Ask five people which Stable Diffusion tools to install and you’ll get five different answers, all delivered with unearned confidence. The honest answer depends on your GPU, how much you enjoy tinkering, and whether you want to make one good image or five hundred consistent ones. Here’s the landscape, with the trade-offs left in.
Three different things get called “Stable Diffusion tools”
Before anything else, untangle the vocabulary, because half of the confusion online comes from mixing these up.
- The interface. The program you type prompts into. Automatic1111, Forge, ComfyUI, Fooocus, InvokeAI, SwarmUI.
- The model file. A multi-gigabyte checkpoint that determines the style and quality. SD 1.5, SDXL, SD 3.5, Flux, plus thousands of community fine-tunes.
- The utilities. Everything that sits beside generation: ControlNet, upscalers, inpainting helpers, LoRA trainers, prompt managers.
You can swap any one of these without touching the others, which is why the ecosystem has stayed so modular. It’s also why the same question gets answered five different ways on Reddit. If you want a wider view of what runs well on your own machine, this breakdown of which open source AI tools are actually good enough to run yourself covers the neighbouring categories.
The interfaces, ranked by how much you enjoy tinkering
Automatic1111 and Forge
A1111 is the grandparent of the whole scene. Its layout (prompt box, negative prompt, sampler dropdown, generate button) became the template everyone else copied. Development has slowed noticeably, and on newer hardware it can feel creaky. Forge is a fork by lllyasviel that keeps the familiar interface but adds speed improvements and better memory handling, and it’s what most people should install if they want the classic experience without the waiting.
ComfyUI
Node-based, so you wire the pipeline together yourself: checkpoint into sampler, sampler into VAE decode, decode into save. It looks intimidating for about twenty minutes and then it clicks, because you can see exactly where every step sits. ComfyUI is now where new features land first, often within days of a paper being published, and saved workflows are portable JSON files you can share or version-control. If you plan to do anything repeatable, this is the one.
Fooocus, InvokeAI, SwarmUI
Fooocus hides almost everything and gives you a prompt box plus a handful of style presets. It’s the fastest route from install to a decent image. InvokeAI adds a proper canvas and layer-style editing, which suits anyone coming from Photoshop. SwarmUI sits on top of ComfyUI and gives you a friendlier front end without losing the node engine underneath.
What your GPU can realistically handle
Numbers matter more than opinions here. SD 1.5 at 512×512 will run on roughly 4GB of VRAM. SDXL wants 8GB to be comfortable and will crawl but function at 6GB with the right flags. Flux at full precision is a 24GB conversation; the quantised and GGUF builds squeeze into 8GB to 12GB with modest quality loss and much longer render times.
On Apple Silicon, Draw Things and DiffusionBee run locally and use unified memory, so a 16GB MacBook Air handles SDXL fine and Flux slowly. And yes, it technically runs on CPU alone. Budget three to six minutes per 512×512 image and decide whether that’s a workflow or a hobby.
Where to get models, and one safety note
Civitai is the main hub for community checkpoints and LoRAs, and Hugging Face hosts the official weights plus most research releases. Both are worth bookmarking.
One thing to internalise early: download .safetensors files, not .ckpt. The older pickle format can execute arbitrary code when loaded, and there have been documented cases of malicious checkpoints on model-sharing sites. Safetensors was designed specifically to close that hole. A flashy preview image is not a security audit.
The add-ons that change what’s actually possible
- ControlNet locks composition to a reference: a depth map, a pose skeleton, a scribbled line drawing. It’s the single biggest quality-of-life jump after the base model.
- Inpainting redoes a masked region without touching the rest. Fixing hands, swapping a jacket colour, removing a lamppost.
- Upscalers such as 4x-UltraSharp and the ESRGAN family take a 1024px render to 4096px without the mush you’d get from resampling.
- LoRAs are small weight patches that teach a face, a style, or an object. Typically 20MB to 200MB instead of 6GB.
- IP-Adapter transfers the look of a reference photo into a generation, useful for matching a brand’s palette across a set of images.
Individually these are optional. Together they’re the difference between a toy and a production pipeline.
When your laptop says no
Hosted options have closed a lot of the gap. RunPod and Vast.ai rent you a 4090 for roughly 40 to 60 cents an hour, and you can run the exact same ComfyUI workflow you use locally. Google Colab still works for lighter jobs, though session limits have tightened. Replicate and Fal wrap specific models behind an API, which is the sane choice if you’re building something rather than making pictures.
On the consumer side, the hosted generators have improved enough that local hardware isn’t always the right call. This roundup of the AI image generators that actually deliver in 2025 is a fair comparison point, and if you want to spend nothing at all there’s a solid list of free AI image generators worth your time. People regularly ask how much they lose by skipping local setup entirely; for clean product shots and stylised portraits, the answer is less than you’d expect.
Small tools that quietly save hours
Prompt engineering gets oversold, but a few utilities genuinely earn their place. Dynamic Prompts handles wildcards and combinatorial variation, so one queue produces forty distinct images instead of forty near-copies. One Button Prompt generates a full prompt for you, which is useful when you’re tired and out of ideas. ComfyUI’s prompt scheduling lets you shift text mid-sampling, which is how a lot of the smoother style blends get made.
For batch work, the boring stuff matters more than the clever stuff: consistent seed handling, folder naming that survives a week, and a spreadsheet or text file logging which checkpoint and LoRA combination produced the image you liked. You will not remember. Nobody does.
What to install first, in order
If you’re starting from nothing, skip the roundups and do this.
- Install Forge if you want the familiar A1111 layout with better speed, or ComfyUI if you don’t mind a learning curve and want the newest features first.
- Grab one SDXL checkpoint and one SD 1.5 checkpoint. Two is enough to learn on. Ten is how people end up with a 400GB folder and no idea what’s in it.
- Add one upscaler and the ControlNet models for depth and pose. That covers most real requests.
- Test a single workflow end to end at 1024×1024, note the time it takes, then decide whether cloud GPUs are worth the money for your volume.
One last piece of advice that saves real pain: pin your versions. A ComfyUI or Forge update can break custom nodes without warning, and an evening lost to a dependency conflict is an evening you don’t get back. Keep a working setup in a separate folder, update the copy, and only migrate once the copy renders something you like.