It’s no exaggeration to say the large language models that power AI bots like ChatGPT and Gemini are changing the world. Many of us are now relying on them for coding, writing, summarizing, and searching the web. Whatever directions AI goes in over the next few years, LLMs will be part of that future.
If you use ChatGPT, Claude, Perplexity, or any of the other AI platforms, then you’re using an LLM. What’s less commonly known is that these LLMs can be run locally, on your own computer. The key benefits are offline access and greater privacy, because you’re not sending anything to the cloud for anyone else to analyze or review.
You’re also not paying any AI companies a monthly subscription or hitting any usage rates. Numerous LLMs are available to download for free, including from big names like Meta and Google. Though they tend not to be as advanced or as speedy as the LLMs inside the apps you have to pay for, they’re capable enough for everyday use, and you can pick and choose between them as needed.
There is more maintenance involved here, and you lose some of the convenience that comes with just loading up the ChatGPT app. (You’ll need to handle updates yourself, for example.) However, you get a more personalized and private AI system all to yourself, and it’s not difficult to get started.
What You Need to Get Started
You can run local LLMs on Windows, macOS, and Linux, though macOS tends to be the preferred platform for most AI enthusiasts. Everything is more unified and consistent on Macs—unlike Windows PCs, only one company makes them—and Apple Silicon chips combine the CPU, GPU, and RAM together, which AI models like.
Whichever platform you go for, having plenty of RAM helps. The bare minimum is 8 GB, though if that’s all you’ve got, you’ll be somewhat limited in terms of the size of the LLMs you can run and how fast they’ll go. Sixteen gigs is better, and 32 or more is required if you want to use the biggest and fastest models. For the best results, you need plenty of VRAM on a dedicated GPU (anything above 8 GB will make a difference), as this type of memory is optimized for the kind of tasks AI models do.
An AI chatbot like ChatGPT relies on the cloud.Photograph: David Nield
