I tried 3 open-source alternatives to LM Studio—here’s the one I’m sticking with

LM Studio is a popular application for running large language models (LLM) locally on your computer. You can download a template, load it into LM Studio and start chatting. LM Studio is not the only option; I tried three open source alternatives, and each of them was useful in different ways.

Jan.ai is the most direct alternative to LM Studio

Download a template and start chatting

If you’re looking for a direct, like-for-like replacement for LM Studio, Jan.ai is the best option of the three I’ve tried. It looks like a real desktop application; when I installed it, it automatically downloaded the default Jan template and I was able to start chatting like I can with ChatGPT or Claude.

The first response was a little slow to appear when loading the model, but after that responses were reasonably fast, even on my M2 MacBook Air with only 8GB of RAM. The default model was pretty decent, although unfortunately knew little about how to use the Jan.ai app, which was one of the first things I wanted to know.

Although the default template is suitable for general discussions, you can download and run other templates. You can browse a list of templates in the Hub, which is similar to how it works in LM Studio. I had trouble getting some models to work properly, with the Apple Silicon-optimized MLX models not loading, meaning I had to resort to downgrading to the previous version to get them working.

As with LM Studio, you can add custom instructions to determine how the chatbot should respond. In Jan.ai, this is done by creating helpers with custom instructions that you call up in chats. If you want a fairly simple way to set up a local chatbot running on your own computer, Jan.ai is a solid choice.

Ollama is the best option for a home server

It’s more an infrastructure than an AI application

Ollama is quite a different beast from Jan.ai or LM Studio. Launching the app gives you a recognizable AI chatbot interface, but it noticeably lacks real features. The menu offers two options: New Chat and Launch, and there’s a drop-down list in the chat window with a small handful of downloadable and cloud templates.

All of the downloadable models listed were too large to run on my M2 MacBook Air with 8GB of RAM, so out of the box the app was effectively useless to me for running a local LLM. The application’s template management is limited; If you want to do more, you’re better off using the command line interface.

You can install other templates in Terminal and use them in the Ollama app, or chat with them directly in Terminal, but that’s not the real purpose of Ollama. This is not primarily a chatbot application; it’s a tool for running and managing local models that you can then access with a local API.

I can run a local LLM on my Mac using Ollama and let other apps and services talk to it. For example, I can use a local LLM running in Ollama as a chat agent in Home Assistant, giving my voice assistant AI-based capabilities without any of my data having to leave my home.

AnythingLLM is great for working with documents

Create a private assistant for your notes and files

AnythingLLM is yet another way to use a local LLM. The main goal of AnythingLLM is to help you create wizards that can work with your own documents and data. Its main features include Local Retrieval Augmented Generation (RAG), which allows you to upload files like PDFs, Word documents, text files, etc. and then talk about their contents to a chatbot.

For example, I downloaded the rules for the game Settlers of Catan and was then able to ask the chatbot questions such as how many victory points I would get for having the largest army, or what resources I would need to build a road, and it was able to extract the information from the PDF. The problem was that on my Mac the response was extremely slow and it was just a search within a single PDF. With more documents, it would quickly become unusable on my hardware.

AnythingLLM can also be used with cloud-based templates and you can combine templates. For example, I could use a local LLM to remove personal information from my bank statements and then use a cloud model to analyze it. It also has web scraping capabilities, although when I tried to get it to find the latest deals on games from Deku Deals, it simply told me it couldn’t do that.

There’s also a pop-up assistant you can call up when using any app, which will take a screenshot of the app you’re using so you can ask questions about it. On my MacBook Air, it was too slow to be useful. If I had better hardware, I could see how AnythingLLM could really be useful, but my Mac just isn’t up to the task.

I stay with Ollama

It’s not really a real alternative to LM Studio

After giving all three a hard time, the one I’m going to stick with is Ollama. While it’s probably the least like LM Studio in terms of easily loading a template and starting a chat, it’s the most useful for what I need.

It essentially gives you API access to an LLM without having to pay. Even though I can only use fairly lightweight models, they can still perform useful tasks. For example, I can use it in an n8n automation to clean the data I pass to the automation before it is sent to a Notion database, without having to worry about my data being exposed outside of my local network.

Local LLMs are a boon for privacy

If you only have modest hardware, a local LLM won’t be able to match the performance of popular cloud-based LLMs. What you lose in power, you gain in intimacy.