ChatGPT has taken the world by storm. We’ve used ChatGPT alternatives like Google’s Bard, and Bing AI Chat. But all of them require an Internet connection to run. Not to mention, advanced models like ChatGPT4 require a paid subscription to use.
Open-source LLMs (Large Language Models) on the other hand have become powerful enough to challenge ChatGPT4 and have become very popular these days. However, an average end user like us doesn’t know how to run them on our own PCs or Macs.
But what if you could run an advanced ChatGPT like LLM locally on your PC or Mac completely offline? That is possible now thanks to software like LM Studio, GPT4All, and Ollama. In this post, I will show you how to run one of these software LM Studio on your computer and chat with an AI model without any internet.
The first step is to go ahead and download the LM Studio software from its website and install it on your device.
Open the installed LM Studio software and you will see a search bar to discover and run Local LLMs.
If you know the LLM model you want to install you can search for it here or you can click on the search icon in the left navigation. In my case, I’ve searched for an LLM model called Mistral. Specifically, because it is the fastest open-source LLM model that is faster than the GPT3.5 turbo.
As the name suggests, these models are really large. Most of them are at least 3GB in size. So take caution If you are running out of hard drive space.
Next, click on the chat icon on the left navigation bar just below the search icon and load the model from the top bar.
You can also add system prompts and modify other settings in the right sidebar. And that’s it. We can now chat with an LLM completely offline on our own devices.
In case you want to try out a different LLM follow the same procedure. Since these LLMs are very large, I would suggest managing the models installed by you by clicking on the folder icon in the left navigation bar.
