Who Needs Baristas? With a Laptop, Starbucks Wi-Fi, and a Cup o’ Joe, You Too Can Brew Your Own Large Language Model!

More individuals than ever are dipping their toes into the waters of running large language models (LLMs) locally. They’re experimenting with various models and tools, carving out new pathways in the field of conversational AI.

1. Transformers with Hugging Face’s PyTorch: A popular choice among AI enthusiasts, PyTorch offers the Transformers library from Hugging Face, which includes pre-trained models like GPT-2 and BERT that can be easily set up and run locally.
2. TensorFlow with TFX: Another powerful framework, TensorFlow, provides the TensorFlow Extended (TFX) platform, useful for training models like BERT and Transformer on local machines.
3. Meta’s LLaMA and Alpaca with Dalai: A more unconventional but intriguing option is using leaked models like Meta’s LLaMA and Alpaca, paired with apps like Dalai found on GitHub.

Few articles for Meta’s LLaMa and Alpaca (with Dalai, Vicuna and others) have been brewing recently:

Beebom: How to Run a ChatGPT-Like LLM on Your PC Offline

ShellyPalmer: Run “ChatGPT” on Your Computer

ReadMultiplex: HOW YOU CAN INSTALL A CHATGPT-LIKE PERSONAL AI ON YOUR OWN COMPUTER AND RUN IT WITH NO INTERNET

Instead of the well-known OpenAI’s ChatGPT, consider using two leaked large language models: Meta’s LLaMA and Alpaca. Add to the mix an app named Dalai found on GitHub, and the setup is complete. (Yes, Dalai LLaMA – quite the playful choice.)

Imagine installing an LLM chat application on a personal computer. A process that takes about 10 minutes, it can deliver impressive outcomes and lead to unexpected insights.

The surprise lies in the performance. These AI apps can operate on modest hardware. Both high-end gaming PCs and M1 MacBook Pros can handle these applications smoothly. However, the adventure of running LLMs on personal devices unfolds both promising and challenging implications.

Pros:

• Data privacy and security are inherently promoted by local installations. All information remains strictly within the device, a significant consideration when dealing with sensitive data.
• The absence of stable internet access does not hinder model usage.
• Performance latency is also less as data doesn’t have to travel to and from a cloud server.
• Perhaps most enticingly, there are no content restrictions. Models can be trained on any content, freeing users from the constraints of outdated information.
• Moreover, these models allow for a level of tinkering and customization that may not be possible with their cloud-based counterparts.

Cons:

• Nevertheless, potential drawbacks lurk. Running these large models demands substantial computational resources.
• The task can be technically challenging, requiring time and expertise.
• Power consumption, energy efficiency, and the necessity for manual updates to keep pace with any improvements or new features added to cloud-based versions are additional factors to consider.

A hammer can be used constructively to build a house, or destructively to break a window.

Running your own AI model comes with a significant degree of responsibility. The action lies in the hands of the person wielding it, not the tool itself. Similarly, AI can be a tool for innovation and advancement, but in the wrong hands, it can be misused.
Your behavior and choices with AI can be reflective of your personal character and maturity. Using AI irresponsibly, such as making it say inappropriate or harmful things, might provide momentary amusement, but it could contribute to broader consequences. , your actions today will shape how you’re remembered tomorrow.
Responsible use of AI is not just about you, but about the entire community and future generations.

Despite these challenges, the power and potential of LLMs operating on personal devices harken back to the 1980s personal computer revolution. PCs transformed society, networking the planet, weaponizing information technology, and revolutionizing communication dynamics. Now, AI is poised to propel this transformation further.

Only recently, it was widely believed that big tech firms like OpenAI, Meta, Google, would control access to leading-edge AI technology. However, a simple experiment changes this perspective.

The future of AI is no longer a distant cloud – it’s right here, on our personal computers. Take a moment and ponder the implications.