🔍 Reviewing Phi-3 LLM on Linux Using Alpaca & Ollama

In this article, I’ll walk you through my experience using Phi-3, a lightweight large language model (LLM) developed by Microsoft, running locally on Linux via Alpaca, a user-friendly front-end for Ollama packaged conveniently as a Flatpak.

Whether you’re exploring on-device AI, seeking open-source alternatives to cloud-based LLMs, or want an affordable and private way to run AI locally, Phi-3 is a great place to start. Let’s dive into the setup, performance, and practical use cases of Phi-3 through Alpaca.

💡 What is Phi-3?

Phi-3 is part of Microsoft’s compact LLM family optimized for speed and memory efficiency. Despite its small size (ranging from 1.3B to 7B parameters), it’s surprisingly capable in reasoning, summarization, and Q&A tasks.

Phi-3 models are released under an MIT license, which makes them ideal for developers, hobbyists, educators, and small businesses looking for unrestricted usage.

🐪 Why Use Alpaca (Ollama) on Linux?

Alpaca is a modern GUI client for Ollama, a framework for running LLMs locally with one command. The Flatpak version makes it easy to install and sandbox on any Linux distribution without dependency conflicts.

🔧 Installation Steps

1. Install Alpaca via Flatpak:
   flatpak install flathub com.jeffser.Alpaca
2. Launch Alpaca and select Phi-3 model:
3. From Alpaca’s UI, you can configure temperature, and other model parameters with ease.

📷 Screenshots

▶️ Live Demo Screencast

Watch my real-time demo of Phi-3 running inside Alpaca on Linux:

Results:

Who is the mayor of Toronto?

Produced inaccurate outdated answer to Olivia Chow as the mayor of Toronto.

I need a PHP code snippet to connect to a MySQL database.

Produced accurate syntax PHP code snippet to connect to a MySQL database.

I need a 1080p screenshot of the gnome desktop environment.

Produced elaborate answer to generate a 1080p screenshot of Gnome desktop environment because it is a text-based AI lacking ability.

I need a kotlin code snippet to open the camera using Camera2 API and place the camera view on a TextureView.

Produced incomplete Kotlin code snippet to open the camera using Camera2 API.

I need a blender blend file for fire animation.

Produced elaborate answer to generate a fire animation, but not a Blender Blend file because it is a text-based AI lacking ability.

💼 Hire Me: LLM Installation & AI Tutoring

Want to run LLMs like Phi-3 on your own machine? Need help understanding how AI works or integrating it into your workflow?

I offer:

• ✅ One-on-one tutoring on local AI and prompt engineering
• ✅ Remote setup and configuration of Ollama, Alpaca, and models
• ✅ Lightweight model recommendations for your hardware
• ✅ Ongoing support and training for your team

👉 Contact me here or via the form to schedule a session!

📚 Conclusion

Running Phi-3 on Linux through Alpaca is an empowering experience. It’s fast, efficient, and respects your privacy. Whether you’re an AI enthusiast or a small business owner, local models like Phi-3 offer real value without the overhead of the cloud.

Let me know in the comments what models you’re experimenting with, or reach out if you’d like help setting one up!