🔍 Reviewing Phi-3 LLM on Linux Using Alpaca & Ollama
In this article, I’ll walk you through my experience using Phi-3, a lightweight large language model (LLM) developed by Microsoft, running locally on Linux via Alpaca, a user-friendly front-end for Ollama packaged conveniently as a Flatpak.
Whether you’re exploring on-device AI, seeking open-source alternatives to cloud-based LLMs, or want an affordable and private way to run AI locally, Phi-3 is a great place to start. Let’s dive into the setup, performance, and practical use cases of Phi-3 through Alpaca.
💡 What is Phi-3?
Phi-3 is part of Microsoft’s compact LLM family optimized for speed and memory efficiency. Despite its small size (ranging from 1.3B to 7B parameters), it’s surprisingly capable in reasoning, summarization, and Q&A tasks.
Phi-3 models are released under an MIT license, which makes them ideal for developers, hobbyists, educators, and small businesses looking for unrestricted usage.
🐪 Why Use Alpaca (Ollama) on Linux?
Alpaca is a modern GUI client for Ollama, a framework for running LLMs locally with one command. The Flatpak version makes it easy to install and sandbox on any Linux distribution without dependency conflicts.
🔧 Installation Steps
- Install Alpaca via Flatpak:
flatpak install flathub com.jeffser.Alpaca - Launch Alpaca and select Phi-3 model:
- From Alpaca’s UI, you can configure temperature, and other model parameters with ease.
📷 Screenshots
▶️ Live Demo Screencast
Watch my real-time demo of Phi-3 running inside Alpaca on Linux:
Results:
Who is the mayor of Toronto?
Produced inaccurate outdated answer to Olivia Chow as the mayor of Toronto.
I need a PHP code snippet to connect to a MySQL database.
Produced accurate syntax PHP code snippet to connect to a MySQL database.
I need a 1080p screenshot of the gnome desktop environment.
Produced elaborate answer to generate a 1080p screenshot of Gnome desktop environment because it is a text-based AI lacking ability.
I need a kotlin code snippet to open the camera using Camera2 API and place the camera view on a TextureView.
Produced incomplete Kotlin code snippet to open the camera using Camera2 API.
I need a blender blend file for fire animation.
Produced elaborate answer to generate a fire animation, but not a Blender Blend file because it is a text-based AI lacking ability.
💼 Hire Me: LLM Installation & AI Tutoring
Want to run LLMs like Phi-3 on your own machine? Need help understanding how AI works or integrating it into your workflow?
I offer:
- ✅ One-on-one tutoring on local AI and prompt engineering
- ✅ Remote setup and configuration of Ollama, Alpaca, and models
- ✅ Lightweight model recommendations for your hardware
- ✅ Ongoing support and training for your team
👉 Contact me here or via the form to schedule a session!
📚 Conclusion
Running Phi-3 on Linux through Alpaca is an empowering experience. It’s fast, efficient, and respects your privacy. Whether you’re an AI enthusiast or a small business owner, local models like Phi-3 offer real value without the overhead of the cloud.
Let me know in the comments what models you’re experimenting with, or reach out if you’d like help setting one up!
Disclosure: Some of the links above are referral (affiliate) links. I may earn a commission if you purchase through them - at no extra cost to you.
Edward is a software engineer, web developer, and author dedicated to helping people achieve their personal and professional goals through actionable advice and real-world tools.