The Setup

First, let’s be clear – you’re not going to run the full 671B parameter model that’s making headlines. That beast needs serious hardware. Instead, we’ll focus on the distilled versions that actually work on our humble Pi 400.

Prerequisites:

            sudo apt update && sudo apt upgrade
            sudo apt install curl
            sudo ufw allow 11434/tcp
        

Installation Steps:

            # Install Ollama
            curl -fsSL https://ollama.com/install.sh | sh

            # Verify installation
            ollama --version

            # Start Ollama server
            ollama serve
        

What to Expect

Here’s the unvarnished truth about performance:

Model Options:

• deepseek-r1:1.5b (Best performer, ~1.1GB storage)
• deepseek-r1:7b (Slower but more capable, ~4.7GB storage)
• deepseek-r1:8b (Even slower, ~4.8GB storage)

The 1.5B model is your best bet for actual usability. You’ll get around 1-2 tokens per second, which means you’ll need some patience, but it’s functional enough for experimentation and learning.

Real Talk

Look, I’ve spent my career telling hard truths about security, and I’ll be straight with you about this: running AI models on a Pi 400 isn’t going to revolutionize your workflow. But that’s not the point. This is about understanding edge AI deployment, learning about model quantization, and getting hands-on experience with local language models.

Think of it like the early days of computer networking – sometimes you need to start small to understand the big picture. Just don’t expect this to replace your ChatGPT subscription, and you won’t be disappointed.

Remember: security is about understanding both capabilities and limitations. This project teaches you both.

Sources