Running AI Models on a Raspberry Pi 5 - Part 1
One fine lazy weekend, I was reading about something called Edge AI - typically means that the AI runs on the end machine than the remote cloud server. As much as it sounded tempting, I knew that you were crazy if you said that I could run a 8b+ param LLM on a mid range phone. Then I looked at my Pi 5 that I haven’t used in quite a while. I haven't written a tech article in a real while and then thought - “Why not waste my Saturday trying to run models on this?” I had to clean up a few stuff to make room for the models that I’ll be downloading. I was using my Pi for retro gaming and as a media streaming hub. Hated to see the NFS and FIFA ROMS go, but I'll download them back again. After several consultations with ChatGPT and some googling (Yeah, I still google and also ignore Gemini’s summaries) to see people have got a few models working on Ollama on a Pi 5. But I was determined to see how far can I push a 8GB RAM and a quad core CPU. At this point, I had no idea what I rea...