In case you’ve missed my posts on social media (I’m on threads/instagram as @sujal, mastodon as @sujal@mastodon.social, @sujal at bluesky), I’m in the process of stepping back from Ohai.ai, the startup I’ve been with for the last 2 years. My plan is to take as much time off as the universe will allow before starting the next thing.
The upside for you, my friendly readers, is that I’m turning my attention back to the fun hobbies and updates that I used to write about here back before things got super-intense at Ohai.
My first order of business has been to finish the Raspberry Pi cluster I’ve been working on for the past 6-8 months. The goal is to end up with 2 clusters of 4 Pis, plus some spares. One cluster running microk8s so I can begin to explore the Kubernetes ecosystem, while self-hosting some small projects: the tranquil.cloud mastodon server will move here, as well as a new project that I’m using to learn Elixir + Phoenix. The second cluster will have AI co-processors on 3 of the Pis, where I’m planning on exploring AI-at-the-edge projects (especially if I can somehow find an M.2 Hailo 10H… can someone help me with that?). The three boards will be running Hailo 8L coprocessors initially.
While I’m not ready to share any of the applications yet, I do have a Raspberry Pi configuration that I really like, and wanted to share in case you’re looking at picking up a Raspberry Pi 5 or two or twelve (yes, I may have a problem). They’re low power, they have PCIe support built in, and have all the flexibility that Pis have been known for. They can also easily be powered via Power-over-Ethernet (POE), minimizing the wire clutter. (I mean, look at the bottom case in the photo - it’s so clean.)
My base setup for each Pi looks like this:
- Raspberry Pi 5 8GB (digikey) - get all the RAM you can afford, it will never hurt. I tend to avoid Amazon when buying Pis - they’re usually marked up and some distributors will offer discounts if you’re buying more. If you prefer Amazon, here’s an affiliate link to a search for 8GB Pi 5’s.
- Pimoroni NVMe Base Duo (amazon, pimoroni) - allows use of 2 NVMe cards with the Pi. Note that this is a base, not a HAT, so may limit your case/mounting options.
- Samsung 990EVO 1TB NVMe M.2 SSD (amazon) - again, get all the storage you can afford, it (mostly) can’t hurt.
- Waveshare POE HAT (amazon, waveshare) - I’m using the F variant, made for the Raspberry Pi 5. It’s been rock solid, if a tiny bit tricky to install.
- (optional) Raspberry Pi AI Kit (pimoroni, waveshare, digikey, amazon) - I did buy these, but it’s a PCIe HAT with a single slot and the Hailo 8L. If you can source the Hailo 8L M.2 separately, it’s likely cheaper and less wasteful if you also want to use an SSD with the Pi. Again, not something I’d buy from Amazon, since there are other US distributors.
- (optional) Pinboards AI Bundle (pineboards) - An alternative, which I’m running on one Pi, is this kit which includes an 8L mounted to the HAT and another PCIe slot for an SSD. Definitely a way to go, if a bit pricey.
For each cluster, this is what you see in the picture above (all links to Amazon):
- C4Labs Cloudlet Case - I love this case - it can rack mount up to 8 Pis, comes with (noisy) fans if I need more cooling, and has room for a small POE switch.
- Netgear POE Switch - I have the 5 and 8 port managed variants of this switch. Both fit nicely in the case, but the 8 port does make it easier to use short, 6" patch cables for boards at the edges of the rack.
- Monoprice SlimRun Cat6A cables - they’re thin, they work well, and they make the case look so neat with the short, 0.5ft cables.
Now, you’re likely doing the math on all of that and wondering about the costs. This is not the cheapest way to get your home lab setup. Pis are not cheap, and there are knockoffs that would be less expensive. Buying a standard Intel N100 mini-pc might also be competitive when you add up the power components and other items above.
So, this is definitely preference and passion on my part, plus wanting to lean on the ecosystem and community around the Pis. It’s full of people doing crazy things like mounting 6 AI coprocessors on a single Pi. Practical? Hell no. Fun? Yes! It’s also inspiring, because in the process of doing the wacky stuff, folks like Jeff Geerling link to other more complex and realistic examples in their write-ups.
I did run into some gotchas which I wanted to document, at least for myself for the next build:
Pimoroni uses proprietary FFC cables: the little “flat flex cable” or FFC that connects the HAT to the PCIe slot on the Pi is finicky and fragile. BUY EXTRAS IF YOU CAN! The biggest drawback of the NVME Base vs the AI Bundle is that the Pimoroni boards use a wider connector on the HAT side. This is simply to prevent users from inserting the cable the wrong way (the FFC is directional). Downside is that, if there’s a cable issue, I can’t just grab an FFC from another PCIe HAT, I have to grab it from a Pimoroni Duo. (My tale of woe with a failed Duo setup will have to wait for another day).
Keep spare standoffs and screws around: I have a bunch of kits like this (different ones with different sizes/materials) in my house from other projects, and I ended up using up all of the 15mm standoffs in one kit, had to order another. Also, the POE hat needs 16mm standoffs, of which I had… zero. So I used 15mm and made sure not to fully tighten or flex the HAT.