Recently whilst doing the weekly round of server maintence in my Homelab, I ran into a problem that every homelabber has encounted at one point. A mundane repetitive task that takes 10 minutes but I could spend 3 days developing a custom automation to save myself that 10 minutes all together.
This task was updating my Raspberry Pi cluster. Specifically I have 4 Raspberry Pi 5’s that all run DietPi OS. To keep things short DietPi is a custom spin of Debian designed to be lightweight and extremly resource effecient. It comes with some custom tools to manage things like services and installation of optimized versions of popular software. It also has a custom tool to update the OS ontop of performing regular software updates with the usual apt upgrade approach.
To perform a full software update of a DietPi node 3 comands need to be run. apt update, apt upgrade and then the custom tool dietpi-update. I got tired of opening 4 terminal windows, using SSH to log into all 4 nodes, and running each of these commands one-by-one and keeping track of which node was on which step, looking for errors, etc. etc. So I went looking for a solution to automate this admittidly simple task, to be as hands-free as possible.
My Solution
Thinking about ways to tackle this issue I had a thought, a miniscule implementation of the core features of Ansible. I decided to create a CLI utility in Rust, that could read a .toml file that lists the different hosts and their authentication parameters aswell as defining the command I wish to run across the configured nodes. This is the file structure I came up with.
command = "apt update -y && apt upgrade -y && /boot/dietpi/dietpi-update"
[[hosts]]
host = "192.168.100.3"
user = "josh"
port = 2202
identity_file = "~/.ssh/id_rsa"
[[hosts]]
host = "192.168.100.5"
user = "admin"
port = 2202
identity_file = "~/.ssh/id_rsa"Now all I have to do is create a program that can parse this file, and create concurrent SSH sessions to each node, and run the command whilst returning the output to stdout in a manner that clearly defines the host and the feedback. I got my inspiration from Laravel’s Sail framework - A docker-compose wrapper that creates a stack of containers powering a full Laravel development environment. When viewing the logs it will tag the log entry with a coloured prefix defining which container is returning which output so I decided to implement the same things.
I now have a simple CLI tool that automates my task like so
sshmux --config sshmux.tomlThe tool will then return the following output..
[192.168.100.3] Get:12 https://archive.raspberrypi.com/debian bookworm/main arm64 Packages [538 kB]
[192.168.100.3] Fetched 1081 kB in 5s (233 kB/s)
[192.168.100.3] Reading package lists...
[192.168.100.3] [ OK ] DietPi-Update | APT update
[192.168.100.3] [ INFO ] DietPi-Update | Storing number of available APT upgrades to file: /run/dietpi/.apt_updates
[192.168.100.5] Get:12 https://archive.raspberrypi.com/debian bookworm/main arm64 Packages [538 kB]
[192.168.100.5] Fetched 1081 kB in 5s (233 kB/s)
[192.168.100.5] Reading package lists...
[192.168.100.5] [ OK ] DietPi-Update | APT update
[192.168.100.5] [ INFO ] DietPi-Update | Storing number of available APT upgrades to file: /run/dietpi/.apt_updatesAs you can tell the output logs are tagged with the host returning it making any errors easy to track down and fix without parsing walls of text.
This project is open source and available on my Github here. Homebrew users can freely install SSHMux by using my personal Tap.
brew tap bepisdev/homebrew
brew install sshmux