frankie-tales

Storage is a pain, or why self-hosting is not always either desirable, or easy.

2026-08-29T15:30:00Z

Self-hosting is the modern mantra, often presented as the final solution to dependence on big companies. Unfortunately, NASes are not a backup, and in most situations you have to arrange a last-resort escape: a B-plan and possibly even a C-plan. This is a collection of horror stories from my personal experience. They all ended (mostly) happily, but usually at the cost of a replacement/integration, and they show what can go wrong when you rely too much on a NAS (or a NAS pool).

At work, we used Synology NASes intensively for years. The very first unit was used as an iSCSI device until the XFS filesystem became locked in read-only mode after a kernel upgrade, due to some odd intersection of stars in the sky (including a kernel-level problem), with no way to run fsck to fix it. The final solution was to buy a new NAS and copy the data, with some weeks of work freezing.

In a few years, the NAS cluster grew, until one expansion started having subtle hardware problems, with sudden, temporary eSATA link timeouts and degradation of the RAID arrays. Again, a unit replacement was due, but it triggered a grave Btrfs consistency corruption (thanks, Synology kernel fork! I love you, damn it) on a data volume first, because shit happens.

Synology NASes are very affordable and reliable units, until they are not.

And when they fail, the failure is generally catastrophic, with no possibility of recovery and corruption of RAID volumes. It doesn't help that they don't have a console mode, either. In the meantime, we also had to replace a series of HDDs (maybe 20% of them) and maintain spare disks in case of failures.

All that has a concrete impact on ownership costs. Even if we use 5-year-warranted disks (enterprise grade), we inevitably have a large portion of them out of warranty. A large portion of almost a hundred HDDs of various sizes: guess the whole cost of that thing.

The proprietary OS and absence of a console are the main reasons why we changed NAS brands (we now moved to Terramaster with plain Debian): the ability to control and look at the low level before trying any recovery is priceless. Disclaimer: we rarely used all the bells and whistles from DSM OS because our storage use is much more aggressive than average, with big files and intensive I/O. The application level is not our primary interest, so in recent years we've simply used a Docker container to run a GlusterFS daemon on the Synology NASes. Also, DSM limitations shaped some technical choices: too many small RAID pools instead of larger ones, and using Btrfs instead of the more mature XFS. And we paid the price, of course: less space, less performance, and finally also less stability.

In the last, maybe 10 years, we adopted GlusterFS as our main carrier to manage pools of NASes. Starting with a low-bandwidth 1Gb network, we had to move towards 10Gb when the number of units increased. We had to learn to manage GlusterFS oddities and problems, which is not a specificity of such a distributed system: all distributed filesystems are weird beasts. I experienced multiple HPC clusters worldwide, and all implementations, like it or not, have issues from time to time, as well as multiple limits one has to consider to avoid problems. The crude reality is that all of them need some form of continuous babysitting, which again impacts the cost of ownership. We had a last event this summer, which stopped us for some weeks.

So the lesson learned is that self-hosting storage has hidden costs and risks: when one has to follow such a siren, (s)he needs to be conscious of all implications, and when possible, a well-managed cloud solution (e.g., using independent encryption for privacy) could be the best alternative.

Because self-hosting storage is only an apparent saving, you could pay the full price later, and that’s often neither evident nor quantifiable.

Is the future federated? Or, why the hyperscalers are here to stay

2026-08-07T19:00:00Z

Sovereignty in the EU is now the trend, at least as a conceptual idea. Someone finally discovered that most of the Western world is dependent on a few US big companies, and the current status of Geopolitics is quite confusing, causing stomachaches and concerns among multiple governments.

Under these conditions, it is a good idea to understand the status of the mainstream internet services, including the so-called cloud and social networks. As explained in a past post, for some of these services, the EU situation is not so bad. We have many good companies that offer hosting and cloud services at a good level, and we have all the skills in-house. Therefore, at least for basic services, including distributed storage, email systems, and computing resources, we are in a position to be independent within a reasonable time frame. Of course, the number and extent of our data centers are not comparable, but if we do not have ambitions of offering such services worldwide, they can be enough for European use.

About social networks and instant messaging systems, among others, the situation is more concerning. While some alternatives exist, in many cases the network effect and first-mover advantage are strong and heavily weigh against building any real alternative. The tragic reality is that a pure distributed network of peers, which has been the core of the Internet since its birth, has been disrupted by the imposition of concentrated, single-company, non-standardized services. Think of WhatsApp, Facebook, LinkedIn, X/Twitter, Instagram, YouTube, or even the so-called alternative like Telegram or Bluesky. Most of the concrete alternatives are simply regional or restricted, simply a different incarnation of the walled garden concept. Think of TikTok, WeChat, Sina Weibo, and other regional services; they are simply different, isolated networks that do not communicate with others.

Like it or not, the only true alternatives for such systems are the fully federated services, such as Mastodon, Matrix, DeltaChat, or Peertube. There are literally dozens of such independent services based on open protocols, and they fully reflect the original idea of the Internet networking. Problem solved? Unfortunately, no, because the scale of such implementations is far from being comparable. We talk about thousands (about 1,5 million monthly active users in the luckiest case) of users against billions: a full order-of-magnitude difference, which causes evident network-effect problems. Common users are simply not incentivized to participate in such alternative services unless strongly motivated.

But imagine for a moment that all Internet users would like to enter the federated worlds.

In most cases, the typical federated server hosts a few hundred to a few thousand users. For such numbers, very small management teams and even small budgets could be viable and easily covered through crowdfunding. The most populated Mastodon instance (Mastodon.social) counts less than 1 million accounts with less than 300,000 daily active users currently. In a different service context, Codeberg supports less than 400,000 users and 700,000 projects, through 1,700 active members for its association (numbers of 2026). This is perfectly viable for a small-scale service and also the ideal situation with a good tradeoff between quality of service and efforts/size.

In our hypothesis, such servers would probably be subject to an average load multiplied by 1000 times, even in the case of fully working federated mutual help by smaller instances. In order to avoid service collapse, such instances would require an intensive architecture improvement and a significant increase in human effort for management. In other words, big money and the spreading of efforts among many individuals and organizations. Finding a tradeoff to avoid a new concentration and consolidation of services in the hands of very few actors would be the next challenge.

Again and again, my consideration is always the same: it is not only a matter of open protocols, mutual peers, and technicalities; it is a matter of governance and sustainability. The crude reality is that concentration among a very few active parties (just another oligarchy of service providers) could be unavoidable as the number of participants scales up. Of course, the only possibility of avoiding such evil concentrations is to ensure sufficient fragmentation by conception, for example, by imposing a maximum number of users per instance, or by maintaining small worlds of strictly thematic/regional instances.

So, federated services could be the answer for the future, but only by ensuring enough diversification of equal-weight instances, which could not scale at the same rate as the number of active users. Small is nice and safe, and this is the natural scale of federated servers to retain. In the opposite case, that would only become the next large walled garden, with serious sustainability problems and the constant temptation of having a new king of the realm. The obvious implication is that the mainstream bigco services will not be threatened by such alternatives, even considering that a lot of people are simply not aware/interested enough in privacy and digital rights. I think the only true limitation to well-known hypertrophic services can and should be found in international and national regulations, implemented with a grain of salt to avoid excessive problems and management load for federated services. Such a grain of salt and reasonability could be the key for the future, or, in the opposite case, establish the end of the Internet dream, which is already threatened by hyperscalers.

To understand the matter, think of Gmail, which is probably the most common email service worldwide. The most credible estimate is that almost 40% of email traffic is managed there, making the whole email thing a fully federated service with a few dominant providers. Most users do not search for alternatives, and many may have multiple alternatives and Gmail in use for different purposes. Even in these conditions, email is an example of a successful federated service. I would be happy if social networks moved in such a direction, frankly.

A completely different matter is the AI drama, where I see most of the risk for the future, for both Europe and most of the free world. Incredibly enough, the only current concrete alternative to the hypertrophy of less than a handful of US giants is represented by the open-weights, Chinese models. They could also be used at a reasonable scale locally, even if not for personal use. Unfortunately, the finest models are also extremely demanding for computing resources (both in terms of memory, storage, and GPUs) and out of reach for individual use. They can be considered for business investments, but it is already evident that a lot of effort is required to create an effective harness, where most of the value is currently present. The whole framework of agentic software and the capability of aligning models to goals will be the challenge of the future, but that will be out of reach for most users (including most companies), who the limits of general providers (OpenAI, Anthropic, Google, or anyone else) will constrain.

More and more, I see personal computing in the AI context will consist of running software on someone else's cluster, with all obvious implications for privacy and security for the average user. We will live sad times about that, with personal computing becoming managed computing, and users reduced to customers. Of course, the biggest problem is the awareness of users in such conditions, which appears more and more complicated due to the absence of a fully independent alphabetization policy in this new(?) digital world.

But maybe this is matter for another post...

A Terramaster NAS with Debian, take two.

2026-01-19T13:00:00Z

After experimenting at home, the very first professional-grade NAS from Terramaster arrived at work, too, with 12 HDD bays and possibly a pair of M2s. NVME cards. In this case, I again installed a plain Debian distribution, but HDD monitoring required some configuration adjustments to run smartd properly.

A decent approach to data safety is to run regularly scheduled short and long SMART tests on all disks to detect potential damage. Running such tests on all disks at once isn't ideal, so I set up a script to create a staggered configuration and test multiple groups of disks at different times. Note that it is mandatory to read the devices at each reboot because their names and order can change.

Of course, the same principle (short/long test at regular intervals along the week) should be applied for a simpler configuration, as in the case of my home NAS with a pair of RAID1 devices.

What follows is a simple script to create a staggered smartd.conf at boot time:

#!/bin/bash
#
# Save this as /usr/local/bin/create-smartd-conf.sh
#
# Dynamically generate smartd.conf with staggered SMART test scheduling
# at boot time based on discovered ATA devices

# HERE IS A LIST OF DIRECTIVES FOR THIS CONFIGURATION FILE.
# PLEASE SEE THE smartd.conf MAN PAGE FOR DETAILS
#
#   -d TYPE Set the device type: ata, scsi[+TYPE], nvme[,NSID],
#           sat[,auto][,N][+TYPE], usbcypress[,X], usbjmicron[,p][,x][,N],
#           usbprolific, usbsunplus, sntasmedia, sntjmicron[,NSID], sntrealtek,
#           ... (platform specific)
#   -T TYPE Set the tolerance to one of: normal, permissive
#   -o VAL  Enable/disable automatic offline tests (on/off)
#   -S VAL  Enable/disable attribute autosave (on/off)
#   -n MODE No check if: never, sleep[,N][,q], standby[,N][,q], idle[,N][,q]
#   -H      Monitor SMART Health Status, report if failed
#   -s REG  Do Self-Test at time(s) given by regular expression REG
#   -l TYPE Monitor SMART log or self-test status:
#           error, selftest, xerror, offlinests[,ns], selfteststs[,ns]
#   -l scterc,R,W  Set SCT Error Recovery Control
#   -e      Change device setting: aam,[N|off], apm,[N|off], dsn,[on|off],
#           lookahead,[on|off], security-freeze, standby,[N|off], wcache,[on|off]
#   -f      Monitor 'Usage' Attributes, report failures
#   -m ADD  Send email warning to address ADD
#   -M TYPE Modify email warning behavior (see man page)
#   -p      Report changes in 'Prefailure' Attributes
#   -u      Report changes in 'Usage' Attributes
#   -t      Equivalent to -p and -u Directives
#   -r ID   Also report Raw values of Attribute ID with -p, -u or -t
#   -R ID   Track changes in Attribute ID Raw value with -p, -u or -t
#   -i ID   Ignore Attribute ID for -f Directive
#   -I ID   Ignore Attribute ID for -p, -u or -t Directive
#   -C ID[+] Monitor [increases of] Current Pending Sectors in Attribute ID
#   -U ID[+] Monitor [increases of] Offline Uncorrectable Sectors in Attribute ID
#   -W D,I,C Monitor Temperature D)ifference, I)nformal limit, C)ritical limit
#   -v N,ST Modifies labeling of Attribute N (see man page)
#   -P TYPE Drive-specific presets: use, ignore, show, showall
#   -a      Default: -H -f -t -l error -l selftest -l selfteststs -C 197 -U 198
#   -F TYPE Use firmware bug workaround:
#           none, nologdir, samsung, samsung2, samsung3, xerrorlba
#   -c i=N  Set interval between disk checks to N seconds
#    #      Comment: text after a hash sign is ignored
#    \      Line continuation character
# Attribute ID is a decimal integer 1 <= ID <= 255
# except for -C and -U, where ID = 0 turns them off.

set -euo pipefail

# Test schedule configuration
BASE_SCHEDULE="L/../../6"  # Long test on Saturdays
TEST_HOURS=(01 03 05 07)   # 4 time slots: 1am, 3am, 5am, 7am

DEVICES_PER_GROUP=3

main() {
    # Get array of device names (e.g., sda, sdb, sdc)
    mapfile -t devices < <(ls -l /dev/disk/by-id/ | grep ata | awk '{print $11}' | grep sd | cut -d/ -f3 | sort -u)

    if [[ ${#devices[@]} -eq 0 ]]; then
        exit 1
    fi

    # Start building config file
    cat << EOF
# smartd.conf - Auto-generated at boot
# Generated: $(date '+%Y-%m-%d %H:%M:%S')
#
# Staggered SMART test scheduling to avoid concurrent disk load
# Long tests run on Saturdays at different times per group
#
EOF

    # Process devices into groups
    local group=0
    local count_in_group=0

    for i in "${!devices[@]}"; do
        local dev="${devices[$i]}"
        local hour="${TEST_HOURS[$group]}"

        # Add group header at start of each group
        if [[ $count_in_group -eq 0 ]]; then
            echo ""
            echo "# Group $((group + 1)) - Tests at ${hour}:00 on Saturdays"
        fi

        # Add device entry
        #echo "/dev/${dev} -a -o on -S on -s (${BASE_SCHEDULE}/${hour}) -m root"
        echo "/dev/${dev} -a -o on -S on -s (L/../../6/${hour}) -s (S/../.././$(((hour + 12) % 24))) -m root"

        # Move to next group when current group is full
        count_in_group=$((count_in_group + 1))
        if [[ $count_in_group -ge $DEVICES_PER_GROUP ]]; then
            count_in_group=0
            group=$(((group + 1) % ${#TEST_HOURS[@]}))
        fi
    done
}

main "$@"

To run such a script at boot, add a unit file to the systemd configuration.

sudo systemctl  edit --full --force regenerate-smartd-conf.service
sudo systemctl enable regenerate-smartd-conf.service

Where the unit service is the following:

[Unit]
Description=Generate smartd.conf with staggered SMART test scheduling
# Wait for all local filesystems and udev device detection
After=local-fs.target systemd-udev-settle.service
Before=smartd.service
Wants=systemd-udev-settle.service
DefaultDependencies=no

[Service]
Type=oneshot
# Only generate the config file, don't touch smartd here
ExecStart=/bin/bash -c '/usr/local/bin/create-smartd-config.sh > /etc/smartd.conf'
StandardOutput=journal
StandardError=journal
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Creating your own Guix substitute server

2025-12-21T16:30:00Z

I lately dedicated some time to setting up my own substitute server for Guix on a foreign distribution. This post is about that experience, after verifying that such a process is currently quite underdocumented. A substitute server is clearly a required step in order to cultivate a personal or unofficial/alternative channel for Guix, at least if one has more than one box (and possibly one physical location) to manage.

For people not up to date with the news from these digital bits, in the last year or so, I elected the Guix package manager and system as my second-preferred OS, after Debian GNU/Linux. I published more posts about it, all readable under my dedicated feed.

What is a Substitute Server?

Guix is mainly a source-level roll-on distribution, so at every pull, there is a concrete risk of having to build your own binaries if no binary provider is available to speed up your update process. That is a certainty if you play with your own packages. This is the concrete reason to possibly have a substitute server in the LAN, even though the official ones can be subject to problems and DDoS. Specifically, I set up a substitute server based on Debian GNU/Linux reachable both at home and at work. While the Guix manual is currently quite verbose about how to use a substitute server from a client, it is bit incomplete about the server side.

Installation of Guix on Debian

The very first step is to install the Guix package manager on Debian, which is currently not more possible using a Debian package from the main archive, as in the past (see this LWN post about). Currently, a binary provided by the Guix project has to be used for that. It is quite straightforward if you use the installer script provided with some caution.

For instance, to install the current Guix version, it is possible to run:

$ wget -O /tmp/guix-binary.tar.xz \
    https://ci.guix.gnu.org/search/latest/archive?query=spec:tarball+status:success+system:x86_64-linux+guix-binary.tar.xz
$ wget -O /tmp/guix-install.sh https://guix.gnu.org/install.sh

# do your homework to check and verify binary and script, then...
$ sudo -i

# export GUIX_BINARY_FILE_NAME=/tmp/guix-binary.tar.xz
# chmod a+x /tmp/guix-install.sh
# apt install uidmap gnupg
# /tmp/guix-install.sh
# rm -f /tmp/guix-install.sh /tmp/guix-binary.tar.xz

That will install a daemon for building sources under an unprivileged guix-daemon user, which has full control over the guix data store at /gnu/store.

Daemon configuration and storage

Such a daemon is run under systemd control with a specific unit file. First, the Guix data store needs plenty of disk space and inodes; I suggest moving the /gnu/store tree under an XFS or even a BTRFS filesystem, for that.

Second, the current installer tries to mount /gnu/store in read-only mode, which can prevent running the guix command as an unprivileged user on foreign distros. This can be disabled if problems occur, as could happen under specific conditions. Once the guix-daemon service is stopped, the gnu-store.mount can be disabled by ensuring the /gnu/store directory is correctly assigned to the guix-daemon user:

$ ls -ld /gnu/store/
drwxrwxr-t 584 guix-daemon guix-daemon 12M 18 dic 07.31 /gnu/store/

The systemd unit file for the Guix daemon

This is the proposed unit file for the guix-daemon process:

[Unit]
Description=Build daemon for GNU Guix
# Start before 'gnu-store.mount' to get a writable view of the store.
Before=gnu-store.mount

[Service]
ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon \
  --discover=yes \
  --substitute-urls='https://bordeaux.guix.gnu.org https://ci.guix.gnu.org'
Environment='GUIX_STATE_DIRECTORY=/var/guix' 'GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale' LC_ALL=en_US.utf8
# Run under a dedicated unprivileged user account.
User=guix-daemon
# Bind-mount the store read-write in a private namespace.
PrivateMounts=true
BindPaths=/gnu/store
MountFlags=private
Type=exec
AmbientCapabilities=CAP_CHOWN
StandardOutput=journal
StandardError=journal
OOMPolicy=continue
Restart=always
TasksMax=8192

[Install]
WantedBy=multi-user.target

Note that, when building from sources in specific cases, it could be a good idea to provide an additional TMPDIR environment variable to bypass the limited size of the /tmp directory (specifically for tmpfs implementation as for Debian 13+). For instance, building a full Linux kernel could require more than 32 GB of storage available in /tmp, which corresponds by default to at least 64 GB of RAM on the system. A typical personal box can be quite limited for that. So adding TMPDIR=/var/tmp to the Environment variable could be a sane idea.

Setting up Guix Publish

To set up the substitute server, a guix publish process must be started via systemd. First, create a publishing asymmetric key pair and a destination for the cache directory:

$ sudo -i guix archive --generate-key
$ sudo mkdir -p /var/cache/guix/publish
$ sudo chown guix-daemon:guix-daemon /var/cache/guix/publish

This is the systemd unit file for the guix-publish.service:

[Unit]
Description=Guix substitute server
After=network.target

[Service]
Type=simple
User=root
Environment='GUIX_LOCPATH=/root/.guix-profile/lib/locale'
ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix publish \
  --listen=0.0.0.0 \
  --port=8080 \
  --user=guix-daemon \
  --compression=gzip:9 \
  --compression=lzip:9 \
  --cache=/var/cache/guix/publish \
  --ttl=30d --negative-ttl=1h \
  --workers=4
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

This will start the service in unprivileged mode on an unprivileged port, which can be easily routed via an NGINX proxy, for instance (my preferred way). I use this system to route the service on both the home and the at-work LANs.

Once done, of course, the next point is defining a policy to populate such a server. While it could be nice running a proper cuirass build service to run continuously building from a custom repository, it is also perfectly fine simply using a manifest file with common packages or the multiple configurations used by personal boxes. Here is a series of alternatives to build a handful of packages.

guix build gcc
guix system build my-laptop-config.scm
guix build -m manifest.scm
guix build make cmake vim emacs perl

A simple manifest example:

(specifications->manifest
 '("gcc-toolchain"
   "python"
   "emacs"
   "git"
   "make"
   "cmake"
   "perl"
   "grass"
   "gdal"
   "geos"
   "gawk"
   "vim"
   ;; Add more packages...
))

Of course, a very simple strategy could consist of running regular pulls & builds for all packages of primary interest and making frequent enough purges of the data store to avoid excessive storage consumption. That can be avoided if plenty of space is available for builds.

Once setup the server and populating it, it is possibile to add its URL to the configurations of all personal/LAN boxes for foreign and native installations, and live peacefully.

Happy guixing, folks!

Installing Debian on a USB stick for a Terramaster NAS

2025-10-15T18:00:00Z

I recently bought a basic NAS for home use. The NAS is a nice Terramaster F2-425, which is a very basic RAID1-only NAS with a decent CPU and 2.5Gb network. Terramaster allows users to either use its custom Linux-based TOS or install any other operating system supported by the x86_64-based platform. Note that this model does not mount any NVME unit for the OS, as for the F2-424.

Common choices include TrueNas, Proxmox, or any other Linux-based distribution. My choice has been a plain Debian stable distribution because I do not have special requirements and prefer a lightweight CLI-only solution over a dashboard. The F2-425 does not have NVME cards, only regular HDDs/SSDs. However, when installing an independent OS, as in my case, you can immediately use an external USB stick for the system, and dedicate HDDs to data. The unit even has a tiny (264MB) internal USB stick for installing TOS, but I simply used a decent 16GB SanDisk thumb drive. The clear advantage is the possibility of having the base system and data perfectly separated, and multiple copies of the stick for safety.

Of course, the installation of such a system can be done without using the Debian installer at all, so I'm describing here how to perform such an installation for my future reference and for other geeks. Of course, you need a running Linux system with debootstrap installed. The process involves partitioning the stick in GPT mode, installing the base system and EFI, and configuring the system to finalize a bootable system with the necessary software to connect the NAS to the network, including OpenSSH.

Note that the 2.5 Gb Ethernet is a RealTek, so a firmware blob (firmware-realtek package on Trixie) is required to properly work with that. Alternatively, another of the USB ports could also be used to add a wireless connection. The OS stick could be simply mounted on the internal port, but it requires opening the chassis for that and using a tiny stick.

At power-on, the internal TOS dongle automagically boots up, so connecting an HDMI display and a keyboard is required to change the setup to boot the Debian EFI image on the stick. On F2-425, press the <F12> key to access the AMI setup and change boot priorities. There are always slight differences among AMI BIOS setups, so it is required to find the right key to access settings and change boot options.

Let's consider /dev/sde as the name of the USB stick device on the host where it will be prepared. A GPT partition can be created via GNU parted, as follows:

parted /dev/sde 	# to create EFI and root primary partition
partprobe --summary /dev/sde
sfdisk -l /dev/sde
mkfs.vfat -F 32 /dev/sde1
mkswap /dev/sde2
mkfs.ext4 /dev/sde3

Once done, installing the base system is immediate.

mount /dev/sde3 /mnt
mount /dev/sde1 /mnt/boot/efi
debootstrap trixie /mnt

mount -o bind /dev /mnt/dev
mount -t devpts devpts /mnt/dev/pts
mount -t proc proc /mnt/proc
mount -t sysfs sysfs /mnt/sys
mount -t tmpfs run /mnt/run

cp /etc/apt/sources.list.d/debian.sources /mnt/etc/apt/sources.list.d/.
cp /etc/resolv.conf /mnt/etc/.

echo "nas" > /mnt/etc/hostname
sed -i -e 's/localhost$/localhost\n127.0.0.1\tnas/' /mnt/etc/hosts

rootfs=$(blkid /dev/sde | grep TYPE=\"ext4\"|awk '{print $2}'|cut -d\" -f2)
vfat=$(blkid /dev/sde|grep TYPE=\"vfat\"|awk '{print $2}'|cut -d\" -f2)
swap=$(blkid /dev/sde|grep TYPE=\"swap\"|awk '{print $2}'|cut -d\" -f2)

cat >/mnt/etc/fstab <<EOF
UUID=$rootfs / ext4 noatime,errors=remount-ro 0 1
UUID=$vfat /boot/efi vfat noatime,umask=0077 0 1
UUID=$swap none swap sw 0 0
EOF

chroot /mnt
apt update
apt upgrade -y
apt install grub-efi-amd64 linux-image-amd64 ssh \
        firmware-misc-nonfree \
        firmware-realtek xfsprogs rsync pmount \
        gddrescue screen util-linux-extra bash-completion \
 	mdadm  parted smartmontools htop ntp unattended-upgrades sudo
useradd -m -G sudo -s /bin/bash -C 'Your Name' your_username 
passwd your_username
adduser your_username plugdev
apt install tzdata locales
dpkg-reconfigure locales
grub-install --target=x86_64-efi --force-extra-removable /dev/sde
update-initramfs -u
apt clean
exit # leave the chroot
umount /mnt/run
umount /mnt/sys/firmware/efi/efivars
umount /mnt/sys
umount /mnt/proc
umount /mnt/dev/pts
umount /mnt/dev

Note that required HDDs can be easily installed later. I manually configured the two disks with GNU parted for a GPT Linux RAID partition. After booting with the stick, a simple install of the md array support suffices. Typically, the USB stick runs as /dev/sdd

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
mkfs.xfs /dev/md0
mkdir /data
data=$(blkid /dev/md0|grep TYPE=\"xfs\"|awk '{print $2}'|cut -d\" -f2)
echo "UUID=$data /data xfs defaults 1 1" >>/etc/fstab

In order to allow shutting down by pressing the power button, it is required to configure systemd-logind as follows.

sed -i -e 's/^#HandlePowerKey=poweroff/HandlePowerKey=poweroff/' \
       -e 's/^#HandlePowerKeyLongPress=ignore/HandlePowerKeyLongPress=ignore/' \
    /etc/systemd/logind.conf

systemctl restart systemd-logind.service

It could also be a good idea to stop the periodic auto-scan on the RAID volume for big disks, which can take ages to run.

sed -i -e 's/^AUTOCHECK=true/AUTOCHECK=false/' /etc/default/mdadm
systemctl restart mdmonitor.service

The network configuration depends definitively on the type of connection used and the home network setup. In my case, the NAS uses a static IPv4 address, so it can be configured through ifupdown, and it is only necessary to correctly write the /etc/network/interfaces for the enp1s0 Realtek 2.5Gb Ethernet interface. Not that it requires a non-free firmware blob to run.

After the initial syncing, a series of software to better manage the NAS can be installed, but that is optional and can be the subject of a different post. For sure, for better convenience, a copy of the USB stick with the complete configuration is a good idea, to allow a fast recovery in case of failures.

Sharing configurations and data among multiple boxes

2025-05-04T23:00:00Z

Following a post detailing my email setup across multiple personal boxes, I'll summarize an outline of my approach for managing data across multiple workstations and individual profiles in this post. This system (or collection of tools and workflow) has evolved over time and continues to adapt to various needs and scenarios, offering high flexibility and control for my use cases.

First of all, it is mandatory to use git to manage multiple configurations with branches for almost every box or family of boxes (e.g., home vs. work, personal vs. shared). Some people tend to use git for the whole home, but in my honest opinion, it is inappropriate for performance reasons, even if Joeyh's git-annex and LFS mode are helpful in those regards. In many cases, it is not required to version all data files, and even if this time-machine-like solution is considered the ideal one for user's homes, it could negatively impact storage requirements and long-term performances. Also, having a mono-repo for all your own stuff could be challenging. In my case, I deal with multiple private and public Git repositories, including GitHub, Gitlab, and others all around, and maintaining a single repo is not a good idea for me.

The good news is that it is generally possible to limit the use of git to a subset of the whole home, specifically the dotfiles used to configure common tools and programs, a helpful set of scripts for personal use, and a few other sparse things. In many cases, some useful scripts to synchronize and update everything is more than enough. Part of my automation workflow is based on Ansible rules, so the whole thing is easily integrated into such management.

It's important to note that I strongly advise against using git for sensitive information, such as credentials and keys. Even with a personal git server, copying such information via directly attached media storage is safer, ensuring they are not exposed on a permanently internet-connected repository. For such cases, specific solutions like Vault/OpenBao could provide an extra layer of security for workgroups, giving you peace of mind and confidence in the system's security measures. That said, the whole topic of the management of passwords, credentials, and other private information would require another post for completeness because it is not straightforward when privacy and security need to be fully conjugated with long-term goals.

Dot files and other stuff maintenance

That said, I currently use at least a couple of different git repositories, one for my dotfiles and one for helpful scripts for personal productivity. These repositories are branched, at least in home/office flavors, and based on multiple languages, including shell, Perl, Python, awk, and others.

I found it helpful to clone such repositories immediately, install them via the GNU stow utility, and install pure symlinks under my $HOME and personal binaries directories. However, for some specific cases (notably the Vim stuff), I prefer maintaining a separate git repository that I update separately: this is partially for historical reasons, but even because, in some cases, the whole stuff is a fork of other repositories.

In the past, I adopted a bare repository for dotfiles and a custom git call to maintain all dotfiles directly in place instead of symlinking them, which is a slightly different approach. Something like the following snippet of shell code:

    $ echo 'alias dotfiles="/usr/bin/git \
         --git-dir=$HOME/.dotfiles.git/ \
         --work-tree=$HOME"' >> $HOME/.bashrc
    $ source $HOME/.bashrc
    $ echo ".dotfiles.git" >> .gitignore
    $ git clone --bare <my_git_host>:/opt/git/dotfiles.git \
            $HOME/.dotfiles.git
    #
    # clean up current dotfiles (with directories) 
    # available in the common repository and backup
    #
    $ mkdir -p .dotfiles-backup && \
    	  dotfiles checkout 2>&1 | egrep "\s+\." | awk {'print $1'} | \
    	  xargs -I{} dirname {} | sort | uniq | xargs -I{} mkdir --parents \
          .dotfiles-backup/{} && dotfiles checkout 2>&1 | \
          egrep "\s+\." | awk {'print $1'} |  \
  		  xargs -I{} mv {} .dotfiles-backup/{}
    $ dotfiles checkout
    $ dotfiles config --local status.showUntrackedFiles no

can be used with a proper alias to maintain all registered dotfiles in each box's HOME. Of course, the previous stow-based approach is much more immediate.

Gnome configuration

My dotfiles repository does not include the main desktop configuration, specifically my Gnome settings, including the few extensions I use. Not all my personal boxes include a desktop, but it is much more immediate to maintain a single dump of a finalized configuration based on the main Gnome version in use instead of branching and following its internals properly (which can largely change from one version to another).

    $ dconf dump / >gnome-settings.dump 	# on the source workstation
    $ tar czvf gnome-shell.tgz .local/share/gnome-shell

    $ dconf load / <gnome-settings.dump 	# on the target workstation
    $ tar xvf gnome-shell.tgz

Data syncing and backups

While code and configurations are better managed via git, I also deal with many generic data files (up to dozens of GBs per file or more) and documents that I synchronize automatically or on demand. Two tools I use intensively — syncthing and unison — can cover both goals. They require at least one master copy to be reachable, and having one master host per site is the best idea.

Finally, my primary backup tool for years has been the venerable borg tool, which allows planning incremental backups and rotation plans of master hosts. At least one encrypted backup is stored on an external cloud for convenience. This is an aspect where I'm still looking for a definitive solution far from the usual big companies for personal stuff (a fight lost at work, unfortunately), specifically for what is attaining the mobile ecosystem and family. That's because, but for strictly FOSS requirements, I'm also interested only in a self-hosted, long-term, and stable solution that I could directly maintain for a lifetime. Currently, I'm simply using rclone to sync third-party cloud storage when required. Of course, the 3-2-1 rule for backup is fully respected, thanks to the use of multiple sites and devices, even without using cloud providers, but I would prefer taking in the loop additional encrypted copies of some stuff (mainly videos and photos) even for sharing with others.

My geeky email setup explained to humans

2025-02-16T11:00:00Z

Being an ancient mariner in the virtual ocean of the Big Network, I started using emails at the very beginning of the Internet era (at least here in Italy) on a SunOS pizza box, which I used at the time as my primary workstation. That was a giant step ahead for me because my first serious use of email in 1991 was on a Digital VT220 terminal under VMS OS and its MAIL client (who remembers the old times of DCL?). At that time, I started using Elm as my Mail User Agent, a software that stopped being developed in the first few years of this millennium.

In 1995, a new attractive MUA started to be developed, and it fascinated my geek inclination: as any smart enough reader can well imagine, it was Mutt, and as anyone knows, it still sucks less as a MUA, and it is nonetheless being developed after almost thirty years. Nowadays, colleagues and friends are always perplexed when they see me still reading and answering emails on a terminal in text mode, but I never found serious motivation to change my email workflow and prefer a graphical MUA instead. I occasionally use a smartphone with K-Mail (now Thunderbird), but I like reading and writing emails on one of my GNU/Linux PCs.

First of all, my mailboxes are read by or received in a single box (let's call it the Collector, with appropriate local backup), permanently connected to the network. It also provides SMTP/IMAP services and a last-resort spam filtering capability, thanks to Spamassassin+Razor and Procmail rules. Mailboxes are provided by third-party services, including (but not exclusively) a series of GMail and MS accounts, and that box is able to communicate with each of them to regularly read new messages, thanks to Fetchmail. Any of my PCs can then connect with the Collector to read remotely archived emails via IMAP and send answers and new messages through any of the legitimate SMTP servers associated with each email account I use: that is because of the SPF/DKIM specs used for all email boxes nowadays.

The Collector archives and rotates multiple mailboxes monthly and yearly, including the main one, where all urgencies are automatically routed when they arrive. I use multiple email accounts for mailing list subscriptions, so many messages are automatically routed off the main track on purpose. For ages, many bad MUAs out there only send HTML emails, so I occasionally use some graphical helper to browse some of them. The result is a long period archive of emails over the last 30 years that I occasionally purge of the mailing list messages to reduce the storage load. Eventually, I use Notmuch to dive into tons of Maildir mailboxes accumulated on the Collector, and that is more than enough because an ancient message is rarely required in my experience.

Thirty years of using Fetchmail, Procmail, Dovecot, Exim, and Mutt seem more than enough to explore new ways of managing emails, but my laziness prevents me from keeping an eye on my second choice, which would be Emacs with mu4e and offlineimap or sync to read emails in disconnected mode when required. Note that Mutt has caching capability, so the initial load of the mailboxes is a fast enough experience.

Finally, I found it helpful in my setup to use other interesting tools, such as lbdb, to collect senders' addresses automatically and have them at my fingertips. Recently, the Collector and all clients started using Simon Robinson's nice email-oauth2-proxy as an intermediate service to access 2FA-enabled email accounts, a reasonable complication of the original setup.

That said, any sane geek should rigorously use a personal domain and email account to avoid being locked in the use of a third-party domain and storage limitations: there is nothing worse than having to change tons of personal contact emails registered in dozens of services because of the dropping of someone else domain (or even worse, being exposed to their changes in policies, features, and limitations). That has been my way in the last 30 years or so. For the same reason, using any webmail system to manage emails is neither practical nor scalable.

Yes, I know, I'm a damn old fashioned geek, but this is the Way.