Recently, I pondered what would happen in case of a black out / internet outage and noticed that I would not even have that much material to read, as books are less and less common, including encyclopedias. Luckily, I discovered the Kiwix project, which I really like. It allows downloading full dumps of all Wikipedia articles in many languages and serve them in the local network. Other wikis/sites are available as well.

As large power banks, laptop batteries and other sources of energy are nowadays available in large quantities, I explored the idea of using a Raspberry Pi as a low power consumption WLAN access point that serves Wikipedia dumps with kiwix-tools.

Currently, I’m using a Raspberry Pi 3 with only 1GB of memory. I inserted a 256GB SD card that can easily contain a dump of the English and German Wikipedia including images.

If you want to set up your own hotspot & Wikipedia server, use the following procedure:

1. Download the .zim files you like, for example wikipedia_en_all_maxi_2025-08.zim.
   mkdir wikipedia
cd wikipedia
wget https://www.mirrorservice.org/sites/download.kiwix.org/zim/wikipedia/wikipedia_en_all_maxi_2025-08.zim
2. Install the kiwix-tools package and also dnsmasq:
   sudo apt install kiwix-tools dnsmasq tmux
3. Create a Kiwix Library and add the downloaded ZIM file:
   kiwix-manage lib.xml add wikipedia_en_all_maxi_2025-08.zim
   You can add additional files, if you like.
4. To set up a WLAN hotspot with NetworkManager, run sudo raspi-config and select NetworkManager to manage your network config.
5. Add a usable domain to the /etc/hosts file, for example wikipedia.local. Replace the IP address with the IP address of your Raspberry Pi. AFAICT NetworkManager will use 10.42.0.1/24:
   sudo bash -c 'echo "10.42.0.1 wikipedia.local" >> /etc/hosts'
6. NetworkManager internally uses dnsmasq as a DNS server when configuring a hotspot. However, it will not use the /etc/hosts file to serve local domain names. Therefore disable the internal dnsmasq by adding the line dns=none in the [main] section of the /etc/NetworkManager/NetworkManager.conf file.
7. Start and enable dnsmasq:
   systemctl start dnsmasq.service
systemctl enable dnsmasq.service
8. Start the hotspot:
   nmcli device wifi hotspot ssid wikipedia.local password "YOUR_SECRET"
   This command prints a UUID, which you need for additional configuration. You can later always retrieve the UUID with nmcli con show --active.
9. Optionally: if you want to make the Hotspot available to neighbors, you can disable the WLAN security/password. Additionally, you can (should) disable routing into your private LAN, if the Pi is connected via network cable.
   sudo nmmtui-edit

   If you want to always start the WLAN network, check the “Automatically connect” option.
10. Start the hotspot with
    nmcli connection up $(nmcli con show --active | grep wifi | awk '{print $2}')
11. Create a systemd service for Kiwix and start it. Run sudo nano /etc/systemd/system/kiwix.service and add the following content:
    [Unit]
Description=Kiwix Serve
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/usr/bin/kiwix-serve --port 80 --library /home/pi/wikipedia/lib.xml
Restart=always
RestartSec=15
[Install]
WantedBy=multi-user.target
12. Start and enable the service:
    systemctl daemon-reload
systemctl start kiwix.service
systemctl enable kiwix.service

You should now be able to connect to the wikipedia.local hotspot and open http://wikipedia.local (not https!) in any device that has a browser (for example smartphone).

Worth noting: kiwix.org sells ready to use hotspots, which probably directly supports the project.

I’m now using an offline network behind a data diode for several years. The data diode allows only transferring data from the internet into the offline network, but there is no back channel to the internet. Since I switched from a serial to a fiber optics connection, the speeds are suitable for day to day use. In this post I want to share my use cases and experiences.

My goals of using an offline network:

• Securely creating and storing private keys, for example PGP. The private keys are copied to hardware security modules (Yubikeys/Nitrokeys) for online usage.
• Backup of highly important data, for example .kdbx files.
• Offline knowledge management for very private information.

Overview of my network topology:

I generally use sshfs to transfer data between the devices in the online and internal offline network.

• In the online network, the desktop devices use sshfs to mount the “send” directory of the public facing Raspberry Pi of the data diode.
• In the offline network, the desktop devices use sshfs to mount the “receive” directory of the internal facing Raspberry Pi of the data diode.
• Additionally, the internal facing Raspberry Pi runs the OpenBSD httpd server to serve a mirror of the OpenBSD install, syspatch, packages and firmware directories for the amd64 and arm64 architectures.

In the offline network, I currently use the following OpenBSD packages for a very light weight setup:

• Zim for managing information
• git for versioning the Notebook directories created by Zim
• sshfs for mounting the diode directories
• KeePassXC for reading and managing kdbx backups and secrets that are only stored in the offline network
• mc (Midnight Commander) for managing files
• i3 with i3lock and i3status as a desktop manager
• gnupg for managing PGP keys

My experiences so far:

• The data diode speed of about 0.5 MB/s is sufficiently fast to transfer OpenBSD updates and the limited set of packages in a reasonable amount of time into the internal network. It takes me 1 to 2 hours to download and transfer all files for a new OpenBSD major release.
• I rarely have issues with OpenBSD in the offline network. Managing a mirror with all required files and packages does not require a lot of effort. I use the normal tools (sysupgrade, syspatch, pkg_add, fw_update) to manage the software.
• The offline desktop devices uses the OpenBSD full disk encryption. As I have a screen and a keyboard, I can directly unlock the disk during boot.
• The offline server only encrypts the /home directory so that it can boot without me attending the process. To mount the /home partition, I enter the disk password via SSH from a desktop device.
• I use the /home directory on the internal server as a backup destination for the files I work with on the offline desktop devices:
  • git repos are directly synchronized with git push
  • other files are copied via sshfs and mc
• I boot up the devices in the offline network only a few times a year. This usually happens when I need to rotate keys, create backups of new important secrets, or update the OpenBSDs in the internal network.

The easiest path for migrating Forgejo to PostgreSQL seems to be with the forgejo dump command. However, the import into the PostgreSQL database failed with

ERROR:  stack depth limit exceeded

I then tried to import the data and structure into an empty PostgreSQL database with pgloader. Forgejo rejected to start after the the import due to an unexpected database schema.

In the end I started Forgejo with an empty PostgreSQL database and waited until all tables were set up and then stopped the Forgejo process again. To not mess with the production data, I first copied the sqlite file. I then configured pgloader to only import the data. The pgloader config:

LOAD DATABASE
FROM sqlite:///root/forgejo.db
INTO postgresql://forgejo:SECRET@127.0.0.1/forgejo
WITH data only, reset sequences, prefetch rows = 10000
SET work_mem TO '16MB', maintenance_work_mem TO '512MB';

There were still some problems with columns that did exist in the SQLite database but not in the PostgreSQL database. I simply removed those columns in the copy of the SQLite database:

ALTER TABLE session DROP COLUMN created_unix;
ALTER TABLE hook_task DROP COLUMN repo_id;

Additionally, the keep_activity_private column did have some NULL values in the SQLite database. I solved this by first allowing NULL values in the PSQL database, then updating all values, and then again disallowing NULL values:

ALTER TABLE public.user ALTER COLUMN keep_activity_private DROP NOT NULL;
# run pgloader now
UPDATE public.user SET keep_activity_private=TRUE WHERE keep_activity_private IS NULL;
ALTER TABLE public.user ALTER COLUMN keep_activity_private SET NOT NULL;

In the end I can definitely recommend the migration. Forgejo is much faster with PSQL.

Today I came across a crashed InnoDB MariaDB database. When trying to restart the process, it immediately failed with the following messages in the systemd journal:

[Note] Starting MariaDB 10.11.6-MariaDB-0+deb12u1 source revision  as process 661197
[Note] InnoDB: Compressed tables use zlib 1.2.13
[Note] InnoDB: Number of transaction pools: 1
[Note] InnoDB: Using crc32 + pclmulqdq instructions
[Note] InnoDB: Using liburing
[Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
[Note] InnoDB: Completed initialization of buffer pool
[Note] InnoDB: File system buffers for log disabled (block size=4096 bytes)
[Note] InnoDB: Starting crash recovery from checkpoint LSN=10862992357
[Note] InnoDB: End of log at LSN=10868013717
[Note] InnoDB: Retry with innodb_force_recovery=5
[ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption
[Note] InnoDB: Starting shutdown...
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
[Note] Plugin 'FEEDBACK' is disabled.
[ERROR] Unknown/unsupported storage engine: InnoDB
[ERROR] Aborting

Before continuing with any recovery process, I created a backup of /var/lib/mysql:

cp -rp /var/lib/mysql /var/lib/mysql.bak

Some posts recommended to start MariaDB with innodb_force_recovery = 4. However, as you can see in the messages, MariaDB automatically tried to recover with innodb_force_recovery = 5 and failed. I manually set the recovery mode to 6 in the /etc/mysql/mariadb.cnf:

[mysqld]
innodb_force_recovery = 6

With that, MariaDB finally started. Sadly, this did not yet fully resolve the issue. In mode 6, MariaDB starts with all tables in read only mode. That means I was able to access the data, but the database was not really usable or fixed. The obvious next step was to do a mysqldump, clean the database and then insert all data again. Running mysqldump failed with the next error:

mysqldump: Error 1034: Index for table 'mytable' is corrupt; try to repair it when dumping table mytable at row: 3

Interestingly, dumping a single table did work. If anyone knows why, please let me know. I dumped a list of all table names in the database into a file called /root/tables, then with bash iterated over all tables and dumped each one into a dedicated file:

while read TABLE; do
   mysqldump -f mydatabase $TABLE > /root/mydatabase/$TABLE-$(date -I).sql;
done < /root/tables

All files can then just be concatenated into one large dump file:

cat /root/mydatabase/* > /root/mydatabase.sql

After the database content was safely stored in a text sql dump file, I bootstrapped a new /var/lib/mysql directory:

rm -rf /var/lib/mysql
apt install --reinstall mariadb-server
sudo -u mysql mariadb-install-db

Then I created the database again and inserted the dump:

MariaDB [(none)]> CREATE DATABASE mydatabase;
MariaDB [(none)]> use mydatabase;
MariaDB [mydatabase]> source /root/mydatabase.sql

After this, the database was healthy again.

In the past I used the Univention Corporate Server for Identity Management in some organizations. However, I find that UCS is relatively huge and I nowadays I prefer to operate a Samba server in combination with Keycloak, because it is easier to integrate into server orchestration tools. Also, Keycloak provides sufficient functionality to manage users within the Samba Active Directory. It took me quite some time to figure out how to migrate the data cleanly. The goal is to migrate the data into clean Active Directory structures. I only want to migrate the username, sn, givenName, displayName and mail attributes while keeping the objectUUID. Also, group memberships should be migrated.

One major caveat was to import the entryUUID of UCS into the objectUUID attribute of Samba. samba-tool has no parameter to set the objectGUID for a new user. A possible workaround is to use ldbmodify to import the user objects first. ldbmodify is available in the ldb-tools package on Debian based distributions.

The import assumes that the Samba server has already been provisioned with

samba-tool domain provision

The following script can be used to export the data in 2 files:

#!/bin/bash

UIDS=$(slapcat -a "(mail=*)" | grep uid: | sed "s/uid: //")

while IFS= read -r USERUID; do
	echo "Exporting '$USERUID'"
	USER_DATA=$(slapcat -a "(uid=$USERUID)")
	USER_DN=$(echo "$USER_DATA" | grep "dn: " | sed "s/dn: //")
	USER_GUID=$(echo "$USER_DATA" | grep "entryUUID: " | sed "s/entryUUID: //")
	USER_MAIL=$(echo "$USER_DATA" | grep -E "^mail:" | sed "s/^mail://")
	USER_FIRST_NAME=$(echo "$USER_DATA" | grep "givenName:" | sed "s/^givenName://")
	USER_LAST_NAME=$(echo "$USER_DATA" | grep "sn:" | sed "s/^sn://")
	USER_DISPLAY_NAME=$(echo "$USER_DATA" | grep "displayName:" | sed "s/^displayName://")
	USER_NTHASH=$(echo "$USER_DATA" | grep "sambaNTPassword: " | sed "s/sambaNTPassword: //")

	echo "Writing $USER_DN"
	echo "dn: CN=$USERUID,CN=Users,DC=example,DC=com" >> /root/users.ldif
	echo "changetype: add" >> /root/users.ldif
	echo "objectclass: user" >> /root/users.ldif
	echo "objectGUID: $USER_GUID" >> /root/users.ldif
	echo "sAMAccountName: $USERUID" >> /root/users.ldif
	echo "mail:$USER_MAIL" >> /root/users.ldif
	echo "displayName:$USER_DISPLAY_NAME" >> /root/users.ldif
	echo "sn:$USER_LAST_NAME" >> /root/users.ldif
	echo "givenName:$USER_FIRST_NAME" >> /root/users.ldif
	echo "" >> /root/users.ldif

	MEMBERSHIPS=$(echo "$USER_DATA" | grep "memberOf: " | sed "s/memberOf: //" | sed "s/cn=//" | cut -d ',' -f 1)
	while IFS= read -r MEMBERSHIP; do
		echo "samba-tool group addmembers '$MEMBERSHIP' $USERUID" >> /root/import.sh
	done <<< "$MEMBERSHIPS"
	echo "pdbedit -u $USERUID --set-nt-hash $USER_NTHASH" >> /root/import.sh
        echo "samba-tool user enable $USERUID" >> /root/import.sh
done <<< "$UIDS"

EXPORT_GROUPS=$(slapcat -a "(objectClass=posixGroup)" | grep "cn: " | sed "s/cn: //")
while IFS= read -r GROUPCN; do
	if [ "$GROUPCN" = "Domain Users" ] || [ "$GROUPCN" = "Domain Admins" ] || [ "$GROUPCN" = "Domain Guests" ] || [ "$GROUPCN" = "Domain Controllers" ]  || [ "$GROUPCN" = "Windows Hosts" ] || [ "$GROUPCN" = "DC Backup Hosts" ] || [ "$GROUPCN" = "DC Slave Hosts" ] || [ "$GROUPCN" = "Computers" ] || [ "$GROUPCN" = "Printer-Admins" ] || [ "$GROUPCN" = "Slave Join" ] || [ "$GROUPCN" = "Backup Join" ] ; then
		continue
	fi
	GROUP_DATA=$(slapcat -a "(cn=$GROUPCN)")
	GROUP_UUID=$(echo "$GROUP_DATA" | grep "entryUUID: " | sed "s/entryUUID: //")
	echo "Writing $GROUPCN"
	echo "dn: CN=$GROUPCN,CN=Users,DC=example,DC=com" >> /root/users.ldif
	echo "changetype: add" >> /root/users.ldif
	echo "objectClass: top" >> /root/users.ldif
	echo "objectClass: group" >> /root/users.ldif
	echo "cn: $GROUPCN" >> /root/users.ldif
	echo "name: $GROUPCN" >> /root/users.ldif
	echo "objectGUID: $GROUP_UUID" >> /root/users.ldif
	echo "sAMAccountName: $GROUPCN" >> /root/users.ldif
	echo "" >> /root/users.ldif
done <<< "$EXPORT_GROUPS"

Now copy the created users.ldif and import.sh files into the /root directory of your new Samba server. To import the data into Samba, first import the user objects with ldbmodify:

ldbmodify -H tdb:///var/lib/samba/private/sam.ldb /root/users.ldif --relax

Finally set the group memberships and password hashes with executing the import.sh:

bash /root/import.sh

The original documentation of Varmilo for its keyboards is rather difficult to understand and plain wrong in some cases. A Reddit thread is incomplete, at least for the 104 keys version. For the 104 keys keyboard, the following combinations can be used:

• Fn + Arrow Right: Switch Backlight mode
• Fn + Arrow Up: increase Backlight
• Fn + Arrow Down: decrease Backlight
• Fn + Right Win: hold for 3 seconds to switch Fn key with Right Win key (to switch back, press Left Win first, then Fn)
• Fn + ESC: hold for 3 seconds to reset settings (if switched with Win key, hold Left Win  + ESC)
• Fn + Left Ctrl: switch Caps Lock and left Ctrl (use Caps Lock key to switch back!)
• Fn + W: hold for 3 seconds to switch to Windows mode
• Fn + A: hold for 3 seconds to switch to Apple mode

Update 2022-10-22: As of the 7.2 release, OpenBSD supports booting from an encrypted RAID 1. The procedure below therefore becomes obsolete.

If you have an OpenBSD running on (mostly) encrypted RAID1 partitions like I described in https://sven-seeberg.de/wp/?p=1018, the unattended system upgrade triggered by sysupgrade will fail after rebooting into install mode. Without interaction, the system is stuck in a reboot loop. To continue with the upgrade process, follow these instructions:

1. When the error message appears that the system cannot continue, hit Control + C to prevent the system from rebooting. You should now have a shell.
2. Create the sd3 device: cd /dev; sh MAKEDEV sd3
3. Decrypt the softraid: bioctl -c C -l /dev/sd3a softraid0
4. Hit Control + D or type exit

The unattended upgrade should continue normally without any further interaction.