OpenArchiver: a self-hosted Gmail archive

Halfway through brainstorming where to put the Gmail archive, it occurred to me that I already have a ZFS pool in my home server, which is snapshotted and backed up nightly.

Once I had the storage sorted, I decided not to bother writing a spec. Instead, I had Claude build the stack on the storage I’d picked, while I made the connector call and ran the setup wizard myself, and it went from idea to running in the same session.

The app is OpenArchiver, an open-source self-hosted email archiving solution from LogicLabs. It pulls mail from your provider and stores each message as a standard .eml file, with full-text search on top. It needs a few helper containers alongside the main app, but I didn’t fork it.. I just adapted the compose file from their docker-compose.yml. I deployed it as a git-backed stack, just like everything else on the server (How a deploy works with Dockhand).

Getting Gmail into the system was the one real decision I had to make. I assumed the Google Workspace connector would be the natural choice (it’s built for Google, after all), but it turns out that relies on OAuth and requires domain-wide delegation.. a Google Workspace admin feature that isn’t available on personal Gmail accounts. So I went with the generic IMAP connector instead, using SSL and a Google app password. No admin console needed.

That app password is the only secret that unlocks my mailbox, and it’s stored in one place and one place only: I type it into the OpenArchiver web UI, where it’s AES-encrypted in Postgres. It’s not in my git repo, not in my password manager, nowhere. Every other secret in this stack is injected at deploy time and kept well out of git, but this Gmail credential (being “per user”) belongs to the app.

I spent most of my time figuring out which parts of the stack are unrecoverable. The .eml files live on a ZFS dataset on the archive pool, backed by Sanoid snapshots and a nightly Backrest job. Postgres stores the metadata and encrypted credential in a named Docker volume, with a logical dump before each backup. The Meilisearch index can be rebuilt from the stored mail, so it’s not backed up. Valkey only holds the job queue, so it’s not backed up either. Apache Tika extracts text from attachments but has no state to back up. The result: I can focus my full backup effort on the mail itself.

Backing up Postgres to a nightly dump is just one line in a registry file read by the pre-backup job. Using named volumes (instead of bind mounts) for all the stateful helper containers keeps me from having to chown host directories to match container user IDs.

openarchiver-postgres:open_archive:<role>

I needed to make it fit on a server where I already run many stacks. I kept the bridge network from upstream so services can discover each other by name (host networking would break that DNS). The upstream DB container is called plain postgres, which would clash on a shared host, so I renamed containers to e.g. openarchiver-postgres while keeping the service names.

The best moment was right after setup. I completed the setup wizard, then added a Gmail source myself, and it immediately started populating. The first-run check showed more than 580 .eml files, over 536 MB, and still growing. It was fun to watch because the stack was just an idea earlier in the same session. After the first pass, it syncs every 15 minutes. I ran it with the all-inclusive setting, so it’s even pulling in Junk and Trash. The next day’s audit showed it fully configured and ingesting (not pending).

    environment:
      SYNC_FREQUENCY: "*/15 * * * *"
      ALL_INCLUSIVE_ARCHIVE: "true"

That same evening, I added a tile to my Glance dashboard and set up Uptime Kuma monitors to alert me via Pushover, just like I did for the other stacks I set up that day, including VictoriaMetrics. A monitor is a good way to know if the archiver ever quietly stops.

This isn’t a light stack. Postgres, Meilisearch, and Tika (JVM) together use about 4 GB RAM on a 32 GB box, so it’s definitely a number to watch if the server comes under memory pressure.