Caddy could not obtain a certificate on first boot: every request to
acme-v02.api.letsencrypt.org timed out. Containers could reach
1.1.1.1:443 by address but resolved no names at all.
Container DNS goes to aardvark-dns on the bridge gateway (10.89.10.1:53).
That traffic terminates on the host, so it takes the input hook, not
forward. Netavark accepts it in its own table, but gitea_filter's input
chain has policy drop, and a packet must be accepted by every base chain
on the hook. Our drop won.
gitea_filter now accepts tcp/udp 53 from the podman subnet to its
gateway. Both come from instance metadata, so the ruleset is rendered
with envsubst, as the fail2ban jail already is. It is not interface-based
because netavark's bridge name (podman1) is not pinned.
setup_nftables also validates the rendered file before installing it.
Previously it installed first and validated second, so a ruleset that
failed to parse stayed in /etc/sysconfig and would fail nftables.service
on the next boot, leaving the host with no gitea_filter table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Self-hosted Gitea on a single e2-small AlmaLinux 10 VM in us-east1,
serving gitea.jasonmross.dev.
Runtime is podman quadlets (systemd .container/.network/.volume units).
Both images are built on Debian 13: Gitea from a GPG-verified release
binary, and Caddy from an xcaddy build carrying the Google Cloud DNS
provider (ACME DNS-01) and the Coraza WAF with the OWASP CRS embedded.
Infrastructure is a Pulumi program in Go against a GCS state backend.
Cloud Build handles CI: a push trigger for images, one for infra, and a
weekly scheduled rebuild. Everything Cloud Build touches is 2nd gen.
Notable design decisions, each documented where it lives:
- Quadlets track a floating :prod tag. AutoUpdate=registry compares
digests for a tag, so a digest-pinned image silently disables
auto-updates.
- Git transport and LFS bypass the WAF. With the bypass removed, a plain
git push returns 403 -- packfiles trip CRS reliably.
- gitea:wafMode drives both SecRuleEngine and whether the fail2ban jail
acting on WAF verdicts exists. Banning on detections that were never
blocks would turn a tuning false positive into an nftables ban.
- fail2ban bans at the nftables prerouting hook. Published container
ports are DNAT'd and never traverse INPUT, where the stock actions
install their rules.
- The DNS zone, backup bucket, and Gitea signing secrets are not
Pulumi-owned, so pulumi destroy cannot take them with it.
- The podman subnet is pinned because it is what Gitea's
REVERSE_PROXY_TRUSTED_PROXIES names.
Three update layers: dnf5-automatic for the OS, podman-auto-update with
health-gated rollback for containers, and a weekly image rebuild that
gives the second layer something to pull.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
JMR-devandClaude Opus 5.5