improved backup script

.env.example

@@ -0,0 +1,24 @@
+# Wasabi S3 endpoint (optional — defaults to s3.<region>.wasabisys.com)
+WASABI_ENDPOINT=s3.us-east-2.wasabisys.com
+
+# Wasabi region, e.g. us-east-2 (used to derive endpoint when WASABI_ENDPOINT is not set)
+WASABI_REGION=
+
+# Wasabi S3 bucket name (used when RESTIC_REPOSITORY is not set)
+WASABI_BUCKET=
+
+# AWS-compatible credentials for Wasabi
+AWS_ACCESS_KEY_ID=
+AWS_SECRET_ACCESS_KEY=
+
+# Restic repository encryption password
+RESTIC_PASSWORD=
+
+# Full restic repository string (optional — overrides bucket/prefix/region)
+RESTIC_REPOSITORY=
+
+# Prefix (folder) inside bucket (optional)
+RESTIC_PREFIX=
+
+# Path to JSON config file containing {"paths": [...]} (optional, backup subcommand only)
+FILE_PATH_CONFIG_PATH=

.gitignore

@@ -0,0 +1,2 @@
+.env.local
+.config/

AGENTS.md

@@ -0,0 +1,86 @@
+# AGENTS.md
+
+## Repo Overview
+
+- Purpose: run `restic backup` against a Wasabi S3-compatible repository.
+- Main entry point: `travel-backup-script.py`.
+- Dependencies are intentionally minimal:
+  - Python standard library
+  - `python-dotenv`
+  - External binary: `restic` must already be installed and available on `PATH`
+- There are currently no tests, package metadata files, or CI config in this repo.
+
+## File Map
+
+- `travel-backup-script.py`: single-script CLI implementation.
+- `requirements.txt`: Python dependency list (`python-dotenv` only).
+- `README.md`: user-facing setup and usage docs.
+- `.env.example`: template for environment variables.
+- `.gitignore`: ignores `.env.*` files (except `.env.example`).
+
+## Current Script Behavior
+
+The script uses two subcommands: `init` and `backup`.
+
+1. Loads `.env.local` automatically via `load_dotenv(".env.local")`.
+2. Parses CLI args into subcommands with shared repo flags and backup-specific source flags.
+3. Resolves the repository string:
+   - `--repository` / `RESTIC_REPOSITORY` takes priority (validated against `s3:s3.<region>.wasabisys.com/...` format).
+   - Otherwise built from `--bucket` / `WASABI_BUCKET`, `--region` / `WASABI_REGION`, `WASABI_ENDPOINT`, and `--prefix` / `RESTIC_PREFIX`.
+4. Constructs a minimal subprocess environment for `restic`:
+   - `WASABI_ENDPOINT`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `RESTIC_PASSWORD`
+   - `PATH`, `HOME` if present
+   - `SYSTEMROOT`, `SystemRoot`, `COMSPEC` (Windows-specific, required for DNS resolution and shell access)
+5. For `init`: runs `restic init` against the resolved repository.
+6. For `backup`: resolves input from `--source`, `--file`, or `FILE_PATH_CONFIG_PATH` env var, then runs `restic backup`.
+7. Prints the repo, command, and redacted env status before optionally executing `restic`.
+
+## Known Drift / Things To Watch
+
+- The JSON config file must contain a top-level `paths` key with a non-empty array of non-empty strings.
+- `--source` and `--file` are mutually exclusive; one of them or `FILE_PATH_CONFIG_PATH` is required for backup.
+- `--region` has no default — it must be provided via CLI or `WASABI_REGION` when not using `--repository`.
+
+## Local Development Workflow
+
+Use this order when changing the repo:
+
+1. Read `travel-backup-script.py` first. Nearly all behavior lives there.
+2. Treat `README.md` as partially authoritative only after comparing it to the script.
+3. If you change CLI flags, env loading, or repository construction, update code, README, and `.env.example` in the same change.
+4. Preserve secret-handling behavior. Do not print raw credentials.
+
+## Suggested Verification
+
+Because the repo has no automated tests, prefer lightweight verification:
+
+- Static review of argument parsing and env handling.
+- Dry-run init:
+  - `python travel-backup-script.py init --bucket <bucket> --region <region> --dry-run`
+- Dry-run backup with a direct source:
+  - `python travel-backup-script.py backup --source <path> --bucket <bucket> --region <region> --dry-run`
+- Dry-run backup with a JSON config file:
+  - `python travel-backup-script.py backup --file <config.json> --bucket <bucket> --region <region> --dry-run`
+
+Avoid real backup runs unless the user explicitly asks for them and valid credentials are present.
+
+## Editing Guidelines
+
+- Keep the implementation as a simple single-file CLI unless the user asks for a larger refactor.
+- Favor backwards-compatible flag changes where practical.
+- Be careful with subprocess environment changes:
+  - `PATH` must remain available so `restic` can be found.
+  - `HOME` is intentionally forwarded when present.
+  - On Windows, `SYSTEMROOT` and `COMSPEC` are required for DNS resolution and shell access.
+- Do not commit real `.env.local` or other secret files.
+- If adding tests later, prefer small CLI/unit tests around:
+  - `build_repo`
+  - `load_sources`
+  - `resolve_repo` (priority order, validation)
+  - missing `--bucket` handling
+  - env minimization/redaction behavior
+
+## Agent Notes
+
+- Start discovery from the script, not the README.
+- Keep `README.md`, `.env.example`, and the script aligned when flags change.

README.md

@@ -1 +1,127 @@
-# Travel Backup Script
+# Wasabi Restic Backup Script
+
+This script automates backups to a [Wasabi](https://wasabi.com/) S3-compatible bucket using [restic](https://restic.net/).
+It loads secrets from `.env.local`, builds a restic repository string, and runs `restic backup` with redacted environment logging.
+
+## Features
+
+- Two subcommands: `init` (initialize a new repository) and `backup` (run a backup).
+- Backup a single source path with `-s/--source`.
+- Backup multiple source paths from a JSON config file with `-f/--file`.
+- Load secrets from `.env.local` via [python-dotenv](https://pypi.org/project/python-dotenv/).
+- Support a prebuilt restic repository via `--repository` or `RESTIC_REPOSITORY`.
+- Redact sensitive environment values in output.
+- Support dry-run mode for command verification.
+
+## Requirements
+
+- Python 3.8+
+- [restic](https://restic.net/) installed and available on `PATH`
+- Install dependencies:
+
+```bash
+pip install -r requirements.txt
+```
+
+## Environment Variables
+
+The script loads `.env.local` automatically and uses these variables:
+
+| Variable | Required | Description |
+|---|---|---|
+| `AWS_ACCESS_KEY_ID` | Yes | Wasabi access key |
+| `AWS_SECRET_ACCESS_KEY` | Yes | Wasabi secret key |
+| `RESTIC_PASSWORD` | Yes | Restic repository encryption password |
+| `WASABI_ENDPOINT` | No | S3 endpoint (defaults to `s3.<region>.wasabisys.com`) |
+| `WASABI_REGION` | No | Wasabi region, e.g. `us-east-2` (used to derive endpoint) |
+| `WASABI_BUCKET` | No | Bucket name (used when `RESTIC_REPOSITORY` is not set) |
+| `RESTIC_REPOSITORY` | No | Full restic repository string (overrides bucket/prefix/region) |
+| `RESTIC_PREFIX` | No | Prefix (folder) inside bucket |
+| `FILE_PATH_CONFIG_PATH` | No | Path to JSON config file (backup subcommand only) |
+
+Only `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `RESTIC_PASSWORD`, and `WASABI_ENDPOINT` are passed to the restic subprocess. The script builds a minimal environment to avoid leaking unrelated variables.
+
+Create a `RESTIC_PASSWORD` via `openssl rand -base64 32` for best security.
+
+## Subcommands
+
+### `init` — Initialize a New Repository
+
+```bash
+python travel-backup-script.py init --bucket my-bucket --region us-east-2 --dry-run
+```
+
+### `backup` — Run a Backup
+
+```bash
+python travel-backup-script.py backup --source /etc --bucket my-bucket --dry-run
+```
+
+## Repository Resolution
+
+The repository string is resolved in this order:
+
+1. `--repository` CLI flag or `RESTIC_REPOSITORY` env var (must match `s3:s3.<region>.wasabisys.com/<bucket>[/<prefix>]`).
+2. Otherwise, built from `--bucket` / `WASABI_BUCKET`, `--region` / `WASABI_REGION`, `WASABI_ENDPOINT`, and `--prefix` / `RESTIC_PREFIX`.
+
+## CLI Options
+
+Shared options (both `init` and `backup`):
+
+- `--bucket`: Wasabi bucket name (or set `WASABI_BUCKET`)
+- `--prefix`: Prefix inside the bucket (or set `RESTIC_PREFIX`)
+- `--region`: Wasabi region, e.g. `us-east-2` (or set `WASABI_REGION`)
+- `--repository`: Full restic repository string (or set `RESTIC_REPOSITORY`)
+- `--dry-run`: Print the command and redacted environment without running `restic`
+
+Backup-only options:
+
+- `-s`, `--source`: Single file or directory to back up
+- `-f`, `--file`: Path to a JSON file containing `{"paths": ["..."]}`
+
+`-s/--source` and `-f/--file` are mutually exclusive. One of them (or the `FILE_PATH_CONFIG_PATH` env var) is required for backup.
+
+## JSON File Format
+
+Example config:
+
+```json
+{
+  "paths": [
+    "/path/to/Documents",
+    "/path/to/Pictures"
+  ]
+}
+```
+
+## Examples
+
+Initialize a repository (dry run):
+
+```bash
+python travel-backup-script.py init --bucket my-bucket --region us-east-2 --dry-run
+```
+
+Backup a single source (dry run):
+
+```bash
+python travel-backup-script.py backup --source /etc --bucket my-bucket --dry-run
+```
+
+Backup from a JSON file (dry run):
+
+```bash
+python travel-backup-script.py backup --file backup-paths.json --bucket my-bucket --dry-run
+```
+
+Use a custom prefix:
+
+```bash
+python travel-backup-script.py backup --source ~/Documents --bucket my-bucket --prefix laptop-backups
+```
+
+Use a full repository string:
+
+```bash
+python travel-backup-script.py backup --source ~/Documents --repository s3:s3.us-east-2.wasabisys.com/my-bucket/laptop-backups
+```

requirements.txt

@@ -0,0 +1 @@
+python-dotenv

travel-backup-script.py

@@ -0,0 +1,199 @@
+#!/usr/bin/env python3
+import argparse
+import json
+import os
+import re
+import subprocess
+import sys
+from dotenv import load_dotenv
+
+def build_repo(endpoint: str, bucket: str, prefix: str = None) -> str:
+    """
+    Build a restic S3 repository URL in the correct format:
+    s3:ENDPOINT/BUCKET[/PREFIX]
+    """
+    parts = [f"s3:{endpoint.rstrip('/')}"]
+    if bucket:
+        parts.append(bucket.strip("/"))
+    if prefix:
+        parts.append(prefix.strip("/"))
+    return "/".join(parts)
+
+def load_sources(source: str = None, source_file: str = None) -> list[str]:
+    """Resolve backup sources from --source or a JSON file."""
+    if source is not None:
+        return [source]
+
+    with open(source_file, "r", encoding="utf-8") as fh:
+        config = json.load(fh)
+
+    paths = config.get("paths")
+    if not isinstance(paths, list) or not paths:
+        raise ValueError('config file must contain a non-empty "paths" array')
+    if not all(isinstance(path, str) and path.strip() for path in paths):
+        raise ValueError('each entry in "paths" must be a non-empty string')
+
+    return paths
+
+def print_env_status(env: dict):
+    """Print redacted environment variable status."""
+    env_vars = ("WASABI_ENDPOINT", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "RESTIC_PASSWORD")
+    for key in env_vars:
+        if key in env and env.get(key) is not None:
+            print(f"  {key}=***REDACTED***")
+        else:
+            print(f"  {key}=NOT SET")
+
+def run_restic(repo: str, sources: list[str], dry_run: bool = False, env: dict = None, source_file: str = None):
+    """Run restic backup command."""
+    cmd = [
+        "restic",
+        "-r", repo,
+        "backup",
+        *sources,
+    ]
+
+    print("Repository:", repo)
+    print("Command:", " ".join(cmd))
+    print("Environment (redacted):")
+
+    # Use provided env or fall back to current process env
+    if env is None:
+        env = os.environ.copy()
+
+    print_env_status(env)
+
+    if dry_run:
+        if source_file:
+            print("Config file:", source_file)
+        print("Dry-run mode: not running restic.")
+        return 0
+
+    # Pass the provided environment explicitly to the restic subprocess
+    subprocess.check_call(cmd, env=env)
+    return 0
+
+def run_restic_init(repo: str, dry_run: bool = False, env: dict = None):
+    """Run restic init command to initialize a new repository."""
+    cmd = [
+        "restic",
+        "-r", repo,
+        "init",
+    ]
+
+    print("Initializing repository:", repo)
+    print("Command:", " ".join(cmd))
+    print("Environment (redacted):")
+
+    if env is None:
+        env = os.environ.copy()
+
+    print_env_status(env)
+
+    if dry_run:
+        print("Dry-run mode: not running restic.")
+        return 0
+
+    subprocess.check_call(cmd, env=env)
+    return 0
+
+def resolve_repo(args) -> str:
+    """Resolve the restic repository string from args and environment."""
+    # CLI --repository takes priority, then RESTIC_REPOSITORY env var
+    repository = getattr(args, "repository", None) or os.getenv("RESTIC_REPOSITORY")
+    if repository:
+        if not re.match(r"^s3:s3\.[a-z0-9-]+\.wasabisys\.com/.+", repository):
+            print(f"error: repository '{repository}' does not match expected format: s3:s3.<region>.wasabisys.com/<bucket>[/<prefix>]")
+            return None
+        return repository
+
+    bucket = getattr(args, "bucket", None) or os.getenv("WASABI_BUCKET")
+    if not bucket:
+        print("error: --bucket or WASABI_BUCKET is required when --repository / RESTIC_REPOSITORY is not provided")
+        return None
+
+    region = getattr(args, "region", None) or os.getenv("WASABI_REGION")
+    if not region:
+        print("error: --region or WASABI_REGION is required when --repository / RESTIC_REPOSITORY is not provided")
+        return None
+
+    endpoint = os.getenv("WASABI_ENDPOINT", f"s3.{region}.wasabisys.com")
+    # CLI --prefix takes priority, then RESTIC_PREFIX env var
+    prefix = getattr(args, "prefix", None) or os.getenv("RESTIC_PREFIX")
+    return build_repo(endpoint, bucket, prefix)
+
+def build_minimal_env() -> dict:
+    """Build a minimal env dict to pass to restic subprocesses."""
+    proc_env = os.environ.copy()
+
+    needed_keys = ("WASABI_ENDPOINT", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "RESTIC_PASSWORD")
+    minimal_env = {}
+    for k in needed_keys:
+        v = proc_env.get(k)
+        if v is not None:
+            minimal_env[k] = v
+
+    # Preserve system env vars needed by subprocesses
+    # PATH/HOME: find restic and resolve home directory
+    # SYSTEMROOT/COMSPEC: required on Windows for DNS resolution and shell access
+    system_keys = ("PATH", "HOME", "SYSTEMROOT", "SystemRoot", "COMSPEC")
+    for k in system_keys:
+        v = proc_env.get(k)
+        if v is not None:
+            minimal_env[k] = v
+
+    return minimal_env
+
+def main():
+    # Load environment variables from .env.local (if present)
+    load_dotenv(".env.local")
+
+    parser = argparse.ArgumentParser(description="Backup files to Wasabi S3 with restic")
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    # Shared repo arguments
+    repo_args = argparse.ArgumentParser(add_help=False)
+    repo_args.add_argument("--bucket", help="Wasabi S3 bucket name (or set WASABI_BUCKET)")
+    repo_args.add_argument("--prefix", help="Prefix (folder) inside bucket (or set RESTIC_PREFIX)", default=None)
+    repo_args.add_argument("--region", default=None, help="Wasabi region, e.g. us-west-1 (or set WASABI_REGION)")
+    repo_args.add_argument("--repository", help="Full restic repository string (or set RESTIC_REPOSITORY)")
+    repo_args.add_argument("--dry-run", action="store_true", help="Print command instead of running it")
+
+    # init subcommand
+    subparsers.add_parser("init", parents=[repo_args], help="Initialize a new restic repository")
+
+    # backup subcommand
+    backup_parser = subparsers.add_parser("backup", parents=[repo_args], help="Run a backup")
+    source_group = backup_parser.add_mutually_exclusive_group(required=False)
+    source_group.add_argument("--source", "-s", help="Path to file or directory to back up")
+    source_group.add_argument("--file", "-f", help='Path to a JSON config file containing {"paths": [...]} (or set FILE_PATH_CONFIG_PATH)')
+
+    args = parser.parse_args()
+
+    repo = resolve_repo(args)
+    if repo is None:
+        return 2
+
+    minimal_env = build_minimal_env()
+
+    try:
+        if args.command == "init":
+            return run_restic_init(repo, args.dry_run, env=minimal_env)
+
+        # backup command — resolve file path from CLI or env var
+        source_file = args.file or os.getenv("FILE_PATH_CONFIG_PATH")
+        if not args.source and not source_file:
+            print("error: --source, --file, or FILE_PATH_CONFIG_PATH is required for backup")
+            return 2
+
+        sources = load_sources(args.source, source_file)
+        return run_restic(repo, sources, args.dry_run, env=minimal_env, source_file=source_file)
+    except subprocess.CalledProcessError as exc:
+        print(f"error: restic exited with code {exc.returncode}: {exc}", file=sys.stderr)
+        return exc.returncode
+    except (OSError, json.JSONDecodeError, ValueError) as exc:
+        print(f"error: {exc}")
+        return 2
+
+if __name__ == "__main__":
+    sys.exit(main())