Migrate, sync, and back up S3 storage with proof every run finished
Godwit Sync is a plan-first tool to migrate, sync, and back up S3-compatible object storage. Move only what changed between any S3 stores or from a local server, keep every version as a plain S3 object, and get proof every run finished with checksum verification and crash-safe resume.
Core Capabilities
Reliable S3 transfers, verified end to end
Every run is controlled, resumable, and verifiable.
Plan Before You Move
Know exactly what will happen before any data moves. Generate migration plans, inspect them, modify as needed, and execute with confidence.
- Deterministic migration plan generation (--plan-only)
- Migration plans stored in SQLite
- Inspect, modify, and reuse plans
- Predictable execution every time
$ godwit sync \
--source s3://production-bucket \
--destination s3://backup-bucket \
--plan-only --brief
────────────────────────────────────────
PLAN CREATED
────────────────────────────────────────
✓ Run-ID: run-1770316589731184186
✓ Objects analyzed: 685,234
✓ Objects to transfer: 620,205
✓ Objects skipped: 65,029
✓ Data to transfer: 40.03 GB
✓ Data skipped: 4.7 GB
────────────────────────────────────────
End-to-End Checksums
Validate every object with real MD5 hashes, not just ETAGs.
Deterministic Runs
Identical inputs produce identical results, with no hidden randomness.
Parallel, Controlled Transfers
High throughput without sacrificing stability or safety.
Metadata Fidelity
Preserve object metadata exactly as stored at the source.
Chunk-Level Recovery
Recover from partial uploads without corrupting large files.
Object Lock Preservation
Replicate GOVERNANCE/COMPLIANCE retention and legal hold settings per version.
Production-Safe by Design
Built for environments where downtime and data loss are not options. Rate-limit operations, resume from any point, and keep your source systems healthy.
- Rate-limited reads and writes
- True stream-to-stream copy (no local staging)
- Resume from exact file or chunk
- Large file stability
- No overload of source systems
Every version, verified byte for byte
Back up local server directories or migrate S3 buckets without losing version history. Godwit Sync keeps every version, writes each change as a new S3 object version, verifies checksums, skips Glacier-class versions, and preserves Object Lock retention.
- Back up a local server directory to any S3-compatible bucket with incremental runs
- Migrate full or partial S3 bucket version history
- Store each change as a new S3 object version, never overwriting older versions
- Verify every version with .md5 sidecars and godwit plan verify
- Skip Glacier-class versions and preserve Object Lock retention and legal hold
# Plan a full version history migration
$ godwit sync \
--source s3://prod-bucket \
--source-endpoint s3.amazonaws.com \
--source-access-key <SOURCE_ACCESS_KEY> \
--source-secret-key <SOURCE_SECRET_KEY> \
--destination s3://archive-bucket \
--destination-endpoint s3.amazonaws.com \
--destination-access-key <DESTINATION_ACCESS_KEY> \
--destination-secret-key <DESTINATION_SECRET_KEY> \
--state-path ./godwit.state.db \
--run-id version-history-prod \
--version-mode all --object-lock --plan-only
Planning...
⚠ Warning: 9 GLACIER objects detected.
Restore required before migration.
────────────────────────────────────────
PLAN CREATED
────────────────────────────────────────
✓ Run-ID: version-history-prod
✓ Objects analyzed: 23
✓ Objects to transfer: 14
✓ Objects skipped: 9
✓ Data to transfer: 48.20 GB
────────────────────────────────────────# Preview a point-in-time restore of the bucket as of 02:00
$ godwit sync \
--source s3://prod-bucket \
--source-endpoint s3.amazonaws.com \
--source-access-key <SOURCE_ACCESS_KEY> \
--source-secret-key <SOURCE_SECRET_KEY> \
--destination /restore/2am \
--state-path ./godwit.state.db \
--run-id pitr-2am \
--version-mode asof:2026-06-01T02:00:00Z --plan-only
Planning...
────────────────────────────────────────
PLAN CREATED
────────────────────────────────────────
✓ Run-ID: pitr-2am
✓ Objects analyzed: 1,268
✓ Objects to transfer: 1,204
✓ Objects skipped: 64
✓ Data to transfer: 62.40 GB
────────────────────────────────────────Rewind a bucket or prefix to any point in time
Reconstruct a bucket, a prefix, or a single object as it was at a chosen instant. Godwit Sync selects the version of each key that was live at your cutoff and writes it as the current object, so your data returns to that moment in one command.
- One command; absolute time or asof:30d
- Restores to a filesystem or plain bucket
- Skips keys deleted or created after cutoff
- Cutoff uses the source server's clock
Sync Only What Changed
The first run is a full sync; every later run compares each source object against confirmed-completed metadata and transfers only what changed, with no destination re-scan. Each run exposes a status endpoint and exit code so automation can confirm it finished.
- Size+ETag change detection
- Portable .md5 checksum sidecars
- Crash-safe confirmed baseline
- Proof every run finished
- Preview delta with --plan-only
# Incremental sync transfers only what changed
$ godwit sync \
--source s3://prod \
--source-endpoint prod-rustfs:9000 \
--source-access-key <SOURCE_ACCESS_KEY> \
--source-secret-key <SOURCE_SECRET_KEY> \
--destination s3://backup \
--destination-endpoint backup-rustfs:9000 \
--destination-access-key <DESTINATION_ACCESS_KEY> \
--destination-secret-key <DESTINATION_SECRET_KEY> \
--state-path ./godwit.state.db \
--run-id daily-backup \
--incremental --override --brief
────────────────────────────────────────
PLAN READY
────────────────────────────────────────
✓ Run-ID: daily-backup
✓ Objects analyzed: 500,234
✓ Objects to transfer: 1,072
✓ Objects skipped: 499,162
✓ Data to transfer: 890 MB
✓ Data skipped: 1.4 TB
────────────────────────────────────────
RUN COMPLETED
────────────────────────────────────────
✓ Run-ID: daily-backup
✓ Objects transferred: 1,072
✓ Data transferred: 890 MB
✓ Total time: 47s
────────────────────────────────────────Observable & Verifiable
Know exactly what's happening at every moment. Integrate with your existing monitoring stack and verify every byte after it lands.
- Verbose CLI UI with real-time progress
- REST status endpoint
- Prometheus-compatible metrics
- Predictable duration and progress estimates
- Built-in MD5 verify after every transfer
Data Movement, Done Right
Godwit Sync turns risky storage migrations into planned, observable, production-safe workflows for modern infrastructure teams.
Leaving MinIO? Move to S3, RustFS, or Ceph without losing the shape of your data.
Keep version history, Object Lock, checksum verification, and resumable transfers while moving away from a storage stack in maintenance mode.
Back up to S3 as plain objects.
Every version remains a normal object you can inspect and restore with familiar S3-compatible tools.