diff --git a/Async-Backup.md b/Async-Backup.md index a4fdd73..8b4b17d 100644 --- a/Async-Backup.md +++ b/Async-Backup.md @@ -80,3 +80,13 @@ The backup destination will have the following directory structure. /2021-03-02/dir1/file1 /2021-03-02/dir1/file4 ``` + +# Server-Side Encryption (SSE) + +`weed filer.backup` transparently decrypts SSE-S3 and SSE-KMS encrypted objects before uploading to the destination sink. The destination receives plaintext — use the destination storage's own encryption if encryption at rest is needed. + +- **SSE-S3**: Decrypted automatically using the KEK from `security.toml` (`WEED_S3_SSE_KEK` / `WEED_S3_SSE_KEY`). +- **SSE-KMS**: Requires a `[kms]` section in `security.toml` or `WEED_KMS_*` environment variables (the S3 API config file is not loaded by `filer.backup`). +- **SSE-C**: Cannot be decrypted during backup (the customer-provided key is not available). These objects will produce an error and be skipped. + +See [Server-Side Encryption — Replication and Backup](Server-Side-Encryption#replication-and-backup-with-sse) for configuration details. diff --git a/Async-Replication-to-Cloud.md b/Async-Replication-to-Cloud.md index 02838b9..3ce466e 100644 --- a/Async-Replication-to-Cloud.md +++ b/Async-Replication-to-Cloud.md @@ -54,5 +54,9 @@ directory = "/" # destination directory * Start the replication. "`weed filer.replicate`" * Start the filer. "`weed filer`" +# Server-Side Encryption (SSE) + +`weed filer.replicate` transparently decrypts SSE-S3 and SSE-KMS encrypted objects before uploading to cloud sinks. SSE-C objects cannot be decrypted (customer key unavailable) and will produce an error. See [Server-Side Encryption — Replication and Backup](Server-Side-Encryption#replication-and-backup-with-sse) for details. + # Replicate existing files See [[Async-Replication-to-another-Filer#replicate-existing-files]] \ No newline at end of file diff --git a/Filer-Active-Active-cross-cluster-continuous-synchronization.md b/Filer-Active-Active-cross-cluster-continuous-synchronization.md index 2da7d00..33b6ac7 100644 --- a/Filer-Active-Active-cross-cluster-continuous-synchronization.md +++ b/Filer-Active-Active-cross-cluster-continuous-synchronization.md @@ -101,6 +101,16 @@ However, it uses volume server IP addresses configured for the local cluster. `f ## Debug log To see all detail of transfers executed by `filer.sync`, options `-a.debug` or/and `-b.debug` can be added to the `weed filer.sync` process starting command line. +## Server-Side Encryption (SSE) + +`weed filer.sync` copies encrypted chunk data and encryption metadata as-is between clusters — no decryption or re-encryption occurs. This means both clusters must share the same SSE key configuration: + +- **SSE-S3**: Both filers must use the same KEK (configured via `WEED_S3_SSE_KEK` or `WEED_S3_SSE_KEY` in `security.toml`, or the same `/etc/s3/sse_kek` file on the filer). +- **SSE-KMS**: Both clusters must have access to the same KMS provider and keys. +- **SSE-C**: Works automatically since the encryption metadata is copied with the chunks. + +> **Warning:** If the destination cluster uses different SSE keys, replicated encrypted objects will be stored but cannot be decrypted. Reads will fail or return corrupt data. + ## Limitations This should be fairly scalable. However, it is limited by network bandwidth and latency. So even though changes are received within milliseconds and replayed right away, there would be data discrepancies if a file is changed quickly in two distant data centers. diff --git a/Server-Side-Encryption.md b/Server-Side-Encryption.md index 545741e..3aae3a7 100644 --- a/Server-Side-Encryption.md +++ b/Server-Side-Encryption.md @@ -173,6 +173,48 @@ If the filer file does not exist either, **SSE-S3 is disabled**. > **Note:** Use `s3.sse.kek` (not `s3.sse.key`) for migration — it uses the same hex format as the filer file. `s3.sse.key` derives a different key via HKDF and refuses to start while the filer file exists. +## Replication and Backup with SSE + +SSE-encrypted objects are handled transparently during replication and backup. The behavior depends on the replication command and destination type: + +### filer.sync (Filer-to-Filer) + +`weed filer.sync` copies encrypted chunk data and SSE metadata as-is between SeaweedFS clusters. No decryption or re-encryption occurs — the destination filer stores the same ciphertext with the same encryption metadata. + +**Requirement:** Both filers must share the same SSE key configuration: +- **SSE-S3:** Same KEK (via `WEED_S3_SSE_KEK`/`WEED_S3_SSE_KEY` or the filer's `/etc/s3/sse_kek`) +- **SSE-KMS:** Same KMS provider with access to the same keys +- **SSE-C:** Works automatically (encryption metadata is copied with the chunk) + +If the destination filer uses different SSE keys, the replicated data will be stored but **cannot be decrypted** — reads will fail silently with corrupt data. + +### filer.backup / filer.replicate (To External Storage) + +`weed filer.backup` and `weed filer.replicate` decrypt SSE-encrypted objects before uploading to external sinks (S3, GCS, Azure, B2, local disk). The destination receives plaintext — the destination storage system's own encryption (e.g., AWS S3 SSE, Azure Storage encryption) should be used if encryption at rest is required. + +| SSE Type | Behavior | +|----------|----------| +| **SSE-S3** | Decrypted automatically using the KEK from `security.toml` or the filer | +| **SSE-KMS** | Decrypted automatically if KMS is configured in `security.toml` `[kms]` section | +| **SSE-C** | **Cannot be decrypted** — the customer key is not available in the backup context. These objects are skipped with an error. | + +#### SSE-KMS Configuration for Backup + +The S3 API server loads KMS configuration from its config file, but `filer.backup` and `filer.replicate` do not load the S3 config. To enable SSE-KMS decryption during backup, add a `[kms]` section to `security.toml` or set the equivalent `WEED_KMS_*` environment variables: + +```toml +[kms] +default_provider = "openbao" + +[kms.providers.openbao] +type = "openbao" +address = "http://localhost:8200" +token = "root-token" +transit_path = "transit" +``` + +If no KMS configuration is provided, SSE-KMS encrypted objects will fail to replicate with a "KMS is not configured" error. + ## Implementation Notes - **SSE-KMS**: Supports AWS KMS, Google Cloud KMS, OpenBao/Vault; Azure Key Vault is available behind the `azurekms` build tag (experimental)