viaprog/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-06-13 23:36:45 +03:00
Code Issues Packages Projects Releases Wiki Activity
Page: S3 Conditional Operations
Pages
A typical step‐by‐step example AWS CLI with SeaweedFS AWS IAM CLI Actual Users Admin UI OIDC Admin UI Amazon IAM API Amazon S3 API Applications Async Backup Async Filer Metadata Backup Async Replication to Cloud Async Replication to another Filer Benchmark SeaweedFS as a GlusterFS replacement Benchmarks from jinleileiking Benchmarks Cache Remote Storage Choosing a Filer Store Client Libraries Cloud Drive Architecture Cloud Drive Benefits Cloud Drive Quick Setup Cloud Monitoring Cloud Tier Cluster Plan Day 2 Operations Cluster Plan Inventory Reference Cluster Plan Workflow Components Configure Remote Storage Cryptography and FIPS Compliance Customize Filer Store Data Backup Data Structure for Large Files Deployment to Kubernetes and Minikube Deployment with seaweed up Directories and Files Distributed POSIX Locks Distributing AI Model Files for Multi GPU Loading Docker Compose for S3 Docker Image Registry with SeaweedFS Doris Iceberg Integration Dremio Iceberg Integration DuckDB Iceberg Integration EC Bitrot Detection Environment Variables Erasure Coding for warm storage Error reporting to sentry FAQ FIO benchmark FUSE Mount Failover Master Server File Operations Quick Reference Filer Active Active cross cluster continuous synchronization Filer Cassandra Setup Filer Change Data Capture Filer Commands and Operations Filer Data Encryption Filer JWT Use Filer Metadata Events Filer Notification Webhook Filer Operation Serialization Filer Redis Setup Filer Server API Filer Setup Filer Store Replication Filer Stores Filer as a Key Large Value Store Gateway to Remote Object Storage Getting Started HDFS via S3 connector Hadoop Benchmark Hadoop Compatible File System Hardware Hobbyest Tinkerer scale on premises tutorial Home Iceberg Table Maintenance Independent Benchmarks Kafka to Kafka Gateway to SMQ to SQL Kubernetes Backups and Recovery with K8up Kubernetes ServiceAccount Authentication Lakekeeper Iceberg Integration Large File Handling Load Command Line Options from a file Master Server API Migrate Maintenance Scripts to Admin Script Plugin Migrate to Filer Store Mount Remote Storage OIDC Integration Optimization for Many Small Buckets Optimization P2P reading in weed mount POSIX Compliance Path Specific Configuration Path Specific Filer Store Plugin Worker Scheduling PostgreSQL compatible Server weed db Production Setup Pub Sub to SMQ to SQL Quick Start with weed mini Replication RisingWave Iceberg Integration Run Blob Storage on Public Internet Run Presto on SeaweedFS Rust Volume Server S3 API Audit log S3 API Benchmark S3 API FAQ S3 Bucket Policies S3 Bucket Quota S3 CORS S3 Conditional Operations S3 Configuration S3 Credentials S3 Lifecycle Architecture S3 Lifecycle Monitoring S3 Lifecycle Operator Guide S3 Lifecycle Troubleshooting S3 Lifecycle vs Volume TTL S3 Lifecycle S3 Nginx Proxy S3 Object Lock and Retention S3 Object Versioning S3 Policy Conditions S3 Policy Variables S3 Rate Limiting S3 Table Bucket Commands S3 Table Bucket S3 Tables Security SFTP Server SQL Queries on Message Queue SQL Quick Reference SRV Service Discovery Seaweed Message Queue SeaweedFS Architecture SeaweedFS Iceberg Catalog SeaweedFS Java Client SeaweedFS in Docker Swarm Security Configuration Security Overview Server Side Encryption SSE C Server Side Encryption SSE KMS Server Side Encryption Server Startup via Systemd Simplest S3 Bucket and User Setup Spark Iceberg Integration Store file with a Time To Live Structured Data Lake with SMQ and SQL Super Large Directories Supported APIs vs Minio System Metrics TUS Resumable Uploads TensorFlow with SeaweedFS Tiered Storage Trino Iceberg Integration UrBackup with SeaweedFS Use Cases Volume Files Structure Volume Management Volume Server API WebDAV Words from SeaweedFS Users Worker fstab and systemd mount nodejs with Seaweed S3 rclone with SeaweedFS restic with SeaweedFS run HBase on SeaweedFS run Spark on SeaweedFS s3cmd with SeaweedFS weed shell
Clone
3
S3 Conditional Operations
Chris Lu edited this page 2026-05-26 13:19:27 -07:00
Table of Contents

S3 Conditional Operations

SeaweedFS supports AWS S3-compatible conditional headers for safe concurrent access patterns, optimistic locking, and efficient conditional operations.

The conditional check and the write are atomic — cluster-wide. The precondition is evaluated against the live object and the write is applied under the same lock, on the filer that owns the key in the cluster's hash ring. Two clients racing an If-Match update never both succeed, regardless of which filer each one talks to. See Filer Operation Serialization for the underlying primitive (ObjectTransaction) and how it stays correct across filer restarts and ring changes.

Supported Conditional Headers

These conditions act on the target object (the object being read or written). All four are supported on GET, HEAD, PUT, POST (CompleteMultipartUpload), and COPY. DELETE supports only If-Match.

Header Type Applies to Special values
If-Match ETag GET, HEAD, PUT, POST, COPY, DELETE * matches any existing object; comma-separated ETag list is honored
If-None-Match ETag GET, HEAD, PUT, POST, COPY * matches only when the object does not exist (compare-and-create); comma-separated ETag list is honored
If-Modified-Since RFC 1123 date GET, HEAD, PUT, POST, COPY —
If-Unmodified-Since RFC 1123 date GET, HEAD, PUT, POST, COPY —

Copy-source conditionals

CopyObject and UploadPartCopy accept a second set of headers that condition on the source object. They are independent of the four standard headers above (which condition on the destination), so a single copy request can carry conditions for both ends.

Header Type
x-amz-copy-source-if-match ETag
x-amz-copy-source-if-none-match ETag
x-amz-copy-source-if-modified-since RFC 1123 date
x-amz-copy-source-if-unmodified-since RFC 1123 date

Evaluation order

Per RFC 7232 §6 and AWS S3, the conditions are evaluated in this order; later conditions are skipped when an earlier one is present:

  1. If-Match
  2. If-Unmodified-Since — ignored if If-Match is present (RFC 7232 §3.4)
  3. If-None-Match
  4. If-Modified-Since — ignored if If-None-Match is present (RFC 7232 §3.3)

Not supported

  • If-Range. A Range request without If-Range works normally; there is no header to fall back to a full-object response when the source has changed.

HTTP Examples

Conditional GET (Caching)

# First, get the object and note its ETag
curl -I "http://localhost:8333/mybucket/myfile.txt"
# Response: ETag: "d41d8cd98f00b204e9800998ecf8427e"

# Conditional GET - only download if object changed
curl -H "If-None-Match: \"d41d8cd98f00b204e9800998ecf8427e\"" \
  "http://localhost:8333/mybucket/myfile.txt"

Conditional PUT (Optimistic Locking)

# Safe update - only modify if object hasn't changed
curl -X PUT -H "If-Match: \"d41d8cd98f00b204e9800998ecf8427e\"" \
  -H "Content-Type: text/plain" \
  --data "Updated content" \
  "http://localhost:8333/mybucket/myfile.txt"

# Prevent overwrites - only create if object doesn't exist
curl -X PUT -H "If-None-Match: *" \
  -H "Content-Type: text/plain" \
  --data "New file content" \
  "http://localhost:8333/mybucket/newfile.txt"

If-Modified-Since

# Only download if modified after specific date
curl -H "If-Modified-Since: Wed, 15 Jan 2024 10:00:00 GMT" \
  "http://localhost:8333/mybucket/log-file.txt"

If-Unmodified-Since

# Update only if not modified since last read
curl -X PUT \
  -H "If-Unmodified-Since: Wed, 15 Jan 2024 10:00:00 GMT" \
  -H "Content-Type: text/plain" \
  --data "Updated content" \
  "http://localhost:8333/mybucket/document.txt"

Copy Operations

# Copy only if source object hasn't changed
curl -X PUT \
  -H "x-amz-copy-source: /source-bucket/source-file.txt" \
  -H "x-amz-copy-source-if-match: \"source-etag\"" \
  "http://localhost:8333/dest-bucket/dest-file.txt"

# Same copy, but also refuse to overwrite the destination
curl -X PUT \
  -H "x-amz-copy-source: /source-bucket/source-file.txt" \
  -H "x-amz-copy-source-if-match: \"source-etag\"" \
  -H "If-None-Match: *" \
  "http://localhost:8333/dest-bucket/dest-file.txt"

Conditional DELETE

# Delete only if the object's ETag still matches what we read
curl -X DELETE \
  -H "If-Match: \"d41d8cd98f00b204e9800998ecf8427e\"" \
  "http://localhost:8333/mybucket/myfile.txt"

DELETE supports If-Match only; the other three conditional headers are ignored on DELETE per the AWS S3 spec.

HTTP Status Codes

Status Code When Meaning
200 OK All conditions met Operation succeeded normally
304 Not Modified GET/HEAD with If-None-Match match, or If-Modified-Since not exceeded Object unchanged; response carries the ETag header but no body
400 Invalid Request If-Modified-Since or If-Unmodified-Since is not a valid RFC 1123 date Header is malformed
412 Precondition Failed Any other failed condition (all writes; GET/HEAD with If-Match mismatch or If-Unmodified-Since exceeded) Operation blocked by condition

Atomicity and concurrency

The contract is the same as AWS S3: a successful PUT with a precondition header means the precondition held at the moment the write was applied, not merely at some earlier read. The two cases worth calling out:

  • If-None-Match: "*" (compare-and-create). At most one of any number of concurrent PUTs for the same key succeeds; the rest return 412 Precondition Failed. Use this as a primitive for distributed leader election, exclusive file creation, or idempotent uploads.
  • If-Match: "<etag>" (optimistic concurrency). A read-modify-write cycle that fails with 412 means another writer changed the object between your read and your write. Retry against the new ETag.

How it stays correct across a multi-filer cluster:

  1. The S3 gateway forwards each write to a filer. That filer issues a single ObjectTransaction RPC describing the precondition plus the mutations (e.g. write the object + flip the version pointer + delete the prior null-version entry).
  2. The transaction is routed by key to the owner filer for that object — the same filer for every concurrent writer of the same key, regardless of which gateway or filer they entered through.
  3. The owner takes an exclusive per-path lock for the key's lifetime of the transaction, evaluates the precondition against the current metadata, and applies the mutations under that lock. The whole sequence is one RPC; there is no caller-held distributed lock that can leak on a slow or crashed client.
  4. During filer membership changes, the same routing primitive applies a cooling-window dual-read and a warm-up period so a new owner never serves a write whose precondition it cannot yet verify. The precondition guarantee survives ring changes.

The detailed mechanism is in Filer Operation Serialization. The same primitive also powers cross-mount POSIX advisory locks; see Distributed POSIX Locks.

Related Documentation

Introduction

API

Configuration

Filer

Filer Stores

Management

Advanced Filer Configurations

FUSE Mount

WebDAV

SFTP Server

Cloud Drive

AWS S3 API

S3 Table Bucket

Iceberg Integrations

S3 Authentication & IAM

Server-Side Encryption

S3 Client Tools

Machine Learning

HDFS

Replication and Backup

Metadata Change Events

Messaging

Use Cases

Operations

Rust Volume Server

Advanced

Security

Misc Use Case Examples