10

S3 Configuration

Chris Lu edited this page 2026-04-06 18:39:18 -07:00

Table of Contents

• S3 Configuration Overview
• Quick Reference
• Zero Configuration (Simplest)
• Security & Persistence
• Basic Credentials (-s3.config)
• Configuration Format
• Available Actions
• S3 with Reverse Proxy
• Advanced IAM (-s3.iam.config)
• Configuration Format
• Key Components
• STS Configuration & Defaults
• Using Both Together
• Common Mistakes
• Wrong: Using identities in -s3.iam.config
• Correct: Using identities in -s3.config
• Configuration Methods Summary
• Embedded IAM API
• Disabling Embedded IAM
• File Permissions for S3 Uploads
• Per-Object Permissions via S3 ACL
• Server-Wide Default File Mode
• Priority Order
• Related Documentation

S3 Configuration Overview

SeaweedFS S3 gateway has two separate configuration systems for different purposes. Understanding the difference is crucial for proper setup.

Quick Reference

Option	Purpose	Use When
No Config	Zero Configuration	Simplest way to use STS features immediately
-s3.config	Basic S3 credentials	You need simple user authentication with access keys
-s3.iam.config	Advanced IAM (STS, OIDC)	You need OIDC integration or role-based access

Zero Configuration (Simplest)

weed s3 enables Advanced IAM (STS) by default with secure, auto-generated keys.

Use this for: Immediate access to STS features like AssumeRoleWithWebIdentity (e.g., for Kubernetes workloads) without any setup.

# S3 server starts with STS enabled, allowing both anonymous and STS access
weed s3 -filer=localhost:8888

Security & Persistence

• Default Effect: Allow: This defaults to Open Access (same as legacy behavior). It does not secure your data by default. To enforce access control, you must provide a config file with "defaultEffect": "Deny".
• Policy Store: Memory: By default, policies are stored in memory and lost on restart. This ensures "Zero Config" instances are isolated and don't accidentally enforce cluster-wide policies. For persistent policies, use -s3.iam.config and configure "storeType": "filer".

Basic Credentials (-s3.config)

Use this for: Simple username/password style authentication with access keys.

Documentation: S3 Credentials

weed s3 -config=/path/to/s3.json -filer=localhost:8888
# OR
weed server -s3 -s3.config=/path/to/s3.json
# OR
weed mini -s3.config=/path/to/s3.json

Configuration Format

{
  "identities": [
    {
      "name": "admin_user",
      "credentials": [
        {
          "accessKey": "your_access_key",
          "secretKey": "your_secret_key"
        }
      ],
      "actions": ["Admin", "Read", "Write", "List", "Tagging"]
    },
    {
      "name": "readonly_user",
      "credentials": [
        {
          "accessKey": "readonly_key",
          "secretKey": "readonly_secret"
        }
      ],
      "actions": ["Read", "List"]
    }
  ]
}

Available Actions

Action	Description
Admin	Full access, create/delete buckets
Read	Read objects
Write	Write/upload objects
List	List buckets and objects
Tagging	Manage object tags
Read:bucket1	Read access to specific bucket
Write:bucket1	Write access to specific bucket

S3 with Reverse Proxy

When SeaweedFS S3 is behind a reverse proxy (Nginx, HAProxy, AWS ALB, etc.), it needs to know the correct host and protocol to verify S3 signatures.

By default, SeaweedFS automatically detects the following headers from your proxy:

• X-Forwarded-Host (e.g. s3.example.com)
• X-Forwarded-Proto (e.g. https)
• X-Forwarded-Port (e.g. 443)

Alternatively, you can explicitly set the public-facing URL using the externalUrl flag. This is recommended for complex proxy setups or when you cannot easily modify proxy headers.

# Explicitly set the external S3 endpoint
weed s3 -s3.externalUrl=https://s3.example.com

For detailed configuration examples, see the S3 Nginx Proxy page.

---

Advanced IAM (-s3.iam.config)

Use this for: Enterprise features like OIDC/Keycloak integration, STS (Security Token Service), IAM policies, and role-based access control.

Documentation: OIDC Integration

weed s3 -filer=localhost:8888 -iam.config=/path/to/iam.json
# OR
weed server -s3 -s3.iam.config=/path/to/iam.json
# OR
weed mini -s3.iam.config=/path/to/iam.json

Configuration Format

{
  "sts": {
    "tokenDuration": "1h",       // Optional, default: "1h"
    "maxSessionLength": "12h",   // Optional, default: "12h"
    "issuer": "seaweedfs-sts",   // Optional, default: "seaweedfs-sts"
    "signingKey": "base64-..."   // Optional, auto-fallback if missing
  },
  "providers": [
    {
      "name": "keycloak",
      "type": "oidc",
      "enabled": true,
      "config": {
        "issuer": "https://keycloak.example.com/realms/myrealm",
        "clientId": "seaweedfs-s3",
        "jwksUri": "https://keycloak.example.com/realms/myrealm/protocol/openid-connect/certs"
      }
    }
  ],
  "policies": [
    {
      "name": "ReadOnlyPolicy",
      "document": {
        "Version": "2012-10-17",
        "Statement": [
          { "Effect": "Allow", "Action": ["s3:Get*", "s3:List*"], "Resource": ["*"] }
        ]
      }
    }
  ],
  "roles": [
    {
      "roleName": "ReadOnlyRole",
      "roleArn": "arn:aws:iam::role/ReadOnlyRole",
      "attachedPolicies": ["ReadOnlyPolicy"],
      "trustPolicy": { ... }
    }
  ]
}

Key Components

Component	Description
sts	Security Token Service configuration for temporary credentials
providers	OIDC identity providers (Keycloak, Okta, Auth0, etc.)
policies	AWS IAM-style policy documents
roles	IAM roles with trust policies for role assumption

STS Configuration & Defaults

The sts section is optional. If omitted or partially configured, the following defaults apply:

• tokenDuration: Defaults to 1h
• maxSessionLength: Defaults to 12h
• issuer: Defaults to seaweedfs-sts

Signing Key Fallback Strategy:

If signingKey is not provided in the IAM config, SeaweedFS attempts to find a key in the following order:

1. Filer Signing Key: The jwt.filer_signing.key from server configuration (security.toml or CLI).
2. SSE-S3 Master Key (KEK): If the filer key is missing, it falls back to the cluster-wide SSE-S3 Master Key (stored in filer at /etc/s3/sse_kek).
   • Auto-Generation: This key is automatically generated if it does not exist, ensuring a fallback is always available in a healthy cluster.
   • Security Note: A specific seaweedfs-sts-signing-key is derived from the master KEK using HKDF-SHA256 to ensure cryptographic isolation between STS and SSE-S3.

Important

: The -s3.iam.config does NOT support the identities field. For basic user credentials, use -s3.config instead.

---

Using Both Together

You can use both configuration options together:

weed s3 \
  -config=/path/to/s3-credentials.json \
  -iam.config=/path/to/iam-advanced.json \
  -filer=localhost:8888

This allows:

• Basic users to authenticate with access keys (from -s3.config)
• OIDC users to authenticate with JWT tokens (from -s3.iam.config)

---

Common Mistakes

Wrong: Using identities in -s3.iam.config

# This will NOT load identities!
weed s3 -iam.config=/path/to/config.json

With config file:

{
  "identities": [...]  // This is IGNORED by -iam.config
}

Correct: Using identities in -s3.config

weed s3 -config=/path/to/config.json

With config file:

{
  "identities": [...]  // This works with -config
}

---

Configuration Methods Summary

Method	Priority	Auto-Reload	Best For
-config file	Highest	SIGHUP	Production static config
Filer storage	Medium	Yes	Dynamic management
Admin UI	Medium	Yes	Web-based management
Environment variables	Fallback	No	Development/testing
Allow All	Lowest (Fallback)	N/A	Quick start / Dev / Local testing

See S3 Credentials for detailed information on each method.

---

Embedded IAM API

Starting with SeaweedFS 3.x, the IAM API is embedded in the S3 server by default. This allows managing users, access keys, and policies using AWS IAM CLI commands on the same endpoint as S3.

# Start S3 with embedded IAM (default)
weed s3 -filer=localhost:8888

# IAM and S3 use the same endpoint
aws --endpoint http://localhost:8333 iam create-user --user-name bob
aws --endpoint http://localhost:8333 s3 ls

Disabling Embedded IAM

If you don't need IAM API functionality, you can disable it:

weed s3 -iam=false -filer=localhost:8888

See Amazon IAM API for detailed IAM usage, or use the weed shell commands for interactive management:

weed shell
> s3.user.create -name alice
> s3.policy -put -name my-policy -file policy.json
> s3.policy.attach -policy my-policy -user alice
> s3.config.show

---

File Permissions for S3 Uploads

By default, objects uploaded via S3 are created with Unix file mode 0660 (-rw-rw----). This can be controlled per-object via the X-Amz-Acl header, or overridden server-wide with the -defaultFileMode flag.

Per-Object Permissions via S3 ACL

The X-Amz-Acl (canned ACL) header is mapped to Unix file permissions:

Canned ACL	File Mode	Permissions
private	0660	-rw-rw----
public-read	0644	-rw-r--r--
public-read-write	0666	-rw-rw-rw-
authenticated-read	0644	-rw-r--r--
bucket-owner-read	0644	-rw-r--r--
bucket-owner-full-control	0660	-rw-rw----

Example with rclone:

# Upload with world-readable permissions
rclone copy myfile.txt remote:mybucket/ --s3-acl=public-read

Example with AWS CLI:

aws s3 cp myfile.txt s3://mybucket/ --acl public-read --endpoint-url http://localhost:8333

Server-Wide Default File Mode

To change the default for all uploads (when no ACL header is provided):

# Standalone S3 server
weed s3 -defaultFileMode=0644

# Embedded in weed server
weed server -s3 -s3.defaultFileMode=0644

# Embedded in weed filer
weed filer -s3 -s3.defaultFileMode=0644

# Mini mode
weed mini -s3.defaultFileMode=0644

The value is an octal file mode string (e.g. 0644, 0666, 0660).

Priority Order

1. Per-object X-Amz-Acl header (if present and recognized)
2. Server-wide -defaultFileMode (if configured)
3. Built-in default 0660

---

Related Documentation

• S3 Credentials - Detailed documentation for basic credentials
• OIDC Integration - OIDC/STS integration guide
• Amazon S3 API - S3 API compatibility reference
• Amazon IAM API - IAM API support (embedded in S3)
• AWS IAM CLI - AWS CLI examples for IAM
• SeaweedFS-Iceberg-Catalog - Iceberg REST Catalog documentation
• Admin UI - Web-based credential management

Introduction

• Quick Start with weed mini
• Simplest S3 Bucket and User Setup
• Components
• Getting Started
• Production Setup
• A typical step‐by‐step example
• Benchmarks
• FAQ
• Applications

API

• Master Server API
• Volume Server API
• Filer Server API
• Client Libraries
• SeaweedFS Java Client

Configuration

• Replication
• Store file with a Time To Live
• Failover Master Server
• Erasure coding for warm storage
• EC Bitrot Detection
• Server Startup via Systemd
• Environment Variables

Filer

• Filer Setup
• Directories and Files
• File Operations Quick Reference
• Data Structure for Large Files
• Filer Data Encryption
• Filer Commands and Operations
• Filer JWT Use
• TUS Resumable Uploads

Filer Stores

• Filer Cassandra Setup
• Filer Redis Setup
• Super Large Directories
• Path-Specific Filer Store
• Choosing a Filer Store
• Customize Filer Store

Management

• Admin UI
• Admin UI OIDC
• Worker
• Plugin Worker Scheduling

Advanced Filer Configurations

• Migrate to Filer Store
• Add New Filer Store
• Filer Store Replication
• Filer Active Active cross cluster continuous synchronization
• Filer as a Key-Large-Value Store
• Path Specific Configuration
• Filer Change Data Capture
• Filer Operation Serialization

FUSE Mount

• FIO benchmark
• fstab and systemd mount
• POSIX Compliance
• Distributed POSIX Locks
• P2P reading in weed mount

WebDAV

SFTP Server

Cloud Drive

• Cloud Drive Benefits
• Cloud Drive Architecture
• Configure Remote Storage
• Mount Remote Storage
• Cache Remote Storage
• Cloud Drive Quick Setup
• Gateway to Remote Object Storage

AWS S3 API

• Amazon S3 API
• Supported APIs vs Minio
• S3 Lifecycle
• S3 Lifecycle vs Volume TTL
• S3 Conditional Operations
• S3 CORS
• S3 Object Lock and Retention
• S3 Object Versioning
• S3 API Benchmark
• S3 API FAQ
• S3 Bucket Quota
• S3 Rate Limiting
• S3 API Audit log
• S3 Nginx Proxy
• Docker Compose for S3

S3 Table Bucket

• S3 Table Bucket
• S3 Table Bucket Commands
• S3 Tables Security
• SeaweedFS Iceberg Catalog
• Iceberg Table Maintenance

Iceberg Integrations

• Spark Iceberg Integration
• Trino Iceberg Integration
• Dremio Iceberg Integration
• DuckDB Iceberg Integration
• Doris Iceberg Integration
• RisingWave Iceberg Integration
• Lakekeeper Iceberg Integration

S3 Authentication & IAM

• S3 Configuration - Start Here
• S3 Credentials (-s3.config)
• OIDC Integration (-s3.iam.config)
• Kubernetes ServiceAccount Authentication (IRSA-style)
• S3 Policy Variables
• S3 Policy Conditions
• S3 Bucket Policies
• Amazon IAM API
• AWS IAM CLI
• weed shell - Shell IAM Commands

Server-Side Encryption

• Server-Side Encryption
• Server-Side Encryption SSE-KMS
• Server-Side Encryption SSE-C

S3 Client Tools

• AWS CLI with SeaweedFS
• s3cmd with SeaweedFS
• rclone with SeaweedFS
• restic with SeaweedFS
• nodejs with Seaweed S3

Machine Learning

• TensorFlow with SeaweedFS
• Distributing AI Model Files for Multi-GPU Loading

HDFS

• Hadoop Compatible File System
• run Spark on SeaweedFS
• run HBase on SeaweedFS
• run Presto on SeaweedFS
• Hadoop Benchmark
• HDFS via S3 connector

Replication and Backup

• Async Replication to another Filer [Deprecated]
• Async Backup
• Async Filer Metadata Backup
• Async Replication to Cloud [Deprecated]
• Kubernetes Backups and Recovery with K8up

Metadata Change Events

• Filer Metadata Events
• Filer Notification Webhook

Messaging

• Structured Data Lake with SMQ and SQL
• Seaweed Message Queue
• SQL Queries on Message Queue
• SQL Quick Reference
• PostgreSQL-compatible Server weed db
• Pub-Sub to SMQ to SQL
• Kafka to Kafka Gateway to SMQ to SQL

Use Cases

• Use Cases
• Actual Users

Operations

• System Metrics
• weed shell
• Data Backup
• Deployment to Kubernetes and Minikube
• Deployment with seaweed-up
  • Cluster Plan Workflow
  • Cluster Plan Inventory Reference
  • Cluster Plan Day 2 Operations

Rust Volume Server

• Rust Volume Server

Advanced

• Large File Handling
• Optimization
• Optimization for Many Small Buckets
• Volume Management
• Tiered Storage
• Cloud Tier
• Cloud Monitoring
• Load Command Line Options from a file
• SRV Service Discovery
• Volume Files Structure

Security

• Security Overview
• Security Configuration
• Cryptography and FIPS Compliance
• Run Blob Storage on Public Internet

Misc Use Case Examples

• UrBackup with SeaweedFS
• Docker Image Registry with SeaweedFS
• SeaweedFS in Docker Swarm
• Words from SeaweedFS Users
• Independent Benchmarks
• Hardware