4

Rust Volume Server

Chris Lu edited this page 2026-05-25 23:47:04 -07:00

Table of Contents

• Rust Volume Server
• Installation
• From Binary Release
• Docker
• Build from Source
• Quick Start
• CLI Compatibility
• Options File
• SIGHUP Reload
• Subtle Differences from Go
• Storage Format
• Index Backend
• HTTP Behavior
• gRPC Behavior
• Security
• Heartbeat & Cluster
• Operational
• Extra CLI Flags (Rust only)
• Extra Index Backend (Rust only)
• Architecture
• Test Coverage
• Protobuf Code Generation

Rust Volume Server

The Rust volume server (weed-volume) is a drop-in replacement for the Go volume server, built for higher throughput and lower tail latency. It reads and writes the same .dat, .idx, and .vif files — you can point it at an existing Go volume directory and it just works.

Installation

From Binary Release

# Install the Go weed binary + Rust volume server
curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bash -s -- --component all

# Or just the Rust volume server
curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bash -s -- --component volume-rust

# Large disk variant (8TB max volume)
curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bash -s -- --component volume-rust --large-disk

Docker

# Run the Rust volume server in a container
docker run -d chrislusf/seaweedfs volume-rust -mserver=localhost:9333

# The standard image includes both Go and Rust volume servers.
# Use "volume" for Go, "volume-rust" for Rust.

The Rust binary is included in the standard Docker image for amd64 and arm64 platforms. On arm and 386, only the Go volume server is available.

Build from Source

cd seaweed-volume
cargo build --release
# Binary at target/release/weed-volume

By default the build enables the 5bytes feature (8TB max volume, matching Go's -tags 5BytesOffset). For 32GB max volume size:

cargo build --release --no-default-features

Quick Start

# Start a Go master server
weed master

# Start the Rust volume server (same flags as Go, but with dashes)
weed-volume --dir /data --mserver localhost:9333 --port 8080

CLI Compatibility

The Rust volume server accepts all 40+ flags from the Go volume server. Both single-dash (-port 8080) and double-dash (--port 8080) formats work.

# Go style
weed-volume -port 8080 -dir /data -mserver localhost:9333

# Standard CLI style
weed-volume --port 8080 --dir /data --mserver localhost:9333

Options File

Load flags from a file (same format as Go's -options flag):

weed-volume --options /etc/seaweedfs/volume.conf

Options file format (one per line, # comments):

# /etc/seaweedfs/volume.conf
port=8080
dir=/data
mserver=localhost:9333
max=0

CLI flags override values from the options file.

SIGHUP Reload

Sending SIGHUP to the Rust volume server reloads:

• Security config — re-reads security.toml and updates the IP whitelist
• New volumes — scans disk directories for newly added volume files

kill -HUP $(pidof weed-volume)

This matches the Go volume server's reload behavior.

Subtle Differences from Go

The Rust server passes all 109 Go integration tests (53 HTTP + 56 gRPC) and is binary-compatible at the storage layer. However, there are minor behavioral differences worth knowing about:

Storage Format

Aspect	Go	Rust
Needle format	V1/V2/V3, CRC32-C	Identical
Index format	16-byte (normal) or 17-byte (5BytesOffset) entries	Identical
SuperBlock	8-byte header	Identical
.vif files	protobuf-JSON, uint64 as strings	Identical
Compaction	Vacuum compact/commit/cleanup	Identical

Index Backend

Backend	Go	Rust
memory	In-memory HashMap	In-memory CompactMap (sorted Vec, binary search)
leveldb	LevelDB via CGo	rusty-leveldb (pure Rust)
leveldbMedium	LevelDB	rusty-leveldb
leveldbLarge	LevelDB	rusty-leveldb
redb	Not available	redb (pure Rust, crash-safe B-tree)

The redb backend is Rust-only. It provides crash-safe disk-backed needle maps without CGo dependencies. If you use redb and later switch back to the Go volume server, delete the .rdb files first — Go will rebuild the index from the .idx file.

HTTP Behavior

Behavior	Go	Rust	Notes
Request ID format	%X%08X (hex timestamp + random)	Identical	Not a UUID despite the x-amz-request-id header name
Range suffix beyond file size	Clamps to file size	Identical	Returns 206 with clamped range, not 416
If-Modified-Since / If-None-Match order	Checks If-Modified-Since first	Identical	Per Go behavior, not RFC 7232 order
JWT extraction precedence	Query ?jwt= > Authorization header > AT cookie	Identical
Chunk manifest MIME	Uses stored MIME, skips extension override	Identical
Method routing	GET/HEAD→read, POST/PUT→write, DELETE→delete, else→400	Identical
Pretty JSON	json.MarshalIndent with 2-space indent	Identical

gRPC Behavior

Behavior	Go	Rust	Notes
All 48 RPCs	Implemented	Implemented
VolumeTierMoveDat progress	No final 100% message	Identical
VolumeUnmount for missing volume	Returns nil	Identical
TierMoveDatFromRemote	No maintenance mode check	Identical
Cookie mismatch in streaming	Returns empty stream, nil error	Identical
BatchDelete EC shard cookie mismatch	Returns 406 status	Identical	HTTP reads return 404 for cookie mismatch

Security

Behavior	Go	Rust	Notes
JWT without exp claim	Accepted	Identical	Go's jwt-go doesn't require expiration
JWT algorithm	HS256	HS256 only
IP whitelist	CIDR matching	Identical
mTLS	Supported	Supported	Same security.toml format

Heartbeat & Cluster

Behavior	Go	Rust	Notes
Master gRPC port	port + 10000	Identical
Duplicate UUID retry	Exponential backoff (2s, 4s, 8s), exit after 3	Identical
Leader redirect	Reconnect via seed list	Sleeps 3s, connects directly to new leader	Functionally equivalent, slightly different reconnection path
GetMasterConfiguration	Called on startup	Identical
Pre-stop shutdown	preStopSeconds default 10	Identical

Operational

Behavior	Go	Rust	Notes
Profiling	Go pprof on debug port	Rust pprof (prost-codec) on debug port	Same /debug/pprof endpoint, different profiler
Prometheus metrics	Go client library	Rust prometheus crate	Same metric names and label patterns
/metrics endpoint	On admin/metrics port	On admin (debug) port	Both serve on admin port, not the main volume port
Disk monitoring	sysinfo package	sysinfo crate + libc
Volume preallocation	fallocate on Linux	fallocate on Linux via libc	Identical behavior
Write batching	Synchronous per-request	Batched write queue (up to 128, mpsc)	Rust groups concurrent writes to reduce lock acquisitions. Each client still blocks until its write is durable — no crash-safety difference
Streaming reads	Files > 1MB	Files > 1MB via spawn_blocking	Same threshold, async I/O

Extra CLI Flags (Rust only)

Flag	Description
--securityFile	Explicit path to security.toml (Go discovers it via viper)
--options	Load CLI flags from a file (Go supports this via fla9 but it's less visible)

Extra Index Backend (Rust only)

Flag value	Description
--index redb	Crash-safe disk-backed needle map using redb. Not available in Go. Files use .rdb extension.

Architecture

seaweed-volume/
├── src/
│   ├── main.rs              # Entry point, signal handling, SIGHUP reload
│   ├── config.rs             # CLI parsing, options file, security.toml
│   ├── security.rs           # JWT validation, IP whitelist
│   ├── images.rs             # JPEG EXIF orientation fix
│   ├── server/
│   │   ├── volume_server.rs  # Axum router, middleware, request ID
│   │   ├── handlers.rs       # HTTP read/write/delete/status/healthz
│   │   ├── grpc_server.rs    # All 48 VolumeServer RPCs
│   │   ├── heartbeat.rs      # Master registration, leader tracking
│   │   └── write_queue.rs    # Async batched write processing
│   ├── storage/
│   │   ├── needle/           # Needle serialization (V1/V2/V3)
│   │   ├── volume.rs         # Volume read/write/delete, .vif
│   │   ├── disk_location.rs  # Per-directory volume + EC shard management
│   │   ├── store.rs          # Multi-disk Store layer
│   │   └── needle_map.rs     # CompactMap, LevelDB, redb backends
│   └── remote_storage/
│       ├── s3.rs             # S3 fetch-and-write for remote needles
│       └── s3_tier.rs        # S3 multipart upload/download for tier moves
├── proto/                    # Shared .proto files (volume_server, master, remote)
├── vendor/                   # Vendored reed-solomon-erasure
└── tests/                    # Integration tests

Test Coverage

Suite	Count	Status
Rust unit tests	137	All pass
Rust integration tests	7	All pass
Go HTTP integration tests	53	All pass
Go gRPC integration tests	56	All pass
S3 remote storage tests	3	All pass

Run the Go integration tests against the Rust server:

cd seaweedfs
VOLUME_SERVER_IMPL=rust go test -v -count=1 -timeout 1200s \
  ./test/volume_server/grpc/... \
  ./test/volume_server/http/...

Run the Rust tests:

cd seaweed-volume
cargo test

Protobuf Code Generation

When updating .proto files, run make in weed/pb/ to regenerate Go code and copy the volume-server-related proto files to the Rust directory:

cd weed/pb && make
# This copies volume_server.proto, master.proto, and remote.proto
# to seaweed-volume/proto/

The Rust server's build.rs runs tonic-build to generate Rust code from these protos at compile time.

Introduction

• Quick Start with weed mini
• Simplest S3 Bucket and User Setup
• Components
• Getting Started
• Production Setup
• A typical step‐by‐step example
• Benchmarks
• FAQ
• Applications

API

• Master Server API
• Volume Server API
• Filer Server API
• Client Libraries
• SeaweedFS Java Client

Configuration

• Replication
• Store file with a Time To Live
• Failover Master Server
• Erasure coding for warm storage
• EC Bitrot Detection
• Server Startup via Systemd
• Environment Variables

Filer

• Filer Setup
• Directories and Files
• File Operations Quick Reference
• Data Structure for Large Files
• Filer Data Encryption
• Filer Commands and Operations
• Filer JWT Use
• TUS Resumable Uploads

Filer Stores

• Filer Cassandra Setup
• Filer Redis Setup
• Super Large Directories
• Path-Specific Filer Store
• Choosing a Filer Store
• Customize Filer Store

Management

• Admin UI
• Admin UI OIDC
• Worker
• Plugin Worker Scheduling

Advanced Filer Configurations

• Migrate to Filer Store
• Add New Filer Store
• Filer Store Replication
• Filer Active Active cross cluster continuous synchronization
• Filer as a Key-Large-Value Store
• Path Specific Configuration
• Filer Change Data Capture
• Filer Operation Serialization

FUSE Mount

• FIO benchmark
• fstab and systemd mount
• POSIX Compliance
• Distributed POSIX Locks
• P2P reading in weed mount

WebDAV

SFTP Server

Cloud Drive

• Cloud Drive Benefits
• Cloud Drive Architecture
• Configure Remote Storage
• Mount Remote Storage
• Cache Remote Storage
• Cloud Drive Quick Setup
• Gateway to Remote Object Storage

AWS S3 API

• Amazon S3 API
• Supported APIs vs Minio
• S3 Lifecycle
• S3 Lifecycle vs Volume TTL
• S3 Conditional Operations
• S3 CORS
• S3 Object Lock and Retention
• S3 Object Versioning
• S3 API Benchmark
• S3 API FAQ
• S3 Bucket Quota
• S3 Rate Limiting
• S3 API Audit log
• S3 Nginx Proxy
• Docker Compose for S3

S3 Table Bucket

• S3 Table Bucket
• S3 Table Bucket Commands
• S3 Tables Security
• SeaweedFS Iceberg Catalog
• Iceberg Table Maintenance

Iceberg Integrations

• Spark Iceberg Integration
• Trino Iceberg Integration
• Dremio Iceberg Integration
• DuckDB Iceberg Integration
• Doris Iceberg Integration
• RisingWave Iceberg Integration
• Lakekeeper Iceberg Integration

S3 Authentication & IAM

• S3 Configuration - Start Here
• S3 Credentials (-s3.config)
• OIDC Integration (-s3.iam.config)
• Kubernetes ServiceAccount Authentication (IRSA-style)
• S3 Policy Variables
• S3 Policy Conditions
• S3 Bucket Policies
• Amazon IAM API
• AWS IAM CLI
• weed shell - Shell IAM Commands

Server-Side Encryption

• Server-Side Encryption
• Server-Side Encryption SSE-KMS
• Server-Side Encryption SSE-C

S3 Client Tools

• AWS CLI with SeaweedFS
• s3cmd with SeaweedFS
• rclone with SeaweedFS
• restic with SeaweedFS
• nodejs with Seaweed S3

Machine Learning

• TensorFlow with SeaweedFS
• Distributing AI Model Files for Multi-GPU Loading

HDFS

• Hadoop Compatible File System
• run Spark on SeaweedFS
• run HBase on SeaweedFS
• run Presto on SeaweedFS
• Hadoop Benchmark
• HDFS via S3 connector

Replication and Backup

• Async Replication to another Filer [Deprecated]
• Async Backup
• Async Filer Metadata Backup
• Async Replication to Cloud [Deprecated]
• Kubernetes Backups and Recovery with K8up

Metadata Change Events

• Filer Metadata Events
• Filer Notification Webhook

Messaging

• Structured Data Lake with SMQ and SQL
• Seaweed Message Queue
• SQL Queries on Message Queue
• SQL Quick Reference
• PostgreSQL-compatible Server weed db
• Pub-Sub to SMQ to SQL
• Kafka to Kafka Gateway to SMQ to SQL

Use Cases

• Use Cases
• Actual Users

Operations

• System Metrics
• weed shell
• Data Backup
• Deployment to Kubernetes and Minikube
• Deployment with seaweed-up
  • Cluster Plan Workflow
  • Cluster Plan Inventory Reference
  • Cluster Plan Day 2 Operations

Rust Volume Server

• Rust Volume Server

Advanced

• Large File Handling
• Optimization
• Optimization for Many Small Buckets
• Volume Management
• Tiered Storage
• Cloud Tier
• Cloud Monitoring
• Load Command Line Options from a file
• SRV Service Discovery
• Volume Files Structure

Security

• Security Overview
• Security Configuration
• Cryptography and FIPS Compliance
• Run Blob Storage on Public Internet

Misc Use Case Examples

• UrBackup with SeaweedFS
• Docker Image Registry with SeaweedFS
• SeaweedFS in Docker Swarm
• Words from SeaweedFS Users
• Independent Benchmarks
• Hardware