Erasure-coded volumes can suffer silent disk corruption (bitrot) that normal serving never notices. EC bitrot detection adds an optional per-volume checksum sidecar so corruption in any shard — including cold parity shards — can be detected and the rebuild path refuses to consume unverified bytes.
The problem
- Data shards (
.ec00–.ec09) are read on the serving path, so corruption in regions that happen to be read is already caught by the per-needle CRC. Padding and unread regions are not. - Parity shards (
.ec10–.ec13) are never read during normal serving. They are only touched during reconstruction, and Reed-Solomon treats every present shard as authoritative — it fills missing shards but does not detect corruption in the shards it reads.
So a parity shard can rot silently for months. The day a data shard is lost and reconstruction runs, the corrupt parity is fed into Reed-Solomon and silently produces wrong data.
How it works
A small sidecar file <volume>.ecsum stores a CRC32C (Castagnoli, the same polynomial used for needles) for every fixed-size block (default 16 MiB) of every shard. It is wrapped in a self-integrity header (magic | format_version | payload_len | payload_crc32c), so a corrupt sidecar is itself detectable and can never be mistaken for shard corruption.
- Tiny: about 11 KB for a 30 GB volume.
- Optional and backward compatible: an absent sidecar means "feature off". Older binaries simply ignore the file — no mount failure, safe to roll back.
At encode time
The checksums are computed inline during the single EC encode pass and written next to the shards. When shards are distributed across servers, the sidecar travels with them.
Detecting corruption — ec.scrub
A new read-only checksum mode reads each local shard in blocks and compares it to the sidecar. This is the only path that exercises cold parity shards.
ec.scrub -mode checksum
ec.scrub -mode checksum -volumeId 7 -node 127.0.0.1:8080
A flagged shard is arbitrated by Reed-Solomon: the shard is reconstructed from the other (clean) shards and compared to what is on disk, so a stale sidecar block can never cause a healthy shard to be reported as corrupt. If more than parity shards mismatch at once, the sidecar itself is treated as suspect rather than reporting mass corruption. ec.scrub is read-only and never deletes anything.
Safe reconstruction
During rebuild, present input shards are verified against the sidecar and corrupt ones are excluded from Reed-Solomon and regenerated (written to a temp file and atomically renamed), then re-verified. This closes the "corrupt parity silently poisons a rebuild" hole.
If the sidecar is present but malformed, the rebuild fails closed rather than rebuilding without verification — pass -unsafeIgnoreSidecar to override. On any verification failure the rebuild removes every file it generated, so it never publishes bytes it could not verify.
Backfilling existing volumes
Volumes encoded before this feature have no sidecar. When such a volume is rebuilt on a server that can reach all of its shards, a sidecar is written automatically (trust-on-first-use), so existing volumes become protected over time.
Configuration
On the volume server:
| Flag | Default | Meaning |
|---|---|---|
-ec.bitrotChecksum |
true |
Write a .ecsum sidecar when generating EC shards |
-ec.bitrotBlockSizeMB |
16 |
Checksum block granularity in MiB (power of two, ≥ 1) |
weed volume -dir=/data -ec.bitrotChecksum -ec.bitrotBlockSizeMB=16
Side notes
- The 10+4 ratio is the default for the open source version.
- The checksum block size is a uniform overlay on the raw shard bytes; it does not need to align with the 1 GB / 1 MB EC block layout.
- The SeaweedFS Enterprise version adds a scheduled background scrubber that continuously verifies cold parity across the fleet with optional auto-repair, ties checksums into the versioned EC vacuum, and provides a coordinator-driven backfill command for protecting existing volumes in bulk.
See also: Erasure coding for warm storage.
Introduction
- Quick Start with weed mini
- Simplest S3 Bucket and User Setup
- Components
- Getting Started
- Production Setup
- A typical step‐by‐step example
- Benchmarks
- FAQ
- Applications
API
Configuration
- Replication
- Store file with a Time To Live
- Failover Master Server
- Erasure coding for warm storage
- EC Bitrot Detection
- Server Startup via Systemd
- Environment Variables
Filer
- Filer Setup
- Directories and Files
- File Operations Quick Reference
- Data Structure for Large Files
- Filer Data Encryption
- Filer Commands and Operations
- Filer JWT Use
- TUS Resumable Uploads
Filer Stores
- Filer Cassandra Setup
- Filer Redis Setup
- Super Large Directories
- Path-Specific Filer Store
- Choosing a Filer Store
- Customize Filer Store
Management
Advanced Filer Configurations
- Migrate to Filer Store
- Add New Filer Store
- Filer Store Replication
- Filer Active Active cross cluster continuous synchronization
- Filer as a Key-Large-Value Store
- Path Specific Configuration
- Filer Change Data Capture
- Filer Operation Serialization
FUSE Mount
- FIO benchmark
- fstab and systemd mount
- POSIX Compliance
- Distributed POSIX Locks
- P2P reading in weed mount
WebDAV
SFTP Server
Cloud Drive
- Cloud Drive Benefits
- Cloud Drive Architecture
- Configure Remote Storage
- Mount Remote Storage
- Cache Remote Storage
- Cloud Drive Quick Setup
- Gateway to Remote Object Storage
AWS S3 API
- Amazon S3 API
- Supported APIs vs Minio
- S3 Lifecycle
- S3 Lifecycle vs Volume TTL
- S3 Conditional Operations
- S3 CORS
- S3 Object Lock and Retention
- S3 Object Versioning
- S3 API Benchmark
- S3 API FAQ
- S3 Bucket Quota
- S3 Rate Limiting
- S3 API Audit log
- S3 Nginx Proxy
- Docker Compose for S3
S3 Table Bucket
- S3 Table Bucket
- S3 Table Bucket Commands
- S3 Tables Security
- SeaweedFS Iceberg Catalog
- Iceberg Table Maintenance
Iceberg Integrations
- Spark Iceberg Integration
- Trino Iceberg Integration
- Dremio Iceberg Integration
- DuckDB Iceberg Integration
- Doris Iceberg Integration
- RisingWave Iceberg Integration
- Lakekeeper Iceberg Integration
S3 Authentication & IAM
- S3 Configuration - Start Here
- S3 Credentials (
-s3.config) - OIDC Integration (
-s3.iam.config) - Kubernetes ServiceAccount Authentication (IRSA-style)
- S3 Policy Variables
- S3 Policy Conditions
- S3 Bucket Policies
- Amazon IAM API
- AWS IAM CLI
- weed shell - Shell IAM Commands
Server-Side Encryption
S3 Client Tools
- AWS CLI with SeaweedFS
- s3cmd with SeaweedFS
- rclone with SeaweedFS
- restic with SeaweedFS
- nodejs with Seaweed S3
Machine Learning
HDFS
- Hadoop Compatible File System
- run Spark on SeaweedFS
- run HBase on SeaweedFS
- run Presto on SeaweedFS
- Hadoop Benchmark
- HDFS via S3 connector
Replication and Backup
- Async Replication to another Filer [Deprecated]
- Async Backup
- Async Filer Metadata Backup
- Async Replication to Cloud [Deprecated]
- Kubernetes Backups and Recovery with K8up
Metadata Change Events
Messaging
- Structured Data Lake with SMQ and SQL
- Seaweed Message Queue
- SQL Queries on Message Queue
- SQL Quick Reference
- PostgreSQL-compatible Server weed db
- Pub-Sub to SMQ to SQL
- Kafka to Kafka Gateway to SMQ to SQL
Use Cases
Operations
- System Metrics
- weed shell
- Data Backup
- Deployment to Kubernetes and Minikube
- Deployment with seaweed-up
Rust Volume Server
Advanced
- Large File Handling
- Optimization
- Optimization for Many Small Buckets
- Volume Management
- Tiered Storage
- Cloud Tier
- Cloud Monitoring
- Load Command Line Options from a file
- SRV Service Discovery
- Volume Files Structure
Security
- Security Overview
- Security Configuration
- Cryptography and FIPS Compliance
- Run Blob Storage on Public Internet