mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-06-13 23:36:45 +03:00
FUSE-Mount: document -writeBufferSizeMB write buffer cap
+25
@@ -328,6 +328,31 @@ For writes:
|
||||
|
||||

|
||||
|
||||
### Bounding the write buffer (`-writeBufferSizeMB`)
|
||||
|
||||
While an application is writing, `weed mount` buffers each outstanding chunk in RAM (for sequential writes) or in a swap file on local disk — by default under `-cacheDir` (i.e. `os.TempDir()`, which is usually `/tmp` on Linux). Once a chunk is full it is handed to an uploader goroutine that pushes it to a volume server; the buffered chunk is only released when the upload has completed and no reader is still referring to it.
|
||||
|
||||
Normally these uploads drain fast enough that the buffer stays small. But if volume servers stall — for example every candidate volume is at the size limit and the master hasn't yet rotated assignments — the upload queue grows, sealed chunks pile up, and the swap file can grow until `/tmp` fills. A large `rclone` sync into such a cluster has been observed to fill a 1.8 TiB `/tmp` partition before the mount process is killed.
|
||||
|
||||
`-writeBufferSizeMB=N` (default `0` = unlimited, preserving the old behavior) installs a single byte budget that is shared across every open file handle on a mount. When the budget is reached, additional writes block inside the FUSE write path until an earlier upload completes and frees its slot — so swap overflow becomes natural back-pressure on the client instead of unbounded disk growth.
|
||||
|
||||
```bash
|
||||
# Cap the total write buffer at 2 GiB, keep swap off /tmp.
|
||||
weed mount \
|
||||
-filer=localhost:8888 -dir=/mnt/seaweedfs -filer.path=/ \
|
||||
-cacheDirWrite=/var/cache/seaweedfs \
|
||||
-writeBufferSizeMB=2048 \
|
||||
-chunkSizeLimitMB=32 \
|
||||
-concurrentWriters=128
|
||||
```
|
||||
|
||||
Notes:
|
||||
* The budget is counted in whole chunks of `-chunkSizeLimitMB`, not in arbitrary bytes, so the effective maximum is `floor(writeBufferSizeMB / chunkSizeLimitMB) * chunkSizeLimitMB`.
|
||||
* Choose `writeBufferSizeMB` ≥ `chunkSizeLimitMB` — otherwise a single chunk would never fit. The accountant allows an oversized reservation only when the budget is empty, to keep a single writer from starving itself.
|
||||
* The flag is independent of `-cacheCapacityMB`, which sizes the *read* chunk cache on disk.
|
||||
* `-cacheDirWrite` (if unset, falls back to `-cacheDir`, which defaults to `os.TempDir()`) is where the swap file lives. When upload stalls are possible it is strongly recommended to point this at a dedicated disk — not `/tmp` — and to set `-writeBufferSizeMB` below the available space on that disk.
|
||||
* The cap is a mount-global limit, not per-file. With many concurrently open writers it will serialize them rather than corrupt any of them.
|
||||
|
||||
### Weed Mount Performance
|
||||
|
||||
Compared to any other distributed file systems, the `weed mount` performance should exceed most other solutions, or at least on par. This is because `weed mount` has multiple optimization techniques:
|
||||
|
||||
Reference in New Issue
Block a user