mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-06-13 23:36:45 +03:00
Chris Lu
594fc667d5
* Filter metadata events before unmarshaling them per subscriber Every subscriber unmarshaled every log entry into a full event just to run the path filter, and entries carry complete chunk lists, so a fleet of path-filtered subscribers spends almost all replay CPU materializing events it then discards. A shallow wire scan now extracts just the directory, entry names and rename destination into a skeleton event, feeds the same matcher, and skips the decode for entries the subscriber cannot match. Any scan surprise (malformed bytes, merged duplicate message fields) falls back to the full decode, and the unsynced-events heartbeat keeps firing for skipped entries. * Raise the legacy replay cap The cap was sized when every replay pinned a private chunk reader per source filer. Replays now share decoded chunks, so sixteen needlessly serializes subscriber catch-up; the expensive part stays bounded by the cache's load gate. * Weight concurrent log-chunk loads by size The flat eight-load gate let eight tiny chunks through as reluctantly as eight full ones. Charge each load's chunk size against a 128MB in-flight budget instead: small chunks decode wide open while full-size ones still serialize enough to cap the transient peak. Oversized weights clamp to the budget so they can always acquire. * Propagate heartbeat send failures and reset the skip counter A failed heartbeat send means the stream is gone, so end the replay instead of scanning on. A delivered event also resets the skip counter, keeping the heartbeat cadence relative to the last thing the client actually received. * Share the unsynced-events counter across the prefilter and delivery Two independent counters could starve the heartbeat: alternating drops reset each side before either reached its threshold. One shared counter increments on every dropped entry, prefiltered or not, and only an actual delivery resets it, restoring the original cadence exactly. * Tighten comments * Benchmark the subscription match paths For a thousand-chunk event that the subscriber filters out, the shallow scan matches in 10us and 9 allocations against 175us and 4031 allocations for the full decode.