Chris Lu 8cc10460b4 fix(remote): correct content and permissions when syncing/caching remote objects (#9879) ...

* fix(remote): reject short reads when caching remote objects

A short read from the remote (stale listing size, truncated or flaky
response) was silently zero-padded: the S3 and Azure clients pre-size
the buffer and discard the downloaded byte count, and the chunk is
recorded with the requested size. The cached file then matched the
expected size but its tail was NULL, and the entry was marked cached
so it never re-fetched.

Check the byte count against the requested size in both clients, and
add a backend-agnostic guard in FetchAndWriteNeedle. The cache now
fails loudly and the entry stays remote-only for a later retry.

* fix(remote): match S3 default modes when syncing remote metadata

Remote object listings carry no POSIX mode, so synced entries were
created with a hardcoded 0644. Against a SeaweedFS remote, whose S3
layer writes objects as 0660 and auto-creates directories as 0771
(0660|0111), the mounted copy ended up 0644/0755 and the permissions
visibly diverged from the source.

Default to the S3 modes instead (files 0660, directories 0771). The
filer derives parent-dir modes from the child as fileMode|0111, so
fixing the file default also brings the directories into line.

Directory mtimes still reflect sync time: S3 listings don't enumerate
directories, so the remote's directory timestamps aren't available.

2026-06-08 13:55:53 -07:00

..

command_cluster_check.go

grpc: don't tear down the shared master connection on a caller's own timeout (#9775)

2026-06-01 15:11:02 -07:00

command_cluster_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_add.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_remove.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_status_test.go

fix(ec): correct multi-disk disk counting and EC balance shard attribution (#9594)

2026-05-20 23:31:21 -07:00

command_cluster_status.go

fix(ec): correct multi-disk disk counting and EC balance shard attribution (#9594)

2026-05-20 23:31:21 -07:00

command_collection_delete.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_collection_list_test.go

fix(s3/shell): factor EC volumes into bucket size metrics and collection.list (#9182)

2026-04-21 20:17:42 -07:00

command_collection_list.go

fix(s3/shell): factor EC volumes into bucket size metrics and collection.list (#9182)

2026-04-21 20:17:42 -07:00

command_ec_balance.go

add admin script worker (#8491)

2026-03-03 15:10:40 -08:00

command_ec_common.go

Treat co-located volume servers as one fault domain when balancing and allocating (#9854)

2026-06-07 14:14:45 -07:00

command_ec_decode.go

fix(ec): pass per-volume data-shard count to the parity-shard split (#9781)

2026-06-01 19:25:15 -07:00

command_ec_encode_test.go

add tests

2026-02-09 01:37:56 -08:00

command_ec_encode.go

fix(ec): pass per-volume data-shard count to the parity-shard split (#9781)

2026-06-01 19:25:15 -07:00

command_ec_rebuild_test.go

fix(ec): gather shards from all disk locations before rebuild (#8633)

2026-03-14 20:59:47 -07:00

command_ec_rebuild.go

Fix ec.rebuild failing on unrepairable volumes instead of skipping (#8632)

2026-03-14 16:18:29 -07:00

command_ec_scrub.go

EC bitrot detection: per-shard checksum sidecars (#9761)

2026-05-31 18:52:44 -07:00

command_ec_test.go

EC placement: shared replica-placement resolver, snapshot + Place core, capacity fixes, tiering (#9621)

2026-05-22 20:22:09 -07:00

command_fs_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_cd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_configure.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_fs_distribute_chunks_test.go

feat(shell): add fs.distributeChunks command for even chunk distribution (#9117)

2026-04-17 21:09:36 -07:00

command_fs_distribute_chunks.go

feat(shell): add fs.distributeChunks command for even chunk distribution (#9117)

2026-04-17 21:09:36 -07:00

command_fs_du.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_log.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_fs_ls.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_merge_volumes.go

fix(mount): sanitize non-UTF-8 filenames; keep marshal errors per-request (#9207)

2026-04-23 19:17:35 -07:00

command_fs_meta_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_meta_change_volume_id.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_load.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_notify.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_meta_save.go

shell: volume.fsck must not skip the system-log subtree (#9764)

2026-06-01 09:54:22 -07:00

command_fs_mkdir.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_mv.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_pwd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_rm.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_tree.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_verify.go

shell: volume.fsck must not skip the system-log subtree (#9764)

2026-06-01 09:54:22 -07:00

command_lock_unlock.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mount_configure.go

…

command_mq_balance.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_compact.go

…

command_mq_topic_configure.go

shell: expose retention flags on mq.topic.configure (#9416)

2026-05-12 12:37:09 -07:00

command_mq_topic_desc.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_list.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_truncate.go

…

command_remote_cache.go

fix(remote): correct content and permissions when syncing/caching remote objects (#9879)

2026-06-08 13:55:53 -07:00

command_remote_configure.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_remote_copy_local.go

fix(shell): set LastLocalSyncTsNs in remote.copy.local so remote.uncache works (#8604)

2026-03-11 12:55:45 -07:00

command_remote_meta_sync.go

fix(remote): correct content and permissions when syncing/caching remote objects (#9879)

2026-06-08 13:55:53 -07:00

command_remote_mount_buckets.go

feat(remote.mount): add -metadataStrategy flag to control metadata caching (#8568)

2026-03-12 15:21:07 -07:00

command_remote_mount.go

feat(filer): add lazy directory listing for remote mounts (#8615)

2026-03-13 09:36:54 -07:00

command_remote_uncache_test.go

shell: add minCacheAge flag to remote.uncache command (#8225)

2026-02-05 12:57:27 -08:00

command_remote_uncache.go

shell: add minCacheAge flag to remote.uncache command (#8225)

2026-02-05 12:57:27 -08:00

command_remote_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_s3_accesskey_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_rotate.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_get.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_set.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_access_test.go

shell: add s3.bucket.access command for anonymous access policy (#8774)

2026-03-25 23:09:53 -07:00

command_s3_bucket_access.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_create.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_delete.go

fix: admin UI bucket delete now properly deletes collection and checks Object Lock (#7734)

2025-12-13 13:41:25 -08:00

command_s3_bucket_lifecycle_fastpath.go

s3: make lifecycle TTL fast path per-bucket opt-in (#9825)

2026-06-06 11:20:15 -07:00

command_s3_bucket_list.go

shell: add -owner flag to s3.bucket.create command (#7728)

2025-12-12 18:06:13 -08:00

command_s3_bucket_lock.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_owner.go

shell: add -owner flag to s3.bucket.create command (#7728)

2025-12-12 18:06:13 -08:00

command_s3_bucket_quota_check.go

s3: auto-enforce bucket quota read-only both ways (#9774)

2026-06-01 13:11:18 -07:00

command_s3_bucket_quota.go

…

command_s3_bucket_versioning.go

Chart createBuckets config #8368: Add TTL, Object Lock, and Versioning support (#8375)

2026-02-26 11:56:10 -08:00

command_s3_circuitbreaker_test.go

…

command_s3_circuitbreaker.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_s3_clean_uploads.go

…

command_s3_config_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_configure.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_add_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_remove_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_client.go

grpc: don't tear down the shared master connection on a caller's own timeout (#9775)

2026-06-01 15:11:02 -07:00

command_s3_iam_export.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_import.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_lifecycle_run_shard_test.go

fix(shell): restore s3.lifecycle.run-shard for CI/integration-test compatibility (#9475)

2026-05-12 18:29:18 -07:00

command_s3_lifecycle_run_shard.go

fix(s3/lifecycle): walker dispatch uses entry.Path for ABORT_MPU (#9477)

2026-05-13 00:19:05 -07:00

command_s3_policy_attach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy_detach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_disable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_enable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_provision.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_versions_audit.go

feat(s3/versioning): grep-able heal logs + scan-anomaly diagnostics + audit cmd (#9468)

2026-05-13 10:48:58 -07:00

command_s3tables_bucket.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_namespace.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_s3tables_table.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_tag.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_sleep.go

add admin script worker (#8491)

2026-03-03 15:10:40 -08:00

command_volume_balance_test.go

Treat co-located volume servers as one fault domain when balancing and allocating (#9854)

2026-06-07 14:14:45 -07:00

command_volume_balance.go

Treat co-located volume servers as one fault domain when balancing and allocating (#9854)

2026-06-07 14:14:45 -07:00

command_volume_check_disk_test.go

Mutex command output writes for volume.check.disk. (#7605)

2025-12-02 10:14:24 -08:00

command_volume_check_disk.go

fix(ec): bring ec.encode worker and EC/volume helpers to parity with shell (#9599)

2026-05-21 02:16:28 -07:00

command_volume_configure_replication.go

Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)

2026-01-16 12:31:48 -08:00

command_volume_copy.go

Fix live volume move tail timestamp (#8440)

2026-02-24 20:07:26 -08:00

command_volume_delete_empty.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_delete.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_fix_replication_test.go

…

command_volume_fix_replication.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_fsck.go

shell: volume.fsck must not skip the system-log subtree (#9764)

2026-06-01 09:54:22 -07:00

command_volume_grow.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_list_test.go

shell: warn in volume.list when a volume id spans collections (#9759)

2026-05-31 11:52:39 -07:00

command_volume_list.go

shell: warn in volume.list when a volume id spans collections (#9759)

2026-05-31 11:52:39 -07:00

command_volume_mark.go

…

command_volume_merge_test.go

fix(shell): verify volume.merge output before overwriting replicas (#9731)

2026-05-28 19:29:25 -07:00

command_volume_merge.go

fix(shell): verify volume.merge output before overwriting replicas (#9731)

2026-05-28 19:29:25 -07:00

command_volume_mount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_move.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_scrub.go

Give the ScrubVolume() RPC an option to flag found broken volumes as read-only. (#8360)

2026-03-26 10:20:57 -07:00

command_volume_server_evacuate_test.go

…

command_volume_server_evacuate.go

fix(shell): volumeServer.evacuate no longer panics on a nil volume (#9587)

2026-05-20 10:27:00 -07:00

command_volume_server_leave.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_volume_server_state.go

Add weed shell command volumeServer.state to query/update volume server state settings. (#8271)

2026-02-11 11:02:37 -08:00

command_volume_tier_compact_test.go

feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715)

2026-03-20 23:52:12 -07:00

command_volume_tier_compact.go

feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715)

2026-03-20 23:52:12 -07:00

command_volume_tier_download.go

…

command_volume_tier_move.go

fix(ec): bring ec.encode worker and EC/volume helpers to parity with shell (#9599)

2026-05-21 02:16:28 -07:00

command_volume_tier_upload.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_vacuum_disable.go

…

command_volume_vacuum_enable.go

…

command_volume_vacuum.go

fix(shell): error on missing volume id in fsck, mergeVolumes, vacuum (#9158)

2026-04-20 15:32:31 -07:00

command.go

shell: add s3.iam.*, s3.config.show, s3.user.provision; hide legacy commands (#8956)

2026-04-07 14:10:15 -07:00

commands.go

fix(ec): bring ec.encode worker and EC/volume helpers to parity with shell (#9599)

2026-05-21 02:16:28 -07:00

common_test.go

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

common.go

Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)

2026-01-16 12:31:48 -08:00

ec_proportional_rebalance.go

chore: remove ~50k lines of unreachable dead code (#8913)

2026-04-03 16:04:27 -07:00

ec_rebalance_slots_test.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

s3tables_helpers.go

grpc: don't tear down the shared master connection on a caller's own timeout (#9775)

2026-06-01 15:11:02 -07:00

shell_liner.go

fix(weed/shell): suppress prompt when piped (#8990)

2026-04-08 13:07:15 -07:00

volume.ecshards.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list2.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00