Commit Graph

554 Commits

Author SHA1 Message Date
chrislu a4b25a642d math/rand => math/rand/v2 2024-08-29 09:52:21 -07:00
chrislu 0cf2c15828 rename 2024-08-27 09:02:48 -07:00
augustazz 0b00706454 EC volume supports expiration and displays expiration message when executing volume.list (#5895)
* ec volume expire

* volume.list show DestroyTime

* comments

* code optimization

---------

Co-authored-by: xuwenfeng <xuwenfeng1@zto.com>
2024-08-16 00:20:00 -07:00
chrislu fdf7193ae7 rename 2024-08-13 13:59:24 -07:00
chrislu 07f4998188 add dat file size into vif for EC 2024-08-13 13:56:00 -07:00
chrislu 7438648d1c Merge branch 'master' into mq 2024-08-10 10:01:57 -07:00
chrislu 4fee496b49 conditional delete 2024-08-01 13:00:54 -07:00
chrislu 9dd008f8f1 add version to filer 2024-07-21 20:13:51 -07:00
chrislu 9dcc576499 Revert "add collection for buckets"
This reverts commit 96af571219.
2024-07-20 12:00:50 -07:00
chrislu 96af571219 add collection for buckets 2024-07-17 23:14:36 -07:00
chrislu c030cb3ce9 bootstrap filer from one peer 2024-06-28 14:57:20 -07:00
Konstantin Lebedev 7988ee0805 [fs.verify] skip failed files if entry not found on filerStore (#5693) 2024-06-19 06:50:55 -07:00
chrislu 49893267e9 Merge branch 'master' into mq 2024-06-15 11:53:19 -07:00
chrislu 464611f614 optionally skip deleting file chunks 2024-06-15 11:39:48 -07:00
chrislu f8a22038c5 passing along publisher name 2024-05-29 23:31:48 -07:00
chrislu edc87c16f9 balance subscribers
need to ensure the offsets are persisted
2024-05-27 17:30:16 -07:00
chrislu d40b350f54 coordinator receives unassignment ack 2024-05-23 08:23:35 -07:00
chrislu 2190d59bc3 refactor
ToDo
1. handle unassignment on client side
2. get notified of completed unassignment
3. in consumer_group.go, balance and send unassignment
2024-05-21 08:05:56 -07:00
chrislu 3b7f24b1fc unused 2024-05-20 12:26:54 -07:00
chrislu d8ab22012a track offset 2024-05-20 09:33:37 -07:00
chrislu c6db3f31a1 start consuming ASAP 2024-05-19 14:52:38 -07:00
chrislu b1871427c3 persist consumer group offset
1. use one follower
2. read write consumer group offset
2024-05-19 00:46:12 -07:00
chrislu 94742b8ace add subscribe follower 2024-05-17 18:17:23 -07:00
chrislu 372bd8d71d consumer instance passing MaxPartitionCount to coordinator 2024-05-13 18:30:10 -07:00
chrislu 7a9b115cc2 rename 2024-05-02 11:14:58 -07:00
chrislu f2f4630029 rename field types 2024-05-02 08:59:22 -07:00
chrislu 111a4e1126 add field index 2024-04-27 23:30:06 -07:00
chrislu cc1ce3de87 all record fields are optional by default 2024-04-18 23:17:52 -07:00
chrislu 0847a17484 convert to parquet schema and value 2024-04-17 23:49:21 -07:00
chrislu 04fb4c34e3 add proto for value 2024-04-12 22:27:16 -07:00
chrislu 48fe50df81 minor 2024-04-12 13:31:54 -07:00
chrislu 0cbf364a3d all fields are optional 2024-04-12 13:29:35 -07:00
chrislu d45a372615 add schema.proto 2024-04-12 01:30:29 -07:00
chrislu 66a878af39 add control message to data message 2024-03-31 16:35:46 -07:00
chrislu 546ae87c39 send is_close message to broker 2024-03-31 01:28:40 -07:00
chrislu c9df613b6b add publisher name for debugging 2024-03-31 00:19:16 -07:00
chrislu ab9fc0ad70 simplify 2024-03-27 23:42:19 -07:00
chrislu 205d2285f3 setup follower by publisher
* the subscriber would getOrGen a local partition and wait
* the publisher would getOrGen a local partition. If localPartition follower is not setup, and init message has follower info, it would create followers based on init.Messages.
2024-03-24 12:57:09 -07:00
chrislu 7003ce7425 publisher is able to fully send data and receive acks
still needs to close the pipes cleanly
2024-03-20 12:25:40 -07:00
chrislu 516cba7053 remove FollowInMemoryMessages 2024-03-17 22:27:13 -07:00
chrislu 6f75a0af55 ask follower to follow 2024-03-10 14:34:28 -07:00
chrislu 28f8f33d6a include key in LogEntry 2024-03-07 10:53:30 -08:00
chrislu 7e443ef0a1 latest protoc-gen-go 2024-02-29 10:06:23 -08:00
chrislu ead27a4f59 publish init add followers 2024-02-29 09:39:12 -08:00
chrislu 1b4484bf0a go fmt 2024-02-29 09:38:52 -08:00
Chris Lu e27deed4bc upgrade protoc 2024-02-05 18:39:08 -08:00
chrislu d41792461c lock returns host and owner 2024-02-02 15:54:57 -08:00
chrislu 34a78ffad0 remove isForPublish from LookupTopicBrokers
also adds a return parameter: whether the topic exists or not
2024-01-16 08:52:42 -08:00
chrislu 49428a303b add batch index for each memory buffer 2024-01-08 00:03:08 -08:00
chrislu e8611ed85d subscribe with partition offset 2024-01-05 15:24:14 -08:00