Consensus-based replicated systems are complex, monolithic, and difficult to upgrade once deployed. As a result, deployed systems do not benefit from innovative research, and new consensus protocols rarely reach production. We propose virtualizing consensus by virtualizing the shared log API, allowing services to change consensus protocols without downtime. Virtualization splits the logic of consensus into the VirtualLog, a generic and reusable reconfiguration layer; and pluggable ordering pr...| www.usenix.org
Comments (e.g. if you would like to share anything about your use case or have any questions)| Google Docs
Update (January 24, 2024) – The original version of this post used the s3 sync command to copy objects from my local directory to the storage bucket. The newest version of this command does not support copying to directory buckets, so I have revised the example to use s3 cp instead. — Jeff; The new […]| Amazon Web Services
Theorem in queueing theory| en.wikipedia.org
S3 Provides Application Programming Interface for Highly Scalable Reliable, Low-Latency Storage at Very Low Costs| US Press Center
The traditional way replicated systems are architected is to physically co-locate the write-ahead log (WAL) on the nodes where the state is being maintained. Then a consensus protocol like Paxos or Raft is used to make sure the log on each replica ag...| unofficial blog
Chances are you probably had a strong reaction to the title of this post. In our experience, Kafka is one of the most polarizing technologies in the data space. Some people hate it, some people swear by it, but almost every technology company uses it.| www.warpstream.com