Log-structured Protocols in Delos

Q3 Computer Science

Operating Systems Review (ACM) Pub Date : 2021-10-26 DOI:10.1145/3477132.3483544

M. Balakrishnan, Chen Shen, AH Jafri, Suyog Mapara, D. Geraghty, J. Flinn, Vidhya Venkat, I. Nedelchev, Santosh K. Ghosh, Mihir Dharamshi, Jingming Liu, Filip Gruszczyński, Jun Li, Rounak Tibrewal, Ali Zaveri, Rajeev Nagar, Ahmed Yossef, Francois Richard, YeeJiun Song

{"title":"Log-structured Protocols in Delos","authors":"M. Balakrishnan, Chen Shen, AH Jafri, Suyog Mapara, D. Geraghty, J. Flinn, Vidhya Venkat, I. Nedelchev, Santosh K. Ghosh, Mihir Dharamshi, Jingming Liu, Filip Gruszczyński, Jun Li, Rounak Tibrewal, Ali Zaveri, Rajeev Nagar, Ahmed Yossef, Francois Richard, YeeJiun Song","doi":"10.1145/3477132.3483544","DOIUrl":null,"url":null,"abstract":"Developers have access to a wide range of storage APIs and functionality in large-scale systems, such as relational databases, key-value stores, and namespaces. However, this diversity comes at a cost: each API is implemented by a complex distributed system that is difficult to develop and operate. Delos amortizes this cost by enabling different APIs on a shared codebase and operational platform. The primary innovation in Delos is a log-structured protocol: a fine-grained replicated state machine executing above a shared log that can be layered into reusable protocol stacks under different databases. We built and deployed two production databases using Delos at Facebook, creating nine different log-structured protocols in the process. We show via experiments and production data that log-structured protocols impose low overhead, while allowing optimizations that can improve latency by up to 100X (e.g., via leasing) and throughput by up to 2X (e.g., via batching).","PeriodicalId":38935,"journal":{"name":"Operating Systems Review (ACM)","volume":"31 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2021-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"6","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Operating Systems Review (ACM)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3477132.3483544","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Computer Science","Score":null,"Total":0}

引用次数: 6

Abstract

Developers have access to a wide range of storage APIs and functionality in large-scale systems, such as relational databases, key-value stores, and namespaces. However, this diversity comes at a cost: each API is implemented by a complex distributed system that is difficult to develop and operate. Delos amortizes this cost by enabling different APIs on a shared codebase and operational platform. The primary innovation in Delos is a log-structured protocol: a fine-grained replicated state machine executing above a shared log that can be layered into reusable protocol stacks under different databases. We built and deployed two production databases using Delos at Facebook, creating nine different log-structured protocols in the process. We show via experiments and production data that log-structured protocols impose low overhead, while allowing optimizations that can improve latency by up to 100X (e.g., via leasing) and throughput by up to 2X (e.g., via batching).

查看原文本刊更多论文

Delos中的日志结构协议

开发人员可以访问大型系统中的各种存储api和功能，例如关系数据库、键值存储和名称空间。然而，这种多样性是有代价的:每个API都是由一个复杂的分布式系统实现的，很难开发和操作。Delos通过在共享的代码库和操作平台上启用不同的api来分摊这一成本。Delos的主要创新是日志结构协议:在共享日志之上执行的细粒度复制状态机，可以将其分层到不同数据库下的可重用协议堆栈中。我们在Facebook上使用Delos构建并部署了两个生产数据库，在此过程中创建了9个不同的日志结构协议。我们通过实验和生产数据表明，日志结构化协议的开销很低，同时允许优化，可以将延迟提高100倍(例如，通过租赁)，吞吐量提高2倍(例如，通过批处理)。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Operating Systems Review (ACM) Computer Science-Computer Networks and Communications

CiteScore

2.80

自引率

0.00%

发文量

期刊介绍： Operating Systems Review (OSR) is a publication of the ACM Special Interest Group on Operating Systems (SIGOPS), whose scope of interest includes: computer operating systems and architecture for multiprogramming, multiprocessing, and time sharing; resource management; evaluation and simulation; reliability, integrity, and security of data; communications among computing processors; and computer system modeling and analysis.