LazyLog: A New Shared Log Abstraction and Design for Modern Low-Latency Applications
Shared logs offer linearizable total order across storage shards. However, they enforce this order eagerly upon ingestion, leading to high latencies. We observe that in many modern shared-log applications, while linearizable ordering is necessary, it is not required eagerly when ingesting data but o...
Saved in:
| Published in | ACM transactions on computer systems |
|---|---|
| Main Authors | , , , , |
| Format | Journal Article |
| Language | English |
| Published |
19.08.2025
|
| Online Access | Get full text |
| ISSN | 0734-2071 1557-7333 1557-7333 |
| DOI | 10.1145/3750445 |
Cover
| Abstract | Shared logs offer linearizable total order across storage shards. However, they enforce this order eagerly upon ingestion, leading to high latencies. We observe that in many modern shared-log applications, while linearizable ordering is necessary, it is not required eagerly when ingesting data but only later when data is consumed. Further, readers are naturally decoupled in time from writers in these applications. Based on this insight, we propose LazyLog, a novel shared log abstraction. LazyLog lazily binds records (across shards) to linearizable global positions and enforces this before a log position can be read. Such lazy ordering enables low ingestion latencies. Given the time decoupling, LazyLog can establish the order well before reads arrive, minimizing overhead upon reads. We build two LazyLog systems that provide linearizable total order across shards. Our experiments show that LazyLog systems deliver significantly lower latencies than conventional, eager-ordering shared logs. |
|---|---|
| AbstractList | Shared logs offer linearizable total order across storage shards. However, they enforce this order eagerly upon ingestion, leading to high latencies. We observe that in many modern shared-log applications, while linearizable ordering is necessary, it is not required eagerly when ingesting data but only later when data is consumed. Further, readers are naturally decoupled in time from writers in these applications. Based on this insight, we propose LazyLog, a novel shared log abstraction. LazyLog lazily binds records (across shards) to linearizable global positions and enforces this before a log position can be read. Such lazy ordering enables low ingestion latencies. Given the time decoupling, LazyLog can establish the order well before reads arrive, minimizing overhead upon reads. We build two LazyLog systems that provide linearizable total order across shards. Our experiments show that LazyLog systems deliver significantly lower latencies than conventional, eager-ordering shared logs. |
| Author | Bhat, Shreesha Gopalakrishna Hu, Jiyu Alagappan, Ramnatthan Ganesan, Aishwarya Luo, Xuhao |
| Author_xml | – sequence: 1 givenname: Xuhao orcidid: 0009-0002-9054-8389 surname: Luo fullname: Luo, Xuhao – sequence: 2 givenname: Shreesha Gopalakrishna orcidid: 0009-0006-1754-5287 surname: Bhat fullname: Bhat, Shreesha Gopalakrishna – sequence: 3 givenname: Jiyu orcidid: 0000-0003-1899-9062 surname: Hu fullname: Hu, Jiyu – sequence: 4 givenname: Ramnatthan orcidid: 0000-0001-9911-4208 surname: Alagappan fullname: Alagappan, Ramnatthan – sequence: 5 givenname: Aishwarya orcidid: 0000-0002-5078-8436 surname: Ganesan fullname: Ganesan, Aishwarya |
| BookMark | eNp10E1PhDAYBODGrInsavwLvekFbWlLqTeyfiaoB7mTl7asGCykxRD89bLZvXqaZPJkDrNGK9c7i9AlJTeUcnHLpCCcixMUUSFkLBljKxQRyXicEEnP0DqEL0LI0icRKgv4nYt-d4dz_GYn_PEJ3hq8NDivw-hBj23vMDiD721odw43vcevvbHeLWqKCxit0zPOh6FrNex1OEenDXTBXhxzg8rHh3L7HBfvTy_bvIgh4yLWpgFR89SmtTDKZFpJZimkXIkUKGgtCVVpAtpk0gqdsCStlRJZRlUDteJsg64Psz9ugHmCrqsG336DnytKqv0Z1fGMhV4dqPZ9CN42_8o_q8JeiQ |
| Cites_doi | 10.1145/2517349.2517350 10.1145/3477132.3483543 10.1145/78969.78972 10.1145/3226595.3226637 |
| ContentType | Journal Article |
| DBID | AAYXX CITATION ADTOC UNPAY |
| DOI | 10.1145/3750445 |
| DatabaseName | CrossRef Unpaywall for CDI: Periodical Content Unpaywall |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | CrossRef |
| Database_xml | – sequence: 1 dbid: UNPAY name: Unpaywall url: https://proxy.k.utb.cz/login?url=https://unpaywall.org/ sourceTypes: Open Access Repository |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1557-7333 |
| ExternalDocumentID | 10.1145/3750445 10_1145_3750445 |
| GroupedDBID | -DZ -~X .4S .DC 23M 2FS 4.4 5GY 5VS 6J9 85S 8US 8VB AAIKC AAKMM AALFJ AAMNW AAYFX AAYXX ABPPZ ACGFO ACGOD ACM ADBCU ADL ADMLS AEBYY AEFXT AEGXH AEJOY AENEX AENSD AETEA AFWIH AFWXC AHQJS AIAGR AIKLT AKRVB AKVCP ALMA_UNASSIGNED_HOLDINGS ARCSS ASPBG AVWKF BDXCO CCLIF CITATION CS3 D0L EBS EBU EDO FEDTE GUFHI HGAVV H~9 I07 IAO ICD IEA IGS IOF K1G LHSKQ MK~ ML~ P1C P2P PQQKQ QWB RNS ROL RXW TAE TH9 TUS U5U UPT WH7 X6Y XH6 XSW ZCA ZL0 9M8 ABFSI ADMHC ADTOC AFFNX AI. E.L EJD HF~ ITC MVM OHT UNPAY VH1 XJT XOL ZCG ZY4 |
| ID | FETCH-LOGICAL-a845-cdfa5b46e6b5d9d8c973e1a64956a1acc701962acd87e5c2326b9958819fab943 |
| IEDL.DBID | UNPAY |
| ISSN | 0734-2071 1557-7333 |
| IngestDate | Sun Sep 07 11:22:31 EDT 2025 Wed Oct 01 05:33:04 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | cc-by |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a845-cdfa5b46e6b5d9d8c973e1a64956a1acc701962acd87e5c2326b9958819fab943 |
| ORCID | 0009-0006-1754-5287 0000-0001-9911-4208 0000-0003-1899-9062 0009-0002-9054-8389 0000-0002-5078-8436 |
| OpenAccessLink | https://proxy.k.utb.cz/login?url=https://doi.org/10.1145/3750445 |
| ParticipantIDs | unpaywall_primary_10_1145_3750445 crossref_primary_10_1145_3750445 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2025-08-19 |
| PublicationDateYYYYMMDD | 2025-08-19 |
| PublicationDate_xml | – month: 08 year: 2025 text: 2025-08-19 day: 19 |
| PublicationDecade | 2020 |
| PublicationTitle | ACM transactions on computer systems |
| PublicationYear | 2025 |
| References | Dobbelaere Philippe (e_1_2_1_42_1) 2017 Falk Eric (e_1_2_1_47_1) 2017 Van Renesse Robbert (e_1_2_1_91_1) 2004 e_1_2_1_60_1 e_1_2_1_81_1 e_1_2_1_20_1 e_1_2_1_87_1 e_1_2_1_89_1 Nightingale B (e_1_2_1_77_1) 2005 Ren Kai (e_1_2_1_84_1) e_1_2_1_24_1 e_1_2_1_45_1 e_1_2_1_62_1 e_1_2_1_83_1 Wang Stephanie (e_1_2_1_92_1) 2019 e_1_2_1_22_1 e_1_2_1_28_1 e_1_2_1_49_1 e_1_2_1_26_1 Akidau Tyler (e_1_2_1_27_1) 2013 Balakrishnan Mahesh (e_1_2_1_37_1) 2013 Eugster Patrick Th (e_1_2_1_44_1) 2003 Li Jialin (e_1_2_1_68_1) 2017 Wang Yang (e_1_2_1_93_1) 2012 e_1_2_1_71_1 Nightingale B (e_1_2_1_78_1) 2006 e_1_2_1_90_1 e_1_2_1_31_1 e_1_2_1_54_1 e_1_2_1_8_1 Lu Lanyue (e_1_2_1_67_1) Ongaro Diego (e_1_2_1_79_1) 2014 e_1_2_1_6_1 e_1_2_1_12_1 e_1_2_1_50_1 e_1_2_1_4_1 e_1_2_1_10_1 e_1_2_1_33_1 e_1_2_1_75_1 e_1_2_1_2_1 e_1_2_1_16_1 e_1_2_1_39_1 Lamport Leslie (e_1_2_1_66_1) 2001; 32 Kotla Ramakrishna (e_1_2_1_64_1) 2007 e_1_2_1_14_1 Goodhope Ken (e_1_2_1_52_1) 2012; 35 e_1_2_1_58_1 Mehdi Syed Akbar (e_1_2_1_73_1) 2017 e_1_2_1_18_1 Balakrishnan Mahesh (e_1_2_1_35_1) 2020 Giantsidi Dimitra (e_1_2_1_51_1) 2023 Faleiro M (e_1_2_1_46_1) 2014 Ding Cong (e_1_2_1_41_1) 2020 Escriva Robert (e_1_2_1_43_1) 2014 Jia Zhipeng (e_1_2_1_57_1) 2021 e_1_2_1_65_1 e_1_2_1_88_1 Herlihy P. (e_1_2_1_55_1) 1990; 12 Zhang Irene (e_1_2_1_95_1) 2015 e_1_2_1_23_1 e_1_2_1_61_1 e_1_2_1_21_1 e_1_2_1_63_1 Ricci Robert (e_1_2_1_85_1) 2014 e_1_2_1_86_1 e_1_2_1_25_1 e_1_2_1_48_1 e_1_2_1_29_1 Ports RK (e_1_2_1_82_1) 2015 Balakrishnan Mahesh (e_1_2_1_36_1) 2012 Hodsdon Christopher (e_1_2_1_56_1) 2023; 3 Cooper F. (e_1_2_1_40_1) 2010 e_1_2_1_70_1 Wester Benjamin (e_1_2_1_94_1) 2009 e_1_2_1_7_1 e_1_2_1_30_1 e_1_2_1_76_1 e_1_2_1_5_1 e_1_2_1_3_1 e_1_2_1_13_1 e_1_2_1_34_1 e_1_2_1_72_1 e_1_2_1_1_1 e_1_2_1_11_1 e_1_2_1_32_1 e_1_2_1_53_1 e_1_2_1_74_1 Li Jialin (e_1_2_1_69_1) 2016 e_1_2_1_17_1 e_1_2_1_38_1 Zhang Irene (e_1_2_1_96_1) 2016; 39 e_1_2_1_15_1 e_1_2_1_59_1 Ousterhout John (e_1_2_1_80_1) 2015; 33 e_1_2_1_9_1 e_1_2_1_19_1 |
| References_xml | – volume-title: et al year: 2020 ident: e_1_2_1_35_1 – ident: e_1_2_1_74_1 – volume: 32 start-page: 18 year: 2001 ident: e_1_2_1_66_1 article-title: Paxos Made Simple publication-title: ACM Sigact News – ident: e_1_2_1_21_1 – volume: 33 start-page: 7 year: 2015 ident: e_1_2_1_80_1 article-title: The RAMCloud storage system publication-title: ACM Transactions on Computer Systems (TOCS) – volume-title: Zyzzyva: Speculative Byzantine Fault Tolerance. In ACM SIGOPS Operating Systems Review, Vol. 41. ACM, 45–58. year: 2007 ident: e_1_2_1_64_1 – ident: e_1_2_1_76_1 doi: 10.1145/2517349.2517350 – ident: e_1_2_1_88_1 – ident: e_1_2_1_19_1 – volume-title: Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI ’04) year: 2004 ident: e_1_2_1_91_1 – ident: e_1_2_1_65_1 – volume-title: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI ’16) year: 2016 ident: e_1_2_1_69_1 – ident: e_1_2_1_26_1 – ident: e_1_2_1_20_1 – ident: e_1_2_1_34_1 – ident: e_1_2_1_18_1 – ident: e_1_2_1_12_1 – volume-title: The many faces of publish/subscribe. ACM computing surveys (CSUR) 35, 2 year: 2003 ident: e_1_2_1_44_1 – ident: e_1_2_1_4_1 – ident: e_1_2_1_81_1 – start-page: 11 volume-title: Proceedings of the VLDB Endowment 6 year: 2013 ident: e_1_2_1_27_1 – volume-title: Gnothi: Separating Data and Metadata for Efficient and Available Storage Replication. In 2012 USENIX Annual Technical Conference (USENIX ATC 12) year: 2012 ident: e_1_2_1_93_1 – ident: e_1_2_1_59_1 – volume-title: Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing. 195–209 year: 2023 ident: e_1_2_1_51_1 – ident: e_1_2_1_53_1 – ident: e_1_2_1_50_1 – volume-title: SC’14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis ident: e_1_2_1_84_1 – ident: e_1_2_1_45_1 – ident: e_1_2_1_75_1 – ident: e_1_2_1_61_1 – ident: e_1_2_1_72_1 – volume-title: 14th USENIX Conference on File and Storage Technologies (FAST 16) ident: e_1_2_1_67_1 – volume-title: Proceedings of the 26th ACM Symposium on Operating Systems Principles (SOSP ’17) year: 2017 ident: e_1_2_1_68_1 – ident: e_1_2_1_8_1 – ident: e_1_2_1_2_1 – volume-title: Boki: Stateful Serverless Computing with Shared Logs, See SOS [25]. year: 2021 ident: e_1_2_1_57_1 – ident: e_1_2_1_83_1 – ident: e_1_2_1_86_1 – ident: e_1_2_1_5_1 – volume-title: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data (SIGMOD ’14) year: 2014 ident: e_1_2_1_46_1 – ident: e_1_2_1_49_1 doi: 10.1145/3477132.3483543 – volume-title: Proceedings of the 11th ACM international conference on distributed and event-based systems. 227–238 year: 2017 ident: e_1_2_1_42_1 – volume-title: Proceedings of the 12th Symposium on Networked Systems Design and Implementation (NSDI ’15) year: 2015 ident: e_1_2_1_82_1 – volume-title: Proceedings of the 25th ACM Symposium on Operating Systems Principles (SOSP ’15) year: 2015 ident: e_1_2_1_95_1 – ident: e_1_2_1_13_1 – volume-title: Proceedings of the ACM Symposium on Cloud Computing (SOCC ’10) year: 2010 ident: e_1_2_1_40_1 – ident: e_1_2_1_10_1 – volume-title: Proceedings of the 27th ACM Symposium on Operating Systems Principles (SOSP ’19) year: 2019 ident: e_1_2_1_92_1 – ident: e_1_2_1_89_1 – volume: 35 start-page: 33 year: 2012 ident: e_1_2_1_52_1 article-title: Building LinkedIn’s Real-time Activity Data Pipeline.IEEE Data publication-title: Eng. Bull. – ident: e_1_2_1_32_1 – ident: e_1_2_1_71_1 – ident: e_1_2_1_7_1 – ident: e_1_2_1_38_1 – volume-title: Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI ’06) year: 2006 ident: e_1_2_1_78_1 – volume: 39 start-page: 27 year: 2016 ident: e_1_2_1_96_1 article-title: When is Operation Ordering Required in Replicated Transactional Storage?IEEE Data publication-title: Eng. Bull. – volume-title: Introducing CloudLab: Scientific infrastructure for advancing cloud architectures and applications. USENIX ;login: 39, 6 year: 2014 ident: e_1_2_1_85_1 – ident: e_1_2_1_24_1 – ident: e_1_2_1_16_1 – volume-title: Proceedings of the 17th Symposium on Networked Systems Design and Implementation (NSDI ’20) year: 2020 ident: e_1_2_1_41_1 – ident: e_1_2_1_23_1 – ident: e_1_2_1_29_1 – ident: e_1_2_1_54_1 – ident: e_1_2_1_62_1 – ident: e_1_2_1_70_1 – ident: e_1_2_1_31_1 – ident: e_1_2_1_1_1 – volume-title: Proceedings of the 14th Symposium on Networked Systems Design and Implementation (NSDI ’17) year: 2017 ident: e_1_2_1_73_1 – volume-title: Proceedings of the EuroSys Conference (EuroSys ’14) year: 2014 ident: e_1_2_1_43_1 – ident: e_1_2_1_15_1 – volume: 3 year: 2023 ident: e_1_2_1_56_1 article-title: MASON: Scalable, Contiguous Sequencing for Building Consistent Services publication-title: Journal of Systems Research (JSys) – ident: e_1_2_1_14_1 – volume: 12 start-page: 3 year: 1990 ident: e_1_2_1_55_1 article-title: Linearizability: A Correctness Condition for Concurrent Objects publication-title: ACM Trans. Program. Lang. Syst. doi: 10.1145/78969.78972 – volume-title: Tango: Distributed Data Structures over a Shared Log, See SOS [22]. year: 2013 ident: e_1_2_1_37_1 – ident: e_1_2_1_25_1 – ident: e_1_2_1_17_1 – volume-title: Proceedings of the VLDB Endowment 10 year: 2017 ident: e_1_2_1_47_1 – volume-title: FireScroll - Source Code. https://github.com/FireScroll/FireScroll/blob/main/api/handler.go#L96 ident: e_1_2_1_48_1 – ident: e_1_2_1_6_1 – ident: e_1_2_1_87_1 doi: 10.1145/3226595.3226637 – ident: e_1_2_1_3_1 – ident: e_1_2_1_22_1 – ident: e_1_2_1_11_1 – volume-title: Proceedings of the 9th Symposium on Networked Systems Design and Implementation (NSDI ’12) year: 2012 ident: e_1_2_1_36_1 – ident: e_1_2_1_30_1 – ident: e_1_2_1_33_1 – ident: e_1_2_1_90_1 – volume-title: Speculative Execution in a Distributed File System. (October year: 2005 ident: e_1_2_1_77_1 – volume-title: Proceedings of the 6th Symposium on Networked Systems Design and Implementation (NSDI ’09) year: 2009 ident: e_1_2_1_94_1 – ident: e_1_2_1_58_1 – ident: e_1_2_1_39_1 – volume-title: 2014 USENIX Annual Technical Conference (USENIX ATC 14) year: 2014 ident: e_1_2_1_79_1 – ident: e_1_2_1_9_1 – ident: e_1_2_1_63_1 – ident: e_1_2_1_28_1 – ident: e_1_2_1_60_1 |
| SSID | ssj0003332 |
| Score | 2.4151955 |
| Snippet | Shared logs offer linearizable total order across storage shards. However, they enforce this order eagerly upon ingestion, leading to high latencies. We... |
| SourceID | unpaywall crossref |
| SourceType | Open Access Repository Index Database |
| Title | LazyLog: A New Shared Log Abstraction and Design for Modern Low-Latency Applications |
| URI | https://doi.org/10.1145/3750445 |
| UnpaywallVersion | publishedVersion |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVEBS databaseName: Inspec with Full Text customDbUrl: eissn: 1557-7333 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003332 issn: 0734-2071 databaseCode: ADMLS dateStart: 20080601 isFulltext: true titleUrlDefault: https://www.ebsco.com/products/research-databases/inspec-full-text providerName: EBSCOhost |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NTwIxEG0QDp7Ez4hRrInXoks_duttIxJiFmIiJHgi023xIFmIgRD49bZ00dVE43Uzm2xm2p03nb43CF1HQhqXyQg0hSbMxoEAiIAERoU0CMY2gzqicLcnOgP2OOTDErrccmGK_fuA8Rvq5McZ30EVwS3YLqPKoPcUv2zENSmzMfYlFechCSmlnhZbfPNbvtldZDNYLWEyKSSRdvVLRMDfHXlrLOaqka5_KDP-8X37aC8HkDj2ET9AJZMdoup2OAPO9-oR6iewXiXT1zscY_snw06Z2Whsn-BYufONDaEBQ6Zxa3OLA1v4iv1oNGu1JAk4OL3CcaHFfYz67Yf-fYfkIxQIRIyTVI-BKyaMUFxLHaUypCYA4aoiCCBNnRi7aEKqo9Dw1KIroaTkkYUJY1CS0RNUzqaZOUX4Vqeu2pBU-l6rFAaaY2Y3vAVkkYhqCG_9PJp5oYyRJz3zUe6kGrr69P9vNmf_sDlH5fn7wlxYNDBXdVSJW93kuZ6vjA81j699 |
| linkProvider | Unpaywall |
| linkToUnpaywall | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LSwMxEA61PXiyPrHiI4LXVLd57MbbopYitXhooZ7K5LEeLNsiLaX99SbNVldB8brMwjKT7HyTyfcNQleJkNZnMgItYQhzcSAAIiKRVTGNosxlUE8UfuqJzoA9Dvmwgi42XJhy_z5i_Jp6-XHGt1BNcAe2q6g26D2nL2txTcpcjENJxXlMYkppoMWW3_yWb7bn-RSWCxiPS0mkXf8SEQh3R96a85lq6tUPZcY_vm8X7RQAEqch4nuoYvN9VN8MZ8DFXj1A_S6slt3J6y1OsfuTYa_MbA12T3Cq_PnGmtCAITf4fn2LAzv4isNoNGe1IF3wcHqJ01KL-xD12w_9uw4pRigQSBgn2mTAFRNWKG6kSbSMqY1A-KoIItDai7GLFmiTxJZrh66EkpInDiZkoCSjR6iaT3J7jPCN0b7akFSGXqsUFloZcxveAbJEJA2EN34eTYNQxiiQnvmocFIDXX76_zebk3_YnKLq7H1uzxwamKnzYkV8AMGYrek |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=LazyLog%3A+A+New+Shared+Log+Abstraction+and+Design+for+Modern+Low-Latency+Applications&rft.jtitle=ACM+transactions+on+computer+systems&rft.au=Luo%2C+Xuhao&rft.au=Bhat%2C+Shreesha+Gopalakrishna&rft.au=Hu%2C+Jiyu&rft.au=Alagappan%2C+Ramnatthan&rft.date=2025-08-19&rft.issn=0734-2071&rft.eissn=1557-7333&rft_id=info:doi/10.1145%2F3750445&rft.externalDBID=n%2Fa&rft.externalDocID=10_1145_3750445 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0734-2071&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0734-2071&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0734-2071&client=summon |