Encoding-Aware Data Placement for Efficient Degraded Reads in XOR-Coded Storage Systems: Algorithms and Evaluation
Modern storage systems adopt erasure coding to maintain fault tolerance with low storage redundancy. However, how to improve the performance of degraded reads in erasure-coded storage has been a critical issue. We revisit this problem from two different perspectives that are neglected by existing st...
Saved in:
| Published in | IEEE transactions on parallel and distributed systems Vol. 29; no. 12; pp. 2757 - 2770 |
|---|---|
| Main Authors | , , , |
| Format | Journal Article |
| Language | English |
| Published |
New York
IEEE
01.12.2018
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects | |
| Online Access | Get full text |
| ISSN | 1045-9219 1558-2183 |
| DOI | 10.1109/TPDS.2018.2842210 |
Cover
| Abstract | Modern storage systems adopt erasure coding to maintain fault tolerance with low storage redundancy. However, how to improve the performance of degraded reads in erasure-coded storage has been a critical issue. We revisit this problem from two different perspectives that are neglected by existing studies: data placement and encoding rules. To this end, we propose encoding-aware data placement (EDP), which mitigates the number of I/Os in degraded reads during a single failure for general XOR-based erasure codes. EDP carefully selects appropriate parity units to be generated by sequential data based on the encoding rules and establishes their generation orders. We further refine the data placement for optimizing the degraded reads to any two sequential data units. Trace-driven evaluation results show that EDP significantly reduces I/Os in degraded reads and hence shortens the read time. |
|---|---|
| AbstractList | Modern storage systems adopt erasure coding to maintain fault tolerance with low storage redundancy. However, how to improve the performance of degraded reads in erasure-coded storage has been a critical issue. We revisit this problem from two different perspectives that are neglected by existing studies: data placement and encoding rules. To this end, we propose encoding-aware data placement (EDP), which mitigates the number of I/Os in degraded reads during a single failure for general XOR-based erasure codes. EDP carefully selects appropriate parity units to be generated by sequential data based on the encoding rules and establishes their generation orders. We further refine the data placement for optimizing the degraded reads to any two sequential data units. Trace-driven evaluation results show that EDP significantly reduces I/Os in degraded reads and hence shortens the read time. |
| Author | Lee, Patrick P. C. Guo, Wenzhong Shen, Zhirong Shu, Jiwu |
| Author_xml | – sequence: 1 givenname: Zhirong orcidid: 0000-0003-2673-5868 surname: Shen fullname: Shen, Zhirong email: zhirong.shen2601@gmail.com organization: Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong – sequence: 2 givenname: Patrick P. C. orcidid: 0000-0002-4501-4364 surname: Lee fullname: Lee, Patrick P. C. email: pclee@cse.cuhk.edu.hk organization: Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong – sequence: 3 givenname: Jiwu surname: Shu fullname: Shu, Jiwu email: shujw@tsinghua.edu.cn organization: Department of Computer Science and Technology, Tsinghua University, Beijing, China – sequence: 4 givenname: Wenzhong orcidid: 0000-0003-4118-8823 surname: Guo fullname: Guo, Wenzhong email: guowenzhong@fzu.edu.cn organization: Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing, Key Laboratory of Spatial Data Mining & Information Sharing, College of Mathematics and Computer Sciences, and Ministry of Education, Fuzhou University, Fuzhou, China |
| BookMark | eNp9kMtOGzEUhq2KSgXKA1TdWOp6UvvYkxl3FyXpRUICEZDYjQ72mdRoYlPboeLtO0NQFyxYnYv-_1y-E3YUYiDGPkkxk1KYr9eXq80MhGxn0GoAKd6xY1nXbQWyVUdjLnRdGZDmAzvJ-V4IqWuhj1laBxudD9tq8RcT8RUW5JcDWtpRKLyPia_73ls_VSvaJnTk-BWhy9wHfntxVS3j1NqUmHBLfPOUC-3yN74YtjH58nuXOQbH14847LH4GD6y9z0Omc5e4im7-b6-Xv6szi9-_FouzisLRpWqaWpz5wRqMBZ7bZwZL1ZEIC3Ytr-btzg-5Rot57VsLAo3B2xAOQcGHFh1yr4c5j6k-GdPuXT3cZ_CuLIDqUDURgkxqpqDyqaYc6K-s74831kS-qGTopsAdxPgbgLcvQAenfKV8yH5HaanNz2fDx5PRP_1rWqE0Fr9A_Q8h-g |
| CODEN | ITDSEO |
| CitedBy_id | crossref_primary_10_1016_j_asoc_2021_107548 crossref_primary_10_1109_TPDS_2023_3306150 crossref_primary_10_1080_17517575_2019_1700552 crossref_primary_10_1007_s12652_020_02582_y crossref_primary_10_1007_s12652_020_02748_8 crossref_primary_10_1109_ACCESS_2020_3037164 crossref_primary_10_1109_ACCESS_2021_3049256 crossref_primary_10_1007_s11432_021_3421_6 crossref_primary_10_1016_j_engappai_2020_103803 crossref_primary_10_1007_s12652_019_01666_8 crossref_primary_10_1080_17517575_2019_1701715 crossref_primary_10_1016_j_asoc_2019_106038 crossref_primary_10_1109_ACCESS_2019_2956067 crossref_primary_10_1109_ACCESS_2019_2956980 crossref_primary_10_1080_17517575_2021_1885743 crossref_primary_10_1007_s12652_020_02543_5 crossref_primary_10_1016_j_asoc_2020_106071 crossref_primary_10_5937_telfor2202056S crossref_primary_10_1007_s11227_020_03511_8 crossref_primary_10_1002_ett_3922 crossref_primary_10_1016_j_asoc_2020_106065 crossref_primary_10_1080_17517575_2020_1722254 crossref_primary_10_1109_ACCESS_2019_2960603 |
| Cites_doi | 10.1109/IPDPS.2011.78 10.1145/1542275.1542326 10.1109/DSN.2011.5958220 10.1109/INFOCOM.2014.6848156 10.1109/ICPP.2015.57 10.1109/NAS.2014.29 10.1109/SRDS.2016.041 10.1109/DSN.2014.47 10.1109/DSN.2014.57 10.1109/TC.2014.2360543 10.1109/MSST.2012.6232371 10.1145/1416944.1416949 10.1145/2658991 10.1109/TC.2007.70830 10.1137/0108018 10.1109/18.746809 10.1007/3-540-45748-8_31 10.1109/IPDPS.2015.46 10.1109/JSAC.2010.100218 10.1109/DSN.2015.19 10.1145/2560013 10.1145/2043556.2043571 10.1109/12.364531 10.1145/1165389.945450 10.1109/CLUSTER.2013.6702621 10.1109/TPDS.2016.2525770 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TPDS.2018.2842210 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1558-2183 |
| EndPage | 2770 |
| ExternalDocumentID | 10_1109_TPDS_2018_2842210 8370044 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Fujian Provincial Natural Science Foundation grantid: 2017J05102 – fundername: Research Grants Council of Hong Kong grantid: GRF 14216316; CRF C7036-15G – fundername: Technology Innovation Platform Project of Fujian Province grantid: 2014H2005 – fundername: National Natural Science Foundation of China grantid: 61602120; 61327902; 61433008; U1435216; 61672159; U1705262 funderid: 10.13039/501100001809 – fundername: Fujian Engineering Research Center of Big Data Analysis and Processing – fundername: Fujian Collaborative Innovation Center for Big Data Application in Governments |
| GroupedDBID | --Z -~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS TN5 TWZ UHB AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D RIG |
| ID | FETCH-LOGICAL-c293t-7759bd0a429caf49d94503ee21c2c8fb68a104d7416517ca0d62a723dd292d2c3 |
| IEDL.DBID | RIE |
| ISSN | 1045-9219 |
| IngestDate | Sun Jun 29 15:25:42 EDT 2025 Thu Apr 24 23:13:01 EDT 2025 Wed Oct 01 04:37:21 EDT 2025 Wed Aug 27 08:33:32 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 12 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c293t-7759bd0a429caf49d94503ee21c2c8fb68a104d7416517ca0d62a723dd292d2c3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-4118-8823 0000-0002-4501-4364 0000-0003-2673-5868 |
| PQID | 2132059300 |
| PQPubID | 85437 |
| PageCount | 14 |
| ParticipantIDs | crossref_citationtrail_10_1109_TPDS_2018_2842210 ieee_primary_8370044 proquest_journals_2132059300 crossref_primary_10_1109_TPDS_2018_2842210 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2018-12-01 |
| PublicationDateYYYYMMDD | 2018-12-01 |
| PublicationDate_xml | – month: 12 year: 2018 text: 2018-12-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on parallel and distributed systems |
| PublicationTitleAbbrev | TPDS |
| PublicationYear | 2018 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref34 ref12 ref37 ref15 ref36 ref14 pinheiro (ref18) 2007 ref30 ref33 ref11 ref32 ford (ref6) 2010 corbett (ref5) 2004 ref1 xia (ref31) 2015 ref17 ref38 ref16 ref19 borthakur (ref3) 2010 rashmi (ref21) 2013 schroeder (ref24) 2007 ref23 ref26 ref25 khan (ref13) 2012 ref28 ref27 huang (ref10) 2012 ref29 plank (ref20) 2008 ref8 ref7 ref9 ref4 rashmi (ref22) 2014 bloemer (ref2) 1995 |
| References_xml | – start-page: 1 year: 2004 ident: ref5 article-title: Row-diagonal parity for double disk failure correction publication-title: Proc 3rd USENIX Conf File Storage Technol – ident: ref30 doi: 10.1109/IPDPS.2011.78 – start-page: 8 year: 2013 ident: ref21 article-title: A solution to the network challenges of data recovery in erasure-coded distributed storage systems: A study on the facebook warehouse cluster publication-title: Proc 3rd USENIX Conf on Hot Topics in Storage and File Syst – ident: ref12 doi: 10.1145/1542275.1542326 – ident: ref29 doi: 10.1109/DSN.2011.5958220 – ident: ref35 doi: 10.1109/INFOCOM.2014.6848156 – year: 2008 ident: ref20 article-title: Jerasure: A library in C/C++ facilitating erasure coding for storage applications-version 1.2 – ident: ref8 doi: 10.1109/ICPP.2015.57 – ident: ref33 doi: 10.1109/NAS.2014.29 – start-page: 61 year: 2010 ident: ref6 article-title: Availability in globally distributed storage systems publication-title: Proc 9th USENIX Conf Operating Syst Des Implementation – ident: ref25 doi: 10.1109/SRDS.2016.041 – start-page: 213 year: 2015 ident: ref31 article-title: A tale of two erasure codes in HDFS publication-title: Proc USENIX Conf File Storage Technol – ident: ref15 doi: 10.1109/DSN.2014.47 – ident: ref26 doi: 10.1109/DSN.2014.57 – ident: ref38 doi: 10.1109/TC.2014.2360543 – ident: ref37 doi: 10.1109/MSST.2012.6232371 – start-page: 331 year: 2014 ident: ref22 article-title: A hitchhiker's guide to fast and efficient data reconstruction in erasure-coded data centers publication-title: Proc ACM Conf SIGCOMM – year: 1995 ident: ref2 article-title: An XOR-based erasure-resilient coding scheme – start-page: 2 year: 2007 ident: ref18 article-title: Failure trends in a large disk drive population publication-title: Proc 5th USENIX Conf File Storage Technol – ident: ref17 doi: 10.1145/1416944.1416949 – ident: ref14 doi: 10.1145/2658991 – ident: ref11 doi: 10.1109/TC.2007.70830 – start-page: 251 year: 2012 ident: ref13 article-title: Rethinking erasure codes for cloud file systems: Minimizing i/o for recovery and degraded reads publication-title: Proc 10th USENIX Conf File Storage Technol – ident: ref23 doi: 10.1137/0108018 – ident: ref34 doi: 10.1109/18.746809 – ident: ref28 doi: 10.1007/3-540-45748-8_31 – ident: ref7 doi: 10.1109/IPDPS.2015.46 – ident: ref16 doi: 10.1109/JSAC.2010.100218 – year: 2010 ident: ref3 article-title: HDFS RAID publication-title: Hadoop User Group Meeting – ident: ref36 doi: 10.1109/DSN.2015.19 – ident: ref19 doi: 10.1145/2560013 – ident: ref4 doi: 10.1145/2043556.2043571 – ident: ref1 doi: 10.1109/12.364531 – ident: ref9 doi: 10.1145/1165389.945450 – start-page: 2 year: 2012 ident: ref10 article-title: Erasure coding in windows azure storage publication-title: Proc USENIX Conf Annu Tech Conf – ident: ref32 doi: 10.1109/CLUSTER.2013.6702621 – ident: ref27 doi: 10.1109/TPDS.2016.2525770 – start-page: 1 year: 2007 ident: ref24 article-title: Disk failures in the real world: What does an mttf of 1, 000, 000 hours mean to you? publication-title: Proc 5th USENIX Conf File Storage Technol |
| SSID | ssj0014504 |
| Score | 2.4033825 |
| Snippet | Modern storage systems adopt erasure coding to maintain fault tolerance with low storage redundancy. However, how to improve the performance of degraded reads... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 2757 |
| SubjectTerms | Coding Decoding degraded reads Distributed databases Encoding Encoding-aware data placement Fault tolerance Fault tolerant systems Layout Performance degradation Performance enhancement Placement Redundancy Storage systems Systems analysis XOR-coded storage systems |
| Title | Encoding-Aware Data Placement for Efficient Degraded Reads in XOR-Coded Storage Systems: Algorithms and Evaluation |
| URI | https://ieeexplore.ieee.org/document/8370044 https://www.proquest.com/docview/2132059300 |
| Volume | 29 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2183 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014504 issn: 1045-9219 databaseCode: RIE dateStart: 19900101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PT9swFH4CTuwwfg2twJAPnCZSEsdJY24VLUKT2NBapN4i_ypU6xLUpkLir-e9xI3QmKbdksi2rHy232f7ve8BnLk0TqU0KtAqxg2KQSyk1ipwKs16mZCZdXQOefs9vbkX3ybJZAPO21gY51ztfOa69Fjf5dvSrOio7IKEWkIhNmGzl6VNrFZ7YyCSOlUg7i6SQOI09DeYUSgvxneDETlxZV1cizmnYNk3NqhOqvJuJa7Ny_UO3K471niV_OquKt01L39oNv5vz3fho-eZrN8MjD3YcMU-7KxzODA_pffhwxtBwgNYDAtTkjUL-s9q4dhAVYrd0Uk7tc-Q4LJhrTlBbwPSmbDOMnLEX7JZwSY_fgZXJX0a4VYeVyrmBdEvWX_-UC5m1ePvJVOFZcNWZPwT3F8Px1c3gc_KEBikBhXS8URqGyo0ZEZNhbQSf33sHI8MN9lUp5lCECwxvSTqGRXalKsej63lkltu4kPYKsrCfQamdcQtkgYlhRKZTaWdIp8R3EVGO-SeHQjXOOXGS5ZT5ox5Xm9dQpkTtDlBm3toO_C1rfLU6HX8q_ABQdUW9Ch14GQ9GHI_o5c5p1jzRMZhePT3WsewTW03ri4nsFUtVu4LEpZKn9Yj9RWfruYo |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PT9swFH5i7LBxgA02UcY2H3aalpI4ThrvVtGibqMMjSL1FvlXBwIS1KZC4q_fe4kbIZim3ZLITqx8tt9n-73vAXxyaZxKaVSgVYwLFINYSK1V4FSa9TIhM-toH3J8ko7OxfdpMl2DL20sjHOudj5zXbqsz_JtaZa0VXZAQi2hEM_geSKESJporfbMQCR1skBcXySBxIHozzCjUB5MTgdn5MaVdXE25pzCZR9YoTqtypO5uDYwR1swXjWt8Su56i4r3TX3j1Qb_7ftr2DTM03Wb7rGa1hzxTZsrbI4MD-ot2HjgSThDsyHhSnJngX9OzV3bKAqxU5pr53ez5DismGtOkF3A1KasM4ycsVfsMuCTX_-Cg5LenSGi3mcq5iXRP_K-te_y_lldXGzYKqwbNjKjL-B86Ph5HAU-LwMgUFyUCEhT6S2oUJTZtRMSCvx18fO8chwk810mikEwRLXS6KeUaFNuerx2FouueUmfgvrRVm4XWBaR9wibVBSKJHZVNoZMhrBXWS0Q_bZgXCFU268aDnlzrjO68VLKHOCNidocw9tBz63VW4bxY5_Fd4hqNqCHqUO7K86Q-7H9CLnFG2eyDgM9_5e6yO8GE3Gx_nxt5Mf7-AlfadxfNmH9Wq-dO-RvlT6Q91r_wDUH-l1 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Encoding-Aware+Data+Placement+for+Efficient+Degraded+Reads+in+XOR-Coded+Storage+Systems%3A+Algorithms+and+Evaluation&rft.jtitle=IEEE+transactions+on+parallel+and+distributed+systems&rft.au=Shen%2C+Zhirong&rft.au=Lee%2C+Patrick+P.+C.&rft.au=Shu%2C+Jiwu&rft.au=Guo%2C+Wenzhong&rft.date=2018-12-01&rft.pub=IEEE&rft.issn=1045-9219&rft.volume=29&rft.issue=12&rft.spage=2757&rft.epage=2770&rft_id=info:doi/10.1109%2FTPDS.2018.2842210&rft.externalDocID=8370044 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1045-9219&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1045-9219&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1045-9219&client=summon |