Fast Scalable Construction of (Minimal Perfect Hash) Functions
Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static functions and minimal perfect hash functions using less space with respect to existing techniques. The main obstruction for any practical application...
Saved in:
Published in | Experimental Algorithms Vol. 9685; pp. 339 - 352 |
---|---|
Main Authors | , , |
Format | Book Chapter |
Language | English |
Published |
Switzerland
Springer International Publishing AG
01.01.2016
Springer International Publishing |
Series | Lecture Notes in Computer Science |
Subjects | |
Online Access | Get full text |
ISBN | 3319388509 9783319388502 |
ISSN | 0302-9743 1611-3349 |
DOI | 10.1007/978-3-319-38851-9_23 |
Cover
Abstract | Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static functions and minimal perfect hash functions using less space with respect to existing techniques. The main obstruction for any practical application of these results is the cubic-time Gaussian elimination required to solve these linear systems: despite they can be made very small, the computation is still too slow to be feasible.
In this paper we describe in detail a number of heuristics and programming techniques to speed up the resolution of these systems by several orders of magnitude, making the overall construction competitive with the standard and widely used MWHC technique, which is based on hypergraph peeling. In particular, we introduce broadword programming techniques for fast equation manipulation and a lazy Gaussian elimination algorithm. We also describe a number of technical improvements to the data structure which further reduce space usage and improve lookup speed.
Our implementation of these techniques yields a minimal perfect hash function data structure occupying 2.24 bits per element, compared to 2.68 for MWHC-based ones, and a static function data structure which reduces the multiplicative overhead from 1.23 to 1.03. |
---|---|
AbstractList | Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static functions and minimal perfect hash functions using less space with respect to existing techniques. The main obstruction for any practical application of these results is the cubic-time Gaussian elimination required to solve these linear systems: despite they can be made very small, the computation is still too slow to be feasible.
In this paper we describe in detail a number of heuristics and programming techniques to speed up the resolution of these systems by several orders of magnitude, making the overall construction competitive with the standard and widely used MWHC technique, which is based on hypergraph peeling. In particular, we introduce broadword programming techniques for fast equation manipulation and a lazy Gaussian elimination algorithm. We also describe a number of technical improvements to the data structure which further reduce space usage and improve lookup speed.
Our implementation of these techniques yields a minimal perfect hash function data structure occupying 2.24 bits per element, compared to 2.68 for MWHC-based ones, and a static function data structure which reduces the multiplicative overhead from 1.23 to 1.03. |
Author | Genuzio, Marco Ottaviano, Giuseppe Vigna, Sebastiano |
Author_xml | – sequence: 1 givenname: Marco surname: Genuzio fullname: Genuzio, Marco organization: Dipartimento di Informatica, Università degli Studi di Milano, Milan, Italy – sequence: 2 givenname: Giuseppe surname: Ottaviano fullname: Ottaviano, Giuseppe organization: Facebook, Menlo Park, USA – sequence: 3 givenname: Sebastiano surname: Vigna fullname: Vigna, Sebastiano email: vigna@di.unimi.it organization: Dipartimento di Informatica, Università degli Studi di Milano, Milan, Italy |
BookMark | eNqNkMtOwzAQRQ0URFv6ByyyhIXB9vi5QUIVpUhFIAFry3EcmhKSErv_T9oCa1Yj3blnpDkjNGjaJiB0TskVJURdG6UxYKAGg9aCYmMZHKBJH0Mf7jJziIZUUooBuDlCo98FMQM0JEAYNorDCRoaQbkUAuQpmsS4IoRQLakENUQ3MxdT9uJd7fI6ZNO2ianb-FS1TdaW2cVj1VSfrs6eQ1cGn7K5i8vLbLZpdpV4ho5LV8cw-Zlj9Da7e53O8eLp_mF6u8ArEDxhWXjBaKFFIT0xgoMHTrkuweuglQqlNJqw3FPNjOSc0KCc47lUuRMkQAFjxPZ347qrmvfQ2bxtP6KlxG5t2V6LBdv_b3dm7NZWD_E9tO7ar02IyYYt5UOTOlf7pVun0EUrmRGaEdsLsyD-jQmhNeXiD_sGYjZ6lg |
ContentType | Book Chapter |
Copyright | Springer International Publishing Switzerland 2016 |
Copyright_xml | – notice: Springer International Publishing Switzerland 2016 |
DBID | FFUUA |
DEWEY | 005.1 |
DOI | 10.1007/978-3-319-38851-9_23 |
DatabaseName | ProQuest Ebook Central - Book Chapters - Demo use only |
DatabaseTitleList | |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Computer Science |
EISBN | 9783319388519 3319388517 |
EISSN | 1611-3349 |
Editor | Kulikov, Alexander S Goldberg, Andrew V |
Editor_xml | – sequence: 1 fullname: Goldberg, Andrew V – sequence: 2 fullname: Kulikov, Alexander S |
EndPage | 352 |
ExternalDocumentID | EBC6295820_302_353 EBC5588145_302_353 |
GroupedDBID | 0D6 0DA 38. AABBV AAMCO AAPIT AAQZU ABBVZ ABMNI ABOWU ACLMJ ADCXD ADPGQ AEDXK AEJGN AEJLV AEKFX AETDV AEZAY ALMA_UNASSIGNED_HOLDINGS AORVH AWFBM AZZ BBABE CZZ FFUUA I4C IEZ SBO SWNTM TPJZQ TSXQS Z81 Z83 Z88 -DT -~X 29L 2HA 2HV ACGFS EJD F5P LAS LDH P2P RSU ~02 |
ID | FETCH-LOGICAL-j354t-6dc521d85d6c09543c34148f3c8e877ef69802bc182964401e7aa4b67ba50e3d3 |
ISBN | 3319388509 9783319388502 |
ISSN | 0302-9743 |
IngestDate | Tue Jul 29 20:12:49 EDT 2025 Thu May 29 00:58:02 EDT 2025 Thu May 29 01:03:55 EDT 2025 |
IsPeerReviewed | true |
IsScholarly | true |
LCCallNum | QA76.9.A43QA76.9.M35 |
Language | English |
LinkModel | OpenURL |
MergedId | FETCHMERGED-LOGICAL-j354t-6dc521d85d6c09543c34148f3c8e877ef69802bc182964401e7aa4b67ba50e3d3 |
OCLC | 951465536 1189767178 |
PQID | EBC5588145_302_353 |
PageCount | 14 |
ParticipantIDs | springer_books_10_1007_978_3_319_38851_9_23 proquest_ebookcentralchapters_6295820_302_353 proquest_ebookcentralchapters_5588145_302_353 |
PublicationCentury | 2000 |
PublicationDate | 2016-01-01 |
PublicationDateYYYYMMDD | 2016-01-01 |
PublicationDate_xml | – month: 01 year: 2016 text: 2016-01-01 day: 01 |
PublicationDecade | 2010 |
PublicationPlace | Switzerland |
PublicationPlace_xml | – name: Switzerland – name: Cham |
PublicationSeriesSubtitle | Theoretical Computer Science and General Issues |
PublicationSeriesTitle | Lecture Notes in Computer Science |
PublicationSeriesTitleAlternate | Lect.Notes Computer |
PublicationSubtitle | 15th International Symposium, SEA 2016, St. Petersburg, Russia, June 5-8, 2016, Proceedings |
PublicationTitle | Experimental Algorithms |
PublicationYear | 2016 |
Publisher | Springer International Publishing AG Springer International Publishing |
Publisher_xml | – name: Springer International Publishing AG – name: Springer International Publishing |
RelatedPersons | Kleinberg, Jon M. Mattern, Friedemann Naor, Moni Mitchell, John C. Terzopoulos, Demetri Steffen, Bernhard Pandu Rangan, C. Kanade, Takeo Kittler, Josef Weikum, Gerhard Hutchison, David Tygar, Doug |
RelatedPersons_xml | – sequence: 1 givenname: David surname: Hutchison fullname: Hutchison, David organization: Lancaster University, Lancaster, United Kingdom – sequence: 2 givenname: Takeo surname: Kanade fullname: Kanade, Takeo organization: Carnegie Mellon University, Pittsburgh, USA – sequence: 3 givenname: Josef surname: Kittler fullname: Kittler, Josef organization: University of Surrey, Guildford, United Kingdom – sequence: 4 givenname: Jon M. surname: Kleinberg fullname: Kleinberg, Jon M. organization: Cornell University, Ithaca, USA – sequence: 5 givenname: Friedemann surname: Mattern fullname: Mattern, Friedemann organization: CNB H 104.2, ETH Zürich, Zürich, Switzerland – sequence: 6 givenname: John C. surname: Mitchell fullname: Mitchell, John C. organization: Stanford, USA – sequence: 7 givenname: Moni surname: Naor fullname: Naor, Moni organization: Weizmann Institute of Science, Rehovot, Israel – sequence: 8 givenname: C. surname: Pandu Rangan fullname: Pandu Rangan, C. organization: Indian Institute of Technology Madr, Chennai, India – sequence: 9 givenname: Bernhard surname: Steffen fullname: Steffen, Bernhard organization: Fakultät Informatik, TU Dortmund, Dortmund, Germany – sequence: 10 givenname: Demetri surname: Terzopoulos fullname: Terzopoulos, Demetri organization: Los Angeles, USA – sequence: 11 givenname: Doug surname: Tygar fullname: Tygar, Doug organization: University of California, Berkeley, USA – sequence: 12 givenname: Gerhard surname: Weikum fullname: Weikum, Gerhard organization: Max Planck Institute for Informatic, Saarbrücken, Germany |
SSID | ssj0001861637 ssj0002792 |
Score | 2.2158287 |
Snippet | Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static... |
SourceID | springer proquest |
SourceType | Publisher |
StartPage | 339 |
SubjectTerms | Algorithms & data structures Chunk Size Construction Time Gaussian Elimination Hash Function Static Function |
Title | Fast Scalable Construction of (Minimal Perfect Hash) Functions |
URI | http://ebookcentral.proquest.com/lib/SITE_ID/reader.action?docID=5588145&ppg=353 http://ebookcentral.proquest.com/lib/SITE_ID/reader.action?docID=6295820&ppg=353 http://link.springer.com/10.1007/978-3-319-38851-9_23 |
Volume | 9685 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LbxMxELZouSAOlJcoL_nAAYSMvPFj7QtSWiVEFUUcWtSb5dfSopIAyam_vuONnc0ulVBRpFVkeS2vv7X3m8-eGYTeSKkjs4KTECUnnAdKXOMYCRp-lHnbtDkjj7_I2Sk_OhNnXW6-1rtk5T74qxv9Sv4HVSgDXJOX7C2Q3TQKBfAf8IUrIAzXAfnty6zl9FwXm398-X0BZv55jj2e34GpXa5g9trL1j8q5eYs0WITRwRueXwxv_gJt3-Nf9ogxjO7PE86wRS-dp2Sl0WBaigKFFFwICtuKVvjTz1DksFMZEoJ2lsZtVyn0_lrmd0-WZG8oNKtFdFm7Trcj2rNxKCw_XhODg7lSAvgH4bRkUmVfv0mKRlY2jTPmVF20E5dw7J1dzw5-vytk86UBBaZsi1uuq3XsZS6x9jykrypmz17YrAF3jKLkz10P3mb4OQGAh1_iO7E-SP0oOTawHnpfYw-JixxwRJvY4kXDX6bkcQZSZyQfIc3OD5Bp9PJyeGM5NwX5AcTfEVk8ECsghJBemDBnHmgG1w1zKuo6jo2Uis6ch7MQw2UllaxtpY7WTsraGSBPUW788U8PkM4xCoISp300I7V0lW1aqBdsBSZsK7ZR6SMhWl36POxYL9-8qURQqmKiwLUP-sPgN1H78sAm1R9aUqobEDGMAPImBYZk5B5fsvWX6B73QR4iXZh7OMr4Ikr9zq_N9fxjmK0 |
linkProvider | Library Specific Holdings |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.title=Experimental+Algorithms&rft.atitle=Fast+Scalable+Construction+of+%28Minimal+Perfect+Hash%29+Functions&rft.date=2016-01-01&rft.pub=Springer+International+Publishing+AG&rft.isbn=9783319388502&rft.volume=9685&rft_id=info:doi/10.1007%2F978-3-319-38851-9_23&rft.externalDBID=353&rft.externalDocID=EBC6295820_302_353 |
thumbnail_s | http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=https%3A%2F%2Febookcentral.proquest.com%2Fcovers%2F5588145-l.jpg http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=https%3A%2F%2Febookcentral.proquest.com%2Fcovers%2F6295820-l.jpg |