Fast Scalable Construction of (Minimal Perfect Hash) Functions

Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static functions and minimal perfect hash functions using less space with respect to existing techniques. The main obstruction for any practical application...

Full description

Saved in:
Bibliographic Details
Published inExperimental Algorithms Vol. 9685; pp. 339 - 352
Main Authors Genuzio, Marco, Ottaviano, Giuseppe, Vigna, Sebastiano
Format Book Chapter
LanguageEnglish
Published Switzerland Springer International Publishing AG 01.01.2016
Springer International Publishing
SeriesLecture Notes in Computer Science
Subjects
Online AccessGet full text
ISBN3319388509
9783319388502
ISSN0302-9743
1611-3349
DOI10.1007/978-3-319-38851-9_23

Cover

Abstract Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static functions and minimal perfect hash functions using less space with respect to existing techniques. The main obstruction for any practical application of these results is the cubic-time Gaussian elimination required to solve these linear systems: despite they can be made very small, the computation is still too slow to be feasible. In this paper we describe in detail a number of heuristics and programming techniques to speed up the resolution of these systems by several orders of magnitude, making the overall construction competitive with the standard and widely used MWHC technique, which is based on hypergraph peeling. In particular, we introduce broadword programming techniques for fast equation manipulation and a lazy Gaussian elimination algorithm. We also describe a number of technical improvements to the data structure which further reduce space usage and improve lookup speed. Our implementation of these techniques yields a minimal perfect hash function data structure occupying 2.24 bits per element, compared to 2.68 for MWHC-based ones, and a static function data structure which reduces the multiplicative overhead from 1.23 to 1.03.
AbstractList Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static functions and minimal perfect hash functions using less space with respect to existing techniques. The main obstruction for any practical application of these results is the cubic-time Gaussian elimination required to solve these linear systems: despite they can be made very small, the computation is still too slow to be feasible. In this paper we describe in detail a number of heuristics and programming techniques to speed up the resolution of these systems by several orders of magnitude, making the overall construction competitive with the standard and widely used MWHC technique, which is based on hypergraph peeling. In particular, we introduce broadword programming techniques for fast equation manipulation and a lazy Gaussian elimination algorithm. We also describe a number of technical improvements to the data structure which further reduce space usage and improve lookup speed. Our implementation of these techniques yields a minimal perfect hash function data structure occupying 2.24 bits per element, compared to 2.68 for MWHC-based ones, and a static function data structure which reduces the multiplicative overhead from 1.23 to 1.03.
Author Genuzio, Marco
Ottaviano, Giuseppe
Vigna, Sebastiano
Author_xml – sequence: 1
  givenname: Marco
  surname: Genuzio
  fullname: Genuzio, Marco
  organization: Dipartimento di Informatica, Università degli Studi di Milano, Milan, Italy
– sequence: 2
  givenname: Giuseppe
  surname: Ottaviano
  fullname: Ottaviano, Giuseppe
  organization: Facebook, Menlo Park, USA
– sequence: 3
  givenname: Sebastiano
  surname: Vigna
  fullname: Vigna, Sebastiano
  email: vigna@di.unimi.it
  organization: Dipartimento di Informatica, Università degli Studi di Milano, Milan, Italy
BookMark eNqNkMtOwzAQRQ0URFv6ByyyhIXB9vi5QUIVpUhFIAFry3EcmhKSErv_T9oCa1Yj3blnpDkjNGjaJiB0TskVJURdG6UxYKAGg9aCYmMZHKBJH0Mf7jJziIZUUooBuDlCo98FMQM0JEAYNorDCRoaQbkUAuQpmsS4IoRQLakENUQ3MxdT9uJd7fI6ZNO2ianb-FS1TdaW2cVj1VSfrs6eQ1cGn7K5i8vLbLZpdpV4ho5LV8cw-Zlj9Da7e53O8eLp_mF6u8ArEDxhWXjBaKFFIT0xgoMHTrkuweuglQqlNJqw3FPNjOSc0KCc47lUuRMkQAFjxPZ347qrmvfQ2bxtP6KlxG5t2V6LBdv_b3dm7NZWD_E9tO7ar02IyYYt5UOTOlf7pVun0EUrmRGaEdsLsyD-jQmhNeXiD_sGYjZ6lg
ContentType Book Chapter
Copyright Springer International Publishing Switzerland 2016
Copyright_xml – notice: Springer International Publishing Switzerland 2016
DBID FFUUA
DEWEY 005.1
DOI 10.1007/978-3-319-38851-9_23
DatabaseName ProQuest Ebook Central - Book Chapters - Demo use only
DatabaseTitleList
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9783319388519
3319388517
EISSN 1611-3349
Editor Kulikov, Alexander S
Goldberg, Andrew V
Editor_xml – sequence: 1
  fullname: Goldberg, Andrew V
– sequence: 2
  fullname: Kulikov, Alexander S
EndPage 352
ExternalDocumentID EBC6295820_302_353
EBC5588145_302_353
GroupedDBID 0D6
0DA
38.
AABBV
AAMCO
AAPIT
AAQZU
ABBVZ
ABMNI
ABOWU
ACLMJ
ADCXD
ADPGQ
AEDXK
AEJGN
AEJLV
AEKFX
AETDV
AEZAY
ALMA_UNASSIGNED_HOLDINGS
AORVH
AWFBM
AZZ
BBABE
CZZ
FFUUA
I4C
IEZ
SBO
SWNTM
TPJZQ
TSXQS
Z81
Z83
Z88
-DT
-~X
29L
2HA
2HV
ACGFS
EJD
F5P
LAS
LDH
P2P
RSU
~02
ID FETCH-LOGICAL-j354t-6dc521d85d6c09543c34148f3c8e877ef69802bc182964401e7aa4b67ba50e3d3
ISBN 3319388509
9783319388502
ISSN 0302-9743
IngestDate Tue Jul 29 20:12:49 EDT 2025
Thu May 29 00:58:02 EDT 2025
Thu May 29 01:03:55 EDT 2025
IsPeerReviewed true
IsScholarly true
LCCallNum QA76.9.A43QA76.9.M35
Language English
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-j354t-6dc521d85d6c09543c34148f3c8e877ef69802bc182964401e7aa4b67ba50e3d3
OCLC 951465536
1189767178
PQID EBC5588145_302_353
PageCount 14
ParticipantIDs springer_books_10_1007_978_3_319_38851_9_23
proquest_ebookcentralchapters_6295820_302_353
proquest_ebookcentralchapters_5588145_302_353
PublicationCentury 2000
PublicationDate 2016-01-01
PublicationDateYYYYMMDD 2016-01-01
PublicationDate_xml – month: 01
  year: 2016
  text: 2016-01-01
  day: 01
PublicationDecade 2010
PublicationPlace Switzerland
PublicationPlace_xml – name: Switzerland
– name: Cham
PublicationSeriesSubtitle Theoretical Computer Science and General Issues
PublicationSeriesTitle Lecture Notes in Computer Science
PublicationSeriesTitleAlternate Lect.Notes Computer
PublicationSubtitle 15th International Symposium, SEA 2016, St. Petersburg, Russia, June 5-8, 2016, Proceedings
PublicationTitle Experimental Algorithms
PublicationYear 2016
Publisher Springer International Publishing AG
Springer International Publishing
Publisher_xml – name: Springer International Publishing AG
– name: Springer International Publishing
RelatedPersons Kleinberg, Jon M.
Mattern, Friedemann
Naor, Moni
Mitchell, John C.
Terzopoulos, Demetri
Steffen, Bernhard
Pandu Rangan, C.
Kanade, Takeo
Kittler, Josef
Weikum, Gerhard
Hutchison, David
Tygar, Doug
RelatedPersons_xml – sequence: 1
  givenname: David
  surname: Hutchison
  fullname: Hutchison, David
  organization: Lancaster University, Lancaster, United Kingdom
– sequence: 2
  givenname: Takeo
  surname: Kanade
  fullname: Kanade, Takeo
  organization: Carnegie Mellon University, Pittsburgh, USA
– sequence: 3
  givenname: Josef
  surname: Kittler
  fullname: Kittler, Josef
  organization: University of Surrey, Guildford, United Kingdom
– sequence: 4
  givenname: Jon M.
  surname: Kleinberg
  fullname: Kleinberg, Jon M.
  organization: Cornell University, Ithaca, USA
– sequence: 5
  givenname: Friedemann
  surname: Mattern
  fullname: Mattern, Friedemann
  organization: CNB H 104.2, ETH Zürich, Zürich, Switzerland
– sequence: 6
  givenname: John C.
  surname: Mitchell
  fullname: Mitchell, John C.
  organization: Stanford, USA
– sequence: 7
  givenname: Moni
  surname: Naor
  fullname: Naor, Moni
  organization: Weizmann Institute of Science, Rehovot, Israel
– sequence: 8
  givenname: C.
  surname: Pandu Rangan
  fullname: Pandu Rangan, C.
  organization: Indian Institute of Technology Madr, Chennai, India
– sequence: 9
  givenname: Bernhard
  surname: Steffen
  fullname: Steffen, Bernhard
  organization: Fakultät Informatik, TU Dortmund, Dortmund, Germany
– sequence: 10
  givenname: Demetri
  surname: Terzopoulos
  fullname: Terzopoulos, Demetri
  organization: Los Angeles, USA
– sequence: 11
  givenname: Doug
  surname: Tygar
  fullname: Tygar, Doug
  organization: University of California, Berkeley, USA
– sequence: 12
  givenname: Gerhard
  surname: Weikum
  fullname: Weikum, Gerhard
  organization: Max Planck Institute for Informatic, Saarbrücken, Germany
SSID ssj0001861637
ssj0002792
Score 2.2158287
Snippet Recent advances in random linear systems on finite fields have paved the way for the construction of constant-time data structures representing static...
SourceID springer
proquest
SourceType Publisher
StartPage 339
SubjectTerms Algorithms & data structures
Chunk Size
Construction Time
Gaussian Elimination
Hash Function
Static Function
Title Fast Scalable Construction of (Minimal Perfect Hash) Functions
URI http://ebookcentral.proquest.com/lib/SITE_ID/reader.action?docID=5588145&ppg=353
http://ebookcentral.proquest.com/lib/SITE_ID/reader.action?docID=6295820&ppg=353
http://link.springer.com/10.1007/978-3-319-38851-9_23
Volume 9685
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LbxMxELZouSAOlJcoL_nAAYSMvPFj7QtSWiVEFUUcWtSb5dfSopIAyam_vuONnc0ulVBRpFVkeS2vv7X3m8-eGYTeSKkjs4KTECUnnAdKXOMYCRp-lHnbtDkjj7_I2Sk_OhNnXW6-1rtk5T74qxv9Sv4HVSgDXJOX7C2Q3TQKBfAf8IUrIAzXAfnty6zl9FwXm398-X0BZv55jj2e34GpXa5g9trL1j8q5eYs0WITRwRueXwxv_gJt3-Nf9ogxjO7PE86wRS-dp2Sl0WBaigKFFFwICtuKVvjTz1DksFMZEoJ2lsZtVyn0_lrmd0-WZG8oNKtFdFm7Trcj2rNxKCw_XhODg7lSAvgH4bRkUmVfv0mKRlY2jTPmVF20E5dw7J1dzw5-vytk86UBBaZsi1uuq3XsZS6x9jykrypmz17YrAF3jKLkz10P3mb4OQGAh1_iO7E-SP0oOTawHnpfYw-JixxwRJvY4kXDX6bkcQZSZyQfIc3OD5Bp9PJyeGM5NwX5AcTfEVk8ECsghJBemDBnHmgG1w1zKuo6jo2Uis6ch7MQw2UllaxtpY7WTsraGSBPUW788U8PkM4xCoISp300I7V0lW1aqBdsBSZsK7ZR6SMhWl36POxYL9-8qURQqmKiwLUP-sPgN1H78sAm1R9aUqobEDGMAPImBYZk5B5fsvWX6B73QR4iXZh7OMr4Ikr9zq_N9fxjmK0
linkProvider Library Specific Holdings
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.title=Experimental+Algorithms&rft.atitle=Fast+Scalable+Construction+of+%28Minimal+Perfect+Hash%29+Functions&rft.date=2016-01-01&rft.pub=Springer+International+Publishing+AG&rft.isbn=9783319388502&rft.volume=9685&rft_id=info:doi/10.1007%2F978-3-319-38851-9_23&rft.externalDBID=353&rft.externalDocID=EBC6295820_302_353
thumbnail_s http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=https%3A%2F%2Febookcentral.proquest.com%2Fcovers%2F5588145-l.jpg
http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=https%3A%2F%2Febookcentral.proquest.com%2Fcovers%2F6295820-l.jpg