d2o: a distributed data object for parallel high-performance computing in Python
We introduce d2o , a Python module for cluster-distributed multi-dimensional numerical arrays. It acts as a layer of abstraction between the algorithm code and the data-distribution logic. The main goal is to achieve usability without losing numerical performance and scalability. d2o ’s global inter...
Saved in:
| Published in | Journal of big data Vol. 3; no. 1; pp. 1 - 34 |
|---|---|
| Main Authors | , , , |
| Format | Journal Article |
| Language | English |
| Published |
Cham
Springer International Publishing
15.09.2016
Springer Nature B.V |
| Subjects | |
| Online Access | Get full text |
| ISSN | 2196-1115 2196-1115 |
| DOI | 10.1186/s40537-016-0052-5 |
Cover
| Abstract | We introduce
d2o
, a Python module for cluster-distributed multi-dimensional numerical arrays. It acts as a layer of abstraction between the algorithm code and the data-distribution logic. The main goal is to achieve usability without losing numerical performance and scalability.
d2o
’s global interface is similar to the one of a numpy.ndarray, whereas the cluster node’s local data is directly accessible for use in customized high-performance modules.
d2o
is written in pure Python which makes it portable and easy to use and modify. Expensive operations are carried out by dedicated external libraries like
numpy
and
mpi4py
. The performance of
d2o
is on a par with numpy for serial applications and scales well when moving to an MPI cluster.
d2o
is open-source software available under the GNU General Public License v3 (GPL-3) at
https://gitlab.mpcdf.mpg.de/ift/D2O
. |
|---|---|
| AbstractList | We introduce
d2o
, a Python module for cluster-distributed multi-dimensional numerical arrays. It acts as a layer of abstraction between the algorithm code and the data-distribution logic. The main goal is to achieve usability without losing numerical performance and scalability.
d2o
’s global interface is similar to the one of a numpy.ndarray, whereas the cluster node’s local data is directly accessible for use in customized high-performance modules.
d2o
is written in pure Python which makes it portable and easy to use and modify. Expensive operations are carried out by dedicated external libraries like
numpy
and
mpi4py
. The performance of
d2o
is on a par with numpy for serial applications and scales well when moving to an MPI cluster.
d2o
is open-source software available under the GNU General Public License v3 (GPL-3) at
https://gitlab.mpcdf.mpg.de/ift/D2O
. We introduce d2o, a Python module for cluster-distributed multi-dimensional numerical arrays. It acts as a layer of abstraction between the algorithm code and the data-distribution logic. The main goal is to achieve usability without losing numerical performance and scalability. d2o’s global interface is similar to the one of a numpy.ndarray, whereas the cluster node’s local data is directly accessible for use in customized high-performance modules. d2o is written in pure Python which makes it portable and easy to use and modify. Expensive operations are carried out by dedicated external libraries like numpy and mpi4py. The performance of d2o is on a par with numpy for serial applications and scales well when moving to an MPI cluster. d2o is open-source software available under the GNU General Public License v3 (GPL-3) at https://gitlab.mpcdf.mpg.de/ift/D2O. |
| ArticleNumber | 17 |
| Author | Enßlin, Torsten Steininger, Theo Beaujean, Frederik Greiner, Maksim |
| Author_xml | – sequence: 1 givenname: Theo orcidid: 0000-0002-6543-5836 surname: Steininger fullname: Steininger, Theo email: theos@mpa-garching.mpg.de organization: Max Planck Institut für Astrophysik, Ludwig-Maximilians-Universität München – sequence: 2 givenname: Maksim surname: Greiner fullname: Greiner, Maksim organization: Max Planck Institut für Astrophysik, Ludwig-Maximilians-Universität München – sequence: 3 givenname: Frederik surname: Beaujean fullname: Beaujean, Frederik organization: Ludwig-Maximilians-Universität München, Exzellenzcluster Universe – sequence: 4 givenname: Torsten surname: Enßlin fullname: Enßlin, Torsten organization: Max Planck Institut für Astrophysik, Ludwig-Maximilians-Universität München |
| BookMark | eNqNkE1rGzEQhkVwIW7qH5CbIOdt9bHSSrmVkH6AIT60ZzHWSrbMWtpIWoL_fTfYh1BoyWmG4X1m3nk_okVM0SF0S8lnSpX8UloieNcQKhtCBGvEFVoyqmVDKRWLN_01WpVyIIRQPiOyXaJNz9I9BtyHUnPYTtX1uIcKOG0PzlbsU8YjZBgGN-B92O2b0eV5eIRoHbbpOE41xB0OEW9OdZ_iJ_TBw1Dc6lJv0O9vj78efjTrp-8_H76uG8u6VjSqZ9wBCM86Dcp2inrOpdNcCettqxkHSdotVyC16LddqyV4Ty1TwD1Rmt8gdt47xRFOL7NBM-ZwhHwylJjXWMw5FjPHYl5jMWKG7s7QmNPz5Eo1hzTlOPs0VKtOMyY1n1X0rLI5lZKdf9fm7i_Ghgo1pFgzhOG_5OWRMl-JO5ffePon9AfcjpOT |
| CitedBy_id | crossref_primary_10_1051_0004_6361_201834642 crossref_primary_10_1088_1475_7516_2018_08_049 crossref_primary_10_1145_3617588 crossref_primary_10_1002_andp_201800290 |
| Cites_doi | 10.7551/mitpress/2946.001.0001 10.1103/PhysRevD.80.105005 10.1007/978-3-540-30218-6_19 10.1051/0004-6361/201526717 10.1145/301618.301661 10.1137/1.9780898719642 10.1109/MCSE.2007.53 10.25080/Majora-ebaa42b7-00d 10.1109/JPROC.2004.840301 10.1051/0004-6361/201323094 10.2514/1.697 10.1109/MCSE.2011.37 10.1051/0004-6361/201321236 10.1016/j.jpdc.2005.03.010 |
| ContentType | Journal Article |
| Copyright | The Author(s) 2016 Journal of Big Data is a copyright of Springer, (2016). All Rights Reserved. |
| Copyright_xml | – notice: The Author(s) 2016 – notice: Journal of Big Data is a copyright of Springer, (2016). All Rights Reserved. |
| DBID | C6C AAYXX CITATION 0-V 3V. 7WY 7WZ 7XB 87Z 88J 8AL 8FE 8FG 8FK 8FL ABUWG AFKRA ALSLI ARAPS AZQEC BENPR BEZIV BGLVJ CCPQU DWQXO FRNLG F~G GNUQQ HCIFZ JQ2 K60 K6~ K7- L.- M0C M0N M2R P5Z P62 PHGZM PHGZT PIMPY PKEHL POGQB PQBIZ PQBZA PQEST PQGLB PQQKQ PQUKI PRQQA Q9U ADTOC UNPAY |
| DOI | 10.1186/s40537-016-0052-5 |
| DatabaseName | Springer Nature OA Free Journals CrossRef ProQuest Social Sciences Premium Collection【Remote access available】 ProQuest Central (Corporate) ABI/INFORM Collection ABI/INFORM Global (PDF only) ProQuest Central (purchase pre-March 2016) ABI/INFORM Collection Social Science Database (Alumni Edition) Computing Database (Alumni Edition) ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) (purchase pre-March 2016) ABI/INFORM Collection (Alumni) ProQuest Central (Alumni) ProQuest Central Social Science Premium Collection Advanced Technologies & Computer Science Collection ProQuest Central Essentials ProQuest Central Business Premium Collection ProQuest Technology Collection ProQuest One ProQuest Central Business Premium Collection (Alumni) ABI/INFORM Global (Corporate) ProQuest Central Student SciTech Premium Collection ProQuest Computer Science Collection ProQuest Business Collection (Alumni Edition) ProQuest Business Collection Computer Science Database ABI/INFORM Professional Advanced ABI/INFORM Global Computing Database Social Science Database Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic (New) Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest Sociology & Social Sciences Collection ProQuest One Business ProQuest One Business (Alumni) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic ProQuest One Academic UKI Edition ProQuest One Social Sciences ProQuest Central Basic Unpaywall for CDI: Periodical Content Unpaywall |
| DatabaseTitle | CrossRef Publicly Available Content Database ABI/INFORM Global (Corporate) ProQuest Business Collection (Alumni Edition) ProQuest One Business ProQuest Sociology & Social Sciences Collection Computer Science Database ProQuest Central Student Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection ProQuest Social Science Journals (Alumni Edition) ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College Sociology & Social Sciences Collection ABI/INFORM Complete ProQuest Central ABI/INFORM Professional Advanced ProQuest One Applied & Life Sciences ProQuest Central Korea ProQuest Central (New) ABI/INFORM Complete (Alumni Edition) Advanced Technologies & Aerospace Collection Business Premium Collection Social Science Premium Collection ABI/INFORM Global ProQuest Computing ProQuest One Social Sciences ABI/INFORM Global (Alumni Edition) ProQuest Central Basic ProQuest Computing (Alumni Edition) ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection ProQuest Business Collection Advanced Technologies & Aerospace Database ProQuest Social Science Journals ProQuest Social Sciences Premium Collection ProQuest One Academic UKI Edition ProQuest One Business (Alumni) ProQuest One Academic ProQuest One Academic (New) ProQuest Central (Alumni) Business Premium Collection (Alumni) |
| DatabaseTitleList | Publicly Available Content Database |
| Database_xml | – sequence: 1 dbid: C6C name: Springer Nature Link url: http://www.springeropen.com/ sourceTypes: Publisher – sequence: 2 dbid: UNPAY name: Unpaywall url: https://proxy.k.utb.cz/login?url=https://unpaywall.org/ sourceTypes: Open Access Repository – sequence: 3 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 2196-1115 |
| EndPage | 34 |
| ExternalDocumentID | 10.1186/s40537-016-0052-5 10_1186_s40537_016_0052_5 |
| GrantInformation_xml | – fundername: Max-Planck-Gesellschaft (DE) – fundername: Studienstiftung des Deutschen Volkes (DE) – fundername: Max-Planck-Gesellschaft funderid: http://dx.doi.org/10.13039/501100004189 – fundername: Ludwig-Maximilians-Universität München funderid: http://dx.doi.org/10.13039/501100005722 |
| GroupedDBID | 0-V 0R~ 3V. 5VS 7WY 8FE 8FG 8FL AAFWJ AAJSJ AAKKN ABEEZ ABFTD ABUWG ACACY ACGFS ACULB ADBBV ADINQ AFGXO AFKRA AFPKN AHBYD AHSBF ALMA_UNASSIGNED_HOLDINGS ALSLI AMKLP ARALO ARAPS ASPBG AZQEC BCNDV BENPR BEZIV BGLVJ BPHCQ C24 C6C CCPQU DWQXO EBLON EBS EJD FRNLG GNUQQ GROUPED_DOAJ HCIFZ K60 K6V K6~ K7- M0C M0N M2R M~E OK1 P62 PIMPY PQBIZ PQBZA PQQKQ PROAC RSV SOJ AASML AAYXX CITATION IAO PHGZM PHGZT PQGLB PRQQA PUEGO 7XB 8AL 8FK JQ2 L.- PKEHL POGQB PQEST PQUKI Q9U ADMLS ADTOC ISR ITC UNPAY |
| ID | FETCH-LOGICAL-c2745-8d23eaa5f279a8c781f336e9385cfc4923a604b38a695db7496aff1c28a3f0893 |
| IEDL.DBID | UNPAY |
| ISSN | 2196-1115 |
| IngestDate | Sun Oct 26 02:27:40 EDT 2025 Tue Oct 14 12:42:38 EDT 2025 Wed Oct 01 01:56:01 EDT 2025 Thu Apr 24 23:09:53 EDT 2025 Fri Feb 21 02:36:08 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | Open source Parallelization Numpy MPI Numerics Python |
| Language | English |
| License | cc-by |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c2745-8d23eaa5f279a8c781f336e9385cfc4923a604b38a695db7496aff1c28a3f0893 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-6543-5836 |
| OpenAccessLink | https://proxy.k.utb.cz/login?url=https://journalofbigdata.springeropen.com/track/pdf/10.1186/s40537-016-0052-5 |
| PQID | 1987922693 |
| PQPubID | 2046140 |
| PageCount | 34 |
| ParticipantIDs | unpaywall_primary_10_1186_s40537_016_0052_5 proquest_journals_1987922693 crossref_primary_10_1186_s40537_016_0052_5 crossref_citationtrail_10_1186_s40537_016_0052_5 springer_journals_10_1186_s40537_016_0052_5 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 20160915 |
| PublicationDateYYYYMMDD | 2016-09-15 |
| PublicationDate_xml | – month: 9 year: 2016 text: 20160915 day: 15 |
| PublicationDecade | 2010 |
| PublicationPlace | Cham |
| PublicationPlace_xml | – name: Cham – name: Heidelberg |
| PublicationTitle | Journal of big data |
| PublicationTitleAbbrev | J Big Data |
| PublicationYear | 2016 |
| Publisher | Springer International Publishing Springer Nature B.V |
| Publisher_xml | – name: Springer International Publishing – name: Springer Nature B.V |
| References | Corporation I. Intel MPI library. 2016. https://software.intel.com/en-us/intel-mpi-library. Accessed 6 June 2016. Team M. MPICH2: high-performance portable MPI. 2016. http://www.mcs.anl.gov/mpich2. Accessed 24 Mar 2016. JunklewitzHBellMRSeligMEnßlinTARESOLVE: a new algorithm for aperture synthesis imaging of extended emission in radio astronomyAstron Astrophys20165867610.1051/0004-6361/201323094 Frigo M. A fast fourier transform compiler. In: Proceedings of the ACM SIGPLAN 1999 conference on programming language design and implementation. PLDI ’99. New York; 1999. doi:10.1145/301618.301661. http://doi.acm.org/10.1145/301618.301661. Ka-Ping Yee GVR. PEP 234—Iterators. 2016. https://www.python.org/dev/peps/pep-0234/. Accessed 12 Apr 2016. Enthought I. DistArray: think globally, act locally. 2016. http://docs.enthought.com/distarray/. Accessed 24 Mar 2016. DadoneAGrossmanBGhost-cell method for inviscid two-dimensional flows on cartesian gridsAIAA J2004422499250710.2514/1.697 SeligMBellMRJunklewitzHOppermannNReineckeMGreinerMPachajoaCEnßlinTANIFTY— numerical information field theory. A versatile PYTHON library for signal inferenceAstron Astrophys20135542610.1051/0004-6361/201321236 Universe EC. Excellence cluster universe. 2016. http://www.universe-cluster.de/c2pap. Accessed 6 Apr 2016. Wiener N. Extrapolation, interpolation and smoothing of stationary time series, with engineering applications. In: note: Originally issued in Feb 1942 as a classified Nat. Council Rep: Defense Res. New York: Technology Press and Wiley; 1949. Strohmaier E, Dongarra J, Simon H, Meuer M. The TOP500 project. 2015. http://www.top500.org/lists/2015/11/. Accessed 24 Mar 2016. DalcínLPazRStortiMMPI for pythonJ Parallel Distrib Comput20056591108111510.1016/j.jpdc.2005.03.010 ForumMPIMPI: a message passing interface standardInt J Supercomput Appl199483/4159416 Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I. Spark: cluster computing with working sets. In: Proceedings of the 2Nd USENIX conference on hot topics in cloud computing. HotCloud’10. Berkeley: USENIX Association. Berkeley; 2010. p. 10–10. http://dl.acm.org/citation.cfm?id=1863103.1863113. BlackfordLSChoiJClearyAD’AzevedoEDemmelJDhillonIDongarraJHammarlingSHenryGPetitetAStanleyKWalkerDWhaleyRCScaLAPACK users’ guide1997PhiladelphiaSociety for Industrial and Applied Mathematics10.1137/1.97808987196420886.65022 Balay S, Abhyankar S, Adams MF, Brown J, Brune P, Buschelman K, Dalcin L, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, McInnes LC, Rupp K, Smith BF, Zampini S, Zhang H. PETSc web page. 2015. http://www.mcs.anl.gov/petsc. Gomersall H. pyFFTW: a pythonic wrapper around FFTW. We use the mpi branch available at https://github.com/fredRos/pyFFTW. 2016. https://hgomersall.github.io/pyFFTW. Accessed 23 Mar 2016. Gabriel E, Fagg GE, Bosilca G, Angskun T, Dongarra JJ, Squyres JM, Sahay V, Kambadur P, Barrett B, Lumsdaine A, Castain RH, Daniel DJ, Graham RL, Woodall TS. Open MPI: goals, concept, and design of a next generation MPI implementation. In: Proceedings, 11th European PVM/MPI users’ group meeting. Budapest; 2004. p. 97–104. Message Passing Interface ForumMPI2: a message passing interface standardHigh Perform Comput Appl1998121–21299 Apache Software Foundation. Hadoop. 2016. https://hadoop.apache.org. Accessed 23 Mar 2016. McKerns MM, Strand L, Sullivan T, Fang A, Aivazis MAG. Building a framework for predictive science. CoRR. 2012. arXiv:1202.1056. Frigo M, Johnson SG. The design and implementation of FFTW3. In: Proceedings of the IEEE, Vol 93(2). Special issue on “Program Generation, Optimization, and Platform Adaptation”. 2005. p. 216–231 EnßlinTAFrommertMKitauraFSInformation field theory for cosmological perturbation reconstruction and non-linear signal analysisPhys Rev D20098010500510.1103/PhysRevD.80.105005 Greiner M, Schnitzeler DHFM, Ensslin TA. Tomography of the galactic free electron density with the square kilometer array. ArXiv e-prints. 2015. 1512.03480. PérezFGrangerBEIPython: a system for interactive scientific computingComput Sci Eng200793212910.1109/MCSE.2007.53 Team S. ScaLAPACK web page. 2016. http://www.netlib.org/scalapack/. Accessed 23 Mar 2016. van der WaltSColbertSCVaroquauxGThe numpy array: a structure for efficient numerical computationComput Sci Eng2011132223010.1109/MCSE.2011.37 52_CR27 H Junklewitz (52_CR2) 2016; 586 52_CR25 L Dalcín (52_CR21) 2005; 65 52_CR26 52_CR23 52_CR22 52_CR20 TA Enßlin (52_CR24) 2009; 80 52_CR8 52_CR7 52_CR1 52_CR16 52_CR17 Message Passing Interface Forum (52_CR6) 1998; 12 52_CR14 52_CR15 52_CR12 A Dadone (52_CR10) 2004; 42 52_CR13 LS Blackford (52_CR9) 1997 F Pérez (52_CR11) 2007; 9 52_CR18 52_CR19 M Selig (52_CR3) 2013; 554 S Walt van der (52_CR4) 2011; 13 MPI Forum (52_CR5) 1994; 8 |
| References_xml | – reference: ForumMPIMPI: a message passing interface standardInt J Supercomput Appl199483/4159416 – reference: Frigo M. A fast fourier transform compiler. In: Proceedings of the ACM SIGPLAN 1999 conference on programming language design and implementation. PLDI ’99. New York; 1999. doi:10.1145/301618.301661. http://doi.acm.org/10.1145/301618.301661. – reference: DadoneAGrossmanBGhost-cell method for inviscid two-dimensional flows on cartesian gridsAIAA J2004422499250710.2514/1.697 – reference: Message Passing Interface ForumMPI2: a message passing interface standardHigh Perform Comput Appl1998121–21299 – reference: JunklewitzHBellMRSeligMEnßlinTARESOLVE: a new algorithm for aperture synthesis imaging of extended emission in radio astronomyAstron Astrophys20165867610.1051/0004-6361/201323094 – reference: Balay S, Abhyankar S, Adams MF, Brown J, Brune P, Buschelman K, Dalcin L, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, McInnes LC, Rupp K, Smith BF, Zampini S, Zhang H. PETSc web page. 2015. http://www.mcs.anl.gov/petsc. – reference: Team M. MPICH2: high-performance portable MPI. 2016. http://www.mcs.anl.gov/mpich2. Accessed 24 Mar 2016. – reference: SeligMBellMRJunklewitzHOppermannNReineckeMGreinerMPachajoaCEnßlinTANIFTY— numerical information field theory. A versatile PYTHON library for signal inferenceAstron Astrophys20135542610.1051/0004-6361/201321236 – reference: Corporation I. Intel MPI library. 2016. https://software.intel.com/en-us/intel-mpi-library. Accessed 6 June 2016. – reference: DalcínLPazRStortiMMPI for pythonJ Parallel Distrib Comput20056591108111510.1016/j.jpdc.2005.03.010 – reference: Strohmaier E, Dongarra J, Simon H, Meuer M. The TOP500 project. 2015. http://www.top500.org/lists/2015/11/. Accessed 24 Mar 2016. – reference: Gabriel E, Fagg GE, Bosilca G, Angskun T, Dongarra JJ, Squyres JM, Sahay V, Kambadur P, Barrett B, Lumsdaine A, Castain RH, Daniel DJ, Graham RL, Woodall TS. Open MPI: goals, concept, and design of a next generation MPI implementation. In: Proceedings, 11th European PVM/MPI users’ group meeting. Budapest; 2004. p. 97–104. – reference: van der WaltSColbertSCVaroquauxGThe numpy array: a structure for efficient numerical computationComput Sci Eng2011132223010.1109/MCSE.2011.37 – reference: Apache Software Foundation. Hadoop. 2016. https://hadoop.apache.org. Accessed 23 Mar 2016. – reference: Enthought I. DistArray: think globally, act locally. 2016. http://docs.enthought.com/distarray/. Accessed 24 Mar 2016. – reference: Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I. Spark: cluster computing with working sets. In: Proceedings of the 2Nd USENIX conference on hot topics in cloud computing. HotCloud’10. Berkeley: USENIX Association. Berkeley; 2010. p. 10–10. http://dl.acm.org/citation.cfm?id=1863103.1863113. – reference: PérezFGrangerBEIPython: a system for interactive scientific computingComput Sci Eng200793212910.1109/MCSE.2007.53 – reference: Universe EC. Excellence cluster universe. 2016. http://www.universe-cluster.de/c2pap. Accessed 6 Apr 2016. – reference: Gomersall H. pyFFTW: a pythonic wrapper around FFTW. We use the mpi branch available at https://github.com/fredRos/pyFFTW. 2016. https://hgomersall.github.io/pyFFTW. Accessed 23 Mar 2016. – reference: Ka-Ping Yee GVR. PEP 234—Iterators. 2016. https://www.python.org/dev/peps/pep-0234/. Accessed 12 Apr 2016. – reference: Greiner M, Schnitzeler DHFM, Ensslin TA. Tomography of the galactic free electron density with the square kilometer array. ArXiv e-prints. 2015. 1512.03480. – reference: Team S. ScaLAPACK web page. 2016. http://www.netlib.org/scalapack/. Accessed 23 Mar 2016. – reference: Frigo M, Johnson SG. The design and implementation of FFTW3. In: Proceedings of the IEEE, Vol 93(2). Special issue on “Program Generation, Optimization, and Platform Adaptation”. 2005. p. 216–231 – reference: McKerns MM, Strand L, Sullivan T, Fang A, Aivazis MAG. Building a framework for predictive science. CoRR. 2012. arXiv:1202.1056. – reference: EnßlinTAFrommertMKitauraFSInformation field theory for cosmological perturbation reconstruction and non-linear signal analysisPhys Rev D20098010500510.1103/PhysRevD.80.105005 – reference: BlackfordLSChoiJClearyAD’AzevedoEDemmelJDhillonIDongarraJHammarlingSHenryGPetitetAStanleyKWalkerDWhaleyRCScaLAPACK users’ guide1997PhiladelphiaSociety for Industrial and Applied Mathematics10.1137/1.97808987196420886.65022 – reference: Wiener N. Extrapolation, interpolation and smoothing of stationary time series, with engineering applications. In: note: Originally issued in Feb 1942 as a classified Nat. Council Rep: Defense Res. New York: Technology Press and Wiley; 1949. – volume: 8 start-page: 159 issue: 3/4 year: 1994 ident: 52_CR5 publication-title: Int J Supercomput Appl – ident: 52_CR22 – ident: 52_CR25 doi: 10.7551/mitpress/2946.001.0001 – ident: 52_CR20 – volume: 80 start-page: 105005 year: 2009 ident: 52_CR24 publication-title: Phys Rev D doi: 10.1103/PhysRevD.80.105005 – ident: 52_CR18 doi: 10.1007/978-3-540-30218-6_19 – ident: 52_CR27 – ident: 52_CR1 doi: 10.1051/0004-6361/201526717 – ident: 52_CR8 doi: 10.1145/301618.301661 – volume-title: ScaLAPACK users’ guide year: 1997 ident: 52_CR9 doi: 10.1137/1.9780898719642 – volume: 9 start-page: 21 issue: 3 year: 2007 ident: 52_CR11 publication-title: Comput Sci Eng doi: 10.1109/MCSE.2007.53 – ident: 52_CR14 doi: 10.25080/Majora-ebaa42b7-00d – ident: 52_CR17 – ident: 52_CR19 – ident: 52_CR15 – ident: 52_CR26 doi: 10.1109/JPROC.2004.840301 – volume: 12 start-page: 1 issue: 1–2 year: 1998 ident: 52_CR6 publication-title: High Perform Comput Appl – ident: 52_CR13 – volume: 586 start-page: 76 year: 2016 ident: 52_CR2 publication-title: Astron Astrophys doi: 10.1051/0004-6361/201323094 – ident: 52_CR7 – volume: 42 start-page: 2499 year: 2004 ident: 52_CR10 publication-title: AIAA J doi: 10.2514/1.697 – volume: 13 start-page: 22 issue: 2 year: 2011 ident: 52_CR4 publication-title: Comput Sci Eng doi: 10.1109/MCSE.2011.37 – ident: 52_CR23 – volume: 554 start-page: 26 year: 2013 ident: 52_CR3 publication-title: Astron Astrophys doi: 10.1051/0004-6361/201321236 – volume: 65 start-page: 1108 issue: 9 year: 2005 ident: 52_CR21 publication-title: J Parallel Distrib Comput doi: 10.1016/j.jpdc.2005.03.010 – ident: 52_CR12 – ident: 52_CR16 |
| SSID | ssj0001340564 |
| Score | 2.0556388 |
| Snippet | We introduce
d2o
, a Python module for cluster-distributed multi-dimensional numerical arrays. It acts as a layer of abstraction between the algorithm code and... We introduce d2o, a Python module for cluster-distributed multi-dimensional numerical arrays. It acts as a layer of abstraction between the algorithm code and... |
| SourceID | unpaywall proquest crossref springer |
| SourceType | Open Access Repository Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 1 |
| SubjectTerms | Big Data Clusters Communications Engineering Computational Science and Engineering Computer Science Data Mining and Knowledge Discovery Database Management High performance computing Information Storage and Retrieval Mathematical Applications in Computer Science Methodology Networks Open source software Source code |
| SummonAdditionalLinks | – databaseName: ProQuest Central dbid: BENPR link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwhV1bSxwxFD5s1wf1oa2X4lpb8uCTEtzJbZJCkSqKFFwWUfBtyG1AGGa3XpD99-bMzuzqg_Z5Mkk4J5fv5CTfB7BvY9SWS0-NkjkVMXdpzmWcChF8xvKoQiMGczlSFzfi76287cGoewuD1yq7NbFZqMPE4xn5EQbHJmEFw4-n_yiqRmF2tZPQsK20QvjdUIx9ghWGzFh9WDk5G42vlqcuPAEUJdr0ZqbV0YNARpMUUafAeihTYPZ2g1qizkWidB1Wn-qpnT3bqnq1F51_hc8tiCR_5l7fgF6sN-FLJ9BA2vm6BePAJr-IJQHZcVHYKgaCV0LJxOHxC0mAlSD3d1XFiiBxMZ0u3xEQ39SXOkPuajKeIcnANtycn12fXtBWQoH6FG5KqgPj0VpZstxY7XOdlZyraLiWvvRIzmbVUDiurTIyuFwYZcsy8yw5sBwmLPMN-vWkjjtAmEqWLZ0LkWfCapZKWi1zF6x1voxmAMPOboVv-cVR5qIqmjhDq2Ju6gLvlKGpCzmAg8Uv0zm5xkeF9zpnFO08eyiWo2IAh52DXn1-v7LDhQ__3_Tux01_hzXWjB9DM7kH_cf7p_gjQZVH97Mdfy9nLOSg priority: 102 providerName: ProQuest – databaseName: Springer Nature OA Free Journals dbid: C24 link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JSwMxFA5aD-rBXawbOXiyDHayTeJNiqUISg8WehuyDQjDtNgW6b83STNtBRc8T5IZ8vIm30ve-z4AbqS1XGKqE8FolhCbKedzKU4IMTpFmWUmiME8v7DegDwN6TDWcU_qbPf6SjL8qYNbc3Y3IZ56xIW-LgJuUxdBbYItBz-Q12voxBKHcLCCXUNG4g3mtz2_7kErYLm8C90F27NqLOcfsizXtpvuAdiLOBE-LAx7CDZsdQT2aw0GGF3yGPQNGt1DCY0nwPXaVdZAn_UJR8qfsECHSaGn9y5LW0LPTZyMV6UCUIfx3MfAtwr2555H4AQMuo-vnV4SVRIS7SJKmnCDsJWSFigTkuuMpwXGzArMqS6051-TrE0U5pIJalRGBJNFkWrkbFS0HVw5BY1qVNkzABFz-KlQylicEsmRayk5zZSRUunCiiZo1_OW60gh7pUsyjyEEpzli6nOfdqYn-qcNsHtsst4wZ_xW-PL2hh5dKVJ7k9FhAOJAjdBqzbQ2uOfB2stbfj3q8__NfYF2EFhOYkkpZegMX2f2SsHTqbqOizGT93y2kk priority: 102 providerName: Springer Nature |
| Title | d2o: a distributed data object for parallel high-performance computing in Python |
| URI | https://link.springer.com/article/10.1186/s40537-016-0052-5 https://www.proquest.com/docview/1987922693 https://journalofbigdata.springeropen.com/track/pdf/10.1186/s40537-016-0052-5 |
| UnpaywallVersion | publishedVersion |
| Volume | 3 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2196-1115 dateEnd: 99991231 omitProxy: true ssIdentifier: ssj0001340564 issn: 2196-1115 databaseCode: DOA dateStart: 20140101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2196-1115 dateEnd: 99991231 omitProxy: true ssIdentifier: ssj0001340564 issn: 2196-1115 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: http://www.proquest.com/pqcentral?accountid=15518 eissn: 2196-1115 dateEnd: 20211231 omitProxy: true ssIdentifier: ssj0001340564 issn: 2196-1115 databaseCode: BENPR dateStart: 20141201 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Technology Collection customDbUrl: eissn: 2196-1115 dateEnd: 99991231 omitProxy: true ssIdentifier: ssj0001340564 issn: 2196-1115 databaseCode: 8FG dateStart: 20150201 isFulltext: true titleUrlDefault: https://search.proquest.com/technologycollection1 providerName: ProQuest – providerCode: PRVAVX databaseName: HAS SpringerNature Open Access 2022 customDbUrl: eissn: 2196-1115 dateEnd: 99991231 omitProxy: true ssIdentifier: ssj0001340564 issn: 2196-1115 databaseCode: AAJSJ dateStart: 20141201 isFulltext: true titleUrlDefault: https://www.springernature.com providerName: Springer Nature – providerCode: PRVAVX databaseName: Springer Nature Link customDbUrl: eissn: 2196-1115 dateEnd: 99991231 omitProxy: true ssIdentifier: ssj0001340564 issn: 2196-1115 databaseCode: C6C dateStart: 20141201 isFulltext: true titleUrlDefault: http://www.springeropen.com/ providerName: Springer Nature – providerCode: PRVAVX databaseName: Springer Nature OA Free Journals customDbUrl: eissn: 2196-1115 dateEnd: 99991231 omitProxy: true ssIdentifier: ssj0001340564 issn: 2196-1115 databaseCode: C24 dateStart: 20141201 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1JaxsxFH4k9qHtoelKnaZGh54a5Hi0jaY318QNhhjT1pCeBq0hZBibxKYkv77SLE5aulDoZQZmpNFI7z3pe1q-B_BWOScV5QZngqeYuVQHm0soZsyahKRO2CoYzOlMnCzY9Iyf7cBpexamacGl1xfncY_koF2ejMGkqp57faXM5dHK-trgpTi6ZpGUJDjFwTce8uBb7UJX8ADNO9BdzOajrzHAXNA0HOyaNyubv8z349h0Bzi3a6SP4MGmXKmbb6oo7g1Dkz0o2wrUu08uB5u1Hpjbn7gd_1sNn8DjBrCiUa1hT2HHlc9grw0GgZq-4TnMLVm-RwrZyMQbg2g5i2LBaKnjVA8K4BhFnvGicAWKJMl4dXdmAZnqe-EH0UWJ5jeR0OAFLCbHX8YnuAnXgE1wbTmWllCnFPckzZQ0qUw8pcJlVHLjTSSCU2LINJVKZNzqlGVCeZ8YEpTFDwNuegmdclm6V4CICEDOa20dTZiSJKRUkqfaKqWNd1kPhq2gctNwmceQGkVe-TRS5HWL5XH_WmyxnPfg3TbLqiby-FPig1b6eSOv6zxOz2QBrWa0B4et0O69_v3HDrdK8_ei9_8p9Wt4SCqtyHDCD6Czvtq4NwElrXUfduXkYx-6o9H08zTcPxzP5p_C0zFh8SrG_Wr-od_YyHe65xGZ |
| linkProvider | Unpaywall |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LT9wwEB4hONAeoE91gbY-tJeiiI1fsSuhqg_QUmC1qkDilvoVqVKUXVgQ2j_X31ZP1tmlh9IT5zh2MjO25_Pj-wDemRCUYcJlWooi46Gwsc_lLOPcu5wWQfpWDOZ0KAfn_PuFuFiB391dGDxW2Y2J7UDtxw7XyPcQHOuYK2j2aXKZoWoU7q52EhomSSv4_ZZiLF3sOA6z2wjhpvtH36K_31N6eHD2dZAllYHMRUQmMuUpC8aIihbaKFeovGJMBs2UcJVD_jIj-9wyZaQW3hZcS1NVuaPxH6u-QjKmOAWsccZ1BH9rXw6Gox_LVR4WEyLJ03ZqruTelCODSkTwEcj3RQSCf0-Iyyx3sTH7GNZvmomZ3Zq6vjP3HT6BjZS0ks_zKHsKK6F5BpudIARJ48NzGHk6_kgM8cjGi0JawRM8gkrGFpd7SEyQCXKN13WoCRIlZ5PlvQXi2vrix5BfDRnNkNTgBZw_iDFfwmozbsIrIFRGT1bW-sBybhSNJY0ShfXGWFcF3YN-Z7fSJT5zlNWoyxbXKFnOTV3iGTY0dSl68GHxymRO5nFf4Z3OGWXq19NyGYU92O0cdOfxvyvbXfjw_01v3d_0W1gfnJ2elCdHw-NteETbWNJZLnZg9frqJryOadK1fZNikcDPhw7_P8PNIMs |
| linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JSwMxFH64gMvBXaxrDp6Uwc5kmcSbVIs7PSh4G7KCMEyLrYj_3mSWVsEFz_OSDHl5vO9l-T6AQ2ktl5jqSDCaRsSmysdcjCNCjI6T1DJTisHc3bPLR3L9RJ9qndNhc9u9OZKs3jQElqZidDIwrgpxzk6GJNCQ-DLYV8Nt6qupaZglPrkFCYMO60w2WbA3ZKQ-zfy25dd8NAGZ43PRRZh_LQby_U3m-afU012BpRozorPKyaswZYs1WG70GFAdnuvQM0n_FElkAhlu0LGyBoUboKivwm4L8vgUBarvPLc5CjzF0WDybADpsj__M-i5QL33wCmwAY_di4fOZVQrJkTaV5c04ibBVkrqklRIrlMeO4yZFZhT7XTgYpOsTRTmkglqVEoEk87FOvH-cm0PXTZhpugXdgtQwjyWckoZi2MieeItJaepMlIq7axoQbuZt0zXdOJB1SLPyrKCs6ya6ixcIQtTndEWHI2bDCoujd-MdxtnZHVYDbOwQyI8YBS4BceNgz59_rmz47EP_x56-199H8Bc77yb3V7d3-zAQlKuLBHFdBdmRi-vds9jlpHaL9flB2M74Zo |
| linkToUnpaywall | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1JTxsxFH5Kw4FyKG2halpAPvQEcpgZjz12bwgRRUigHBopPY28IsRoEsFECH499ixhES2q1PN4Gfs929_z8n0AP6S1XBKqsWA0w6nNlB9zMcFpanScZJaZWgzm7JyNp-npjM56cNa9hWl7cO7U5UW4IznsjieDmFQ9c1fXUl8dLoxrBjxnhzdpICXxQbGPjSPqY6t3sMaoh-Z9WJueT45-B4E572nYj2vanmy-mu_52vQIOFdnpBuwviwX8u5WFsWTZWi0CWXXgOb2ydVwWamhvn_B7fjfWvgRPrSAFR01HvYJerb8DJudGARq54YtmJhk_hNJZAITbxDRsgaFitFcha0e5MExCjzjRWELFEiS8eLxzQLSdXn-B9FliSZ3gdBgG6ajk1_HY9zKNWDtQ1uKuUmIlZK6JBOS64zHjhBmBeFUOx2I4CSLUkW4ZIIalaWCSedinXhncZHHTV-gX85L-xVQwjyQc0oZS-JU8sSnlJxmykiptLNiAFFnqFy3XOZBUqPI65iGs7zpsTzcXws9ltMB7K-yLBoij78l3umsn7f2usnD9ozwaFWQARx0Rnvy-c-FHayc5u2qv_1T6u_wPqm9QuCY7kC_ul7aXY-SKrXX-v4DjbcK8Q |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=d2o%3A+a+distributed+data+object+for+parallel+high-performance+computing+in+Python&rft.jtitle=Journal+of+big+data&rft.au=Steininger%2C+Theo&rft.au=Greiner%2C+Maksim&rft.au=Beaujean%2C+Frederik&rft.au=En%C3%9Flin%2C+Torsten&rft.date=2016-09-15&rft.pub=Springer+Nature+B.V&rft.eissn=2196-1115&rft.volume=3&rft.issue=1&rft.spage=1&rft.epage=34&rft_id=info:doi/10.1186%2Fs40537-016-0052-5 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2196-1115&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2196-1115&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2196-1115&client=summon |