面向算力网络的跨域数据管理方法
跨域算力网络希望整合多个算力中心的计算和数据资源，但现有的方案对跨域文件和数据管理关注不够。提出了一种轻量级的跨域算力网络数据管理方案：通过文件系统协议转换，接入远程算力中心的并行文件系统存储资源；算力中心内部的存储资源作为一种补充，应对高IOPS应用；通过容器绑定技术，将远程存储挂载并绑定到指定目录。基于该方案的原型系统已经在高校校级计算平台部署运行。实测数据和用户体验显示，该方案能够满足常见高性能计算应用需求。...
Saved in:
| Published in | 大数据 Vol. 10; no. 2; pp. 94 - 108 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | Chinese |
| Published |
人民邮电出版社有限公司
15.03.2024
中国人民大学大型科学仪器共享平台,北京 100083%联旌智能科技(上海)有限公司,上海 200051%华中科技大学网络与计算中心,湖北 武汉 430074 China InfoCom Media Group |
| Subjects | |
| Online Access | Get full text |
| ISSN | 2096-0271 |
| DOI | 10.11959/j.issn.2096-0271.2023068 |
Cover
| Abstract | 跨域算力网络希望整合多个算力中心的计算和数据资源，但现有的方案对跨域文件和数据管理关注不够。提出了一种轻量级的跨域算力网络数据管理方案：通过文件系统协议转换，接入远程算力中心的并行文件系统存储资源；算力中心内部的存储资源作为一种补充，应对高IOPS应用；通过容器绑定技术，将远程存储挂载并绑定到指定目录。基于该方案的原型系统已经在高校校级计算平台部署运行。实测数据和用户体验显示，该方案能够满足常见高性能计算应用需求。 |
|---|---|
| AbstractList | 跨域算力网络希望整合多个算力中心的计算和数据资源，但现有的方案对跨域文件和数据管理关注不够。提出了一种轻量级的跨域算力网络数据管理方案：通过文件系统协议转换，接入远程算力中心的并行文件系统存储资源；算力中心内部的存储资源作为一种补充，应对高IOPS应用；通过容器绑定技术，将远程存储挂载并绑定到指定目录。基于该方案的原型系统已经在高校校级计算平台部署运行。实测数据和用户体验显示，该方案能够满足常见高性能计算应用需求。 TP316; 跨域算力网络希望整合多个算力中心的计算和数据资源,但现有的方案对跨域文件和数据管理关注不够.提出了一种轻量级的跨域算力网络数据管理方案:通过文件系统协议转换,接入远程算力中心的并行文件系统存储资源;算力中心内部的存储资源作为一种补充,应对高IOPS应用;通过容器绑定技术,将远程存储挂载并绑定到指定目录.基于该方案的原型系统已经在高校校级计算平台部署运行.实测数据和用户体验显示,该方案能够满足常见高性能计算应用需求. |
| Abstract_FL | Cross-domain computing power networks wish to integrate computational and data resources from multiple computing centers,but existing methods do not pay enough attention to cross-domain file and data management.In this paper,a lightweight data access scheme for cross-domain computing power networks was proposed:(1)accessing parallel file system storage resources of remote computing centers through file system protocol conversion;(2)local caching as a supplement to cope with high IOPS applications;and(3)mounting remote or local storage to specified directories through container binding technology.The prototype system based on this scheme had been deployed on high-performance computing centers in multiple universities.The measured data and user experience showed that the scheme in this paper could meet the requirements of common high-performance computing applications. |
| Author | 张策 鲁蔚征 戴奇志 |
| AuthorAffiliation | 中国人民大学大型科学仪器共享平台,北京 100083%联旌智能科技(上海)有限公司,上海 200051%华中科技大学网络与计算中心,湖北 武汉 430074 |
| AuthorAffiliation_xml | – name: 中国人民大学大型科学仪器共享平台,北京 100083%联旌智能科技(上海)有限公司,上海 200051%华中科技大学网络与计算中心,湖北 武汉 430074 |
| Author_FL | LU Weizheng DAI Qizhi ZHANG Ce |
| Author_FL_xml | – sequence: 1 fullname: LU Weizheng – sequence: 2 fullname: DAI Qizhi – sequence: 3 fullname: ZHANG Ce |
| Author_xml | – sequence: 1 fullname: 鲁蔚征 organization: 中国人民大学大型科学仪器共享平台,北京100083 – sequence: 2 fullname: 戴奇志 organization: 联旌智能科技(上海)有限公司,上海200051 – sequence: 3 fullname: 张策 organization: 华中科技大学网络与计算中心,湖北武汉430074 |
| BookMark | eNo9kL1KA0EUhaeIYIx5BcHCcuPszOxkppT4FwlYqPUwe2cSNsRd2VHEXiwEISmi4A9BK6sgWIiIj5NdfQvXRKzuved-HA5nAZXiJLYILfu45vsykKvdWuRcXCNYcg-Tul9shGIuSqj8r82jqnNRiClnlAtOy8j7fnia9Af5-GZyeZd_DvKP-_z2_OvteTIaZcOX7Gqcjx_z_kV2_Z69DhfRXFv3nK3-zQo62NzYb2x7rd2tZmOt5UGRRXqEWbAcrABNw3YdAjBEQ933_ZAJyoorMIJhMMZwabgvgYUyEAYMCCwlphXUnPmaRHfVURod6vRMJTpSUyFJO0qnxxH0rLLE2pCHVAaYsjpw0WZAAEPANSFGhoXXyszrVMdtHXdUNzlJ4yK9Mtp1i5IYJhjLgluacaCdU7FzRq3v7Uz_v4CgPxeKef0 |
| ClassificationCodes | TP316 |
| ContentType | Journal Article |
| Copyright | Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| Copyright_xml | – notice: Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| DBID | NSCOK 2B. 4A8 92I 93N PSX TCJ DOA |
| DOI | 10.11959/j.issn.2096-0271.2023068 |
| DatabaseName | 国家哲学社会科学文献中心 (National Center for Philosophy and Social Sciences Documentation) Wanfang Data Journals - Hong Kong WANFANG Data Centre Wanfang Data Journals 万方数据期刊 - 香港版 China Online Journals (COJ) China Online Journals (COJ) DOAJ Directory of Open Access Journals |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Open Access Journal Directory url: https://www.doaj.org/ sourceTypes: Open Website |
| DeliveryMethod | fulltext_linktorsrc |
| DocumentTitle_FL | Cross-domain data management for computing power networks |
| EndPage | 108 |
| ExternalDocumentID | oai_doaj_org_article_e2eeb6b3950347c68f4c2c0c56a22d9b dasj202402009 DSJ2024002008 |
| GroupedDBID | -0I -SI -S~ 2RA 5VR 92M 9D9 9DI AAXDM ALMA_UNASSIGNED_HOLDINGS CAJEI FA0 GROUPED_DOAJ JUIAU NSCOK NTYSC PB1 PB5 Q-- Q-8 R-I RT9 S.. T8Y U1F U5I ~NM ~NO 2B. 4A8 92I 93N AAITT AFUIB CQIGP PSX TCJ |
| ID | FETCH-LOGICAL-c1199-24ece6ce8ca3bf7c5cd2ac7111b4834cd25d840cddd69d619c4b958dcdc809903 |
| IEDL.DBID | DOA |
| ISSN | 2096-0271 |
| IngestDate | Tue Oct 14 18:56:23 EDT 2025 Thu May 29 03:56:13 EDT 2025 Tue Jan 21 20:55:07 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | true |
| Issue | 2 |
| Keywords | 并行文件系统 数据管理 算力网络 异构存储资源 data management parallel file system heterogeneous storage resource computing power network |
| Language | Chinese |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c1199-24ece6ce8ca3bf7c5cd2ac7111b4834cd25d840cddd69d619c4b958dcdc809903 |
| OpenAccessLink | https://doaj.org/article/e2eeb6b3950347c68f4c2c0c56a22d9b |
| PageCount | 15 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_e2eeb6b3950347c68f4c2c0c56a22d9b wanfang_journals_dasj202402009 cass_nssd_DSJ2024002008 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-03-15 |
| PublicationDateYYYYMMDD | 2024-03-15 |
| PublicationDate_xml | – month: 03 year: 2024 text: 2024-03-15 day: 15 |
| PublicationDecade | 2020 |
| PublicationTitle | 大数据 |
| PublicationTitle_FL | Big Data Research |
| PublicationYear | 2024 |
| Publisher | 人民邮电出版社有限公司 中国人民大学大型科学仪器共享平台,北京 100083%联旌智能科技(上海)有限公司,上海 200051%华中科技大学网络与计算中心,湖北 武汉 430074 China InfoCom Media Group |
| Publisher_xml | – name: 人民邮电出版社有限公司 – name: 中国人民大学大型科学仪器共享平台,北京 100083%联旌智能科技(上海)有限公司,上海 200051%华中科技大学网络与计算中心,湖北 武汉 430074 – name: China InfoCom Media Group |
| SSID | ssib036436863 ssib024184013 ssib051371281 ssib057785240 ssib035218648 ssj0002857271 ssib058759006 |
| Score | 2.3774107 |
| Snippet | 跨域算力网络希望整合多个算力中心的计算和数据资源，但现有的方案对跨域文件和数据管理关注不够。提出了一种轻量级的跨域算力网络数据管理方案：通过文件系统协议转换，接... TP316; 跨域算力网络希望整合多个算力中心的计算和数据资源,但现有的方案对跨域文件和数据管理关注不够.提出了一种轻量级的跨域算力网络数据管理方案:通过文件系统协议转... |
| SourceID | doaj wanfang cass |
| SourceType | Open Website Aggregation Database |
| StartPage | 94 |
| SubjectTerms | 并行文件系统 异构存储资源 数据管理 算力网络 |
| Title | 面向算力网络的跨域数据管理方法 |
| URI | https://www.ncpssd.cn/Literature/articleinfo?id=DSJ2024002008&type=journalArticle https://d.wanfangdata.com.cn/periodical/dasj202402009 https://doaj.org/article/e2eeb6b3950347c68f4c2c0c56a22d9b |
| Volume | 10 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Open Access Journal Directory issn: 2096-0271 databaseCode: DOA dateStart: 20150101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.doaj.org/ omitProxy: true ssIdentifier: ssj0002857271 providerName: Directory of Open Access Journals – providerCode: PRVEBS databaseName: Academic Search Ultimate issn: 2096-0271 databaseCode: ABDBF dateStart: 20220301 customDbUrl: https://search.ebscohost.com/login.aspx?authtype=ip,shib&custid=s3936755&profile=ehost&defaultdb=asn isFulltext: true dateEnd: 99991231 titleUrlDefault: https://search.ebscohost.com/direct.asp?db=asn omitProxy: true ssIdentifier: ssib035218648 providerName: EBSCOhost – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources issn: 2096-0271 databaseCode: M~E dateStart: 20150101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://road.issn.org omitProxy: true ssIdentifier: ssib058759006 providerName: ISSN International Centre |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV29axRBFB8kAbERxa_z40gg7Sa7sztfZc7kCIHYxEC6ZebNrJJiFS82FlZiIQhJkQSMErSyOgQLkeCfc7f6X_je3HK5zsZm2dmFnXnzZt7HvLe_x9iSAVQCghdJblyg0yqZuAzvbDA65IVxLhab2HokN3aKzV2xO1Pqi3LCJvDAk4lbCTwEJ11uRJoXCqSuCuCQgpCWc28cSd9UmxlnClcSqiU96ziglZFpeWH455KA1y-Ap0SWKwopTdtKaSQgnZ7WcC1Q0ZP3xtOYtauyy2wxShwjzMpe3LHL03fLPJr2GuU8oA3c1gOI_wXVla2fzKiw_jV2tbU9F1YnNF9nl149vcGSP5--jA4Om-HJ6N1p8-uwOf_YfHjz-8fX0dnZ-Ojb-P2wGX5uDt6Oj3-Ovx_dZDv99ccPN5K2hkICODQKngQIEoIGm7tKgQDPLSiUcI6OEbElPE4VeO-l8ehNQeGM0B48aIqZ5bfYXP2sDnfYAsHcoDQIVapcAZnVTiIDrNNWKC9k1WG3idiyHgx8uba9yQk-LaX8ig7rEf3l8wmARkmQ1vEBMrpsGV3-i9Ed1m1nr2y32aD0drAXu6EYz93_0cs9doU-SFlmmbjP5vZfvAwP0OzYd102v9pb6_W7caXhdev1-l99hNmw |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%E9%9D%A2%E5%90%91%E7%AE%97%E5%8A%9B%E7%BD%91%E7%BB%9C%E7%9A%84%E8%B7%A8%E5%9F%9F%E6%95%B0%E6%8D%AE%E7%AE%A1%E7%90%86%E6%96%B9%E6%B3%95&rft.jtitle=%E5%A4%A7%E6%95%B0%E6%8D%AE&rft.au=%E9%B2%81%E8%94%9A%E5%BE%81&rft.au=%E6%88%B4%E5%A5%87%E5%BF%97&rft.au=%E5%BC%A0%E7%AD%96&rft.date=2024-03-15&rft.pub=%E4%BA%BA%E6%B0%91%E9%82%AE%E7%94%B5%E5%87%BA%E7%89%88%E7%A4%BE%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8&rft.issn=2096-0271&rft.volume=10&rft.issue=2&rft.spage=94&rft.epage=108&rft_id=info:doi/10.11959%2Fj.issn.2096-0271.2023068&rft.externalDocID=DSJ2024002008 |
| thumbnail_s | http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=http%3A%2F%2Fwww.wanfangdata.com.cn%2Fimages%2FPeriodicalImages%2Fdasj%2Fdasj.jpg |