执行时间预测驱动的工作流作业调度

TP393; 针对工作流作业调度问题,提出使用关键路径法进行工作流的执行时间预测和资源分配.工作流执行时间预测算法使用并行应用有向无环图描述工作流中子作业的执行顺序.基于此顺序,为子作业进行系统资源的逻辑分配.根据子作业的特征和资源分配信息,使用梯度提升决策树进行子作业执行时间预测,并计算工作流的关键路径.关键路径上所有子作业的完成时间之和即为工作流的执行时间.若预测的工作流执行时间满足用户要求,则根据子作业执行顺序和资源分配方案进行作业调度,执行工作流.对比实验表明,两个工作流的执行时间预测误差分别为5.72％和1.57％.与Spark默认调度算法相比,工作流调度算法将两个工作流的完成时间分...

Full description

Saved in:
Bibliographic Details
Published in国防科技大学学报 Vol. 46; no. 5; pp. 228 - 238
Main Authors 胡亚红, 邱圆圆, 毛家发
Format Journal Article
LanguageChinese
Published 浙江工业大学计算机科学与技术学院,浙江杭州 310023 01.10.2024
Subjects
工作流
time prediction
workflow
critical path
关键路径
调度算法
时间预测
梯度提升决策树
scheduling algorithm
gradient boosting decision tree
Online AccessGet full text
ISSN1001-2486
DOI10.11887/j.cn.202405024

Cover

Abstract TP393; 针对工作流作业调度问题,提出使用关键路径法进行工作流的执行时间预测和资源分配.工作流执行时间预测算法使用并行应用有向无环图描述工作流中子作业的执行顺序.基于此顺序,为子作业进行系统资源的逻辑分配.根据子作业的特征和资源分配信息,使用梯度提升决策树进行子作业执行时间预测,并计算工作流的关键路径.关键路径上所有子作业的完成时间之和即为工作流的执行时间.若预测的工作流执行时间满足用户要求,则根据子作业执行顺序和资源分配方案进行作业调度,执行工作流.对比实验表明,两个工作流的执行时间预测误差分别为5.72％和1.57％.与Spark默认调度算法相比,工作流调度算法将两个工作流的完成时间分别缩短了 15.71％和15.44％.
AbstractList TP393; 针对工作流作业调度问题,提出使用关键路径法进行工作流的执行时间预测和资源分配.工作流执行时间预测算法使用并行应用有向无环图描述工作流中子作业的执行顺序.基于此顺序,为子作业进行系统资源的逻辑分配.根据子作业的特征和资源分配信息,使用梯度提升决策树进行子作业执行时间预测,并计算工作流的关键路径.关键路径上所有子作业的完成时间之和即为工作流的执行时间.若预测的工作流执行时间满足用户要求,则根据子作业执行顺序和资源分配方案进行作业调度,执行工作流.对比实验表明,两个工作流的执行时间预测误差分别为5.72％和1.57％.与Spark默认调度算法相比,工作流调度算法将两个工作流的完成时间分别缩短了 15.71％和15.44％.
Abstract_FL For the problem of workflow job scheduling,the critical path method was proposed to predict the execution time of the workflow and allocate resources.The parallel application directed acyclic graph was used to describe the relationships among the sub-jobs of a workflow in the workflow execution time prediction algorithm.Based on this order,the system resources were logically allocated to the sub-jobs.According to the characteristics and resource allocation information of sub-jobs,the gradient boosting decision tree-based algorithm was used to predict the execution time of sub-jobs,and the critical path of workflow was calculated.The sum of the completion time of all sub-jobs on the critical path is the execution time of the workflow.If the predicted workflow execution time satisfies the user's requirements,job scheduling was executed according to the sub-job execution sequence and resource allocation scheme,and the workflow was executed.Comparative experiments show that the prediction errors of the execution time of two workflows are 5.72％and 1.57％,respectively.Compared with the default scheduling algorithm of Spark,the workflow scheduling algorithm reduces the completion time of the two workflows by 15.71％and 15.44％,respectively.
Author 胡亚红
毛家发
邱圆圆
AuthorAffiliation 浙江工业大学计算机科学与技术学院,浙江杭州 310023
AuthorAffiliation_xml – name: 浙江工业大学计算机科学与技术学院,浙江杭州 310023
Author_FL MAO Jiafa
HU Yahong
QIU Yuanyuan
Author_FL_xml – sequence: 1
  fullname: HU Yahong
– sequence: 2
  fullname: QIU Yuanyuan
– sequence: 3
  fullname: MAO Jiafa
Author_xml – sequence: 1
  fullname: 胡亚红
– sequence: 2
  fullname: 邱圆圆
– sequence: 3
  fullname: 毛家发
BookMark eNo9j7tKA0EYhaeIYIypfQaLXeef-5YSvEHARuswOzMbXGUCLmJaMY2XPhDSKJhop2BUBN8mu2bfwgVFOHyn-w5nBdV8zzuE1gCHAErJjTQ0PiSYMMwr1FAdMIaAMCWWUTPLjmJMKAgJEuoIF1fTxd1tMXwrh6_l_aCY3ZRPL_n14_dokL8_zL_Gxeyi4vxjtHi-zD8nq2gp0SeZa_51Ax1ubx20doP2_s5ea7MdZIAZCwzXUjAXxdZZbpUkThHNDWHEgZMsoZgzLSIqI14FqObCWIGFJpLFwA1toPVf77n2ifbdTto7O_XVYqebHKe234__H9If80ZYQg
ClassificationCodes TP393
ContentType Journal Article
Copyright Copyright © Wanfang Data Co. Ltd. All Rights Reserved.
Copyright_xml – notice: Copyright © Wanfang Data Co. Ltd. All Rights Reserved.
DBID 2B.
4A8
92I
93N
PSX
TCJ
DOI 10.11887/j.cn.202405024
DatabaseName Wanfang Data Journals - Hong Kong
WANFANG Data Centre
Wanfang Data Journals
万方数据期刊 - 香港版
China Online Journals (COJ)
China Online Journals (COJ)
DatabaseTitleList
DeliveryMethod fulltext_linktorsrc
Discipline Military & Naval Science
DocumentTitle_FL Workflows scheduling powered by execution time prediction model
EndPage 238
ExternalDocumentID gfkjdxxb202405024
GrantInformation_xml – fundername: 国家重点研发计划
  funderid: (2018YFB0204003)
GroupedDBID -03
2B.
4A8
5XA
5XD
92H
92I
93N
ABJNI
ACGFS
ALMA_UNASSIGNED_HOLDINGS
CCEZO
CEKLB
CW9
PSX
TCJ
TGT
TN5
U1G
U5M
ID FETCH-LOGICAL-s1044-c5a764e9bded5d872e82a5c242e1e74f3054a69379579513a56cd606a274b15c3
ISSN 1001-2486
IngestDate Thu May 29 04:04:51 EDT 2025
IsPeerReviewed false
IsScholarly true
Issue 5
Keywords 工作流
time prediction
workflow
critical path
关键路径
调度算法
时间预测
梯度提升决策树
scheduling algorithm
gradient boosting decision tree
Language Chinese
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-s1044-c5a764e9bded5d872e82a5c242e1e74f3054a69379579513a56cd606a274b15c3
PageCount 11
ParticipantIDs wanfang_journals_gfkjdxxb202405024
PublicationCentury 2000
PublicationDate 2024-10-01
PublicationDateYYYYMMDD 2024-10-01
PublicationDate_xml – month: 10
  year: 2024
  text: 2024-10-01
  day: 01
PublicationDecade 2020
PublicationTitle 国防科技大学学报
PublicationTitle_FL Journal of National University of Defense Technology
PublicationYear 2024
Publisher 浙江工业大学计算机科学与技术学院,浙江杭州 310023
Publisher_xml – name: 浙江工业大学计算机科学与技术学院,浙江杭州 310023
SSID ssib023167171
ssib057620141
ssib051370975
ssib001129263
ssj0000556656
Score 2.429948
Snippet TP393; 针对工作流作业调度问题,提出使用关键路径法进行工作流的执行时间预测和资源分配.工作流执行时间预测算法使用并行应用有向无环图描述工作流中子作业的执行顺序.基...
SourceID wanfang
SourceType Aggregation Database
StartPage 228
Title 执行时间预测驱动的工作流作业调度
URI https://d.wanfangdata.com.cn/periodical/gfkjdxxb202405024
Volume 46
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVEBS
  databaseName: Inspec with Full Text
  issn: 1001-2486
  databaseCode: ADMLS
  dateStart: 20181028
  customDbUrl:
  isFulltext: true
  dateEnd: 99991231
  titleUrlDefault: https://www.ebsco.com/products/research-databases/inspec-full-text
  omitProxy: false
  ssIdentifier: ssib057620141
  providerName: EBSCOhost
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1Lb9QwELb6uHBBPFWeWiHMBQUSrx3bR2c3qwrRXmil3qq8thSkRWq3UukN0QsPca1UcQGJ1w0kCgiJf9Mu7b9gxnE3Ke2hcIlmJxPP55lkZ-zYE0Kua6VznYWZp3kqPZ4o4SWsKLykqdKMdTlWUMfVFtPh5Cy_MyfmRkZf1VYtrfTTW9nakftK_serwAO_4i7Zf_DssFFgAA3-hSN4GI7H8jGNQ6o0NZLGipqAqhZytKRRSGNtCY6EYVRxPBUJqiLL0TQKaAw_DTWKxpJqY2UEjSQ1gsacRm2qW_tXBTUOEArlQWnkU9W0V0E7YT3TRaaO8BJEApIMtQBUHVjYBtdYgIzhFj8QbWihRlgZI_bvCNQGqkwJxFj90qpllYimirmOAVAVDolKJKSmg8BQUWwNBb0DTlCf_2B8uJKuvGOdGbS2REB156CpSnsc7o1CJYhZIgEOQf-0EHZljKGwbUfFNfNYYdOpyWjEAP5kraMggXwbJUtsOr6JL1bKndYu8uDaNsadQVxocrOzi_WX_2WccTvqy5SFlQVyDkdDhRNKEA4zrPQLuZvwyw3rf5UYX-g-fJCvrqZDmVEyziBK-mNk3LSn7t6rEmxID1lVsI1hHYWgGlCIoCl9XSWkAhsJXMJdVtuH4YP9pPKwv67eFmK9fRCp3UfX6ya9hVrKN3OKnHRjtYYpH7zTZGTt_hkyMWXr2i89btxoTCfwNDdceDxL_MGzD7tvXg42vu1tfN17uz7YerH36cvO84-_N9d3vr_b_vV6sPUEjts_Nnc_P935-f4cme3EM61Jz32RxFsOfM69TCQy5IVO8yIXuZKsUCwRGaS5RVBI3oXgyZMQMn58-Q3GSESY5aEfJkzyNBBZ8zwZ6z3qFROkIZscTuShSHMYNOk8LXjqZ5xnRZY2oZ0L5Jrr_Lz7x1meP-Sni8cRukROVM_NZTLWX1oprkAm3U-vOvf-AaVekJw
linkProvider EBSCOhost
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%E6%89%A7%E8%A1%8C%E6%97%B6%E9%97%B4%E9%A2%84%E6%B5%8B%E9%A9%B1%E5%8A%A8%E7%9A%84%E5%B7%A5%E4%BD%9C%E6%B5%81%E4%BD%9C%E4%B8%9A%E8%B0%83%E5%BA%A6&rft.jtitle=%E5%9B%BD%E9%98%B2%E7%A7%91%E6%8A%80%E5%A4%A7%E5%AD%A6%E5%AD%A6%E6%8A%A5&rft.au=%E8%83%A1%E4%BA%9A%E7%BA%A2&rft.au=%E9%82%B1%E5%9C%86%E5%9C%86&rft.au=%E6%AF%9B%E5%AE%B6%E5%8F%91&rft.date=2024-10-01&rft.pub=%E6%B5%99%E6%B1%9F%E5%B7%A5%E4%B8%9A%E5%A4%A7%E5%AD%A6%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%A7%91%E5%AD%A6%E4%B8%8E%E6%8A%80%E6%9C%AF%E5%AD%A6%E9%99%A2%2C%E6%B5%99%E6%B1%9F%E6%9D%AD%E5%B7%9E+310023&rft.issn=1001-2486&rft.volume=46&rft.issue=5&rft.spage=228&rft.epage=238&rft_id=info:doi/10.11887%2Fj.cn.202405024&rft.externalDocID=gfkjdxxb202405024
thumbnail_s http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=http%3A%2F%2Fwww.wanfangdata.com.cn%2Fimages%2FPeriodicalImages%2Fgfkjdxxb%2Fgfkjdxxb.jpg