PDAA3C: An A3C-Based Multi-Path Data Scheduling Algorithm
In this letter, a path dynamics assessment asynchronous advantage actor-critic scheduling algorithm (PDAA3C) is proposed to solve the MPTCP scheduling problem by using deep reinforcement learning Actor-Critic framework. The algorithm picks out the optimal transmitting path faster by multi-core async...
Saved in:
| Published in | IEICE Transactions on Information and Systems Vol. E105.D; no. 12; pp. 2127 - 2130 |
|---|---|
| Main Authors | , , , |
| Format | Journal Article |
| Language | English |
| Published |
Tokyo
The Institute of Electronics, Information and Communication Engineers
01.12.2022
Japan Science and Technology Agency |
| Subjects | |
| Online Access | Get full text |
| ISSN | 0916-8532 1745-1361 1745-1361 |
| DOI | 10.1587/transinf.2022EDL8052 |
Cover
| Abstract | In this letter, a path dynamics assessment asynchronous advantage actor-critic scheduling algorithm (PDAA3C) is proposed to solve the MPTCP scheduling problem by using deep reinforcement learning Actor-Critic framework. The algorithm picks out the optimal transmitting path faster by multi-core asynchronous updating and also guarantee the network fairness. Compared with the existing algorithms, the proposed algorithm achieves 8.6% throughput gain over RLDS algorithm, and approaches the theoretic upper bound in the NS3 simulation. |
|---|---|
| AbstractList | In this letter, a path dynamics assessment asynchronous advantage actor-critic scheduling algorithm (PDAA3C) is proposed to solve the MPTCP scheduling problem by using deep reinforcement learning Actor-Critic framework. The algorithm picks out the optimal transmitting path faster by multi-core asynchronous updating and also guarantee the network fairness. Compared with the existing algorithms, the proposed algorithm achieves 8.6% throughput gain over RLDS algorithm, and approaches the theoretic upper bound in the NS3 simulation. |
| ArticleNumber | 2022EDL8052 |
| Author | ZHAN, Ao WANG, Zhengqiang WU, Chengyu LIANG, Teng |
| Author_xml | – sequence: 1 fullname: WU, Chengyu organization: School of Information Science and Engineering, Zhejiang Sci-Tech University – sequence: 1 fullname: WANG, Zhengqiang organization: School of Communication and Information Enginnering, Chongqing University of Posts and Telecommunication – sequence: 1 fullname: LIANG, Teng organization: School of Information Science and Engineering, Zhejiang Sci-Tech University – sequence: 1 fullname: ZHAN, Ao organization: School of Information Science and Engineering, Zhejiang Sci-Tech University |
| BookMark | eNqFkM1OGzEURq0qSA2BN-hipK4n-Gdsz7CbJimtlAokYG1dPHbiyHhS2yOUt-9EAVp10a6uF9_57j0-R5PQB4PQJ4LnhNfyKkcIyQU7p5jS1XJdY04_oCmRFS8JE2SCprghoqw5ox_ReUo7jElNCZ-i5m7ZtmxxXbShGGf5BZLpih-Dz668g7wtlpChuNdb0w3ehU3R-k0fXd4-X6AzCz6Zy9c5Q49fVw-Lb-X69ub7ol2XmguRS8GfqLRdg6HDQKHmvLLQSNZQkLaSVpDGaquxEFgyzkDWoxEw_QS1aDgxbIb4qXcIezi8gPdqH90zxIMiWB391Zu_Ovqbzh_9R-7zidvH_udgUla7fohhPFVRWUlMGoLZmKpOKR37lKKx_y5__dwRu_4L0y5Ddn0Y487_D74_wbuUYWPeN0LMTnvzG1oRzNVSEfr2-qPlPa23EJUJ7BdIRZ4s |
| CitedBy_id | crossref_primary_10_3390_a17040145 |
| Cites_doi | 10.1109/TVT.2020.3047877 10.1109/ICRA.2017.7989385 10.1145/3302505.3312590 10.1109/JSAC.2020.3000365 10.1109/CCWC.2019.8666496 10.1145/2630088.2631977 10.1109/LCN44214.2019.8990831 10.1109/COMST.2016.2586112 10.1145/2534169.2486020 |
| ContentType | Journal Article |
| Copyright | 2022 The Institute of Electronics, Information and Communication Engineers Copyright Japan Science and Technology Agency 2022 |
| Copyright_xml | – notice: 2022 The Institute of Electronics, Information and Communication Engineers – notice: Copyright Japan Science and Technology Agency 2022 |
| DBID | AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D ADTOC UNPAY |
| DOI | 10.1587/transinf.2022EDL8052 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional Unpaywall for CDI: Periodical Content Unpaywall |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| Database_xml | – sequence: 1 dbid: UNPAY name: Unpaywall url: https://proxy.k.utb.cz/login?url=https://unpaywall.org/ sourceTypes: Open Access Repository |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1745-1361 |
| EndPage | 2130 |
| ExternalDocumentID | 10.1587/transinf.2022edl8052 10_1587_transinf_2022EDL8052 article_transinf_E105_D_12_E105_D_2022EDL8052_article_char_en |
| GroupedDBID | -~X 5GY ABJNI ABZEH ACGFS ADNWM AENEX ALMA_UNASSIGNED_HOLDINGS CS3 DU5 EBS EJD F5P ICE JSF JSH KQ8 OK1 P2P RJT RZJ TN5 ZKX AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D 1TH ADTOC AFFNX C1A CKLRP H13 RYL UNPAY VOH ZE2 ZY4 |
| ID | FETCH-LOGICAL-c566t-65b27fd90ad0a2a8554fa97392a7f47f619fcfc06607353a78587a3cba86951e3 |
| IEDL.DBID | UNPAY |
| ISSN | 0916-8532 1745-1361 |
| IngestDate | Wed Oct 01 15:48:38 EDT 2025 Mon Jun 30 03:20:44 EDT 2025 Thu Apr 24 23:07:08 EDT 2025 Tue Jul 01 02:54:08 EDT 2025 Wed Sep 03 06:30:44 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 12 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c566t-65b27fd90ad0a2a8554fa97392a7f47f619fcfc06607353a78587a3cba86951e3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| OpenAccessLink | https://proxy.k.utb.cz/login?url=https://www.jstage.jst.go.jp/article/transinf/E105.D/12/E105.D_2022EDL8052/_pdf |
| PQID | 2747019103 |
| PQPubID | 2048497 |
| PageCount | 4 |
| ParticipantIDs | unpaywall_primary_10_1587_transinf_2022edl8052 proquest_journals_2747019103 crossref_primary_10_1587_transinf_2022EDL8052 crossref_citationtrail_10_1587_transinf_2022EDL8052 jstage_primary_article_transinf_E105_D_12_E105_D_2022EDL8052_article_char_en |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2022-12-01 |
| PublicationDateYYYYMMDD | 2022-12-01 |
| PublicationDate_xml | – month: 12 year: 2022 text: 2022-12-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Tokyo |
| PublicationPlace_xml | – name: Tokyo |
| PublicationTitle | IEICE Transactions on Information and Systems |
| PublicationTitleAlternate | IEICE Trans. Inf. & Syst. |
| PublicationYear | 2022 |
| Publisher | The Institute of Electronics, Information and Communication Engineers Japan Science and Technology Agency |
| Publisher_xml | – name: The Institute of Electronics, Information and Communication Engineers – name: Japan Science and Technology Agency |
| References | [10] H. Wu, O. Alay, A. Brunstrom, S. Ferlin, and G. Caso, “Peekaboo: Learning-based multipath scheduling for dynamic heterogeneous environments,” IEEE J. Sel. Areas Commun., vol.38, no.10, pp.2295-2310, 2020. 10.1109/jsac.2020.3000365 [4] H. Shi, Y. Cui, X. Wang, Y. Hu, M. Dai, F. Wang, and K. Zheng, “STMS: Improving MPTCP throughput under heterogeneous networks,” Proc. 2018 USENIX Conference on Usenix Annual Technical Conference, Boston, MA, USA, p.719-730, 2018. [11] B. Liao, G. Zhang, Z. Diao, and G. Xie, “Precise and adaptable: Leveraging deep reinforcement learning for GAP-based multipath scheduler,” IEEE IFIP Networking Conference, Paris, France, pp.154-162, 2020. [6] C. Paasch, S. Ferlin, O. Alay, and O. Bonaventure, “Experimental evaluation of multipath TCP schedulers,” Proc. 2014 ACM SIGCOMM Workshop on Capacity Sharing Workshop, New York, NY, USA, p.27-32, 2014. 10.1145/2630088.2631977 [13] K. Winstein and H. Balakrishnan, “TCP ex machina: Computer-generated congestion control,” Proc. ACM SIGCOMM, New York, NY, USA, vol.43, no.4, pp.123-134, 2013. 10.1145/2534169.2486020 [9] J. Luo, X. Su, and B. Liu, “A reinforcement learning approach for multipath TCP data scheduling,” IEEE 9th Annual Computing and Communication Workshop and Conference, pp.0276-0280, 2019. 10.1109/ccwc.2019.8666496 [3] H. Zhang, Z. Yang, and P. Mohapatra, “Wireless access to ultimate virtual reality 360-degree video: poster abstract,” IoTDI '19: International Conference on Internet-of-Things Design and Implementation, pp.271-272, 2019. 10.1145/3302505.3312590 [7] W. Wei, K. Xue, J. Han, Y. Xing, D.S.L. Wei, and P. Hong, “BBR-based congestion control and packet scheduling for bottleneck fairness considered multipath TCP in heterogeneous wireless networks,” IEEE Trans. Veh. Technol., vol.70, no.1, pp.914-927, 2021. 10.1109/tvt.2020.3047877 [12] V. Mnih, A.P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” International Conference on Machine Learning, pp.1928-1937, PMLR, 2016. [1] M. Handley, O. Bonaventure, C. Raiciu, and A. Ford, “TCP extensions for multipath operation with multiple addresses,” IETF RFC 6824, 2013. 10.17487/rfc6824 [2] L. Ming, A. Lukyanenko, Z. Ou, A. Ylä-Jääski, S. Tarkoma, M. Coudron, and S. Secci, “Multipath transmission for the internet: A survey,” IEEE Communications Surveys & Tutorials, vol.18, no.4, pp.2887-2925, 2016. 10.1109/comst.2016.2586112 [5] R. Lübben and J. Morgenroth, “An odd couple: Loss-based congestion control and minimum RTT scheduling in MPTCP,” IEEE 44th Conference on Local Computer Networks, pp.300-307, 2019. 10.1109/lcn44214.2019.8990831 [8] S. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” 2017 IEEE International Conference on Robotics and Automation (ICRA), pp.3389-3396, 2017. 10.1109/icra.2017.7989385 11 12 13 1 2 3 4 5 6 7 8 9 10 |
| References_xml | – reference: [12] V. Mnih, A.P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” International Conference on Machine Learning, pp.1928-1937, PMLR, 2016. – reference: [6] C. Paasch, S. Ferlin, O. Alay, and O. Bonaventure, “Experimental evaluation of multipath TCP schedulers,” Proc. 2014 ACM SIGCOMM Workshop on Capacity Sharing Workshop, New York, NY, USA, p.27-32, 2014. 10.1145/2630088.2631977 – reference: [10] H. Wu, O. Alay, A. Brunstrom, S. Ferlin, and G. Caso, “Peekaboo: Learning-based multipath scheduling for dynamic heterogeneous environments,” IEEE J. Sel. Areas Commun., vol.38, no.10, pp.2295-2310, 2020. 10.1109/jsac.2020.3000365 – reference: [11] B. Liao, G. Zhang, Z. Diao, and G. Xie, “Precise and adaptable: Leveraging deep reinforcement learning for GAP-based multipath scheduler,” IEEE IFIP Networking Conference, Paris, France, pp.154-162, 2020. – reference: [9] J. Luo, X. Su, and B. Liu, “A reinforcement learning approach for multipath TCP data scheduling,” IEEE 9th Annual Computing and Communication Workshop and Conference, pp.0276-0280, 2019. 10.1109/ccwc.2019.8666496 – reference: [2] L. Ming, A. Lukyanenko, Z. Ou, A. Ylä-Jääski, S. Tarkoma, M. Coudron, and S. Secci, “Multipath transmission for the internet: A survey,” IEEE Communications Surveys & Tutorials, vol.18, no.4, pp.2887-2925, 2016. 10.1109/comst.2016.2586112 – reference: [8] S. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” 2017 IEEE International Conference on Robotics and Automation (ICRA), pp.3389-3396, 2017. 10.1109/icra.2017.7989385 – reference: [5] R. Lübben and J. Morgenroth, “An odd couple: Loss-based congestion control and minimum RTT scheduling in MPTCP,” IEEE 44th Conference on Local Computer Networks, pp.300-307, 2019. 10.1109/lcn44214.2019.8990831 – reference: [13] K. Winstein and H. Balakrishnan, “TCP ex machina: Computer-generated congestion control,” Proc. ACM SIGCOMM, New York, NY, USA, vol.43, no.4, pp.123-134, 2013. 10.1145/2534169.2486020 – reference: [1] M. Handley, O. Bonaventure, C. Raiciu, and A. Ford, “TCP extensions for multipath operation with multiple addresses,” IETF RFC 6824, 2013. 10.17487/rfc6824 – reference: [3] H. Zhang, Z. Yang, and P. Mohapatra, “Wireless access to ultimate virtual reality 360-degree video: poster abstract,” IoTDI '19: International Conference on Internet-of-Things Design and Implementation, pp.271-272, 2019. 10.1145/3302505.3312590 – reference: [4] H. Shi, Y. Cui, X. Wang, Y. Hu, M. Dai, F. Wang, and K. Zheng, “STMS: Improving MPTCP throughput under heterogeneous networks,” Proc. 2018 USENIX Conference on Usenix Annual Technical Conference, Boston, MA, USA, p.719-730, 2018. – reference: [7] W. Wei, K. Xue, J. Han, Y. Xing, D.S.L. Wei, and P. Hong, “BBR-based congestion control and packet scheduling for bottleneck fairness considered multipath TCP in heterogeneous wireless networks,” IEEE Trans. Veh. Technol., vol.70, no.1, pp.914-927, 2021. 10.1109/tvt.2020.3047877 – ident: 7 doi: 10.1109/TVT.2020.3047877 – ident: 8 doi: 10.1109/ICRA.2017.7989385 – ident: 4 – ident: 1 – ident: 12 – ident: 11 – ident: 3 doi: 10.1145/3302505.3312590 – ident: 10 doi: 10.1109/JSAC.2020.3000365 – ident: 9 doi: 10.1109/CCWC.2019.8666496 – ident: 6 doi: 10.1145/2630088.2631977 – ident: 5 doi: 10.1109/LCN44214.2019.8990831 – ident: 2 doi: 10.1109/COMST.2016.2586112 – ident: 13 doi: 10.1145/2534169.2486020 |
| SSID | ssj0018215 |
| Score | 2.3071365 |
| Snippet | In this letter, a path dynamics assessment asynchronous advantage actor-critic scheduling algorithm (PDAA3C) is proposed to solve the MPTCP scheduling problem... |
| SourceID | unpaywall proquest crossref jstage |
| SourceType | Open Access Repository Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 2127 |
| SubjectTerms | Algorithms data scheduler Machine learning MPTCP network fairness Scheduling throughput Upper bounds |
| Title | PDAA3C: An A3C-Based Multi-Path Data Scheduling Algorithm |
| URI | https://www.jstage.jst.go.jp/article/transinf/E105.D/12/E105.D_2022EDL8052/_article/-char/en https://www.proquest.com/docview/2747019103 https://www.jstage.jst.go.jp/article/transinf/E105.D/12/E105.D_2022EDL8052/_pdf |
| UnpaywallVersion | publishedVersion |
| Volume | E105.D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| ispartofPNX | IEICE Transactions on Information and Systems, 2022/12/01, Vol.E105.D(12), pp.2127-2130 |
| journalDatabaseRights | – providerCode: PRVAFT databaseName: Open Access Digital Library customDbUrl: eissn: 1745-1361 dateEnd: 99991231 omitProxy: true ssIdentifier: ssj0018215 issn: 0916-8532 databaseCode: KQ8 dateStart: 20080101 isFulltext: true titleUrlDefault: http://grweb.coalliance.org/oadl/oadl.html providerName: Colorado Alliance of Research Libraries |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3db9QwDLe2GxLbA4PBxMGY-sBrv5LrF29lvWmCMQ6Jk8ZT5KbNja3rVaOnCf56nDUtNx5ASDw1lewosVP75zR2AF57yJOJ0tYv0L8ZOec2Bgna5AwZ4WdWslzvd3w4C0_mk3fnwfkGfOxzYfSxykvCRYtSP5zF0rlsXCNEt9XmmwTvTgkVOJnrM9MSFMKzaXaq6_O7oinUJmyFAYHzEWzNz2bpl7uKe36oR8S6FMnA9nnom2S6II6Gzh3dV1lUuq97zupBN657UPThqm7w-y1W1ZpXOt6Fpp9Pdxjlylm1uSN__Fbq8T9O-DE8MgjWSjvmJ7BR1nuw298OYRljsQc7a6UOn0Iyy9KUH72x0tqip_2WvGdh3aX_2jNCoVaGLRLzBbk-nSFvpdViefO1vbh-BvPj6eejE9tc22BLwoatHQY5i1SReFh4yFCfg1OYRKR8jNQkUhSyKakkYR0yLwHHKCbxI5c5xiHhvZLvw6he1uVzsFSsWBFKTyFFdaqQuUy4pHggLjBEFntj4L1-hDQ1zfXVGpXQsQ11K3oJrgtsDPbA1XQ1Pf5Cf9ppaKA2-vlFrbUiMuGzvrXGPlDrPDoyRmM46BeQMAbjm9CbA4S2fY-PwRkW1Z-HZxbpi39leAnb-q07lHMAo_ZmVb4iaNXmh7D5_lN8aD6an4tvHlQ |
| linkProvider | Unpaywall |
| linkToUnpaywall | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1Lb9QwEB6VLRLlQKFQsVBQDlydh715cQvNVhUqZQ-sVE6W48RbSshGJSsEv56ZxglbDiAkTnGkGcuecWa-cTxjgFe-EunMkPUL6TejEIKpMFUMnSFH_MwrXtB-x7vz6HQ5e3sRXuzA-yEXho5VXiEuWlX0cFdr96r1rBC9jsw3Ct6bIypwcy_gtiUxhOfz_Izq83uyLc0d2I1CBOcT2F2eL7KPNxX3gohGxPsUyZAFIgpsMl2YxGPnLvVVlTX1dctZ3e3HdQuK3ts0rfr-TdX1llc62Yd2mE9_GOWzu-kKV__4rdTjf5zwQ3hgEayT9cyPYKdqDmB_uB3CscbiAO5vlTp8DOkizzJx_NrJGgef7A16z9K5Sf9lC0ShTq46hcyX6PooQ97J6tX6-lN3-eUJLE_mH45Pmb22gWnEhh2LwoLHpkx9VfqKKzoHZ1Qao_JVbGaxwZDNaKMR66B5CYWKExS_ErpQSYR4rxKHMGnWTfUUHJMYXkbaNwqjOlPqQqdCYzyQlCpSPPGnIAb9SG1rmtPVGrWk2Aa7lYMEtwU2BTZytX1Nj7_Qn_UaGqmtfn5Rk1ZkLgM-tLbYR2rKo0NjNIWjYQFJazC-StocQLQd-GIK7rio_jw8u0if_SvDc9ijt_5QzhFMuutN9QKhVVe8tJ_LT9sjHV8 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=PDAA3C%3A+An+A3C-Based+Multi-Path+Data+Scheduling+Algorithm&rft.jtitle=IEICE+transactions+on+information+and+systems&rft.au=LIANG%2C+Teng&rft.au=ZHAN%2C+Ao&rft.au=WU%2C+Chengyu&rft.au=WANG%2C+Zhengqiang&rft.date=2022-12-01&rft.issn=0916-8532&rft.eissn=1745-1361&rft.volume=E105.D&rft.issue=12&rft.spage=2127&rft.epage=2130&rft_id=info:doi/10.1587%2Ftransinf.2022EDL8052&rft.externalDBID=n%2Fa&rft.externalDocID=10_1587_transinf_2022EDL8052 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0916-8532&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0916-8532&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0916-8532&client=summon |