Decentralized Approximate Newton Methods for Convex Optimization on Networked Systems
In this article, a class of decentralized approximate Newton (DEAN) methods for addressing convex optimization on a networked system is developed, where nodes in the networked system seek a consensus that minimizes the sum of their individual objective functions through local interactions only. The...
Saved in:
| Published in | IEEE transactions on control of network systems Vol. 8; no. 3; pp. 1489 - 1500 |
|---|---|
| Main Authors | , , , , |
| Format | Journal Article |
| Language | English |
| Published |
Piscataway
IEEE
01.09.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects | |
| Online Access | Get full text |
| ISSN | 2325-5870 2372-2533 |
| DOI | 10.1109/TCNS.2021.3070663 |
Cover
| Abstract | In this article, a class of decentralized approximate Newton (DEAN) methods for addressing convex optimization on a networked system is developed, where nodes in the networked system seek a consensus that minimizes the sum of their individual objective functions through local interactions only. The proposed DEAN algorithms allow each node to repeatedly perform a local approximate Newton update, which leverages tracking the global Newton direction and dissipating the discrepancies among the nodes. Under less restrictive problem assumptions in comparison with most existing second-order methods, the DEAN algorithms enable the nodes to reach a consensus that can be arbitrarily close to the optimum. Moreover, for a particular DEAN algorithm, the nodes linearly converge to a common suboptimal solution with an explicit error bound. Finally, simulations demonstrate the competitive performance of DEAN in convergence speed, accuracy, and efficiency. |
|---|---|
| AbstractList | In this article, a class of decentralized approximate Newton (DEAN) methods for addressing convex optimization on a networked system is developed, where nodes in the networked system seek a consensus that minimizes the sum of their individual objective functions through local interactions only. The proposed DEAN algorithms allow each node to repeatedly perform a local approximate Newton update, which leverages tracking the global Newton direction and dissipating the discrepancies among the nodes. Under less restrictive problem assumptions in comparison with most existing second-order methods, the DEAN algorithms enable the nodes to reach a consensus that can be arbitrarily close to the optimum. Moreover, for a particular DEAN algorithm, the nodes linearly converge to a common suboptimal solution with an explicit error bound. Finally, simulations demonstrate the competitive performance of DEAN in convergence speed, accuracy, and efficiency. |
| Author | Qu, Zhihai Wu, Xuyang Lu, Jie Wei, Hejie Wang, Hao |
| Author_xml | – sequence: 1 givenname: Hejie surname: Wei fullname: Wei, Hejie email: 20190057@lixin.edu.cn organization: School of Statistics and Mathematics, Shanghai Lixin University of Accounting and Finance, Shanghai, China – sequence: 2 givenname: Zhihai orcidid: 0000-0003-4996-4583 surname: Qu fullname: Qu, Zhihai email: quzhh1@shanghaitech.edu.cn organization: College of Electronics and Information Engineering, Tongji University, Shanghai, China – sequence: 3 givenname: Xuyang orcidid: 0000-0001-7409-9611 surname: Wu fullname: Wu, Xuyang email: xuyangw@kth.se organization: Division of Decision and Control Systems, KTH Royal Institute of Technology, Stockholm, Sweden – sequence: 4 givenname: Hao orcidid: 0000-0001-8821-7260 surname: Wang fullname: Wang, Hao email: wanghao1@shanghaitech.edu.cn organization: School of Information Science and Technology, ShanghaiTech University, Shanghai, China – sequence: 5 givenname: Jie orcidid: 0000-0002-4362-6543 surname: Lu fullname: Lu, Jie email: lujie@shanghaitech.edu.cn organization: School of Information Science and Technology, ShanghaiTech University, Shanghai, China |
| BackLink | https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-303362$$DView record from Swedish Publication Index |
| BookMark | eNp9kEFP2zAYhi0EEh30B6BdIu2czvYXx_GxKmObxMoBytVKnC_D0MaZ7a7Ar5-zdjtwQLJkW34e6_3eD-S4dz0ScsHojDGqPt8tlrczTjmbAZW0LOGITDhInnMBcDyeuchFJekpmYbwSCllXKQ7TMjqEg320ddr-4ptNh8G757tpo6YLXEXXZ_9wPjg2pB1zmcL1__G5-xmiHZjX-to03taS4w755-Sf_sSIm7COTnp6nXA6WE_I6urL3eLb_n1zdfvi_l1bgBYzDtTmIbVtVGyNLTpsDRMSZBF2RRoTCux6SpRGV61DCouq0ZVZdtxJUShqBFwRvL9v2GHw7bRg0_R_Yt2tdWX9n6unf-pn-KDBgpQ8sR_2vNpyl9bDFE_uq3vU0TNhQRRqqIYKbmnjHcheOy0sfHvsKknu9aM6rF2Pdaux9r1ofZksjfmv0TvOR_3jkXE_7wCVUih4A-qwJDB |
| CODEN | ITCNAY |
| CitedBy_id | crossref_primary_10_1109_TSP_2024_3514676 crossref_primary_10_1109_TSIPN_2023_3290397 |
| Cites_doi | 10.1109/TSP.2019.2951216 10.1109/TAC.2015.2449811 10.1137/14096668X 10.1109/TAC.2012.2184199 10.1109/ICARCV.2018.8581380 10.1109/TAC.2019.2907711 10.1109/TAC.2011.2167817 10.1109/TCNS.2017.2698261 10.1016/j.automatica.2015.11.014 10.1109/TCNS.2014.2309751 10.1109/TSP.2016.2617829 10.1137/16M1084316 10.1109/TAC.2017.2737582 10.1109/CACSD.2004.1393890 10.1109/TSIPN.2016.2613678 10.1109/TAC.2004.834113 10.1145/984622.984626 10.1109/TAC.2019.2933607 10.1137/130943170 10.1137/15M1038049 10.1016/j.automatica.2018.04.010 10.1109/TAC.2010.2041686 10.1109/TSP.2016.2548989 10.1109/TSP.2017.2666776 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D ADTPV AOWAS D8V |
| DOI | 10.1109/TCNS.2021.3070663 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional SwePub SwePub Articles SWEPUB Kungliga Tekniska Högskolan |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2372-2533 |
| EndPage | 1500 |
| ExternalDocumentID | oai_DiVA_org_kth_303362 10_1109_TCNS_2021_3070663 9394759 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Natural Science Foundation of Shanghai grantid: 16ZR1422500 funderid: 10.13039/100007219 – fundername: National Natural Science Foundation of China grantid: 61603254; 12001367 funderid: 10.13039/501100001809 |
| GroupedDBID | 0R~ 4.4 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF M43 OCL PQQKQ RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D ADTPV AOWAS D8V |
| ID | FETCH-LOGICAL-c331t-fc4cb1aac976c0bfe6c1973746b4eccd7ebf858c28d138278b986df2955490c53 |
| IEDL.DBID | RIE |
| ISSN | 2325-5870 |
| IngestDate | Thu Aug 21 07:31:08 EDT 2025 Mon Jun 30 03:26:51 EDT 2025 Thu Apr 24 23:02:35 EDT 2025 Wed Oct 01 04:44:49 EDT 2025 Wed Aug 27 02:27:19 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c331t-fc4cb1aac976c0bfe6c1973746b4eccd7ebf858c28d138278b986df2955490c53 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-4996-4583 0000-0001-8821-7260 0000-0002-4362-6543 0000-0001-7409-9611 |
| PQID | 2573569442 |
| PQPubID | 2040410 |
| PageCount | 12 |
| ParticipantIDs | proquest_journals_2573569442 crossref_citationtrail_10_1109_TCNS_2021_3070663 crossref_primary_10_1109_TCNS_2021_3070663 swepub_primary_oai_DiVA_org_kth_303362 ieee_primary_9394759 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2021-09-01 |
| PublicationDateYYYYMMDD | 2021-09-01 |
| PublicationDate_xml | – month: 09 year: 2021 text: 2021-09-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE transactions on control of network systems |
| PublicationTitleAbbrev | TCNS |
| PublicationYear | 2021 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 bach (ref3) 2014; 15 ref12 ref15 ref14 ref11 ref10 ref2 ref1 ref17 ref16 zhu (ref5) 2012; 57 ref19 ref18 ref24 ref23 ref25 ref20 ref22 ref21 ref8 ref7 ref9 ref4 ref6 |
| References_xml | – ident: ref15 doi: 10.1109/TSP.2019.2951216 – ident: ref21 doi: 10.1109/TAC.2015.2449811 – ident: ref6 doi: 10.1137/14096668X – ident: ref22 doi: 10.1109/TAC.2012.2184199 – ident: ref23 doi: 10.1109/ICARCV.2018.8581380 – ident: ref16 doi: 10.1109/TAC.2019.2907711 – volume: 57 start-page: 151 year: 2012 ident: ref5 article-title: On distributed convex optimization under inequality and equality constraints publication-title: IEEE Trans Autom Control doi: 10.1109/TAC.2011.2167817 – ident: ref9 doi: 10.1109/TCNS.2017.2698261 – ident: ref7 doi: 10.1016/j.automatica.2015.11.014 – ident: ref2 doi: 10.1109/TCNS.2014.2309751 – volume: 15 start-page: 595 year: 2014 ident: ref3 article-title: Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression publication-title: J Mach Learn Res – ident: ref18 doi: 10.1109/TSP.2016.2617829 – ident: ref10 doi: 10.1137/16M1084316 – ident: ref11 doi: 10.1109/TAC.2017.2737582 – ident: ref25 doi: 10.1109/CACSD.2004.1393890 – ident: ref13 doi: 10.1109/TSIPN.2016.2613678 – ident: ref24 doi: 10.1109/TAC.2004.834113 – ident: ref1 doi: 10.1145/984622.984626 – ident: ref19 doi: 10.1109/TAC.2019.2933607 – ident: ref8 doi: 10.1137/130943170 – ident: ref20 doi: 10.1137/15M1038049 – ident: ref12 doi: 10.1016/j.automatica.2018.04.010 – ident: ref4 doi: 10.1109/TAC.2010.2041686 – ident: ref14 doi: 10.1109/TSP.2016.2548989 – ident: ref17 doi: 10.1109/TSP.2017.2666776 |
| SSID | ssj0001255873 |
| Score | 2.2194169 |
| Snippet | In this article, a class of decentralized approximate Newton (DEAN) methods for addressing convex optimization on a networked system is developed, where nodes... |
| SourceID | swepub proquest crossref ieee |
| SourceType | Open Access Repository Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 1489 |
| SubjectTerms | Algorithms Approximation Approximation algorithms Computational geometry Convergence Convex analysis Convex functions Convexity Distributed optimization Linear programming network optimization Newton method Newton methods Nodes Optimization Radio frequency second-order methods |
| Title | Decentralized Approximate Newton Methods for Convex Optimization on Networked Systems |
| URI | https://ieeexplore.ieee.org/document/9394759 https://www.proquest.com/docview/2573569442 https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-303362 |
| Volume | 8 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2372-2533 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001255873 issn: 2325-5870 databaseCode: RIE dateStart: 20140101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PSxwxGP1QT-2ham3p-oscSg_FWTP5NclxWRUpuD3UFW9hk0nool2LzoL41_tlJruspYgwhzlMQsj7JnlJXt4H8LUO1HvJYiFcJQvBQihMVKbA2NGmZFGoSavyHanzsfhxLa_X4Gh5FyaE0IrPQj-9tmf59Z2fp62yY8NNsqdbh_VKq-6u1sp-ipS64vngsqTm-HI4-oULQFb2U1wrxV9MPW0ulZe0ctUqtJ1ezjbhYtGwTlVy0583ru-f_vFsfGvLt-BD5plk0AXGNqyF2Ud4v-I-uAPjk5ClmdOnUJNBchd_nCKDDQSHPuSE5KJNL_1AkNiSYZKnP5KfOMT8yXc3CT6jTkaO5bP3-ScYn51eDs-LnGWh8JyXTRG98K6cTDwSE09dDMqXpuKVUE4gvnUVXNRSe6br5FdYaWe0qiMzSEQM9ZJ_ho3Z3Sx8ARJlGSmdKB91WudIx5SMkVHPTO3KWvSALgCwPluQp0wYt7ZdilBjE2Y2YWYzZj34vizyt_PfeO3jndT1yw9zr_dgf4GyzX_og8WhiktlhGA9-NYhvyyXLLdPplcDixjam-Y31s9xot_9f_V78C41ohOe7cNGcz8PB8hUGnfYhugz59rnoQ |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PT9swGP3E4LBx2C-GKGObD2iHaSmOfyU-VmWobLQc1k7crNqxtYpREKQS4q_f58StyjRNk3LIIbYsvy_2s_38PoDDylPnJAuZsIXMBPM-00HpDGOn1DkLQk0ble9IDSbi64W82IDPq7sw3vtGfOa78bU5y6-u3SJulR1prqM93RPYkkII2d7WWttRkbIseDq6zKk-GvdH33EJyPJujGyl-KPJp8mm8phYrpuFNhPMyQsYLpvW6kouu4vadt3DH66N_9v2l_A8MU3Sa0PjFWz4-WvYXvMf3IHJsU_izNmDr0gv-ovfz5DDeoKDH7JCMmwSTN8RpLakHwXq9-QcB5mrdHuT4DNqheRYPrmfv4HJyZdxf5ClPAuZ4zyvs-CEs_l06pCaOGqDVy7XBS-EsgIRrgpvQylLx8oqOhYWpdWlqgLTSEU0dZLvwub8eu73gASZB0qnyoUyrnSkZUqGwKhjurJ5JTpAlwAYl0zIYy6MX6ZZjFBtImYmYmYSZh34tCpy0zpw_Ovjndj1qw9Tr3fgYImySf_oncHBikulhWAd-NgivyoXTbePZz96BjE0l_VPrJ_jVL__9-o_wNPBeHhmzk5H397Cs9igVoZ2AJv17cK_Q95S2_dNuP4GKp3q7g |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Decentralized+Approximate+Newton+Methods+for+Convex+Optimization+on+Networked+Systems&rft.jtitle=IEEE+transactions+on+control+of+network+systems&rft.au=Hejie+Wei&rft.au=Qu%2C+Zhihai&rft.au=Wu%2C+Xuyang&rft.au=Wang%2C+Hao&rft.date=2021-09-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.eissn=2372-2533&rft.volume=8&rft.issue=3&rft.spage=1489&rft_id=info:doi/10.1109%2FTCNS.2021.3070663&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2325-5870&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2325-5870&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2325-5870&client=summon |