Sub-100 $\mu$W Multispectral Riemannian Classification for EEG-Based Brain–Machine Interfaces
Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on en...
Saved in:
Published in | IEEE transactions on biomedical circuits and systems Vol. 15; no. 6; pp. 1149 - 1160 |
---|---|
Main Authors | , , , |
Format | Journal Article |
Language | English |
Published |
United States
01.12.2021
|
Subjects | |
Online Access | Get full text |
ISSN | 1932-4545 1940-9990 1940-9990 |
DOI | 10.1109/TBCAS.2021.3137290 |
Cover
Abstract | Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification. |
---|---|
AbstractList | Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification. Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification.Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification. |
Author | Wang, Xiaying Cavigelli, Lukas Schneider, Tibor Benini, Luca |
Author_xml | – sequence: 1 givenname: Xiaying orcidid: 0000-0003-3467-5033 surname: Wang fullname: Wang, Xiaying – sequence: 2 givenname: Lukas orcidid: 0000-0003-1767-7715 surname: Cavigelli fullname: Cavigelli, Lukas – sequence: 3 givenname: Tibor surname: Schneider fullname: Schneider, Tibor – sequence: 4 givenname: Luca orcidid: 0000-0001-8068-3806 surname: Benini fullname: Benini, Luca |
BackLink | https://www.ncbi.nlm.nih.gov/pubmed/34932486$$D View this record in MEDLINE/PubMed |
BookMark | eNp9kL1OwzAURi0EgrbwAgwoQweWFP828dhWpVQqQoIiFiTLcW6EUeIUOxnYeAfekCchhcLAwHTv1T3fN5w-2ne1A4ROCR4RguXFejqb3I0opmTECEuoxHuoRyTHsZQS7293RmMuuDhC_RCeMRZjKukhOmK8-_B03EPqrs1ignE0fKza4UN03ZaNDRswjddldGuh0s5Z7aJZqUOwhTW6sbWLitpH8_kinuoAeTT12rqPt_drbZ6sg2jpGvCFNhCO0UGhywAnuzlA95fz9ewqXt0slrPJKjaMJ03MMpamnIlckyxhjILJC8JFmhQCKBUkl8CNEQQylhPdHZBwIzkr8gwLagQboPPv3o2vX1oIjapsMFCW2kHdBkXHhHbFnKcderZD26yCXG28rbR_VT9SOoB-A8bXIXgofhGC1da8-jKvtubVznwXSv-EjG2-XHUmbflf9BMmVYdo |
CitedBy_id | crossref_primary_10_1109_JSEN_2024_3353146 crossref_primary_10_1109_TBCAS_2022_3215962 |
Cites_doi | 10.1109/EMBC.2018.8512520 10.1145/567806.567809 10.1109/TAES.2018.2799758 10.23919/EUSIPCO.2018.8553378 10.1109/ASAP.2018.8445101 10.1109/MCI.2018.2840738 10.1109/TNSRE.2017.2757519 10.1109/SMARTCOMP50058.2020.00065 10.1002/tee.22742 10.1109/ACCESS.2020.3011969 10.1002/hbm.23730 10.1109/IJCNN.2017.7965927 10.1109/SMC42975.2020.9283028 10.1109/ISCAS51556.2021.9401564 10.1016/j.neucom.2012.12.039 10.1109/TBCAS.2019.2927551 10.3389/fnhum.2021.595723 10.1109/TNSRE.2016.2627016 10.1109/TKDE.2020.3028705 10.3389/fnins.2017.00400 10.1016/B978-0-444-63934-9.00008-1 10.1038/s41467-018-04673-z 10.1109/JIOT.2020.2976702 10.1109/ISCAS51556.2021.9401427 10.1109/TNSRE.2021.3099908 10.1098/rsta.2019.0155 10.1088/1741-2552/aab2f2 10.1016/j.neucom.2017.10.013 10.1016/B978-0-12-819246-7.00014-0 10.1109/TNNLS.2020.3048385 10.1109/ECAI50035.2020.9223246 10.1109/CVPR.2018.00286 10.1007/s12021-020-09473-9 10.1109/TNSRE.2020.2979464 10.1016/j.jneumeth.2018.04.013 10.1109/JSSC.2019.2912307 10.1109/ICASSP.2013.6639344 10.1109/TBCAS.2020.3004544 10.2172/4412175 10.1007/s11263-015-0816-y 10.1371/journal.pone.0234178 10.1137/0110014 10.1109/MeMeA49120.2020.9137134 10.1109/BIBM49941.2020.9313336 10.1109/IWW-BCI.2019.8737349 10.3389/fnins.2019.01275 10.1088/1741-2552/aace8c 10.1080/2326263X.2017.1297192 |
ContentType | Journal Article |
DBID | AAYXX CITATION CGR CUY CVF ECM EIF NPM 7X8 |
DOI | 10.1109/TBCAS.2021.3137290 |
DatabaseName | CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
DatabaseTitle | CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
DatabaseTitleList | MEDLINE MEDLINE - Academic |
Database_xml | – sequence: 1 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: EIF name: MEDLINE url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search sourceTypes: Index Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Engineering |
EISSN | 1940-9990 |
EndPage | 1160 |
ExternalDocumentID | 34932486 10_1109_TBCAS_2021_3137290 |
Genre | Research Support, Non-U.S. Gov't Journal Article |
GroupedDBID | --- 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AASAJ AAWTH AAYXX ABAZT ABQJQ ABVLG ACIWK ACPRK AENEX AETIX AFRAH AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CITATION CS3 DU5 EBS EJD F5P HZ~ IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS CGR CUY CVF ECM EIF NPM RIG 7X8 |
ID | FETCH-LOGICAL-c347t-3b388435da1b7332ecdf14587f5e2251d9e4cc51eb3d1a9e4e74c943fdb052c53 |
ISSN | 1932-4545 1940-9990 |
IngestDate | Sun Sep 28 01:09:55 EDT 2025 Thu Jan 02 22:55:50 EST 2025 Wed Oct 01 04:06:01 EDT 2025 Thu Apr 24 22:55:49 EDT 2025 |
IsDoiOpenAccess | false |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 6 |
Language | English |
License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
LinkModel | OpenURL |
MergedId | FETCHMERGED-LOGICAL-c347t-3b388435da1b7332ecdf14587f5e2251d9e4cc51eb3d1a9e4e74c943fdb052c53 |
Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
ORCID | 0000-0003-3467-5033 0000-0001-8068-3806 0000-0003-1767-7715 |
OpenAccessLink | https://hdl.handle.net/11585/963485 |
PMID | 34932486 |
PQID | 2612733448 |
PQPubID | 23479 |
PageCount | 12 |
ParticipantIDs | proquest_miscellaneous_2612733448 pubmed_primary_34932486 crossref_primary_10_1109_TBCAS_2021_3137290 crossref_citationtrail_10_1109_TBCAS_2021_3137290 |
ProviderPackageCode | CITATION AAYXX |
PublicationCentury | 2000 |
PublicationDate | 2021-12-00 20211201 |
PublicationDateYYYYMMDD | 2021-12-01 |
PublicationDate_xml | – month: 12 year: 2021 text: 2021-12-00 |
PublicationDecade | 2020 |
PublicationPlace | United States |
PublicationPlace_xml | – name: United States |
PublicationTitle | IEEE transactions on biomedical circuits and systems |
PublicationTitleAlternate | IEEE Trans Biomed Circuits Syst |
PublicationYear | 2021 |
References | ref57 ref13 ref12 ref59 ref15 ref58 ref14 ref53 ref11 ref10 ref17 ref19 ref18 valentin (ref6) 2019; 13 cavigelli (ref50) 2020 wilkinson (ref56) 1965; 662 ref45 burden (ref55) 2004 ref48 ref47 ref42 ref41 ref44 ref43 ref49 ref8 ref7 ref9 ref4 ref3 ref5 ref40 ref35 ref34 ref37 ref36 ref31 ref30 ref33 ref32 ref2 ref1 ref39 brunner (ref46) 0 spallanzani (ref51) 2019 ref24 ref23 ref26 ang (ref29) 0 ref25 ref20 ref22 ref21 rossi (ref16) 0; 64 ref28 ref27 wang (ref54) 2019 smith (ref52) 2007 malekmohammadi (ref38) 2019; 26 |
References_xml | – ident: ref3 doi: 10.1109/EMBC.2018.8512520 – ident: ref58 doi: 10.1145/567806.567809 – ident: ref22 doi: 10.1109/TAES.2018.2799758 – volume: 13 start-page: 103 year: 2019 ident: ref6 article-title: Validation and benchmarking of a wearable EEG acquisition platform for real-world applications publication-title: IEEE Trans Biomed Circuits Syst – ident: ref40 doi: 10.23919/EUSIPCO.2018.8553378 – ident: ref11 doi: 10.1109/ASAP.2018.8445101 – ident: ref20 doi: 10.1109/MCI.2018.2840738 – volume: 662 year: 1965 ident: ref56 publication-title: The Algebraic Eigenvalue Problem – ident: ref30 doi: 10.1109/TNSRE.2017.2757519 – ident: ref13 doi: 10.1109/SMARTCOMP50058.2020.00065 – year: 2019 ident: ref51 article-title: Additive noise annealing and approximation properties of quantized neural networks publication-title: CoRR – ident: ref1 doi: 10.1002/tee.22742 – ident: ref41 doi: 10.1109/ACCESS.2020.3011969 – volume: 26 start-page: 72 year: 2019 ident: ref38 article-title: An efficient hardware implementation for a motor imagery brain computer interface system publication-title: Scientia Iranica – ident: ref23 doi: 10.1002/hbm.23730 – ident: ref18 doi: 10.1109/IJCNN.2017.7965927 – ident: ref26 doi: 10.1109/SMC42975.2020.9283028 – ident: ref39 doi: 10.1109/ISCAS51556.2021.9401564 – ident: ref48 doi: 10.1016/j.neucom.2012.12.039 – ident: ref15 doi: 10.1109/TBCAS.2019.2927551 – volume: 64 start-page: 60 year: 0 ident: ref16 article-title: A 1.3TOPS/W, 32GOPS fully integrated 10-core SoC for IoT end-nodes with 1.7uW cognitive wake-up from MRAM-based state-retentive sleep mode publication-title: Proc IEEE Int Solid- State Circuits Conf – ident: ref32 doi: 10.3389/fnhum.2021.595723 – ident: ref35 doi: 10.1109/TNSRE.2016.2627016 – ident: ref21 doi: 10.1109/TKDE.2020.3028705 – year: 2020 ident: ref50 article-title: RPR: Random partition relaxation for training; Binary and ternary weight neural networks publication-title: CoRR – ident: ref4 doi: 10.3389/fnins.2017.00400 – ident: ref2 doi: 10.1016/B978-0-444-63934-9.00008-1 – ident: ref5 doi: 10.1038/s41467-018-04673-z – ident: ref12 doi: 10.1109/JIOT.2020.2976702 – ident: ref10 doi: 10.1109/ISCAS51556.2021.9401427 – ident: ref45 doi: 10.1109/TNSRE.2021.3099908 – ident: ref59 doi: 10.1098/rsta.2019.0155 – ident: ref28 doi: 10.1088/1741-2552/aab2f2 – ident: ref31 doi: 10.1016/j.neucom.2017.10.013 – ident: ref8 doi: 10.1016/B978-0-12-819246-7.00014-0 – ident: ref44 doi: 10.1109/TNNLS.2020.3048385 – ident: ref7 doi: 10.1109/ECAI50035.2020.9223246 – ident: ref49 doi: 10.1109/CVPR.2018.00286 – ident: ref36 doi: 10.1007/s12021-020-09473-9 – ident: ref43 doi: 10.1109/TNSRE.2020.2979464 – ident: ref37 doi: 10.1016/j.jneumeth.2018.04.013 – year: 2007 ident: ref52 publication-title: Introduction to Digital Filters with Audio Applications – ident: ref14 doi: 10.1109/JSSC.2019.2912307 – ident: ref19 doi: 10.1109/ICASSP.2013.6639344 – ident: ref9 doi: 10.1109/TBCAS.2020.3004544 – year: 2019 ident: ref54 article-title: DSP library for PULP – start-page: 2390 year: 0 ident: ref29 article-title: Filter bank common spatial pattern (FBCSP) in brain-computer interface publication-title: Proc IEEE Int Joint Conf Neural Netw – ident: ref57 doi: 10.2172/4412175 – year: 2004 ident: ref55 publication-title: Numerical Analysis – year: 0 ident: ref46 article-title: BCI competition 2008 - Graz data set A – ident: ref17 doi: 10.1007/s11263-015-0816-y – ident: ref27 doi: 10.1371/journal.pone.0234178 – ident: ref53 doi: 10.1137/0110014 – ident: ref47 doi: 10.1109/MeMeA49120.2020.9137134 – ident: ref42 doi: 10.1109/BIBM49941.2020.9313336 – ident: ref33 doi: 10.1109/IWW-BCI.2019.8737349 – ident: ref25 doi: 10.3389/fnins.2019.01275 – ident: ref24 doi: 10.1088/1741-2552/aace8c – ident: ref34 doi: 10.1080/2326263X.2017.1297192 |
SSID | ssj0056292 |
Score | 2.330497 |
Snippet | Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a... |
SourceID | proquest pubmed crossref |
SourceType | Aggregation Database Index Database Enrichment Source |
StartPage | 1149 |
SubjectTerms | Algorithms Brain-Computer Interfaces Electroencephalography Imagination Neural Networks, Computer |
Title | Sub-100 $\mu$W Multispectral Riemannian Classification for EEG-Based Brain–Machine Interfaces |
URI | https://www.ncbi.nlm.nih.gov/pubmed/34932486 https://www.proquest.com/docview/2612733448 |
Volume | 15 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE/IET Electronic Library customDbUrl: eissn: 1940-9990 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0056292 issn: 1932-4545 databaseCode: RIE dateStart: 20070101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3bbtMwGLbKuIELxJlykpEYN1VKHNs5XK5VxoS2IbFUq7iJYseBCEjHaJDgKXggnoFn4rfjHCoKGtxErVUnqf_P_8H_CaGnoSKCZJQ4gsN2YxFVTpgz7hSSZqGQPM9MkaSjY_9gwV4u-XI0-j6IWqrXYiq_bc0r-R-qwhjQVWfJ_gNlu5vCAHwG-sIVKAzXC9EYdj0wOHeyO493Z_PTicmmNbmTOu3-dak-6o5EsINN60sdFNSHFsbxC2cGIiwH8mZl5RyZqEobSVnoQK2h3qptQt1Oou0tbpwMTep-U2KkPJe19kGYc_hBFXRzVt_wk2WZfW0FpfF6fCnf6nKg5migfp91E07ku0qX37JgEqsugHimqrKyE2yMkT2x8Mgg-qNhshFzHVBMG3-M2jLWcmY-QOCQzYIRF23n_6Z8ajKb751M9aOnlGi_pNtLu9bDf_wq3V8cHqZJvEyenX1ydB8y7a-3TVkuocte4PtekxHYynZQFk2b7e5t2zQsN3r--0M3VZ0_2C9Gj0muo2vWAMF7DZpuoJGqbqKrg7KUt9Abiyv888cp3sAU7jGFNzGFAVO4wxTewBTuMXUbLfbjZH7g2B4cjqQsWDtU0DAElTrPiAgo9ZTMC8J4GBRcgSggeaSYlJwoQXOSwRcVMBkxWuTC5Z7k9A7aqVaVuodwwApKcuoWVBWMRyDdhK6GmPvKozQI2RiRdrlSaQvU6z4pH1JjqLpRapY41Uuc2iUeo0k356wpz_LXXz9pqZACF9WusaxSq_pzqgvpwd9jLByjuw15uvtR4GAeC_37F5j9AF3pEf8Q7azPa_UItNa1eGyA9AtyK5W2 |
linkProvider | IEEE |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Sub-100+%CE%BCW+Multispectral+Riemannian+Classification+for+EEG-Based+Brain-Machine+Interfaces&rft.jtitle=IEEE+transactions+on+biomedical+circuits+and+systems&rft.au=Wang%2C+Xiaying&rft.au=Cavigelli%2C+Lukas&rft.au=Schneider%2C+Tibor&rft.au=Benini%2C+Luca&rft.date=2021-12-01&rft.issn=1940-9990&rft.eissn=1940-9990&rft.volume=15&rft.issue=6&rft.spage=1149&rft_id=info:doi/10.1109%2FTBCAS.2021.3137290&rft.externalDBID=NO_FULL_TEXT |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1932-4545&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1932-4545&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1932-4545&client=summon |