Sub-100 $\mu$W Multispectral Riemannian Classification for EEG-Based Brain–Machine Interfaces

Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on en...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on biomedical circuits and systems Vol. 15; no. 6; pp. 1149 - 1160
Main Authors Wang, Xiaying, Cavigelli, Lukas, Schneider, Tibor, Benini, Luca
Format Journal Article
LanguageEnglish
Published United States 01.12.2021
Subjects
Online AccessGet full text
ISSN1932-4545
1940-9990
1940-9990
DOI10.1109/TBCAS.2021.3137290

Cover

Abstract Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification.
AbstractList Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification.
Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification.Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification.
Author Wang, Xiaying
Cavigelli, Lukas
Schneider, Tibor
Benini, Luca
Author_xml – sequence: 1
  givenname: Xiaying
  orcidid: 0000-0003-3467-5033
  surname: Wang
  fullname: Wang, Xiaying
– sequence: 2
  givenname: Lukas
  orcidid: 0000-0003-1767-7715
  surname: Cavigelli
  fullname: Cavigelli, Lukas
– sequence: 3
  givenname: Tibor
  surname: Schneider
  fullname: Schneider, Tibor
– sequence: 4
  givenname: Luca
  orcidid: 0000-0001-8068-3806
  surname: Benini
  fullname: Benini, Luca
BackLink https://www.ncbi.nlm.nih.gov/pubmed/34932486$$D View this record in MEDLINE/PubMed
BookMark eNp9kL1OwzAURi0EgrbwAgwoQweWFP828dhWpVQqQoIiFiTLcW6EUeIUOxnYeAfekCchhcLAwHTv1T3fN5w-2ne1A4ROCR4RguXFejqb3I0opmTECEuoxHuoRyTHsZQS7293RmMuuDhC_RCeMRZjKukhOmK8-_B03EPqrs1ignE0fKza4UN03ZaNDRswjddldGuh0s5Z7aJZqUOwhTW6sbWLitpH8_kinuoAeTT12rqPt_drbZ6sg2jpGvCFNhCO0UGhywAnuzlA95fz9ewqXt0slrPJKjaMJ03MMpamnIlckyxhjILJC8JFmhQCKBUkl8CNEQQylhPdHZBwIzkr8gwLagQboPPv3o2vX1oIjapsMFCW2kHdBkXHhHbFnKcderZD26yCXG28rbR_VT9SOoB-A8bXIXgofhGC1da8-jKvtubVznwXSv-EjG2-XHUmbflf9BMmVYdo
CitedBy_id crossref_primary_10_1109_JSEN_2024_3353146
crossref_primary_10_1109_TBCAS_2022_3215962
Cites_doi 10.1109/EMBC.2018.8512520
10.1145/567806.567809
10.1109/TAES.2018.2799758
10.23919/EUSIPCO.2018.8553378
10.1109/ASAP.2018.8445101
10.1109/MCI.2018.2840738
10.1109/TNSRE.2017.2757519
10.1109/SMARTCOMP50058.2020.00065
10.1002/tee.22742
10.1109/ACCESS.2020.3011969
10.1002/hbm.23730
10.1109/IJCNN.2017.7965927
10.1109/SMC42975.2020.9283028
10.1109/ISCAS51556.2021.9401564
10.1016/j.neucom.2012.12.039
10.1109/TBCAS.2019.2927551
10.3389/fnhum.2021.595723
10.1109/TNSRE.2016.2627016
10.1109/TKDE.2020.3028705
10.3389/fnins.2017.00400
10.1016/B978-0-444-63934-9.00008-1
10.1038/s41467-018-04673-z
10.1109/JIOT.2020.2976702
10.1109/ISCAS51556.2021.9401427
10.1109/TNSRE.2021.3099908
10.1098/rsta.2019.0155
10.1088/1741-2552/aab2f2
10.1016/j.neucom.2017.10.013
10.1016/B978-0-12-819246-7.00014-0
10.1109/TNNLS.2020.3048385
10.1109/ECAI50035.2020.9223246
10.1109/CVPR.2018.00286
10.1007/s12021-020-09473-9
10.1109/TNSRE.2020.2979464
10.1016/j.jneumeth.2018.04.013
10.1109/JSSC.2019.2912307
10.1109/ICASSP.2013.6639344
10.1109/TBCAS.2020.3004544
10.2172/4412175
10.1007/s11263-015-0816-y
10.1371/journal.pone.0234178
10.1137/0110014
10.1109/MeMeA49120.2020.9137134
10.1109/BIBM49941.2020.9313336
10.1109/IWW-BCI.2019.8737349
10.3389/fnins.2019.01275
10.1088/1741-2552/aace8c
10.1080/2326263X.2017.1297192
ContentType Journal Article
DBID AAYXX
CITATION
CGR
CUY
CVF
ECM
EIF
NPM
7X8
DOI 10.1109/TBCAS.2021.3137290
DatabaseName CrossRef
Medline
MEDLINE
MEDLINE (Ovid)
MEDLINE
MEDLINE
PubMed
MEDLINE - Academic
DatabaseTitle CrossRef
MEDLINE
Medline Complete
MEDLINE with Full Text
PubMed
MEDLINE (Ovid)
MEDLINE - Academic
DatabaseTitleList MEDLINE
MEDLINE - Academic
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: EIF
  name: MEDLINE
  url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search
  sourceTypes: Index Database
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1940-9990
EndPage 1160
ExternalDocumentID 34932486
10_1109_TBCAS_2021_3137290
Genre Research Support, Non-U.S. Gov't
Journal Article
GroupedDBID ---
0R~
29I
4.4
5GY
5VS
6IK
97E
AAJGR
AASAJ
AAWTH
AAYXX
ABAZT
ABQJQ
ABVLG
ACIWK
ACPRK
AENEX
AETIX
AFRAH
AGQYO
AGSQL
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CITATION
CS3
DU5
EBS
EJD
F5P
HZ~
IFIPE
IPLJI
JAVBF
LAI
M43
O9-
OCL
P2P
RIA
RIE
RNS
CGR
CUY
CVF
ECM
EIF
NPM
RIG
7X8
ID FETCH-LOGICAL-c347t-3b388435da1b7332ecdf14587f5e2251d9e4cc51eb3d1a9e4e74c943fdb052c53
ISSN 1932-4545
1940-9990
IngestDate Sun Sep 28 01:09:55 EDT 2025
Thu Jan 02 22:55:50 EST 2025
Wed Oct 01 04:06:01 EDT 2025
Thu Apr 24 22:55:49 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 6
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c347t-3b388435da1b7332ecdf14587f5e2251d9e4cc51eb3d1a9e4e74c943fdb052c53
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ORCID 0000-0003-3467-5033
0000-0001-8068-3806
0000-0003-1767-7715
OpenAccessLink https://hdl.handle.net/11585/963485
PMID 34932486
PQID 2612733448
PQPubID 23479
PageCount 12
ParticipantIDs proquest_miscellaneous_2612733448
pubmed_primary_34932486
crossref_primary_10_1109_TBCAS_2021_3137290
crossref_citationtrail_10_1109_TBCAS_2021_3137290
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2021-12-00
20211201
PublicationDateYYYYMMDD 2021-12-01
PublicationDate_xml – month: 12
  year: 2021
  text: 2021-12-00
PublicationDecade 2020
PublicationPlace United States
PublicationPlace_xml – name: United States
PublicationTitle IEEE transactions on biomedical circuits and systems
PublicationTitleAlternate IEEE Trans Biomed Circuits Syst
PublicationYear 2021
References ref57
ref13
ref12
ref59
ref15
ref58
ref14
ref53
ref11
ref10
ref17
ref19
ref18
valentin (ref6) 2019; 13
cavigelli (ref50) 2020
wilkinson (ref56) 1965; 662
ref45
burden (ref55) 2004
ref48
ref47
ref42
ref41
ref44
ref43
ref49
ref8
ref7
ref9
ref4
ref3
ref5
ref40
ref35
ref34
ref37
ref36
ref31
ref30
ref33
ref32
ref2
ref1
ref39
brunner (ref46) 0
spallanzani (ref51) 2019
ref24
ref23
ref26
ang (ref29) 0
ref25
ref20
ref22
ref21
rossi (ref16) 0; 64
ref28
ref27
wang (ref54) 2019
smith (ref52) 2007
malekmohammadi (ref38) 2019; 26
References_xml – ident: ref3
  doi: 10.1109/EMBC.2018.8512520
– ident: ref58
  doi: 10.1145/567806.567809
– ident: ref22
  doi: 10.1109/TAES.2018.2799758
– volume: 13
  start-page: 103
  year: 2019
  ident: ref6
  article-title: Validation and benchmarking of a wearable EEG acquisition platform for real-world applications
  publication-title: IEEE Trans Biomed Circuits Syst
– ident: ref40
  doi: 10.23919/EUSIPCO.2018.8553378
– ident: ref11
  doi: 10.1109/ASAP.2018.8445101
– ident: ref20
  doi: 10.1109/MCI.2018.2840738
– volume: 662
  year: 1965
  ident: ref56
  publication-title: The Algebraic Eigenvalue Problem
– ident: ref30
  doi: 10.1109/TNSRE.2017.2757519
– ident: ref13
  doi: 10.1109/SMARTCOMP50058.2020.00065
– year: 2019
  ident: ref51
  article-title: Additive noise annealing and approximation properties of quantized neural networks
  publication-title: CoRR
– ident: ref1
  doi: 10.1002/tee.22742
– ident: ref41
  doi: 10.1109/ACCESS.2020.3011969
– volume: 26
  start-page: 72
  year: 2019
  ident: ref38
  article-title: An efficient hardware implementation for a motor imagery brain computer interface system
  publication-title: Scientia Iranica
– ident: ref23
  doi: 10.1002/hbm.23730
– ident: ref18
  doi: 10.1109/IJCNN.2017.7965927
– ident: ref26
  doi: 10.1109/SMC42975.2020.9283028
– ident: ref39
  doi: 10.1109/ISCAS51556.2021.9401564
– ident: ref48
  doi: 10.1016/j.neucom.2012.12.039
– ident: ref15
  doi: 10.1109/TBCAS.2019.2927551
– volume: 64
  start-page: 60
  year: 0
  ident: ref16
  article-title: A 1.3TOPS/W, 32GOPS fully integrated 10-core SoC for IoT end-nodes with 1.7uW cognitive wake-up from MRAM-based state-retentive sleep mode
  publication-title: Proc IEEE Int Solid- State Circuits Conf
– ident: ref32
  doi: 10.3389/fnhum.2021.595723
– ident: ref35
  doi: 10.1109/TNSRE.2016.2627016
– ident: ref21
  doi: 10.1109/TKDE.2020.3028705
– year: 2020
  ident: ref50
  article-title: RPR: Random partition relaxation for training; Binary and ternary weight neural networks
  publication-title: CoRR
– ident: ref4
  doi: 10.3389/fnins.2017.00400
– ident: ref2
  doi: 10.1016/B978-0-444-63934-9.00008-1
– ident: ref5
  doi: 10.1038/s41467-018-04673-z
– ident: ref12
  doi: 10.1109/JIOT.2020.2976702
– ident: ref10
  doi: 10.1109/ISCAS51556.2021.9401427
– ident: ref45
  doi: 10.1109/TNSRE.2021.3099908
– ident: ref59
  doi: 10.1098/rsta.2019.0155
– ident: ref28
  doi: 10.1088/1741-2552/aab2f2
– ident: ref31
  doi: 10.1016/j.neucom.2017.10.013
– ident: ref8
  doi: 10.1016/B978-0-12-819246-7.00014-0
– ident: ref44
  doi: 10.1109/TNNLS.2020.3048385
– ident: ref7
  doi: 10.1109/ECAI50035.2020.9223246
– ident: ref49
  doi: 10.1109/CVPR.2018.00286
– ident: ref36
  doi: 10.1007/s12021-020-09473-9
– ident: ref43
  doi: 10.1109/TNSRE.2020.2979464
– ident: ref37
  doi: 10.1016/j.jneumeth.2018.04.013
– year: 2007
  ident: ref52
  publication-title: Introduction to Digital Filters with Audio Applications
– ident: ref14
  doi: 10.1109/JSSC.2019.2912307
– ident: ref19
  doi: 10.1109/ICASSP.2013.6639344
– ident: ref9
  doi: 10.1109/TBCAS.2020.3004544
– year: 2019
  ident: ref54
  article-title: DSP library for PULP
– start-page: 2390
  year: 0
  ident: ref29
  article-title: Filter bank common spatial pattern (FBCSP) in brain-computer interface
  publication-title: Proc IEEE Int Joint Conf Neural Netw
– ident: ref57
  doi: 10.2172/4412175
– year: 2004
  ident: ref55
  publication-title: Numerical Analysis
– year: 0
  ident: ref46
  article-title: BCI competition 2008 - Graz data set A
– ident: ref17
  doi: 10.1007/s11263-015-0816-y
– ident: ref27
  doi: 10.1371/journal.pone.0234178
– ident: ref53
  doi: 10.1137/0110014
– ident: ref47
  doi: 10.1109/MeMeA49120.2020.9137134
– ident: ref42
  doi: 10.1109/BIBM49941.2020.9313336
– ident: ref33
  doi: 10.1109/IWW-BCI.2019.8737349
– ident: ref25
  doi: 10.3389/fnins.2019.01275
– ident: ref24
  doi: 10.1088/1741-2552/aace8c
– ident: ref34
  doi: 10.1080/2326263X.2017.1297192
SSID ssj0056292
Score 2.330497
Snippet Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a...
SourceID proquest
pubmed
crossref
SourceType Aggregation Database
Index Database
Enrichment Source
StartPage 1149
SubjectTerms Algorithms
Brain-Computer Interfaces
Electroencephalography
Imagination
Neural Networks, Computer
Title Sub-100 $\mu$W Multispectral Riemannian Classification for EEG-Based Brain–Machine Interfaces
URI https://www.ncbi.nlm.nih.gov/pubmed/34932486
https://www.proquest.com/docview/2612733448
Volume 15
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE/IET Electronic Library
  customDbUrl:
  eissn: 1940-9990
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0056292
  issn: 1932-4545
  databaseCode: RIE
  dateStart: 20070101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3bbtMwGLbKuIELxJlykpEYN1VKHNs5XK5VxoS2IbFUq7iJYseBCEjHaJDgKXggnoFn4rfjHCoKGtxErVUnqf_P_8H_CaGnoSKCZJQ4gsN2YxFVTpgz7hSSZqGQPM9MkaSjY_9gwV4u-XI0-j6IWqrXYiq_bc0r-R-qwhjQVWfJ_gNlu5vCAHwG-sIVKAzXC9EYdj0wOHeyO493Z_PTicmmNbmTOu3-dak-6o5EsINN60sdFNSHFsbxC2cGIiwH8mZl5RyZqEobSVnoQK2h3qptQt1Oou0tbpwMTep-U2KkPJe19kGYc_hBFXRzVt_wk2WZfW0FpfF6fCnf6nKg5migfp91E07ku0qX37JgEqsugHimqrKyE2yMkT2x8Mgg-qNhshFzHVBMG3-M2jLWcmY-QOCQzYIRF23n_6Z8ajKb751M9aOnlGi_pNtLu9bDf_wq3V8cHqZJvEyenX1ydB8y7a-3TVkuocte4PtekxHYynZQFk2b7e5t2zQsN3r--0M3VZ0_2C9Gj0muo2vWAMF7DZpuoJGqbqKrg7KUt9Abiyv888cp3sAU7jGFNzGFAVO4wxTewBTuMXUbLfbjZH7g2B4cjqQsWDtU0DAElTrPiAgo9ZTMC8J4GBRcgSggeaSYlJwoQXOSwRcVMBkxWuTC5Z7k9A7aqVaVuodwwApKcuoWVBWMRyDdhK6GmPvKozQI2RiRdrlSaQvU6z4pH1JjqLpRapY41Uuc2iUeo0k356wpz_LXXz9pqZACF9WusaxSq_pzqgvpwd9jLByjuw15uvtR4GAeC_37F5j9AF3pEf8Q7azPa_UItNa1eGyA9AtyK5W2
linkProvider IEEE
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Sub-100+%CE%BCW+Multispectral+Riemannian+Classification+for+EEG-Based+Brain-Machine+Interfaces&rft.jtitle=IEEE+transactions+on+biomedical+circuits+and+systems&rft.au=Wang%2C+Xiaying&rft.au=Cavigelli%2C+Lukas&rft.au=Schneider%2C+Tibor&rft.au=Benini%2C+Luca&rft.date=2021-12-01&rft.issn=1940-9990&rft.eissn=1940-9990&rft.volume=15&rft.issue=6&rft.spage=1149&rft_id=info:doi/10.1109%2FTBCAS.2021.3137290&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1932-4545&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1932-4545&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1932-4545&client=summon