Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays
High-accuracy, low-latency gaze tracking is becoming one of the indispensable features in augmented reality (AR) head-mounted devices (HMDs). Researchers have proposed different approaches to predict gaze positions from eye images. However, since only the eye modality is focused, these appearance-ba...
Saved in:
Published in | 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) pp. 82 - 91 |
---|---|
Main Authors | , , , , , , , , |
Format | Conference Proceeding |
Language | English |
Published |
IEEE
01.10.2022
|
Subjects | |
Online Access | Get full text |
DOI | 10.1109/ISMAR55827.2022.00022 |
Cover
Abstract | High-accuracy, low-latency gaze tracking is becoming one of the indispensable features in augmented reality (AR) head-mounted devices (HMDs). Researchers have proposed different approaches to predict gaze positions from eye images. However, since only the eye modality is focused, these appearance-based algorithms are still struggle to trade off the accuracy and running speed in HMDs. In this paper, we propose a lightweight multi-modal network (HE-Tracker) to regress gaze positions. By fusing head-movement features with eye features, HE-Tracker achieves comparable accuracy (3.655° in all subjects) and 27 \times speedup (48 fps in the specialized AR HMD) compared to the state-of-the-art gaze tracking algorithm. We further demonstrate that when applying our head-eye coordination strategy to other baseline models, all these models achieve at least 6.36% performance improvement without a pronounced effect on running speed. Moreover, we construct HE-Gaze, the first multi-modal dataset with eye images and head-movement data for near-eye gaze tracking. This dataset is currently made of 757,360 frames and 15 persons, providing an opportunity to foster research in multi-modal gaze tracking approaches. Our dataset is available at DOWNLOAD LINK 1 . |
---|---|
AbstractList | High-accuracy, low-latency gaze tracking is becoming one of the indispensable features in augmented reality (AR) head-mounted devices (HMDs). Researchers have proposed different approaches to predict gaze positions from eye images. However, since only the eye modality is focused, these appearance-based algorithms are still struggle to trade off the accuracy and running speed in HMDs. In this paper, we propose a lightweight multi-modal network (HE-Tracker) to regress gaze positions. By fusing head-movement features with eye features, HE-Tracker achieves comparable accuracy (3.655° in all subjects) and 27 \times speedup (48 fps in the specialized AR HMD) compared to the state-of-the-art gaze tracking algorithm. We further demonstrate that when applying our head-eye coordination strategy to other baseline models, all these models achieve at least 6.36% performance improvement without a pronounced effect on running speed. Moreover, we construct HE-Gaze, the first multi-modal dataset with eye images and head-movement data for near-eye gaze tracking. This dataset is currently made of 757,360 frames and 15 persons, providing an opportunity to foster research in multi-modal gaze tracking approaches. Our dataset is available at DOWNLOAD LINK 1 . |
Author | Xie, Liang Yan, Ye Bai, Xiaowei Yin, Erwei Chen, Lingling Wang, Xiaodong Song, Mingwu Hu, Yongqiang Li, Yingxi |
Author_xml | – sequence: 1 givenname: Lingling surname: Chen fullname: Chen, Lingling organization: Hebei University of Technology,School of Artificial Intelligence and Data Science – sequence: 2 givenname: Yingxi surname: Li fullname: Li, Yingxi organization: Hebei University of Technology,School of Artificial Intelligence and Data Science – sequence: 3 givenname: Xiaowei surname: Bai fullname: Bai, Xiaowei email: xwbai@ipp.ac.cn organization: Defense Innovation Institute,Academy of Military Sciences,Beijing,China – sequence: 4 givenname: Xiaodong surname: Wang fullname: Wang, Xiaodong organization: Tianjin Artificial Intelligence Innovation Center,Tianjin,China – sequence: 5 givenname: Yongqiang surname: Hu fullname: Hu, Yongqiang organization: Tianjin Artificial Intelligence Innovation Center,Tianjin,China – sequence: 6 givenname: Mingwu surname: Song fullname: Song, Mingwu organization: Tianjin Artificial Intelligence Innovation Center,Tianjin,China – sequence: 7 givenname: Liang surname: Xie fullname: Xie, Liang organization: Defense Innovation Institute,Academy of Military Sciences,Beijing,China – sequence: 8 givenname: Ye surname: Yan fullname: Yan, Ye organization: Defense Innovation Institute,Academy of Military Sciences,Beijing,China – sequence: 9 givenname: Erwei surname: Yin fullname: Yin, Erwei organization: Defense Innovation Institute,Academy of Military Sciences,Beijing,China |
BookMark | eNotjMtKw0AUQEfQha1-gQjzA4kzd3InmWWJ2hYiQq3rcpO50cE8ShKR-PUW6uacxYGzEJdd37EQ91rFWiv3sH17We0QM0hjUACxUideiIW2FhM0gOm1KHZMTTSFluWaflnuB6q-Qvchf8L0KTdMPuKZZd73gw8dTaHvZN0P59L2393EXj6G8djQPN6Iq5qakW__vRTvz0_7fBMVr-ttviqiAMpMkUF0nFmoNYL2JgXEhJS1leW0NKUi0mw91pQhJ5ahzJz2AFllMPPekFmKu_M3MPPhOISWhvngnMMkAfMHkp9KTQ |
CODEN | IEEPAD |
ContentType | Conference Proceeding |
DBID | 6IE 6IL CBEJK RIE RIL |
DOI | 10.1109/ISMAR55827.2022.00022 |
DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
EISBN | 1665453257 9781665453257 |
EndPage | 91 |
ExternalDocumentID | 9995442 |
Genre | orig-research |
GrantInformation_xml | – fundername: National Natural Science Foundation of China funderid: 10.13039/501100001809 – fundername: National Science Foundation funderid: 10.13039/100000001 |
GroupedDBID | 6IE 6IL CBEJK RIE RIL |
ID | FETCH-LOGICAL-i203t-3559e862f1521d372554a066c6e7b3b0aa1e6d5fa85e46e2b891d228c358dd3a3 |
IEDL.DBID | RIE |
IngestDate | Thu Jan 18 11:15:05 EST 2024 |
IsPeerReviewed | false |
IsScholarly | true |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-i203t-3559e862f1521d372554a066c6e7b3b0aa1e6d5fa85e46e2b891d228c358dd3a3 |
PageCount | 10 |
ParticipantIDs | ieee_primary_9995442 |
PublicationCentury | 2000 |
PublicationDate | 2022-Oct. |
PublicationDateYYYYMMDD | 2022-10-01 |
PublicationDate_xml | – month: 10 year: 2022 text: 2022-Oct. |
PublicationDecade | 2020 |
PublicationTitle | 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) |
PublicationTitleAbbrev | ISMAR |
PublicationYear | 2022 |
Publisher | IEEE |
Publisher_xml | – name: IEEE |
Score | 2.2890763 |
Snippet | High-accuracy, low-latency gaze tracking is becoming one of the indispensable features in augmented reality (AR) head-mounted devices (HMDs). Researchers have... |
SourceID | ieee |
SourceType | Publisher |
StartPage | 82 |
SubjectTerms | Artificial intelligence Augmented reality Computer vision Computer vision problems Computing methodologies Gaze tracking Head Human computer interaction (HCI) Human-centered computing Interaction paradigms Performance evaluation Prediction algorithms Predictive models Rendering (computer graphics) Resists |
Title | Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays |
URI | https://ieeexplore.ieee.org/document/9995442 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NTwIxEG2Qkyc1YPxODx4t7PZrt0eDEjRiDErCjfRjmhAVCCwH_PW2u4jGePC26R626ST73pvOzEPo0mUy0GThicwtEO40JSYLKkUpJw31DNLSY6n_KHtDfj8Soxq62vbCAEBZfAat-Fje5buZXcVUWVvF4WU8_HB3skxVvVqbppw0Ue275_71QIicZkH20XIOZ7TE_WGaUmJGdw_1v75WlYq8tlaFadmPX4MY_7udfdT87s7DT1vcOUA1mDbQwyAwPhKd4nGs38EBg2zMguOYaMW9EEkCa8CdWRCbkyoDiANfrd68R8MIcPhmspy_6fWyiYbd25dOj2ycEsiEJqwggTQoCNrERzR2LAs6getAJqyEzDCTaJ2CdMLrXACXQE2uUkdpbpnInWOaHaL6dDaFI4QFWB5vXLyRhifW56mPxn2JZtwLL9Jj1IgnMZ5XwzDGm0M4-Xv5FO3GWFTVb2eoXixWcB5QvDAXZfg-AZzZnZ8 |
linkProvider | IEEE |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PT8IwFG4IHvSkBoy_7cGjhW39se5oUALKiEFIuJG1fU2ICkTGAf962w3RGA-etmyHLX1Jv-97fe99CF2bWDiazC0RUgNhJouIip1KSRIjVGQphIXHUtoXnRF7GPNxBd1se2EAoCg-g4a_Lc7yzVyvfKqsmfjhZcxtuDvuIuOyW2vTlhMGSbP7nN4OOJdR7IRfVEzi9Ka4P2xTCtRo76P063tlschLY5Wrhv74NYrxvz90gOrf_Xn4aYs8h6gCsxrqDRznI94rHvsKHuxQSPs8OPapVtxxsSSwBtyaO7k5LXOA2DHW8s2bt4wAg--my8Vrtl7W0ah9P2x1yMYrgUyjgObE0YYEnDqxHo8NjZ1SYJmjE1pArKgKsiwEYbjNJAcmIFIyCU0USU25NIZm9AhVZ_MZHCPMQTN_5mKVUCzQVobWW_cFGWWWWx6eoJpficmiHIcx2SzC6d-Pr9BuZ5j2Jr1u__EM7fm4lLVw56iav6_gwmF6ri6LUH4CFrmg8A |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2022+IEEE+International+Symposium+on+Mixed+and+Augmented+Reality+%28ISMAR%29&rft.atitle=Real-time+Gaze+Tracking+with+Head-eye+Coordination+for+Head-mounted+Displays&rft.au=Chen%2C+Lingling&rft.au=Li%2C+Yingxi&rft.au=Bai%2C+Xiaowei&rft.au=Wang%2C+Xiaodong&rft.date=2022-10-01&rft.pub=IEEE&rft.spage=82&rft.epage=91&rft_id=info:doi/10.1109%2FISMAR55827.2022.00022&rft.externalDocID=9995442 |