Null Distribution of Volume Under Ordered Three-Class ROC Surface (VUS) With Continuous Measurements

Receiver operating characteristic (ROC) analysis has become an indispensable tool in medical care, with a major application to characterizing the performance of binary diagnostic tests in clinical practice. In many circumstances, however, the diagnostic test has three outcomes, that is, the abnormal...

Full description

Saved in:
Bibliographic Details
Published inIEEE signal processing letters Vol. 25; no. 12; pp. 1855 - 1859
Main Authors Liu, Shun, Sun, Xu, Xu, Weichao, Zhang, Yun, Dai, Jisheng
Format Journal Article
LanguageEnglish
Published New York IEEE 01.12.2018
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Online AccessGet full text
ISSN1070-9908
1558-2361
DOI10.1109/LSP.2018.2877930

Cover

Abstract Receiver operating characteristic (ROC) analysis has become an indispensable tool in medical care, with a major application to characterizing the performance of binary diagnostic tests in clinical practice. In many circumstances, however, the diagnostic test has three outcomes, that is, the abnormalities are two sided. To deal with this scenario, this letter develops a recursive algorithm for computing the exact null distribution of the volume under the ordered three-class ROC surface (VUS) for samples following continuous distributions. Based on the asymptotic normality, an approximately normal distribution with exact mean and variance is also proposed, which is hoped to be useful for large-sample scenarios. Moreover, an efficient rank-based formula, in linearithmic time, is established for nonparametric estimation of VUS. Monte Carlo simulations verify the usefulness of the theoretical and algorithmic findings in this letter.
AbstractList Receiver operating characteristic (ROC) analysis has become an indispensable tool in medical care, with a major application to characterizing the performance of binary diagnostic tests in clinical practice. In many circumstances, however, the diagnostic test has three outcomes, that is, the abnormalities are two sided. To deal with this scenario, this letter develops a recursive algorithm for computing the exact null distribution of the volume under the ordered three-class ROC surface (VUS) for samples following continuous distributions. Based on the asymptotic normality, an approximately normal distribution with exact mean and variance is also proposed, which is hoped to be useful for large-sample scenarios. Moreover, an efficient rank-based formula, in linearithmic time, is established for nonparametric estimation of VUS. Monte Carlo simulations verify the usefulness of the theoretical and algorithmic findings in this letter.
Author Shun Liu
Xu Sun
Weichao Xu
Jisheng Dai
Yun Zhang
Author_xml – sequence: 1
  givenname: Shun
  orcidid: 0000-0002-8751-7509
  surname: Liu
  fullname: Liu, Shun
– sequence: 2
  givenname: Xu
  surname: Sun
  fullname: Sun, Xu
– sequence: 3
  givenname: Weichao
  orcidid: 0000-0001-6516-0927
  surname: Xu
  fullname: Xu, Weichao
– sequence: 4
  givenname: Yun
  surname: Zhang
  fullname: Zhang, Yun
– sequence: 5
  givenname: Jisheng
  orcidid: 0000-0002-0462-4414
  surname: Dai
  fullname: Dai, Jisheng
BookMark eNp9kD1PHDEQhi0EEh9JHymNpTSk2MvM-rxrl2ghJNLBIY4j5cq3OxZGe2vwR8G_Z0-HUqRIMzPF-8xonlN2OPqRGPuCMEME_WOxupuVgGpWqrrWAg7YCUqpilJUeDjNUEOhNahjdhrjMwAoVPKE9bd5GPiliym4TU7Oj9xb_uiHvCW-HnsKfBmmSj1_eApERTOYGPn9suGrHKzpiJ8_rlff-R-Xnnjjx-TG7HPkN2RiDrSlMcVP7MiaIdLnj37G1j-vHppfxWJ5_bu5WBRdqTEVc2OoAltTvbG6R5ISbKfqaeik0aBRkCrNXPawkdgrAOzmIMBSiSB6W4sz9m2_9yX410wxtc8-h3E62ZYoSqhQV2pKVftUF3yMgWzbuWR2r6dg3NAitDuj7WS03RltP4xOIPwDvgS3NeHtf8jXPeKI6G9cSajmAsU7YrSCXQ
CODEN ISPLEM
CitedBy_id crossref_primary_10_1109_ACCESS_2019_2959620
crossref_primary_10_12677_airr_2024_132019
crossref_primary_10_1007_s10651_024_00633_7
crossref_primary_10_1002_sta4_249
crossref_primary_10_1016_j_eswa_2024_123194
crossref_primary_10_1109_TASLP_2023_3313447
crossref_primary_10_1109_ACCESS_2020_3011159
crossref_primary_10_1016_j_dsp_2022_103500
Cites_doi 10.1016/j.jspi.2010.07.004
10.1109/ICMLC.2010.5580746
10.1016/j.jspi.2009.05.043
10.1016/j.csda.2013.07.007
10.1002/sim.1917
10.1016/j.csda.2007.12.001
10.1109/TIT.1954.1057460
10.1006/jmps.1996.0024
10.1214/aoms/1177730030
10.1109/JBHI.2013.2293059
10.1109/LSP.2014.2337313
10.1109/TSP.2007.916127
10.1109/JBHI.2014.2328870
10.1016/j.csda.2013.01.004
10.1109/TSP.2007.899374
10.1109/TIT.1954.1057471
10.1109/JBHI.2014.2353031
10.1016/j.sigpro.2013.05.010
10.1177/0272989X9901900110
10.1109/TSP.2009.2032448
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/LSP.2018.2877930
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005-present
IEEE All-Society Periodicals Package (ASPP) 1998-Present
IEEE Xplorer
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1558-2361
EndPage 1859
ExternalDocumentID 10_1109_LSP_2018_2877930
8506431
Genre orig-research
GrantInformation_xml – fundername: National Natural Science Foundation of China
  grantid: 61771148; U1501251; 61571211
  funderid: 10.13039/501100001809
– fundername: Guangzhou Science and Technology Plan
  grantid: 201607010290
GroupedDBID -~X
.DC
0R~
29I
3EH
4.4
5GY
5VS
6IK
85S
97E
AAJGR
AARMG
AASAJ
AAWTH
AAYJJ
ABAZT
ABFSI
ABQJQ
ABVLG
ACGFO
ACGFS
ACIWK
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
E.L
EBS
EJD
F5P
HZ~
H~9
ICLAB
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
O9-
OCL
P2P
RIA
RIE
RNS
TAE
TN5
VH1
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
RIG
ID FETCH-LOGICAL-c291t-4aae60f7e7bf9d1e550fc871e5c5a90913e82a45d0b51d8001c4030fe2103df73
IEDL.DBID RIE
ISSN 1070-9908
IngestDate Mon Jun 30 10:27:40 EDT 2025
Thu Apr 24 23:06:44 EDT 2025
Wed Oct 01 01:11:35 EDT 2025
Wed Aug 27 02:57:17 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 12
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c291t-4aae60f7e7bf9d1e550fc871e5c5a90913e82a45d0b51d8001c4030fe2103df73
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0001-6516-0927
0000-0002-8751-7509
0000-0002-0462-4414
PQID 2132061968
PQPubID 75747
PageCount 5
ParticipantIDs proquest_journals_2132061968
crossref_citationtrail_10_1109_LSP_2018_2877930
crossref_primary_10_1109_LSP_2018_2877930
ieee_primary_8506431
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2018-12-01
PublicationDateYYYYMMDD 2018-12-01
PublicationDate_xml – month: 12
  year: 2018
  text: 2018-12-01
  day: 01
PublicationDecade 2010
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE signal processing letters
PublicationTitleAbbrev LSP
PublicationYear 2018
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref20
ref11
ref10
ref21
ref2
ref1
ref17
ref16
ref19
ref18
ref8
witte (ref22) 2010
ref7
ref9
ref3
ref6
ref5
gibbons (ref23) 1992
hanley (ref4) 1989; 29
References_xml – ident: ref17
  doi: 10.1016/j.jspi.2010.07.004
– volume: 29
  start-page: 307
  year: 1989
  ident: ref4
  article-title: Receiver operating characteristic (ROC) methodology: The state of the art
  publication-title: Crit Rev Diagnostic Imag
– ident: ref21
  doi: 10.1109/ICMLC.2010.5580746
– ident: ref13
  doi: 10.1016/j.jspi.2009.05.043
– ident: ref15
  doi: 10.1016/j.csda.2013.07.007
– ident: ref12
  doi: 10.1002/sim.1917
– ident: ref14
  doi: 10.1016/j.csda.2007.12.001
– ident: ref1
  doi: 10.1109/TIT.1954.1057460
– ident: ref11
  doi: 10.1006/jmps.1996.0024
– ident: ref3
  doi: 10.1214/aoms/1177730030
– year: 1992
  ident: ref23
  publication-title: Nonparametric Statistical Inference
– ident: ref7
  doi: 10.1109/JBHI.2013.2293059
– ident: ref10
  doi: 10.1109/LSP.2014.2337313
– ident: ref19
  doi: 10.1109/TSP.2007.916127
– ident: ref8
  doi: 10.1109/JBHI.2014.2328870
– ident: ref16
  doi: 10.1016/j.csda.2013.01.004
– ident: ref18
  doi: 10.1109/TSP.2007.899374
– ident: ref2
  doi: 10.1109/TIT.1954.1057471
– ident: ref6
  doi: 10.1109/JBHI.2014.2353031
– year: 2010
  ident: ref22
  publication-title: Statistics
– ident: ref9
  doi: 10.1016/j.sigpro.2013.05.010
– ident: ref5
  doi: 10.1177/0272989X9901900110
– ident: ref20
  doi: 10.1109/TSP.2009.2032448
SSID ssj0008185
Score 2.2853484
Snippet Receiver operating characteristic (ROC) analysis has become an indispensable tool in medical care, with a major application to characterizing the performance...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 1855
SubjectTerms Abnormalities
Algorithms
Approximation algorithms
Computer simulation
Diagnostic systems
Gaussian distribution
Normal distribution
Normality
Null distribution
Signal processing algorithms
three-class ROC surface
Time complexity
volume under the ROC surface (VUS)
Title Null Distribution of Volume Under Ordered Three-Class ROC Surface (VUS) With Continuous Measurements
URI https://ieeexplore.ieee.org/document/8506431
https://www.proquest.com/docview/2132061968
Volume 25
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1558-2361
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0008185
  issn: 1070-9908
  databaseCode: RIE
  dateStart: 19940101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3JTsMwFLRKT3BgK4hCQT5woBJunT0-okJVIdoiutBbFG-iAqWoTS58PbaThlWIWw5OZGlenj32ezMAnHNGHcwpRlLYLlJBESJKuEQuUXSEBMzhgW5O7g_83sS9nXmzCrgse2GEEKb4TLT0o7nL5wuW6aOytlFX003TG0Ho571aZdbVC09eX4iRyrDh-koSk_bd6F7XcIUtxQ5UOOIvS5DxVPmRiM3q0t0B_fW88qKS51aW0hZ7-ybZ-N-J74LtYpsJr_K42AMVkeyDrU_igzXAB4p9wmstnFt4XsGFhFOTraCxQ4LDpbHyhGMFuEDGPxM-DDtwlC1lzAS8mE5GTfg4T5-gVrmaJ9kiW8H-x7Hj6gBMujfjTg8VnguI2cRKkRvHwscyEAGVhFtCERjJFKkSHvNiokVERWjHrscx9SyudpsWc1WeUEhb2OEycA5BNVkk4ghAz5fSCqnHnDh2MVWZQXBLhj7lNgkoD-qgvYYhYoUgufbFeIkMMcEkUsBFGrioAK4OmuUbr7kYxx9jaxqHclwBQR001khHxd-6imzdR66YpB8e__7WCdjU387LWBqgmi4zcao2Iyk9M1H4Dluz2sQ
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PT9swFH5C7DA4jPFLFNjwgcOQcOskdhIfUbeqg7Yg2gK3KP6loU0tapPL_npsJy1sIMQtB0ex9L08-7Pf-z6AYyVFRJQg2OiQYhsUKRZcGUy5pSM8kZFKXHNyfxB3x_T8jt2twOmyF0Zr7YvPdNM9-rt8NZWlOypreXU11zT9gVFKWdWttcy7bumpKgwJtjk2XVxKEt7qDa9cFVfatPzABiT5ZxHyriovUrFfXzob0F_MrCor-d0sC9GUf_8TbXzv1D_Dp3qjic6qyNiEFT3ZgvVn8oPboAaWf6LvTjq3dr1CU4NufL5C3hAJXc68mScaWcg19g6a6PqyjYblzORSo2834-EJur0vfiGnc3U_KaflHPWfDh7nOzDu_Bi1u7h2XcAy5EGBaZ7rmJhEJ8JwFWhLYYy0tEozyXLuZER1GuaUKSJYoOx-M5DUZgqLdUAiZZJoF1Yn04neA8RiY4JUMBnlOSXC5gatApPGQoU8ESppQGsBQyZrSXLnjPEn89SE8MwClzngshq4Bpws33io5DjeGLvtcFiOqyFowOEC6az-X-dZ6DrJLZeM0_3X3zqCj91Rv5f1fg4uDmDNfacqajmE1WJW6i92a1KIrz4iHwH5W94R
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Null+Distribution+of+Volume+Under+Ordered+Three-Class+ROC+Surface+%28VUS%29+With+Continuous+Measurements&rft.jtitle=IEEE+signal+processing+letters&rft.au=Shun+Liu&rft.au=Xu+Sun&rft.au=Weichao+Xu&rft.au=Yun+Zhang&rft.date=2018-12-01&rft.pub=IEEE&rft.issn=1070-9908&rft.volume=25&rft.issue=12&rft.spage=1855&rft.epage=1859&rft_id=info:doi/10.1109%2FLSP.2018.2877930&rft.externalDocID=8506431
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1070-9908&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1070-9908&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1070-9908&client=summon