Capacity and Random-Coding Exponents for Channel Coding With Side Information
Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with...
Saved in:
| Published in | IEEE transactions on information theory Vol. 53; no. 4; pp. 1326 - 1347 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | English |
| Published |
New York, NY
IEEE
01.04.2007
Institute of Electrical and Electronics Engineers The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects | |
| Online Access | Get full text |
| ISSN | 0018-9448 1557-9654 |
| DOI | 10.1109/TIT.2007.892789 |
Cover
| Abstract | Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally, to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized maximum a posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning |
|---|---|
| AbstractList | Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally, to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized maximum a posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning Capacity formulas and random-coding exponents are derived for a generalized family of Gelafand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally, to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized maximum a posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning. Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally, to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized maximum a posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning. [PUBLICATION ABSTRACT] |
| Author | Ying Wang Moulin, P. |
| Author_xml | – sequence: 1 givenname: P. surname: Moulin fullname: Moulin, P. organization: Dept. of Electr. & Comput. Eng., Illinois Univ., Urbana, IL – sequence: 2 surname: Ying Wang fullname: Ying Wang |
| BackLink | http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=18621882$$DView record in Pascal Francis |
| BookMark | eNp9kc1r3DAQxUVJoZu05x56MYWWXLzRyPoYHYtJ24WUQrKlRyHbUqPglbeSF5r_PtrskkOguYwY3u_NoHmn5CRO0RHyHugSgOqL9Wq9ZJSqJWqmUL8iCxBC1VoKfkIWlALWmnN8Q05zvistF8AW5Edrt7YP831l41BdlzJt6nYaQvxTXf7blhVxzpWfUtXe2hjdWB3F32G-rW7C4KpVLPLGzmGKb8lrb8fs3h3fM_Lr6-W6_V5f_fy2ar9c1X2DbK6dsOC1dB3VSLnSTjONyAf0imnvB-hkB2g5850UAqjzVKAdlESJfadFc0Y-H-Zu0_R35_JsNiH3bhxtdNMum4ZzJSjFAp6_CIJU0DRcMSjox2fo3bRLsXzDgBaaUYm0QJ-OkM29HX2ysQ_ZbFPY2HRvACUDRFY4ceD6NOWcnDflxo8nmpMNowFq9qGZEprZh2YOoRXfxTPf0-j_Oj4cHME590RzaPZi8wDkd6Fc |
| CODEN | IETTAW |
| CitedBy_id | crossref_primary_10_1109_TIT_2015_2400994 crossref_primary_10_1016_j_proeng_2010_11_068 crossref_primary_10_1109_TIT_2012_2201346 crossref_primary_10_1109_TIT_2015_2400999 crossref_primary_10_1109_TCOMM_2014_2319071 crossref_primary_10_1109_TIT_2008_921684 crossref_primary_10_1109_TIT_2016_2572725 crossref_primary_10_1109_TIT_2021_3063905 crossref_primary_10_1109_TIT_2017_2708112 crossref_primary_10_1155_2014_345892 crossref_primary_10_1109_TIFS_2012_2188797 crossref_primary_10_1109_TIT_2017_2753249 crossref_primary_10_1109_TIT_2009_2021383 crossref_primary_10_1109_TIT_2015_2449844 crossref_primary_10_1587_transfun_E96_A_2324 crossref_primary_10_1109_TIT_2010_2070350 crossref_primary_10_1109_TIT_2023_3314210 |
| Cites_doi | 10.1109/18.720535 10.1109/TIT.2004.824920 10.1109/ITW.2004.1405328 10.1109/TIT.1983.1056723 10.1007/978-3-662-03315-9 10.1109/TIT.2003.810622 10.1109/TIT.1983.1056659 10.1109/TIT.1985.1056995 10.1109/TIT.2002.1003844 10.1109/18.481780 10.1109/TIT.2004.834787 10.1109/18.923725 10.1109/TIT.2005.862077 10.1109/18.2598 10.1007/11889342_10 10.1109/18.720546 10.1109/TIT.2002.1003843 10.1109/TIT.1986.1057222 10.1109/TIT.2002.808132 10.1109/TIT.2002.808134 10.1109/18.904523 10.1109/TIT.2003.810639 |
| ContentType | Journal Article |
| Copyright | 2007 INIST-CNRS Copyright Institute of Electrical and Electronics Engineers, Inc. (IEEE) Apr 2007 |
| Copyright_xml | – notice: 2007 INIST-CNRS – notice: Copyright Institute of Electrical and Electronics Engineers, Inc. (IEEE) Apr 2007 |
| DBID | 97E RIA RIE AAYXX CITATION IQODW 7SC 7SP 8FD JQ2 L7M L~C L~D F28 FR3 |
| DOI | 10.1109/TIT.2007.892789 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Pascal-Francis Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional ANTE: Abstracts in New Technology & Engineering Engineering Research Database |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional Engineering Research Database ANTE: Abstracts in New Technology & Engineering |
| DatabaseTitleList | Technology Research Database Technology Research Database Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science Applied Sciences |
| EISSN | 1557-9654 |
| EndPage | 1347 |
| ExternalDocumentID | 1246392001 18621882 10_1109_TIT_2007_892789 4137893 |
| Genre | orig-research Feature |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACGOD ACIWK AENEX AETEA AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 VH1 VJK AAYXX CITATION IQODW RIG 7SC 7SP 8FD JQ2 L7M L~C L~D F28 FR3 |
| ID | FETCH-LOGICAL-c382t-e5a1f96eb0980479e929884d8f729ffd1b6b18a42fb65510ef058ad76868cb953 |
| IEDL.DBID | RIE |
| ISSN | 0018-9448 |
| IngestDate | Sun Sep 28 00:38:37 EDT 2025 Sat Sep 27 23:14:13 EDT 2025 Sun Sep 07 03:49:48 EDT 2025 Mon Jul 21 09:13:53 EDT 2025 Wed Oct 01 03:15:52 EDT 2025 Thu Apr 24 22:57:03 EDT 2025 Tue Aug 26 16:39:36 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Keywords | Alphabet Channel with memory data hiding Error probability Image processing Maximum mutual information Transmitter Random coding capacity Universal coding Codec Arbitrarily varying channels Information protection A posteriori estimation Broadcast channels random binning maximum a posteriori probability (MAP) decoding Data communication Digital watermarking Discrete channel error exponents Two channel system randomized codes Decoding reliability function Channel coding watermarking Random sequence Upper bound Memoryless channel universal coding and decoding Posterior probability Noisy channel channel coding with side information method of types |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html CC BY 4.0 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c382t-e5a1f96eb0980479e929884d8f729ffd1b6b18a42fb65510ef058ad76868cb953 |
| Notes | SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-1 ObjectType-Feature-2 content type line 23 ObjectType-Article-2 |
| PQID | 195920680 |
| PQPubID | 23500 |
| PageCount | 22 |
| ParticipantIDs | proquest_miscellaneous_34475008 crossref_primary_10_1109_TIT_2007_892789 pascalfrancis_primary_18621882 proquest_miscellaneous_1671334721 crossref_citationtrail_10_1109_TIT_2007_892789 ieee_primary_4137893 proquest_journals_195920680 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2007-04-01 |
| PublicationDateYYYYMMDD | 2007-04-01 |
| PublicationDate_xml | – month: 04 year: 2007 text: 2007-04-01 day: 01 |
| PublicationDecade | 2000 |
| PublicationPlace | New York, NY |
| PublicationPlace_xml | – name: New York, NY – name: New York |
| PublicationTitle | IEEE transactions on information theory |
| PublicationTitleAbbrev | TIT |
| PublicationYear | 2007 |
| Publisher | IEEE Institute of Electrical and Electronics Engineers The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: Institute of Electrical and Electronics Engineers – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 ref15 ref14 ref11 ref32 rudolph (ref33) 1997 ref2 ref17 ref19 ref18 wang (ref31) 2006 haroutunian (ref24) 2000; 36 willems (ref10) 2000 somekh-baruch (ref27) 2004 ref23 ref20 ref21 haroutunian (ref25) 2001; 22 ref28 ref29 csiszr (ref16) 1981 ref8 haroutunian (ref22) 1988; 17 ref7 ref9 ref4 ref3 ref6 moulin (ref30) 2004 ref5 haroutunian (ref26) 2004 gel'fand (ref1) 1980; 9 |
| References_xml | – ident: ref18 doi: 10.1109/18.720535 – ident: ref15 doi: 10.1109/TIT.2004.824920 – ident: ref28 doi: 10.1109/ITW.2004.1405328 – ident: ref2 doi: 10.1109/TIT.1983.1056723 – volume: 22 start-page: 44 year: 2001 ident: ref25 article-title: new bounds on e-capacity of arbitrarily varying channel and channels with random parameters publication-title: Trans IIAP NAS RA and YSU (Institute for Informatics and Automation Problems of the Nat Acad Sci Rep Armenia and Yerevan State Univ ) Math Probl Comp Sci – volume: 17 start-page: 99 year: 1988 ident: ref22 article-title: e-capacity upper bound for a channel with random parameter publication-title: Probl Contr Inf Theory – year: 1997 ident: ref33 publication-title: Convergence Properties of Evolutionary Algorithms – ident: ref32 doi: 10.1007/978-3-662-03315-9 – ident: ref9 doi: 10.1109/TIT.2003.810622 – ident: ref3 doi: 10.1109/TIT.1983.1056659 – ident: ref19 doi: 10.1109/TIT.1985.1056995 – ident: ref13 doi: 10.1109/TIT.2002.1003844 – ident: ref20 doi: 10.1109/18.481780 – ident: ref5 doi: 10.1109/TIT.2004.834787 – ident: ref11 doi: 10.1109/18.923725 – start-page: 448 year: 2004 ident: ref27 article-title: on the random coding error exponents of the single-user and the multiple-access gel'fandpinsker channels publication-title: Proc IEEE Int Symp Information Theory – year: 2004 ident: ref30 article-title: new results on steganographic capacity publication-title: Proc Conf Information Science and Systems – year: 2006 ident: ref31 publication-title: Detection- and information-theoretic analysis of steganography and fingerprinting – ident: ref6 doi: 10.1109/TIT.2005.862077 – start-page: 255 year: 2000 ident: ref10 article-title: an information theoretical approach to information embedding publication-title: Proc 21st Symp Information Theory in the Benelux – ident: ref17 doi: 10.1109/18.2598 – ident: ref23 doi: 10.1007/11889342_10 – volume: 9 start-page: 19 year: 1980 ident: ref1 article-title: coding for channel with random parameters publication-title: Probl Contr Inf Theory – ident: ref29 doi: 10.1109/18.720546 – ident: ref7 doi: 10.1109/TIT.2002.1003843 – ident: ref21 doi: 10.1109/TIT.1986.1057222 – ident: ref14 doi: 10.1109/TIT.2002.808132 – ident: ref12 doi: 10.1109/TIT.2002.808134 – ident: ref4 doi: 10.1109/18.904523 – start-page: 536 year: 2004 ident: ref26 article-title: random coding bound of information hiding e-capacity publication-title: Proc IEEE Int Symp Information Theory – ident: ref8 doi: 10.1109/TIT.2003.810639 – year: 1981 ident: ref16 publication-title: Information Theory Coding Theory for Discrete Memoryless Systems – volume: 36 start-page: 140 year: 2000 ident: ref24 article-title: letter to the editor publication-title: Probl Inf Transm |
| SSID | ssj0014512 |
| Score | 2.0298088 |
| Snippet | Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper... Capacity formulas and random-coding exponents are derived for a generalized family of Gelafand-Pinsker coding problems. These exponents yield asymptotic upper... |
| SourceID | proquest pascalfrancis crossref ieee |
| SourceType | Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 1326 |
| SubjectTerms | Alphabets Applied sciences Arbitrarily varying channels Asymptotic properties broadcast channels Broadcasting capacity Channel coding channel coding with side information Channels Coding Coding, codes Costs Cryptography Data encapsulation data hiding Decoders Decoding Digital transmission Electrical engineering Electronics error exponents Exact sciences and technology Exponents Information theory Information, signal and communications theory maximum a posteriori probability (MAP) decoding Memoryless systems method of types Mutual information Radiocommunications random binning randomized codes reliability function Signal and communications theory Telecommunications Telecommunications and information theory Transmitters Transmitters. Receivers universal coding and decoding Upper bound Watermarking |
| Title | Capacity and Random-Coding Exponents for Channel Coding With Side Information |
| URI | https://ieeexplore.ieee.org/document/4137893 https://www.proquest.com/docview/195920680 https://www.proquest.com/docview/1671334721 https://www.proquest.com/docview/34475008 |
| Volume | 53 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1557-9654 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014512 issn: 0018-9448 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1Nb9QwEB21PcGBQgsiFIqROHAg28RJHPuIVq0K0nKAregt8qeoWmUrmpUQv56Z2BsoUIlLFClO5Hg8njf2zBuA15xrZ1Sjc4u2Iq8bb3IdqiK3obAh2MpyQ9nIi4_i9Kz-cN6cb8HbKRfGez8Gn_kZ3Y5n-W5l17RVdoQLbov2dRu221bFXK3pxKBuysgMXqICo8-RaHzKQh0t3y8jV6FUlPd5ywKNJVUoIFLf4JiEWMzir3V5NDYnu7DYdDPGmFzO1oOZ2R9_MDj-7388hAcJdbJ3cZo8gi3f78HupqIDSwq-B_d_oyfch8UcLalFmM5079gnvOCCOl-RtWPH369XPQVhMES9jHIUen_F0sMvF8NX9vnCeZaynUj6j-Hs5Hg5P81T-YXcVpIPuW90GZTwplCSiOg9IikpaycDAvIQXGmEKaWueTACcVfhQ9FI7dB_EdKi_KsnsNNjV54CE63iPrjGcq7qoIWqlLAIfXQrnDBCZjDbiKSziZucSmRcdaOPUqgOZUgVM9suyjCDN9ML15GW4-6m-ySBqVka_AwOb8n812fQxyvR78jgYDMJuqTXNx1R8XAqV5LBq-kpKiSdsujer9bYRJDfX6NnncHLO9qMNIuIvp79u28HcC9uIVOA0HPYGb6t_QvEPoM5HCf9TzrVAHg |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1Lb9QwEB6VcgAOFFoQodAaiQMHss3TsY9o1WoL3R5gK3qL_BQVVbaiWQnx65mJvYEClbhEkexETsbj-cae-QbgdVEoq2WtUoO2Iq1qp1Plyyw1PjPem9IUmrKR56d8dla9P6_PN-DtmAvjnBuCz9yEboezfLs0K9oqO8AFt0H7egfu1uhVNCFbazwzqOo8cIPnqMLYHol88kweLI4Xga1QSMr8vGGDhqIqFBKprvGv-FDO4q-VeTA3R1swXw80RJl8nax6PTE__uBw_N8veQQPI-5k78JEeQwbrtuGrXVNBxZVfBse_EZQuAPzKdpSg0Cdqc6yj3jBJXW6JHvHDr9fLTsKw2CIexllKXTuksXGzxf9F_bpwjoW851I_k_g7OhwMZ2lsQBDakpR9KmrVe4ldzqTgqjoHWIpISorPEJy722uuc6FqgqvOSKvzPmsFsqiB8OFwRlQPoXNDofyDBhvZOG8rU1RyMorLkvJDYIf1XDLNRcJTNYiaU1kJ6ciGZft4KVkskUZUs3Mpg0yTODN-MBVIOa4vesOSWDsFn9-Ans3ZP7rNejl5eh5JLC7ngRt1Ozrlsh4CipYksCrsRVVks5ZVOeWK-zCyfOv0LdOYP-WPgPRIuKv5_8e2z7cmy3mJ-3J8emHXbgfNpQpXOgFbPbfVu4lIqFe7w0K8BPVHwPJ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Capacity+and+Random-Coding+Exponents+for+Channel+Coding+With+Side+Information&rft.jtitle=IEEE+transactions+on+information+theory&rft.au=Moulin%2C+Pierre&rft.au=Wang%2C+Ying&rft.date=2007-04-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0018-9448&rft.eissn=1557-9654&rft.volume=53&rft.issue=4&rft.spage=1326&rft_id=info:doi/10.1109%2FTIT.2007.892789&rft.externalDBID=NO_FULL_TEXT&rft.externalDocID=1246392001 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9448&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9448&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9448&client=summon |