lambda-Domain Perceptual Rate Control for 360-Degree Video Compression

The 360-degree video is projected to 2-D formats using various projection methods for efficient compression. As a necessary part of general-video compression, rate control is also indispensable for the projected 360-degree video compression. However, the current rate control algorithm has not been o...

Full description

Saved in:
Bibliographic Details
Published inIEEE journal of selected topics in signal processing Vol. 14; no. 1; pp. 130 - 145
Main Authors Li, Li, Yan, Ning, Li, Zhu, Liu, Shan, Li, Houqiang
Format Journal Article
LanguageEnglish
Published IEEE 01.01.2020
Subjects
360-degree video compression
<inline-formula xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <tex-math notation="LaTeX"> boldsymbol {\lambda }</tex-math> </inline-formula>-domain rate control
Approximation algorithms
Bit rate
Encoding
High Efficiency Video Coding
Rate control
Rate distortion optimization
Signal processing algorithms
Software algorithms
Video coding
Video compression
Online AccessGet full text
ISSN1932-4553
1941-0484
DOI10.1109/JSTSP.2019.2963154

Cover

Abstract The 360-degree video is projected to 2-D formats using various projection methods for efficient compression. As a necessary part of general-video compression, rate control is also indispensable for the projected 360-degree video compression. However, the current rate control algorithm has not been optimized for the 360-degree video compression yet. The Coding Tree Unit (CTU) level bit allocation in the rate control algorithm has not taken into consideration the characteristic that various pixels in 2-D formats have different influences on the visual experiences. In this article, we first propose an optimal CTU level weight taking this characteristic into consideration. The CTU level weight is an approximation to the pixel level weight since the smallest granularity of a rate control algorithm is usually CTU. Second, based on the CTU level weight, a weighted CTU level bit allocation algorithm is proposed to achieve better coding performance. The bits of each CTU are assigned that the Lagrange multiplier λ of a CTU is inversely proportional to its CTU level weight. This CTU level bit allocation scheme is applied to all the 360-degree video projection formats. Third, we propose a CTU row (CR) level rate control algorithm for the Equi-Rectangle Projection (ERP) format. Different CTUs in the same row in the ERP format are combined into a CR to provide more stable model parameters. The proposed algorithms are implemented in the newest video coding standard High Efficiency Video Coding (HEVC) reference software. The experimental results show that the proposed algorithm is able to achieve much better subjective and objective qualities as well as smaller bitrate errors compared with the state-of-the-art rate control algorithm.
AbstractList The 360-degree video is projected to 2-D formats using various projection methods for efficient compression. As a necessary part of general-video compression, rate control is also indispensable for the projected 360-degree video compression. However, the current rate control algorithm has not been optimized for the 360-degree video compression yet. The Coding Tree Unit (CTU) level bit allocation in the rate control algorithm has not taken into consideration the characteristic that various pixels in 2-D formats have different influences on the visual experiences. In this article, we first propose an optimal CTU level weight taking this characteristic into consideration. The CTU level weight is an approximation to the pixel level weight since the smallest granularity of a rate control algorithm is usually CTU. Second, based on the CTU level weight, a weighted CTU level bit allocation algorithm is proposed to achieve better coding performance. The bits of each CTU are assigned that the Lagrange multiplier λ of a CTU is inversely proportional to its CTU level weight. This CTU level bit allocation scheme is applied to all the 360-degree video projection formats. Third, we propose a CTU row (CR) level rate control algorithm for the Equi-Rectangle Projection (ERP) format. Different CTUs in the same row in the ERP format are combined into a CR to provide more stable model parameters. The proposed algorithms are implemented in the newest video coding standard High Efficiency Video Coding (HEVC) reference software. The experimental results show that the proposed algorithm is able to achieve much better subjective and objective qualities as well as smaller bitrate errors compared with the state-of-the-art rate control algorithm.
Author Yan, Ning
Li, Li
Liu, Shan
Li, Zhu
Li, Houqiang
Author_xml – sequence: 1
  givenname: Li
  orcidid: 0000-0002-7163-6263
  surname: Li
  fullname: Li, Li
  email: lil1@umkc.edu
  organization: University of Missouri-Kansas City, 5100 Rockhill Road, Kansas City, MO, USA
– sequence: 2
  givenname: Ning
  orcidid: 0000-0002-6771-111X
  surname: Yan
  fullname: Yan, Ning
  email: nyan@mail.ustc.edu.cn
  organization: CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, University of Science and Technology of China, No. 443 Huangshan Road, Hefei, China
– sequence: 3
  givenname: Zhu
  orcidid: 0000-0002-8246-177X
  surname: Li
  fullname: Li, Zhu
  email: lizhu@umkc.edu
  organization: University of Missouri-Kansas City, 5100 Rockhill Road, Kansas City, MO, USA
– sequence: 4
  givenname: Shan
  orcidid: 0000-0002-1442-1207
  surname: Liu
  fullname: Liu, Shan
  email: shanl@tencent.com
  organization: Tencent America, 661 Bryant St, Palo Alto, CA, USA
– sequence: 5
  givenname: Houqiang
  orcidid: 0000-0003-2188-3028
  surname: Li
  fullname: Li, Houqiang
  email: lihq@ustc.edu.cn
  organization: CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, University of Science and Technology of China, No. 443 Huangshan Road, Hefei, China
BookMark eNp9kL1OwzAUhS1UJNrCC8CSF3DxX5x4RCktoEpUtLBGjnONjJI4ssPA29PQioGB6Vzp6Ls6-mZo0vkOELqmZEEpUbdPu_1uu2CEqgVTktNUnKEpVYJiInIxGW_OsEhTfoFmMX4QkmaSiilaNbqtao2XvtWuS7YQDPTDp26SFz1AUvhuCL5JrA8JlwQv4T0AJG-uBn8o2z5AjM53l-jc6ibC1Snn6HV1vy8e8OZ5_VjcbbBhMhswz7gSNiVMpZoZWVkthamEAVkzW1GqMm5zw7jNJKkzUxuoqlxwIVVtFRDO5yg__jXBxxjAlsYNenDjTO2akpJy9FH--ChHH-XJxwFlf9A-uFaHr_-hmyPkAOAXyJWQh4X8G_NQbiY
CODEN IJSTGY
CitedBy_id crossref_primary_10_3390_app132312758
crossref_primary_10_1145_3551641
crossref_primary_10_1007_s11042_023_17795_4
crossref_primary_10_1109_ACCESS_2021_3132294
crossref_primary_10_1109_LSP_2022_3219359
crossref_primary_10_1109_TBC_2022_3147103
crossref_primary_10_1109_TCSVT_2022_3231335
crossref_primary_10_1109_TBC_2023_3332019
crossref_primary_10_3390_electronics13245028
crossref_primary_10_1016_j_jvcir_2021_103242
crossref_primary_10_1109_TCSVT_2022_3192665
crossref_primary_10_1007_s00530_022_00942_6
crossref_primary_10_1016_j_jvcir_2020_103000
crossref_primary_10_1117_1_JEI_31_3_033026
crossref_primary_10_1109_TIP_2020_3004714
crossref_primary_10_1109_JIOT_2023_3263687
Cites_doi 10.1109/VCIP.2017.8305106
10.1109/76.554439
10.1109/TCSVT.2012.2221191
10.1109/LSP.2017.2720693
10.1016/j.jvcir.2018.03.001
10.1109/TIP.2018.2885482
10.1109/ICME.2017.8019460
10.1109/TMM.2015.2477682
10.1109/ICIP.2017.8296517
10.1109/TCSVT.2015.2444671
10.1007/s11554-011-0237-2
10.1007/s11042-017-5507-y
10.1109/TBC.2018.2865647
10.1109/DCC.2019.00051
10.1109/TCSVT.2002.804883
10.1109/MMSP.2017.8122231
10.1109/TMM.2016.2595264
10.1109/TCSVT.2005.857300
10.1016/j.jvcir.2019.02.031
10.1109/TCSVT.2007.894053
10.2991/icadme-16.2016.118
10.1016/j.jvcir.2018.05.001
10.1109/JSTSP.2013.2272240
10.1109/ISMAR.2015.12
10.1109/TIP.2019.2911180
10.1109/TBC.2018.2847445
10.1109/LSP.2014.2377032
10.1109/TIP.2014.2336550
10.1109/76.937431
10.1109/TBC.2019.2917402
10.1109/ICME.2017.8019492
10.1007/s11760-018-01411-2
10.1109/ISM.2008.71
10.1109/TCSVT.2017.2658024
10.1109/VCIP.2017.8305050
10.1109/TCSVT.2016.2598672
10.1109/TCSVT.2016.2589878
10.1109/TCSVT.2007.913757
10.1109/TBC.2014.2361964
10.1109/TMM.2016.2535254
10.1109/TCSVT.2019.2914100
10.1109/TCSVT.2002.805511
10.1109/TCSVT.2007.905532
ContentType Journal Article
DBID 97E
RIA
RIE
AAYXX
CITATION
DOI 10.1109/JSTSP.2019.2963154
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1941-0484
EndPage 145
ExternalDocumentID 10_1109_JSTSP_2019_2963154
8946760
Genre orig-research
GrantInformation_xml – fundername: National Key Research and Development Plan
  grantid: 2017YFB1002401
– fundername: Big Learning and Tencent Media Lab
GroupedDBID -~X
0R~
29I
4.4
5GY
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACIWK
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
EJD
F5P
HZ~
IFIPE
IPLJI
JAVBF
LAI
M43
O9-
OCL
RIA
RIE
RNS
AAYXX
CITATION
ID FETCH-LOGICAL-c267t-37394f50295a2c6bfa64cb4ce6d2fb11973f8c23f760d7cdcebb843469df9e033
IEDL.DBID RIE
ISSN 1932-4553
IngestDate Thu Apr 24 23:03:25 EDT 2025
Wed Oct 01 03:34:39 EDT 2025
Wed Aug 27 02:41:42 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 1
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c267t-37394f50295a2c6bfa64cb4ce6d2fb11973f8c23f760d7cdcebb843469df9e033
ORCID 0000-0003-2188-3028
0000-0002-1442-1207
0000-0002-8246-177X
0000-0002-6771-111X
0000-0002-7163-6263
PageCount 16
ParticipantIDs crossref_citationtrail_10_1109_JSTSP_2019_2963154
ieee_primary_8946760
crossref_primary_10_1109_JSTSP_2019_2963154
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2020-Jan.
2020-1-00
PublicationDateYYYYMMDD 2020-01-01
PublicationDate_xml – month: 01
  year: 2020
  text: 2020-Jan.
PublicationDecade 2020
PublicationTitle IEEE journal of selected topics in signal processing
PublicationTitleAbbrev JSTSP
PublicationYear 2020
Publisher IEEE
Publisher_xml – name: IEEE
References ref13
ref15
ref14
ref53
ref52
ref17
ref16
hanhart (ref57) 2018
ref19
ref18
ref51
ref50
ref46
ref45
ref48
ref47
ref42
ref41
ref44
ye (ref4) 2017
ref43
pitrey (ref29) 0
ref49
ref8
(ref2) 0
ref7
ref9
ref5
ref40
ref35
ref34
ref37
ref36
ref31
ref30
ref33
ref32
li (ref54) 0
bjontegaard (ref56) 2001
sauer (ref11) 2017
ref39
ref38
boyce (ref3) 2016
(ref1) 0
coban (ref12) 2017
bross (ref6) 2019
li (ref20) 2018
ref24
ref23
ref26
ref25
ref22
ref21
(ref58) 2019
ref28
ref27
he (ref10) 2016
(ref55) 2019
References_xml – ident: ref52
  doi: 10.1109/VCIP.2017.8305106
– ident: ref16
  doi: 10.1109/76.554439
– year: 2001
  ident: ref56
  article-title: Calculation of Average PSNR Differences between RD-Curves
– year: 2019
  ident: ref6
  article-title: Versatile Video Coding (Draft 5)
– ident: ref5
  doi: 10.1109/TCSVT.2012.2221191
– ident: ref7
  doi: 10.1109/LSP.2017.2720693
– year: 2019
  ident: ref58
  article-title: IEEE 1857 working group
– ident: ref21
  doi: 10.1016/j.jvcir.2018.03.001
– ident: ref9
  doi: 10.1109/TIP.2018.2885482
– ident: ref53
  doi: 10.1109/ICME.2017.8019460
– year: 2017
  ident: ref4
  article-title: Algorithm Descriptions of Projection Format Conversion and Video Quality Metrics in 360Lib Version 5
– ident: ref49
  doi: 10.1109/TMM.2015.2477682
– ident: ref13
  doi: 10.1109/ICIP.2017.8296517
– ident: ref32
  doi: 10.1109/TCSVT.2015.2444671
– ident: ref25
  doi: 10.1007/s11554-011-0237-2
– ident: ref42
  doi: 10.1007/s11042-017-5507-y
– ident: ref43
  doi: 10.1109/TBC.2018.2865647
– ident: ref47
  doi: 10.1109/DCC.2019.00051
– ident: ref27
  doi: 10.1109/TCSVT.2002.804883
– year: 2018
  ident: ref57
  article-title: JVET Common Test Conditions and Evaluation Procedures for 360 Video
– ident: ref14
  doi: 10.1109/MMSP.2017.8122231
– ident: ref44
  doi: 10.1109/TMM.2016.2595264
– ident: ref15
  doi: 10.1109/TCSVT.2005.857300
– ident: ref45
  doi: 10.1016/j.jvcir.2019.02.031
– ident: ref22
  doi: 10.1109/TCSVT.2007.894053
– ident: ref23
  doi: 10.2991/icadme-16.2016.118
– ident: ref50
  doi: 10.1016/j.jvcir.2018.05.001
– ident: ref31
  doi: 10.1109/JSTSP.2013.2272240
– year: 2017
  ident: ref11
  article-title: AHG8: Results for Geometry Correction for Motion Compensation of Planar-Projected 360VR Video with JEM4.1 and 360Lib
– ident: ref8
  doi: 10.1109/ISMAR.2015.12
– year: 2019
  ident: ref55
  article-title: High Efficiency Video Coding test model, HM-16.20
– ident: ref38
  doi: 10.1109/TIP.2019.2911180
– year: 2017
  ident: ref12
  article-title: AHG8: Reference Picture Extension of ACP Format 360-degree Video
– ident: ref35
  doi: 10.1109/TBC.2018.2847445
– ident: ref40
  doi: 10.1109/LSP.2014.2377032
– ident: ref18
  doi: 10.1109/TIP.2014.2336550
– ident: ref26
  doi: 10.1109/76.937431
– ident: ref36
  doi: 10.1109/TBC.2019.2917402
– ident: ref19
  doi: 10.1109/ICME.2017.8019492
– start-page: 1
  year: 2018
  ident: ref20
  article-title: Spherical domain rate-distortion optimization for omnidirectional video coding
  publication-title: IEEE Trans Circuits Syst Video Technol
– start-page: 477
  year: 0
  ident: ref54
  article-title: QP refinement according to lagrange multiplier for high efficiency video coding
  publication-title: Proc IEEE Int Symp Circuits Syst (ISCAS2013)
– ident: ref46
  doi: 10.1007/s11760-018-01411-2
– year: 0
  ident: ref2
– year: 0
  ident: ref1
– ident: ref28
  doi: 10.1109/ISM.2008.71
– ident: ref48
  doi: 10.1109/TCSVT.2017.2658024
– ident: ref41
  doi: 10.1109/VCIP.2017.8305050
– ident: ref33
  doi: 10.1109/TCSVT.2016.2598672
– ident: ref34
  doi: 10.1109/TCSVT.2016.2589878
– year: 2016
  ident: ref3
  article-title: JVET Common Test Conditions and Evaluation Procedures for 360 Video
– ident: ref24
  doi: 10.1109/TCSVT.2007.913757
– ident: ref51
  doi: 10.1109/TBC.2014.2361964
– ident: ref39
  doi: 10.1109/TMM.2016.2535254
– ident: ref37
  doi: 10.1109/TCSVT.2019.2914100
– year: 2016
  ident: ref10
  article-title: AHG8: Geometry Padding for 360 Video Coding
– ident: ref17
  doi: 10.1109/TCSVT.2002.805511
– start-page: 1
  year: 0
  ident: ref29
  article-title: One-pass bitrate control for MPEG-4 scalable video coding using $\rho$-Domain
  publication-title: Proc IEEE Int Symp Broadband Multimedia Syst Broadcast
– ident: ref30
  doi: 10.1109/TCSVT.2007.905532
SSID ssj0057614
Score 2.38024
Snippet The 360-degree video is projected to 2-D formats using various projection methods for efficient compression. As a necessary part of general-video compression,...
SourceID crossref
ieee
SourceType Enrichment Source
Index Database
Publisher
StartPage 130
SubjectTerms 360-degree video compression
<inline-formula xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <tex-math notation="LaTeX"> boldsymbol {\lambda }</tex-math> </inline-formula>-domain rate control
Approximation algorithms
Bit rate
Encoding
High Efficiency Video Coding
Rate control
Rate distortion optimization
Signal processing algorithms
Software algorithms
Video coding
Video compression
Title lambda-Domain Perceptual Rate Control for 360-Degree Video Compression
URI https://ieeexplore.ieee.org/document/8946760
Volume 14
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1941-0484
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0057614
  issn: 1932-4553
  databaseCode: RIE
  dateStart: 20070101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LS8NAEF5qT3rwVcX6Igdvumn2mexRWksRKkWt9Bb2FRA1EUku_np3k7RUEfEWwi4Ms8M8v5kB4EILirBECCoWU-gsBIEywRoqpCNLiFFR5hP60zs-mdPbBVt0wNWqF8ZaW4PPbOg_61q-KXTlU2WDxO-D5y5A34gT3vRqLbWuc5tRW0HGkDJGlg0ykRg4EX-YeRSXCLGTN8ToNyO0tlWlNirjHTBdktNgSV7CqlSh_vwxqfG_9O6C7da7DK4bcdgDHZvvg621mYM9MHYioIyEo-JNPufBrAG2VO7WvXM7g2EDXQ-cLxsQHsGRdQG5DZ6ejS0Crzwa3Gx-AObjm8fhBLbLFKDGPC6dIiGCZizCgkmsucokp1pRbbnBmfLFRJIlGpPMEWxibbRVKqHERc8mEzYi5BB08yK3RyDAsRXKWKT8YBfNMyV5oiRLBFYmlhHqA7TkbqrbSeN-4cVrWkcckUjrF0n9i6Tti_TB5erOezNn48_TPc_t1cmW0ce__z4Bm9hHyXXi5BR0y4_KnjlXolTntQx9AbAaxEM
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LS8NAEF6KHtSDryrWZw7edNvsM9mjtJaqbSnaSm9hX4GipiLtxV_vbpKWKiLeQtiFYXaY5zczAFxqQRGWCEHFIgqdhSBQxlhDhXRoCTEqTH1Cv9fnnRG9H7NxBVwve2GstTn4zNb9Z17LN1M996myRuz3wXMXoK8zSikrurUWetc5zqisIWNIGSOLFplQNJyQPw08jkvUsZM4xOg3M7SyVyU3K-0d0FsQVKBJXurzmarrzx-zGv9L8S7YLv3L4KYQiD1Qsdk-2FqZOlgFbScEykjYmr7JSRYMCmjL3N16dI5n0CzA64HzZgPCQ9iyLiS3wfPE2Gng1UeBnM0OwKh9O2x2YLlOAWrMo5lTJUTQlIVYMIk1V6nkVCuqLTc4Vb6cSNJYY5I6gk2kjbZKxZS4-NmkwoaEHIK1bJrZIxDgyAplLFJ-tIvmqZI8VpLFAisTyRDVAFpwN9HlrHG_8uI1yWOOUCT5iyT-RZLyRWrgannnvZi08efpquf28mTJ6OPff1-Ajc6w1026d_2HE7CJfcycp1FOwdrsY27PnGMxU-e5PH0BJaTHkA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=lambda-Domain+Perceptual+Rate+Control+for+360-Degree+Video+Compression&rft.jtitle=IEEE+journal+of+selected+topics+in+signal+processing&rft.au=Li%2C+Li&rft.au=Yan%2C+Ning&rft.au=Li%2C+Zhu&rft.au=Liu%2C+Shan&rft.date=2020-01-01&rft.pub=IEEE&rft.issn=1932-4553&rft.volume=14&rft.issue=1&rft.spage=130&rft.epage=145&rft_id=info:doi/10.1109%2FJSTSP.2019.2963154&rft.externalDocID=8946760
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1932-4553&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1932-4553&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1932-4553&client=summon