MEE : An Automatic Metric for Evaluation Using Embeddings for Machine Translation

We propose MEE, an approach for automatic Machine Translation (MT) evaluation which leverages the similarity between embeddings of words in candidate and reference sentences to assess translation quality. Unigrams are matched based on their surface forms, root forms and meanings which aids to captur...

Full description

Saved in:
Bibliographic Details
Published in2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA) pp. 292 - 299
Main Authors Mukherjee, Ananya, Ala, Hema, Shrivastava, Manish, Sharma, Dipti Misra
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.10.2020
Subjects
Automatic Metrics
Conferences
Data science
Embeddings
Frequency modulation
Morphological Languages
MT Evaluation
Semantic Evaluation
Online AccessGet full text
DOI10.1109/DSAA49011.2020.00042

Cover

Abstract We propose MEE, an approach for automatic Machine Translation (MT) evaluation which leverages the similarity between embeddings of words in candidate and reference sentences to assess translation quality. Unigrams are matched based on their surface forms, root forms and meanings which aids to capture lexical, morphological and semantic equivalence. We perform experiments for MT from English to four Indian Languages (Telugu, Marathi, Bengali and Hindi) on a robust dataset comprising simple and complex sentences with good and bad translations. Further, it is observed that the proposed metric correlates better with human judgements than the existing widely used metrics.
AbstractList We propose MEE, an approach for automatic Machine Translation (MT) evaluation which leverages the similarity between embeddings of words in candidate and reference sentences to assess translation quality. Unigrams are matched based on their surface forms, root forms and meanings which aids to capture lexical, morphological and semantic equivalence. We perform experiments for MT from English to four Indian Languages (Telugu, Marathi, Bengali and Hindi) on a robust dataset comprising simple and complex sentences with good and bad translations. Further, it is observed that the proposed metric correlates better with human judgements than the existing widely used metrics.
Author Shrivastava, Manish
Mukherjee, Ananya
Sharma, Dipti Misra
Ala, Hema
Author_xml – sequence: 1
  givenname: Ananya
  surname: Mukherjee
  fullname: Mukherjee, Ananya
  email: ananya.mukherjee@research.iiit.ac.in
  organization: IIIT,Language Technology Research Centre,Hyderabad,India
– sequence: 2
  givenname: Hema
  surname: Ala
  fullname: Ala, Hema
  email: hema.ala@research.iiit.ac.in
  organization: IIIT,Language Technology Research Centre,Hyderabad,India
– sequence: 3
  givenname: Manish
  surname: Shrivastava
  fullname: Shrivastava, Manish
  email: m.srivastava@iiit.ac.in
  organization: IIIT,Language Technology Research Centre,Hyderabad,India
– sequence: 4
  givenname: Dipti Misra
  surname: Sharma
  fullname: Sharma, Dipti Misra
  email: dipti@iiit.ac.in
  organization: IIIT,Language Technology Research Centre,Hyderabad,India
BookMark eNotj81OwzAQhI0EB1p4Ajj4BRJ2bceOuUUl_EiNUNX2XDn1BiwlDkpSJN6eqHD6RjOjkWbBLmMfibF7hBQR7MPTtiiUBcRUgIAUAJS4YAs0IsdcgLbXbFOVJX_kReTFaeo7N4Ujr2gaZjT9wMtv155ms498P4b4wcuuJu9nNZ7zyh0_QyS-G1wc23Pxhl01rh3p9p9Ltn8ud6vXZP3-8rYq1kkQIKdEZrWXjaTaWNmoLBdGOwGOpCSFNTrvCLXNrDSNV5g5rXMkaTRYMEblXi7Z3d9uIKLD1xA6N_wcrNDzy0z-AmmFSr0
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/DSAA49011.2020.00042
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 1728182069
9781728182063
EndPage 299
ExternalDocumentID 9260045
Genre orig-research
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i203t-35bd3f3eb793f458276a20ae33e41b1adae1695937fd415a6681e3760907748d3
IEDL.DBID RIE
IngestDate Thu Jun 29 18:38:36 EDT 2023
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-35bd3f3eb793f458276a20ae33e41b1adae1695937fd415a6681e3760907748d3
PageCount 8
ParticipantIDs ieee_primary_9260045
PublicationCentury 2000
PublicationDate 2020-Oct.
PublicationDateYYYYMMDD 2020-10-01
PublicationDate_xml – month: 10
  year: 2020
  text: 2020-Oct.
PublicationDecade 2020
PublicationTitle 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA)
PublicationTitleAbbrev DSAA
PublicationYear 2020
Publisher IEEE
Publisher_xml – name: IEEE
Score 1.8479505
Snippet We propose MEE, an approach for automatic Machine Translation (MT) evaluation which leverages the similarity between embeddings of words in candidate and...
SourceID ieee
SourceType Publisher
StartPage 292
SubjectTerms Automatic Metrics
Conferences
Data science
Embeddings
Frequency modulation
Morphological Languages
MT Evaluation
Semantic Evaluation
Title MEE : An Automatic Metric for Evaluation Using Embeddings for Machine Translation
URI https://ieeexplore.ieee.org/document/9260045
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PS8MwFA7bTp5UNvE3OXi0W9O8po23oh1DqCg62G0kzSuI2Im0F_96k7ROFA9CDiE9JCSU7yXv-75HyIUEiLWIVRCKCgIAaQLFQAXahsbShIpxb5JU3InFEm5X8WpALrdaGET05DOcuq7P5ZtN2bqnspl0buoQD8kwSWSn1erVcCyUs5vHLAOnpLS3vij0NpzRj5opHjLmu6T4mqxjirxM20ZPy49fPoz_Xc0emXyL8-j9Fnb2yQDrMXko8pxe0aymWdtsvA0rLVyxrJLaqJTmW09v6jkCNH_VaHzayX8vPKUSqUeujh03Ict5_nS9CPpqCcFzFPIm4LE2vOKo7R9XuWxYIlQUKuQcgWmmjEImnA1xUhmL2kqIlKGjxNjrcQKp4QdkVG9qPCSU64rbyCo2AAmIyKQyNDJRtgErAdMjMnbbsX7rDDHW_U4c_z18QnbcgXQMuFMyat5bPLNI3uhzf4SfUeudNg
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PS8MwFA5zHvSksom_zcGj3ZrmNW29Fe2Yug7FDXYbyfIKInYi7cW_3iSdE8WDkENJDi0J4Xuv7_u-R8hFAhAqEUrPFwV4AIn2JAPpKRMaJ9qXjDuTpHwshlO4m4WzFrlca2EQ0ZHPsGcfXS1fLxe1_VXWT6ybOoQbZDM0WUXUqLVWejjmJ_2bpzQFq6U0eV_gOyPO4EfXFAcagx2Sf72u4Yq89OpK9RYfv5wY__s9u6T7Lc-jD2vg2SMtLDvkMc8yekXTkqZ1tXRGrDS37bIW1MSlNFu7elPHEqDZq0LtCk9uPXekSqQOuxp-XJdMB9nkeuit-iV4z4HPK4-HSvOCozJ3rrD1sEjIwJfIOQJTTGqJTFgj4qjQBrelEDFDS4oxCXIEseb7pF0uSzwglKuCm9gq1AARiEDHia-TSJoBbAEYH5KO3Y75W2OJMV_txNHf0-dkazjJR_PR7fj-mGzbw2n4cCekXb3XeGpwvVJn7jg_AceIoIc
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2020+IEEE+7th+International+Conference+on+Data+Science+and+Advanced+Analytics+%28DSAA%29&rft.atitle=MEE+%3A+An+Automatic+Metric+for+Evaluation+Using+Embeddings+for+Machine+Translation&rft.au=Mukherjee%2C+Ananya&rft.au=Ala%2C+Hema&rft.au=Shrivastava%2C+Manish&rft.au=Sharma%2C+Dipti+Misra&rft.date=2020-10-01&rft.pub=IEEE&rft.spage=292&rft.epage=299&rft_id=info:doi/10.1109%2FDSAA49011.2020.00042&rft.externalDocID=9260045