面向点击率预测的自注意力深度域嵌入因子分解机
TP391; 点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐.针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-atten-tion deep field-embedded factorization machine,Self-AtDFEFM)模型.首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶...
Saved in:
| Published in | 工程科学与技术 Vol. 56; no. 5; pp. 287 - 296 |
|---|---|
| Main Authors | , , , , , |
| Format | Journal Article |
| Language | Chinese |
| Published |
华东交通大学信息与软件工程学院,江西南昌 330013
01.09.2024
|
| Subjects | |
| Online Access | Get full text |
| ISSN | 2096-3246 |
| DOI | 10.12454/j.jsuese.202201373 |
Cover
| Abstract | TP391; 点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐.针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-atten-tion deep field-embedded factorization machine,Self-AtDFEFM)模型.首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶特征交互;然后,围绕精化后的嵌入向量,联合多头自注意力与残差机制堆叠多个显式高阶特征交互层,通过自注意力捕获同一特征在不同子空间上的互补信息,完成显示高阶特征交互;最后,联合显式与隐式高阶特征交互实现点击率预测.在Criteo和Avazu两大公开数据集上,将Self-AtDFEFM模型与主流基线模型在AUC和Lo-gLoss指标上进行对比实验;为Self-AtDFEFM模型调制显式高阶特征交互层层数、注意力头数量、嵌入层维度及隐式高阶特征交互层层数等参数;对Self-AtDFEFM模型进行消融实验.实验结果表明:在两大数据集上,Self-AtDFEFM 模型的AUC、LogLoss均优于主流基线模型;Self-AtDFEFM模型的全部参数已调为最佳;各模块形成合力以促使Self-AtDFEFM模型性能达到最优,其中显示高阶特征交互层的作用最大.Self-AtDFEFM模型各模块即插即用,易于构建和部署,且在性能与复杂度之间取得平衡,具备较高实用性. |
|---|---|
| AbstractList | TP391; 点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐.针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-atten-tion deep field-embedded factorization machine,Self-AtDFEFM)模型.首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶特征交互;然后,围绕精化后的嵌入向量,联合多头自注意力与残差机制堆叠多个显式高阶特征交互层,通过自注意力捕获同一特征在不同子空间上的互补信息,完成显示高阶特征交互;最后,联合显式与隐式高阶特征交互实现点击率预测.在Criteo和Avazu两大公开数据集上,将Self-AtDFEFM模型与主流基线模型在AUC和Lo-gLoss指标上进行对比实验;为Self-AtDFEFM模型调制显式高阶特征交互层层数、注意力头数量、嵌入层维度及隐式高阶特征交互层层数等参数;对Self-AtDFEFM模型进行消融实验.实验结果表明:在两大数据集上,Self-AtDFEFM 模型的AUC、LogLoss均优于主流基线模型;Self-AtDFEFM模型的全部参数已调为最佳;各模块形成合力以促使Self-AtDFEFM模型性能达到最优,其中显示高阶特征交互层的作用最大.Self-AtDFEFM模型各模块即插即用,易于构建和部署,且在性能与复杂度之间取得平衡,具备较高实用性. |
| Abstract_FL | Objective Click-through rate(CTR)prediction realizes accurate recommendation of digital advertisements by predicting the user's click probabil-ity on advertisements or commodities.However,current CTR prediction models have the following key issues.First,the raw embedding vectors have not been fully refined.Second,the corresponding feature interaction method is too simple.As a result,the performance of the models is heavily restricted.To alleviate these issues,a novel CTR model named self-attention deep field-embedded factorization machine(Self-AtDFEFM)is proposed.
Methods First,a well-known multi-head self-attention mechanism is employed to capture the implicit information of the raw embedding vectors on different sub-spaces,and the corresponding weight is calculated to further refine the key low-level features.Second,a novel field-embedded factorization machine(FEFM)is designed to strengthen the interaction intensity between different feature fields by the field pair symmetric mat-rix.The key low-order feature combinations are fully optimized by the FEFM module for the subsequent high-order feature interaction.Third,a deep neural network(DNN)is built based on the low-order feature combinations to complete implicit high-order feature interaction.Finally,both the explicit and implicit feature interactions are combined together to implement CTR prediction.
Results and Discussions Extensive experiments have been performed on the two public available datasets,namely Criteo and Avazu.First,the proposed Self-AtDFEFM is compared with numerous state-of-the-art baselines on the AUC(area under curve)and LogLoss metrics.Second,all parameters of Self-AtDFEFM was tuned,and the parameters included the number of the explicit high-order feature interaction layers,the number of the attention heads,the embedding dimension,and the number of the implicit high-order feature interaction layers.Further,ablation experi-ments of our model were completed.The results of the experiments showed that:the Self-AtDFEFM model outperformed mainstream baseline models on the AUC and LogLoss metrics;all parameters of Self-AtDFEFM have been adjusted to their optimal values;each module form a kind of joint force to improve the final CTR prediction performance.Notably,the explicit high-order feature interaction layer plays the most important role in Self-AtDFEFM.
Conclusions Each module of Self-AtDFEFM is plug-and-play,that is,the Self-AtDFEFM is easier to build and deploy.Hence,Self-AtDFEFM achieves a good trade-off between prediction performance and model complexity,making it highly practical. |
| Author | 李广丽 许广鑫 张红斌 吴光庭 叶艺源 吕敬钦 |
| AuthorAffiliation | 华东交通大学信息与软件工程学院,江西南昌 330013 |
| AuthorAffiliation_xml | – name: 华东交通大学信息与软件工程学院,江西南昌 330013 |
| Author_FL | ZHANG Hongbin LYU Jingqin XU Guangxin LI Guangli WU Guangting YE Yiyuan |
| Author_FL_xml | – sequence: 1 fullname: LI Guangli – sequence: 2 fullname: YE Yiyuan – sequence: 3 fullname: XU Guangxin – sequence: 4 fullname: ZHANG Hongbin – sequence: 5 fullname: WU Guangting – sequence: 6 fullname: LYU Jingqin |
| Author_xml | – sequence: 1 fullname: 李广丽 – sequence: 2 fullname: 叶艺源 – sequence: 3 fullname: 许广鑫 – sequence: 4 fullname: 张红斌 – sequence: 5 fullname: 吴光庭 – sequence: 6 fullname: 吕敬钦 |
| BookMark | eNotj01LAkEch-dgkJmfoK_QbjP_eVnnWNIbCF3qvMzOzkoWKzRInsMySsquWRhGLwQdIiSyzzOrfouEOv3gOTwPvwWUS-upQWiJYJ8A42yl5tdsw1jjAwbAhAY0h_KApfAoMDGPitbuR5gKRjkXPI_WpvcD170Zn3y79s_4qj0dtLLh5fi2NWm_ZZ-vWevaXfSyrw83enb9vht23OmT6z249647P5u8PGZ3o0U0l6hDa4r_W0B7G-u75S2vsrO5XV6teJbgQHqEUQLaRIKD1oGRmErGmC4loCKhBOEl0DwGBVIrHmipZUQSykwygwxiQgto-c97rNJEpdWwVm8cpbNiaHXcbEZhVR80Z7cZ5hgk_QXNtWfv |
| ClassificationCodes | TP391 |
| ContentType | Journal Article |
| Copyright | Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| Copyright_xml | – notice: Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| DBID | 2B. 4A8 92I 93N PSX TCJ |
| DOI | 10.12454/j.jsuese.202201373 |
| DatabaseName | Wanfang Data Journals - Hong Kong WANFANG Data Centre Wanfang Data Journals 万方数据期刊 - 香港版 China Online Journals (COJ) China Online Journals (COJ) |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| DocumentTitle_FL | Self-attention Deep Field-embedded Factorization Machine for Click-through Rate Prediction |
| EndPage | 296 |
| ExternalDocumentID | scdxxb_gckx202405029 |
| GrantInformation_xml | – fundername: (国家自然科学基金); (国家自然科学基金); (江西省重点研发计划重点项目); (教育部人文社会科学研究项目); (江西省自然科学基金面上项目); (江西省社会科学研究规划基金项目); (江西省高校人文社科基金项目); (江西省教育厅科技项目); (江西省教育厅科技项目); (江西省研究生创新专项资金项目) funderid: (国家自然科学基金); (国家自然科学基金); (江西省重点研发计划重点项目); (教育部人文社会科学研究项目); (江西省自然科学基金面上项目); (江西省社会科学研究规划基金项目); (江西省高校人文社科基金项目); (江西省教育厅科技项目); (江西省教育厅科技项目); (江西省研究生创新专项资金项目) |
| GroupedDBID | -0C -SC -S~ 2B. 2RA 4A8 5VR 92I 92M 93N 9D9 9DC AFUIB ALMA_UNASSIGNED_HOLDINGS CAJEC CQIGP GROUPED_DOAJ PB1 PB9 PSX Q-- R-C RT3 T8S TCJ U1F U5C |
| ID | FETCH-LOGICAL-s1079-14312ceb652cc7e9039444c8f2ab6a61582c5d2a29ca57c9c9b1f34ef5d242d13 |
| ISSN | 2096-3246 |
| IngestDate | Thu May 29 03:53:57 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Keywords | field-embedded factorization machine click-through rate prediction multi-head self-attention 深度神经网络 feature interaction 点击率预测 多头自注意力 deep neural network 特征交互 域嵌入因子分解机 |
| Language | Chinese |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-s1079-14312ceb652cc7e9039444c8f2ab6a61582c5d2a29ca57c9c9b1f34ef5d242d13 |
| PageCount | 10 |
| ParticipantIDs | wanfang_journals_scdxxb_gckx202405029 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-09-01 |
| PublicationDateYYYYMMDD | 2024-09-01 |
| PublicationDate_xml | – month: 09 year: 2024 text: 2024-09-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | 工程科学与技术 |
| PublicationTitle_FL | Advanced Engineering Sciences |
| PublicationYear | 2024 |
| Publisher | 华东交通大学信息与软件工程学院,江西南昌 330013 |
| Publisher_xml | – name: 华东交通大学信息与软件工程学院,江西南昌 330013 |
| SSID | ssib036435565 ssib050593459 ssib041261190 ssib030194745 ssib051371919 ssj0003313526 ssib027967859 |
| Score | 2.3949916 |
| Snippet | TP391; 点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐.针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深... |
| SourceID | wanfang |
| SourceType | Aggregation Database |
| StartPage | 287 |
| Title | 面向点击率预测的自注意力深度域嵌入因子分解机 |
| URI | https://d.wanfangdata.com.cn/periodical/scdxxb-gckx202405029 |
| Volume | 56 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: Directory of Open Access Journals issn: 2096-3246 databaseCode: DOA dateStart: 20220101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.doaj.org/ omitProxy: true ssIdentifier: ssj0003313526 providerName: Directory of Open Access Journals |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3Na9VAEA9tvXgRRcVvenBPEk02u9nscfe9hOLBUwu9lWRfUqnwBPsKpUepVlS0Xq1SqfiB4EGkiPVf8ZrX9r9wZpO-RNpDFR5hmZ3MzG_nbXYm2Q_HuR6mqd9LaeqyQhiXCSHclPd6bpEXacqMxC2tcLbFnXBqht2e5bNj479bs5aWBtlNs3LkupL_8SrQwK-4SvYfPDsSCgQog3_hCh6G67F8TGJJZJcoSmJOpEekT2JBIkq0RArk-VpbSoxlYAbOiJE4JBpqbZVUlhIhg1K2KiAqwgLQo8TKUURqWwUCfaRoRVRolSb4QwqwdSwzJ4rbKk2UhwXVRduwCrSEqEuBrgAFyg6pXxQsHCDiqAUlCDSjMhL4pV-LQr2M6AhBoZEK52pUotRolrIldC0Lx8bQSX2T7jYsHOFpa1AkERIiBKhewwKmxnhbI0WiJUq3peiOBSpsq1CrGuzqtN-nUDaaMFb1AKu9MtDaBdZjAQQw1IHTT5QFzBD8X8gTonyLHOxIWm2B4A4o2uI63JQjOQDD_htox6L2rRcjZNZJbZsUFgrc2LkRBBjEN4MGhZTUhSA5bI9w1dbtdU_m7eGqDnaqyIdWZwsfGlQp48yOqgt4HiTuLUsp7lQZNDHEaGbnouktL2dz8-beMjauxz0qx50TFMZcr_XCA572VEgIlJpgGkYayVpnLwQQK3PefINnPiT7fvNNl-NhlK2dkDhY5Mt6c0KMw4LAx8Mf8HTJg0aptx1DQLcOw7Hr9vpF2p9vhZjTp51TdW44qaqOfsYZW7l71tH7b7fK9Ve7D3-Wa792X6ztb60Ot5_tvl7dW_sy_P55uPqyfLox_PGt3PlYbm6W28_LRx_KjXfl1_XyyeO9T--Hb3bOOTNJPN2ZcuuDT9xF3xPShRzGpybPQk6NEbn0cPU6M1FB0yxMIQeJqOE9mlJpUi6MNDLzi4DlBRAZ7fnBeWeif7-fX3AmZZAzaSARMFGIwXiW8UwYDnlXwEVB84sOqTHP1Q-2xbmjfHjpmHyXnZNNp7riTAweLOVXIWwfZNes9_8Ah6mykA |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%E9%9D%A2%E5%90%91%E7%82%B9%E5%87%BB%E7%8E%87%E9%A2%84%E6%B5%8B%E7%9A%84%E8%87%AA%E6%B3%A8%E6%84%8F%E5%8A%9B%E6%B7%B1%E5%BA%A6%E5%9F%9F%E5%B5%8C%E5%85%A5%E5%9B%A0%E5%AD%90%E5%88%86%E8%A7%A3%E6%9C%BA&rft.jtitle=%E5%B7%A5%E7%A8%8B%E7%A7%91%E5%AD%A6%E4%B8%8E%E6%8A%80%E6%9C%AF&rft.au=%E6%9D%8E%E5%B9%BF%E4%B8%BD&rft.au=%E5%8F%B6%E8%89%BA%E6%BA%90&rft.au=%E8%AE%B8%E5%B9%BF%E9%91%AB&rft.au=%E5%BC%A0%E7%BA%A2%E6%96%8C&rft.date=2024-09-01&rft.pub=%E5%8D%8E%E4%B8%9C%E4%BA%A4%E9%80%9A%E5%A4%A7%E5%AD%A6%E4%BF%A1%E6%81%AF%E4%B8%8E%E8%BD%AF%E4%BB%B6%E5%B7%A5%E7%A8%8B%E5%AD%A6%E9%99%A2%2C%E6%B1%9F%E8%A5%BF%E5%8D%97%E6%98%8C+330013&rft.issn=2096-3246&rft.volume=56&rft.issue=5&rft.spage=287&rft.epage=296&rft_id=info:doi/10.12454%2Fj.jsuese.202201373&rft.externalDocID=scdxxb_gckx202405029 |
| thumbnail_s | http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=http%3A%2F%2Fwww.wanfangdata.com.cn%2Fimages%2FPeriodicalImages%2Fscdxxb-gckx%2Fscdxxb-gckx.jpg |