Integrating transformer and autoencoder techniques with spectral graph algorithms for the prediction of scarcely labeled molecular data

• Published in: Computers in biology and medicine Vol. 153; p. 106479
• Main Authors: Hayes, Nicole, Merkurjev, Ekaterina, Wei, Guo-Wei
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.02.2023; Elsevier Limited
• Subjects: Algorithms; Autoencoder; Computer applications; Datasets; Decision trees; Internal Medicine; Other; R-S scores; Random Forest; Scarcely labeled data; Small data; Spectral graph; Support Vector Machine; Support vector machines; Transfer learning; Transformer; Transformers; Spectral graph; Small data; Scarcely labeled data; R-S scores; Transformer; Autoencoder
• ISSN: 0010-4825; 1879-0534; 1879-0534
• DOI: 10.1016/j.compbiomed.2022.106479

In molecular and biological sciences, experiments are expensive, time-consuming, and often subject to ethical constraints. Consequently, one often faces the challenging task of predicting desirable properties from small data sets or scarcely-labeled data sets. Although transfer learning can be advantageous, it requires the existence of a related large data set. This work introduces three graph-based models incorporating Merriman–Bence–Osher (MBO) techniques to tackle this challenge. Specifically, graph-based modifications of the MBO scheme are integrated with state-of-the-art techniques, including a home-made transformer and an autoencoder, in order to deal with scarcely-labeled data sets. In addition, a consensus technique is detailed. The proposed models are validated using five benchmark data sets. We also provide a thorough comparison to other competing methods, such as support vector machines, random forests, and gradient boosting decision trees, which are known for their good performance on small data sets. The performances of various methods are analyzed using residue-similarity (R-S) scores and R-S indices. Extensive computational experiments and theoretical analysis show that the new models perform very well even when as little as 1% of the data set is used as labeled data. [Display omitted] •Three new machine learning models for data classification are proposed in this paper.•The proposed methods are very useful for the case of a small amount of labeled data.•The proposed techniques incorporate molecular fingerprints for multi-task learning.•We integrate autoencoders, bidirectional encoder transformers, circular fingerprints.•The performances are analyzed using residue-similarity (R-S) scores and R-S indices.