DLBW: an end-to-end multi-task adaptive gradient balancing method

• Published in: Cluster computing Vol. 28; no. 9; p. 610
• Main Authors: Shi, Yayong, Cui, Zenghao, Zhang, Yida, Wang, Rui, Zhu, Haiyan
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.10.2025; Springer Nature B.V
• Subjects: Accuracy; Algorithms; Artificial intelligence; Balancing; Bias; Computer Communication Networks; Computer Science; Datasets; Decision trees; Electronic health records; Machine learning; Medical research; Neural networks; Operating Systems; Optimization; Parameter estimation; Perturbation; Processor Architectures; Saddle points; Sampling techniques; Uncertainty; Gradient descent; AUC maximization; Electronic health record; Class imbalance; Cross-entropy
• ISSN: 1386-7857; 1573-7543
• DOI: 10.1007/s10586-025-05279-z

Medical datasets often suffer from class imbalance, causing AI models optimized via traditional Cross-Entropy (CE) loss to exhibit bias against minority classes. While direct optimization of the Area Under the Curve (AUC) mitigates this bias, it may degrade the discriminative power of learned features. Furthermore, joint optimization of CE and AUC introduces gradient conflicts due to divergent gradient surfaces, making it challenging to balance the two losses. Existing two-stage methods (CE pre-training followed by AUC fine-tuning) struggle to determine the optimal transition point for end-to-end AUC maximization. To address this, we propose DLBW, a dynamic loss balancing method that adaptively switches based on optimization uncertainty. First, a gradient balancing strategy aligns conflicting gradients to maintain synergy. Second, stochastic perturbations help parameters escape saddle points and local optima. Third, an uncertainty-aware weighting mechanism accelerates convergence while guiding models toward global optima via controlled perturbations. Experiments on two medical datasets show that DLBW outperforms CE baselines, improving AUC by 0.7%–2.6% and Precision–Recall Curve (PRC) by 2.4%–2.6%. By enhancing minority class recognition, DLBW enables equitable treatment in clinical applications such as disease diagnosis, reducing bias-induced decision risks for diverse patient populations.