Deep Statistical Comparison for Multi-Objective Stochastic Optimization Algorithms

• Published in: Swarm and evolutionary computation Vol. 61; p. 100837
• Main Authors: Eftimov, Tome, Korošec, Peter
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.03.2021
• Subjects: benchmarking; high-dimensional data; multi-objective problems; statistical comparison; stochastic algorithms; multi-objective problems; stochastic algorithms; benchmarking; high-dimensional data; statistical comparison
• ISSN: 2210-6502; 2210-6510
• DOI: 10.1016/j.swevo.2020.100837

Making a statistical comparison of meta-heuristic multi-objective optimization algorithms is crucial for identifying the strengths and weaknesses of a newly proposed algorithm. Currently, state-of-the-art comparison approaches involve user-preference-based selection of a single quality indicator or an ensemble of quality indicators as a comparison metric. Using these quality indicators, high-dimensional data is transformed into one-dimensional data. By doing this, information contained in the high-dimensional space can be lost, which will affect the results of the comparison. To avoid losing this information, we propose a novel ranking scheme that compares the distributions of high-dimensional data. Experimental results show that the proposed approach reduces potential information loss when statistical significance is not observed in high-dimensional data. Consequently, the selection of a quality indicator is required only in cases when statistical significance is observed in high-dimensional data. With this the cases that are affected by the user preference selection are reduced.