DSCTool: A web-service-based framework for statistical comparison of stochastic optimization algorithms

• Published in: Applied soft computing Vol. 87; p. 105977
• Main Authors: Eftimov, Tome, Petelin, Gašper, Korošec, Peter
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.02.2020
• Subjects: Benchmarking; DSCTool; Statistical tool; Stochastic optimization algorithms; Web service; Stochastic optimization algorithms; Benchmarking; Statistical tool; DSCTool; Web service
• ISSN: 1568-4946; 1872-9681; 1872-9681
• DOI: 10.1016/j.asoc.2019.105977

DSCTool is a statistical tool for comparing performance of stochastic optimization algorithms on a single benchmark function (i.e. single-problem analysis) or a set of benchmark functions (i.e., multiple-problem analysis). DSCTool implements a recently proposed approach, called Deep Statistical Comparison (DSC), and its variants. DSC ranks optimization algorithms by comparing distributions of obtained solutions for a problem instead of using a simple descriptive statistic such as the mean or the median. The rankings obtained for an individual problem give the relations between the performance of the applied algorithms. To compare optimization algorithms in the multiple-problem scenario, an appropriate statistical test must be applied to the rankings obtained for a set of problems. The main advantage of DSCTool are its REST web services, which means all its functionalities can be accessed from any programming language. In this paper, we present the DSCTool in detail with examples for its usage. •DSCTool - a statistical tool for comparing stochastic optimization algorithms.•REST web services implementation allows access from any programming language.•Identification of statistical and practical significance.•Understanding of exploitation and exploration powers of single-objective algorithms.•Ranking of multi-objective algorithms using an ensemble of quality indicators.