SpySMAC: Automated Configuration and Performance Analysis of SAT Solvers

• Published in: Theory and Applications of Satisfiability Testing -- SAT 2015 Vol. 9340; pp. 215 - 222
• Main Authors: Falkner, Stefan, Lindauer, Marius, Hutter, Frank
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2015; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Artificial intelligence; Bayesian Optimization; Computer science; Local Search; Machine Learning Model; Mathematical theory of computation; Parameter Importance; Random Forest
• ISBN: 3319243179; 9783319243177
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-24318-4_16

Most modern SAT solvers expose a range of parameters to allow some customization for improving performance on specific types of instances. Performing this customization manually can be challenging and time-consuming, and as a consequence several automated algorithm configuration methods have been developed for this purpose. Although automatic algorithm configuration has already been applied successfully to many different SAT solvers, a comprehensive analysis of the configuration process is usually not readily available to users. Here, we present SpySMAC to address this gap by providing a lightweight and easy-to-use toolbox for (i) automatic configuration of SAT solvers in different settings, (ii) a thorough performance analysis comparing the best found configuration to the default one, and (iii) an assessment of each parameter’s importance using the fANOVA framework. To showcase our tool, we apply it to Lingeling and probSAT, two state-of-the-art solvers with very different characteristics.