Automated Scene-Adaptive Rock Fragment Recognition Based on the Enhanced Segment Anything Model and Fine-Tuning RTMDet

• Published in: Rock mechanics and rock engineering Vol. 58; no. 3; pp. 3973 - 3999
• Main Authors: Tang, Yudi, Wang, Yulin, Wang, Xin, Oh, Joung, Si, Guangyao
• Format: Journal Article
• Language: English
• Published: Vienna Springer Vienna 01.03.2025; Springer Nature B.V
• Subjects: Algorithms; Annotations; Artificial intelligence; Automation; Bibliographic information; Civil Engineering; Computer vision; Earth and Environmental Science; Earth Sciences; Fragments; Geophysics/Geodesy; Image contrast; Modules; Original Paper; Particle size distribution; Real time; Rock; Rocks; Scene analysis; Segments; Size distribution; Optimal mask selection; Automated annotation; Scene-adaptive framework; Segment anything model; Rock fragment recognition
• ISSN: 0723-2632; 1434-453X; 1434-453X
• DOI: 10.1007/s00603-024-04360-0

The particle-size distribution (PSD) of rock fragments is crucial for various engineering practices. Recently, there has been increasing attention towards new methods for analysing rock fragment PSD based on artificial intelligence and computer vision. However, most existing image-based studies on this topic are case-based, while heavily relying on manual annotation. The proposed algorithms or trained models often exhibit poor performance across different geological and lighting conditions. In this research, an automatic scene-adaptive framework for real-time rock fragment recognition (FragAdaptixAI) is introduced. First, a diverse foundation image set of rock fragments is built, and an automated annotation module based on an improved segment anything model (SAM) is employed to create the foundation rock fragment dataset, which is then utilised for training the foundation model (RTMDet). When faced with new cases, the foundation model is rapidly fine-tuned using a small amount of current case data to derive an application model, supporting real-time automation in rock fragment recognition. FragAdaptixAI was tested using two cases, and the test results demonstrate that (1) the automated annotation module, developed based on the improved SAM, is effective, and (2) FragAdaptixAI achieves excellent results with short tuning time for new cases. The approach introduces a new concept to address the generalisation issues in rock fragment recognition by combining the zero-shot capability of large models with the real-time processing of small models. Highlights FragAdaptixAI is proposed for automatic scene-adaptive real-time rock fragment recognition. Segment anything model is utilized for automatic annotation. The PCCS algorithm for rock fragment image contrast enhancement is proposed. The OMS algorithm is proposed to replace the traditional NMS algorithm.