Trainable segmentation for transmission electron microscope images of inorganic nanoparticles

• Published in: Journal of microscopy (Oxford) Vol. 288; no. 3; pp. 169 - 184
• Main Authors: Bell, Cameron G., Treder, Kevin P., Kim, Judy S., Schuster, Manfred E., Kirkland, Angus I., Slater, Thomas J. A.
• Format: Journal Article
• Language: English
• Published: England Wiley Subscription Services, Inc 01.12.2022; John Wiley and Sons Inc
• Subjects: Algorithms; Bayes Theorem; Bayesian analysis; Classifiers; Electron microscopes; Electron microscopy; Image contrast; Image processing; Image Processing, Computer-Assisted - methods; Image segmentation; Kernels; Learning algorithms; Machine learning; metal nanoparticles; Microscopy, Electron, Transmission; Nanoparticles; Neural networks; Neural Networks, Computer; Pixels; scanning transmission electron microscopy; Themed Issue; Transmission electron microscopy; scanning transmission electron microscopy; transmission electron microscopy; machine learning; metal nanoparticles; image segmentation
• ISSN: 0022-2720; 1365-2818; 1365-2818
• DOI: 10.1111/jmi.13110

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user‐labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a random forest classifier performs best for high‐contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low‐contrast TEM images. Lay description Measurement of the size, shape and composition of nanoparticles is routinely performed using transmission electron microscopy and related techniques. Typically, distinguishing particles from the background in an image is performed using the intensity of each pixel, creating two sets of pixels to separate particles from background. However, this separation of intensity can be difficult if the contrast in an image is low, or if the intensity of the background varies significantly. In this study, an approach that takes into account additional image features (such as boundaries and texture) was investigated to study electron microscope images of metallic nanoparticles. In this ‘trainable segmentation’ approach, the user labels examples of particle and background pixels in order to train a machine learning algorithm to distinguish between particles and background. The performance of different machine learning algorithms was investigated, in addition to the effect of using different features to aid the segmentation. Overall, a trainable segmentation approach was found to perform better than use of an intensity threshold to distinguish between particles and background in electron microscope images.