Transfer Learning Aurora Image Classification and Magnetic Disturbance Evaluation

• Published in: Journal of geophysical research. Space physics Vol. 127; no. 1
• Main Authors: Sado, P., Clausen, L. B. N., Miloch, W. J., Nickisch, H.
• Format: Journal Article
• Language: English
• Published: 01.01.2022
• Subjects: all sky imager; aurora; auroral imaging; machine learning
• ISSN: 2169-9380; 2169-9402; 2169-9402
• DOI: 10.1029/2021JA029683

We develop an open source algorithm to apply Transfer learning to Aurora image classification and Magnetic disturbance Evaluation (TAME). For this purpose, we evaluate the performance of 80 pretrained neural networks using the Oslo Auroral THEMIS (OATH) data set of all‐sky images, both in terms of runtime and their features' predictive capability. From the features extracted by the best network, we retrain the last neural network layer using the Support Vector Machine (SVM) algorithm to distinguish between the labels “arc,” “diffuse,” “discrete,” “cloud,” “moon” and “clear sky/ no aurora”. This transfer learning approach yields 73% accuracy in the six classes; if we aggregate the 3 auroral and 3 non‐aurora classes, we achieve up to 91% accuracy. We apply our classifier to a new dataset of 550,000 images and evaluate the classifier based on these previously unseen images. To show the potential usefulness of our feature extractor and classifier, we investigate two test cases: First, we compare our predictions for the “cloudy” images to meteorological data and second we train a linear ridge model to predict perturbations in Earth's locally measured magnetic field. We demonstrate that the classifier can be used as a filter to remove cloudy images from datasets and that the extracted features allow to predict magnetometer measurements. All procedures and algorithms used in this study are publicly available, and the code and classifier are provided, which opens possibility for large scale studies of all‐sky images. Plain Language Summary In the interest of auroral research and space physics, many images capturing the night sky have been taken automatically over the last decades. Sifting through these images manually takes a lot of time and is generally impractical. We use Convolutional Neural Networks (CNN), which are good at image classification to extract a set of numbers per image (“features”) that capture the essential contents of the image. A Support Vector Machine (SVM) is trained to interpret these features and assign labels to the images. We search for the best configuration between different CNNs and SVMs and achieve up to 91% accuracy. To show that our method can be extended to other datasets, we classify half a million images from a different dataset and evaluate the performance of our classifier based on these results. We show that our classifier also excels at detecting clouds in images. It can therefore be used to filter unusable images from this kind of datasets. Based on the images' features, we create a model to predict disturbances in the Earth's local magnetic field. To enable other researches to work with our results, we use industry‐standard, open‐source software and make our algorithms and results available the same way. Key Points A pretrained feature extractor and a subsequent classifier can successfully detect aurora in all‐sky images A validation on unknown, partially manually categorized images achieved a classification accuracy of 91% We predict physical quantities such as magnetic disturbance and cloud height from the underlying image feature representation