Monitoring machine learning models: a categorization of challenges and methods

The importance of software based on machine learning is growing rapidly, but the potential of prototypes may not be realized in operation. This study identified six categories of challenges for verification and validation of machine learning applications during production. Subsequently, monitoring w...

Full description

Saved in:
Bibliographic Details
Published inData science and management Vol. 5; no. 3; pp. 105 - 116
Main Authors Schröder, Tim, Schulz, Michael
Format Journal Article
LanguageEnglish
Published Elsevier B.V 01.09.2022
KeAi Communications Co. Ltd
Subjects
Machine learning
Monitoring
Operations
Taxonomy
Taxonomy
Operations
Monitoring
Machine learning
Online AccessGet full text
ISSN2666-7649
2666-7649
DOI10.1016/j.dsm.2022.07.004

Cover

More Information
Summary:The importance of software based on machine learning is growing rapidly, but the potential of prototypes may not be realized in operation. This study identified six categories of challenges for verification and validation of machine learning applications during production. Subsequently, monitoring was analyzed as a possible solution to mitigate those challenges. Capturing relevant data and model metrics may reveal problems at an early stage, allowing for targeted countermeasures. This study presents a taxonomy of methods and metrics currently addressed in scientific literature and compares these categories with case studies from practice.
ISSN:2666-7649
2666-7649
DOI:10.1016/j.dsm.2022.07.004