ML-ModelExplorer: An Explorative Model-Agnostic Approach to Evaluate and Compare Multi-class Classifiers - Machine Learning and Knowledge Extraction
Conference Papers Year : 2020

ML-ModelExplorer: An Explorative Model-Agnostic Approach to Evaluate and Compare Multi-class Classifiers

Abstract

A major challenge during the development of Machine Learning systems is the large number of models resulting from testing different model types, parameters, or feature subsets. The common approach of selecting the best model using one overall metric does not necessarily find the most suitable model for a given application, since it ignores the different effects of class confusions. Expert knowledge is key to evaluate, understand and compare model candidates and hence to control the training process. This paper addresses the research question of how we can support experts in the evaluation and selection of Machine Learning models, alongside the reasoning about them. ML-ModelExplorer is proposed – an explorative, interactive, and model-agnostic approach utilising confusion matrices. It enables Machine Learning and domain experts to conduct a thorough and efficient evaluation of multiple models by taking overall metrics, per-class errors, and individual class confusions into account. The approach is evaluated in a user-study and a real-world case study from football (soccer) data analytics is presented.ML-ModelExplorer and a tutorial video are available online for use with own data sets: www.ml-and-vis.org/mex
Fichier principal
Vignette du fichier
497121_1_En_16_Chapter.pdf (5.47 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03414731 , version 1 (04-11-2021)

Licence

Identifiers

Cite

Andreas Theissler, Simon Vollert, Patrick Benz, Laurentius A. Meerhoff, Marc Fernandes. ML-ModelExplorer: An Explorative Model-Agnostic Approach to Evaluate and Compare Multi-class Classifiers. 4th International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE), Aug 2020, Dublin, Ireland. pp.281-300, ⟨10.1007/978-3-030-57321-8_16⟩. ⟨hal-03414731⟩
86 View
68 Download

Altmetric

Share

More