Exploring the Hidden Challenges Associated with the Evaluation of Multi-class Datasets Using Multiple Classifiers

Shamaila Iram, Dhiya Al-jumeily, Paul Fergus, Abir Hussain

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

The optimization and evaluation of a pattern recognition system requires different problems like multi-class and imbalanced datasets be addressed. This paper presents the classification of multi-class datasets which present more challenges when compare to binary class datasets in machine learning. Furthermore, it argues that the performance evaluation of a classification model for multi-class imbalanced datasets in terms of simple "accuracy rate" can possibly provide misleading results. Other parameters such as failure avoidance, true identification of positive and negative instances of a class and class discrimination are also very important. We, in this paper, hypothesize that "misclassification of true positive patterns should not necessarily be categorized as false negative while evaluating a classifier for multi-class datasets", a common practice that has been observed in the existing literature. In order to address these hidden challenges for the generalization of a particular classifier, several evaluation metrics are compared for a multi-class dataset with four classes, three of them belong to different neurodegenerative diseases and one to control subjects. Three classifiers, linear discriminant, quadratic discriminant and Parzen are selected to demonstrate the results with examples.
Original languageEnglish
Title of host publication2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems (CISIS)
PublisherIEEE
Pages346-352
Number of pages7
ISBN (Electronic)9781479943258
DOIs
Publication statusPublished - Jul 2014
Externally publishedYes
Event8th International Conference on Complex, Intelligent and Software Intensive Systems - Birmingham City University, Birmingham, United Kingdom
Duration: 2 Jul 20144 Jul 2014
Conference number: 8
http://voyager.ce.fit.ac.jp/conf/cisis/2014/ (Link to Conference Website)

Conference

Conference8th International Conference on Complex, Intelligent and Software Intensive Systems
Abbreviated titleCISIS
CountryUnited Kingdom
CityBirmingham
Period2/07/144/07/14
Internet address

    Fingerprint

Cite this

Iram, S., Al-jumeily, D., Fergus, P., & Hussain, A. (2014). Exploring the Hidden Challenges Associated with the Evaluation of Multi-class Datasets Using Multiple Classifiers. In 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems (CISIS) (pp. 346-352). IEEE. https://doi.org/10.1109/CISIS.2014.48