Dissertation_Jieqiong_Zhao_deposit.pdf (13.53 MB)

Visual Analytics for Decision Making in Performance Evaluation

Download (13.53 MB)
thesis
posted on 05.05.2020, 23:05 by Jieqiong Zhao
Performance analysis often considers numerous factors contributing to performance, and the relative importance of these factors is evolving based on dynamic conditions and requirements. Investigating large numbers of factors and understanding individual factors' predictability within the ultimate performance are challenging tasks. A visual analytics approach that integrates interactive analysis, novel visual representations, and predictive machine learning models can provide new capabilities to examine performance effectively and thoroughly. Currently, only limited research has been done on the possible applications of visual analytics for performance evaluation. In this dissertation, two specific types of performance analysis are presented: (1) organizational employee performance evaluation and (2) performance improvement of machine learning models with interactive feature selection. Both application scenarios leverage the human-in-the-loop approach to assist the identification of influential factors. For organizational employee performance evaluation, a novel visual analytics system, MetricsVis, is developed to support exploratory organizational performance analysis. MetricsVis incorporates hybrid evaluation metrics that integrate quantitative measurements of observed employee achievements and subjective feedback on the relative importance of these achievements to demonstrate employee performance at and between multiple levels regarding the organizational hierarchy. MetricsVis II extends the original system by including actual supervisor ratings and user-guided rankings to capture preferences from users through derived weights. Comparing user preferences with objective employee workload data enables users to relate user evaluation to historical observations and even discover potential bias. For interactive feature selection and model evaluation, a visual analytics system, FeatureExplorer, allows users to refine and diagnose a model iteratively by selecting features based on their domain knowledge, interchangeable features, feature importance, and the resulting model performance. FeatureExplorer enables users to identify stable, trustable, and credible predictive features that contribute significantly to a prediction model.

Funding

U.S. Department of Homeland Security VACCINE Center under Award Number 2009-ST-061-CI0003

Automated Sorghum Phenotyping and Trait Development Platform

Advanced Research Projects Agency-Energy

Find out more...

History

Degree Type

Doctor of Philosophy

Department

Electrical and Computer Engineering

Campus location

West Lafayette

Advisor/Supervisor/Committee Chair

David S. Ebert

Advisor/Supervisor/Committee co-chair

Niklas Elmqvist

Additional Committee Member 2

Alexander J. Quinn

Additional Committee Member 3

Edward J. Delp

Additional Committee Member 4

Melba M. Crawford

Licence

Exports