Comprehensive Multidimensional Evaluation of Deep Models

My Contributions

• Led the investigation of deep model evaluation techniques, establishing a comprehensive, rigorous, and insightful evaluation framework for modern deep learning models.
• Led the creation of four survey papers and one research paper, with two currently under major revision at ACM Computing Surveys.
• Led the development of an online deep model evaluation platform, integrating multidimensional assessments across correctness, robustness, fairness, and transferability.

Papers

1. How Correct Is Your Vision Model? A Comprehensive Survey
   Jiacong Hu, Haoze Jiang, Jinxun Wu, Guoxiang Li, Zunlei Feng, Mingli Song

2. Robustness Evaluation for Deep Vision Models: A Comprehensive Survey
   Guoxiang Li, Jiacong Hu, Yongcheng Jing, Haoze Jiang, Jinxun Wu, Zunlei Feng, Mingli Song

3. A Comprehensive Survey of Fairness Evaluation in Computer Vision Tasks
   Haoze Jiang, Jiacong Hu, Guoxiang Li, Jinxun Wu, Jingwen Ye, Zunlei Feng, Mingli Song

4. A Comprehensive Survey and Benchmarking on Deep Model Transferability Evaluation
   Jinxun Wu, Jiacong Hu, Haoze Jiang, Guoxiang Li, Lechao Cheng, Zunlei Feng, Mingli Song

5. Beyond the Label: Unveiling Fairness through Dynamic Attribute Projections in Classification
   Haoze Jiang, Zunlei Feng, Jiacong Hu, Bingde Hu, Mingli Song, Yuanyu Wan

Model Evaluation Platform

The online model evaluation platform (Project Link) offers a robust suite of tools for assessing key model attributes, including correctness, robustness, fairness, and transferability. Each attribute is evaluated through multiple perspectives and metrics, providing a thorough and flexible assessment of model performance.

Key features of the platform include:

• Customizable evaluation: Users can define visual tasks, datasets, metrics, and model parameters to tailor assessments to their needs.
• Comprehensive benchmarking: Supports in-depth comparisons across core computer vision tasks, including image classification, object detection, and image segmentation.
• Intuitive result visualization: Enables clear interpretation of evaluation outcomes, offering actionable insights for model optimization.

Looking ahead, we plan to expand the platform’s capabilities to keep pace with advancements in deep learning and evolving application needs, ensuring a more comprehensive and powerful evaluation tool for researchers and practitioners.