ICIP-2022-EDUCATION-SHORT-COURSE

ICIP 2022 Education Short Course (Live)

Title: Exploring the Impact of Explanatory Paradigms in Machine Learning

Presenters: Ghassan AlRegib, and Mohit Prabhushankar

Omni Lab for Intelligent Visual Engineering and Science (OLIVES)

School of Electrical and Computer Engineering

Georgia Institute of Technology, Atlanta, USA

1. Course Overview

Visual explanations have traditionally acted as rationales used to justify the decisions made by machine learning systems. With the advent of large-scale neural networks, the role of visual explanations has been to shed interpretability on black-box models. We view this role as the process for the network to answer the question `Why P?’, where P is a trained network’s prediction. Recently however, with increasingly capable models, the role of explainability has expanded. Neural networks are asked to justify `What if?’ counterfactual and `Why P, rather than Q?’ contrastive question modalities that the network did not explicitly train to answer. This allows explanations to act as reasons to make further prediction. The short course provides a principled and rational introduction into Explainability within machine learning and justifies them as reasons to make decisions. Such a reasoning framework allows for robust machine learning as well as trustworthy AI to be accepted in everyday lives. Applications like robust recognition, image quality assessment, visual saliency, anomaly detection, out-of-distribution detection, adversarial image detection, seismic interpretation, semantic segmentation, and machine teaching among others will be discussed together with several hands-on experiments in topics that relate to explainability and anomaly detection.

2. Learning Outcome

  • Basics of explainability in neural networks – their function, types, shortcomings, evaluation, and reasoning paradigms
  • Understanding applicability of reason-based explainability across applications and data modalities. Applications include robust recognition, anomaly detection, visual saliency, and machine teaching. Data modalities include natural images, and computed seismic and biomedical images
  • Hands-on google colab codes for correlation, counterfactual, and contrastive explanations, and anomaly detection

3. Syllabus and Presenters

Day 1 (4 hrs)

Lecture 1: Introduction to Explainability in Neural Networks (1hr 30 mins) (Ghassan AlRegib)

  • Explainability in ML: Definition, role, and need for Explainability 
  • Categorizations of Explainability
    • Implicit vs Explicit explanations 
    • Black box vs White box explanations 
    • Interventionist vs Non-interventionist explanations 
  • Overview of Explainability in Neural Networks
    • Explainability in neural networks 
    • Dimensionality reduction in last layer embedding
    • Visualizing activations 
    • Gradient-based visualizations 
    • Saliency maps and intermediate feature visualizations 
    • CAM visualizations and explanations 
    • GradCAM visualization and explanations 
    • Examples of applications: robust recognition, image quality assessment, visual saliency, anomaly detection, out-of-distribution detection, adversarial image detection, seismic interpretation, semantic segmentation, and machine teaching

Lecture 2: Explanations as Reasons: Towards explanatory paradigms (1hr 30 mins) (Mohit Prabhushankar)

  • Reasoning in AI
    • Deductive reasoning 
    • Inductive reasoning 
    • Abductive Reasoning 
  • Significance of Explanations
    • As justifications of decisions 
    • As assistants in making decisions  
  • Explanatory Paradigms
    • Types of Explanations 
    • Indirect and Direct Explanations 
    • Targeted Explanations 
    • Explanatory Paradigms  
    • Examples
      • Recognition in natural and seismic images
      • Image Quality Assessment
  • Complete explanations 

Outlook for Day 1 (1hr) (Mohit Prabhushankar)

Google Colab codes and hands-on experience with Grad-CAM, contrastive and counterfactual explanations 

Day 2 (4 hrs)

Lecture 3: Impact of Explanations 1: Towards Robust Neural Networks (1hr 30 mins) (Ghassan AlRegib)

  • Recap of significance of explanations from Day 1
    • As justifications of decisions 
    • As assistants in making decisions  
  • Utilizing explanations in making decisions
    • The effectiveness of contrastive reasoning to provide better representation space than Inductive reasoning 
    • Robustness in Neural Networks
      • Robust classification in the presence of noise 
      • Robust adversarial image detection 

Lecture 4: Impact of Explanations 2: Towards Robust Neural Networks (1hr 30 mins) (Mohit Prabhushankar)

  • Utilizing explanations in making decisions
    • Robust out-of-distribution detection 
  • Anomaly Detection in Neural Networks
    • Anomaly detection in images 
    • Statistical analysis of anomalies 
    • Performance metrics 
    • Anomaly Detection settings 
    • Gradient-based explanation-based anomaly detection  

Outlook for Day 2 (1hr) (Mohit Prabhushankar)

Google Colab codes and hands-on experience for robust detection of anomalies on CURE-OR dataset

Day 3 (2 hrs)

Lecture 5: Impact of Explanations 3: Towards Trust and Evaluation (1hr 30 mins) (Mohit Prabhushankar)

  • Explanatory evaluation taxonomy
    • Direct evaluations 
    • Indirect evaluations 
    • Targeted evaluations 
  • Direct evaluations
    • Human evaluations 
    • Ethical considerations for human evaluations 
    • Examples for direct evaluations: Amazon Mechanical Turk 
  • Indirect evaluations
    • Human Visual Saliency 
    • Attention in Transformers 
  • Targeted Evaluations
    • Examples of targeted evaluations through robustness 
    • Machine Teaching and examples on seismic data 

Course Evaluation (30 mins) (Mohit Prabhushankar)

4. Target audience, and the expected prerequisite technical knowledge

The targeted audiences are senior-year undergraduate, postgraduate, engineers and practitioners, with some background in python and machine learning.

It is also preferred that the audience are aware of the basics of building and analyzing deep learning models in PyTorch in a google collab environment. If not, then we provide resources in the pre-reading section to familiarize audience with the requisite skill set.

5. Supporting course resources, software, tools and readings

  • Lecture notes from the materials presented in the course.
  • References to papers for specific details taught during the course
  • Hands-on Google Colab codes for the attendees who can implement the discussed explanatory concepts either synchronously in real time (on their personal laptops) or asynchronously at their convenience. 

6. Pre-reading:

If you are new to colab and PyTorch (highly recommended that the audience complete this tutorial beforehand):

Suggested Reading:

  • AlRegib, G., & Prabhushankar, M. (2022). Explanatory Paradigms in Neural Networks: Towards relevant and contextual explanations. IEEE Signal Processing Magazine, 39(4), 59-72
  • Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision (pp. 618-626)
  • Prabhushankar, M., & AlRegib, G. (2021). Contrastive Reasoning in Neural Networks. arXiv preprint arXiv:2103.12329.
  • Goyal, Y., Wu, Z., Ernst, J., Batra, D., Parikh, D., & Lee, S. (2019, May). Counterfactual visual explanations. In International Conference on Machine Learning (pp. 2376-2384). PMLR
  • G. Kwon*, M. Prabhushankar*, D. Temel, and G. AlRegib, “Distorted Representation Space Characterization Through Backpropagated Gradients,” in IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, Sep. 2019.
  • G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, “Backpropagated Gradient Representations for Anomaly Detection,” in Proceedings of the European Conference on Computer Vision (ECCV), SEC, Glasgow, Aug. 23-28 2020
  • Temel, D., Lee, J., & AlRegib, G. (2018, December). Cure-or: Challenging unreal and real environments for object recognition. In 2018 17th IEEE international conference on machine learning and applications (ICMLA) (pp. 137-144). IEEE

7. Hands-on or lab components of the short course (For virtual course, please indicate how this would work over Zoom)

Google Colab codes and libraries will be provided for hands-on practice for correlation, counterfactual, and contrastive explanations, and anomaly detection

8. Presenters’ contact information and short biography

PresenterShort Biography
Ghassan AlRegib
alregib@gatech.edu		Prof. AlRegib is currently the Marilu and John McCarty Chair Professor of Electrical and Computer Engineering at the Georgia Institute of Technology. His group is the Omni Lab for Intelligent Visual Engineering and Science (OLIVES) at Georgia Tech. He has authored and co-authored more than 275 articles in international journals and conference proceedings. He has been issued several U.S. patents and invention disclosures. He is a Fellow of the IEEE. Prof. AlRegib received the ECE Outstanding Graduate Teaching Award in 2001 and both the CSIP Research and the CSIP Service Awards in 2003. In 2008, he received the ECE Outstanding Junior Faculty Member Award. In 2017, he received the 2017 Denning Faculty Award for Global Engagement. He served the community at various capacities including TPC chair for ICIP’20 and GlobalSIP’14. He on the Editorial Boards for IEEE Transactions on Image Processing and Elsevier Journal on Signal Processing: Image Communications. He has served as a consultant for a number of global education organizations and as a witness expert on a number of patent infringement cases.
Mohit Prabhushankar
mohit.p@gatech.edu		Mohit Prabhushankar received his Ph.D. degree in electrical engineering from the Georgia Institute of Technology (Georgia Tech), Atlanta, Georgia, 30332, USA, in 2021. He is currently a Postdoctoral Research Fellow in the School of Electrical and Computer Engineering at the Georgia Institute of Technology in the Omni Lab for Intelligent Visual Engineering and Science (OLIVES). He is working in the fields of image processing, machine learning, active learning, healthcare, and robust and explainable AI. He is the recipient of the Best Paper award at ICIP 2019 and Top Viewed Special Session Paper Award at ICIP 2020. He is the recipient of the ECE Outstanding Graduate Teaching Award, the CSIP Research award, and of the Roger P Webb ECE Graduate Research Assistant Excellence award, all in 2022.

9. Recent related publications

  1. G. AlRegib and M. Prabhushankar, “Explanatory Paradigms in Neural Networks,” in IEEE Signal Processing Magazine, Special Issue on Explainability in Data Science, Feb. 18 2022. [PDF][Code]
  2. G. AlRegib, M. Deriche, Z. Long, H. Di, Z. Wang, Y. Alaudah, M. A. Shafiq, M. Alfarraj, “Subsurface Structure Analysis Using Computational Interpretation and Learning,” in IEEE Signal Processing Magazine, vol. 35, no. 2, pp. 82-98, Mar. 2018. [PDF]
  3. D. Temel, M-H. Chen, and G. AlRegib, “Traffic Sign Detection Under Challenging Conditions: A Deeper Look Into Performance Variations and Spectral Characteristics,” in IEEE Transactions on Intelligent Transportation Systems, Jul. 2019. [PDF][Code]
  4. D. Temel*, J. Lee*, and G. AlRegib, “CURE-OR: Challenging Unreal and Real Environments for Object Recognition,” in IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, Dec. 2018 [PDF][Code]
  5. D. Temel, G. Kwon*, M. Prabhushankar*, and G. AlRegib, “CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition,” in Advances in Neural Information Processing Systems (NIPS) Workshop on Machine Learning for Intelligent Transportation Systems, Long Beach, CA, Dec. 2017 [PDF][Code]
  6. M. Prabhushankar, G. Kwon, D. Temel, and G. AlRegib, “Contrastive Explanations in Neural Networks,” in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020. [PDF][Code][Video]
  7. M. Prabhushankar and G. AlRegib, “Extracting Causal Visual Features for Limited Label Classification,” in IEEE International Conference on Image Processing (ICIP), Anchorage, AK, Sep. 19-22 2021. [PDF] [Code]
  8. G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, “Backpropagated Gradient Representations for Anomaly Detection,” in Proceedings of the European Conference on Computer Vision (ECCV), SEC, Glasgow, Aug. 23-28 2020. [PDF][Code][Short Video]
  9. G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, “Novelty Detection Through Model-Based Characterization of Neural Networks,” in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020. [PDF][Code][Video]
  10. Y. Sun, M. Prabhushankar, and G. AlRegib, “Implicit Saliency in Deep Neural Networks,” in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020. [PDF][Code][Video]
  11. J. Lee and G. AlRegib, “Gradients as a Measure of Uncertainty in Neural Networks,” in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020. [PDF][Video]
  12. J. Lee and G. AlRegib, “Open-Set Recognition with Gradient-Based Representations,” in IEEE International Conference on Image Processing (ICIP), Anchorage, AK, Sep. 19-22 2021. [PDF]
  13. G. Kwon*, M. Prabhushankar*, D. Temel, and G. AlRegib, “Distorted Representation Space Characterization Through Backpropagated Gradients,” in IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, Sep. 2019. [PDF][Code]
  14. S. Liu, C. Lehman, and G. AlRegib, “Robustness and Overfitting Behavior of Implicit Background Models,” in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020. [PDF][Code][Video]
Print Friendly, PDF & Email