Location: Coda, 756 W Peachtree St NW, Atlanta, GA 30308
Date: November 13-15, 2024
Click on each item for more details
Session: Fault & Leakage Detection and Segmentation Approaches
-
Dropdown Example J. Quesada, M. Prabhushankar, and G. AlRegib, “Crowdsourcing Annotations for Fault Segmentation: Benchmarking Label Sources,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Segmenting faults is of paramount importance in the seismic interpretation pipeline, albeit involving both costly and labor-intensive expert annotation. Alternatives to expertly labeled data imply often relying on synthetic data or weakly labeled data. In this work, we present the CRACKS dataset, a comprehensive fault segmentation dataset spanning labels across multiple levels of expertise and confidence. We benchmark the effectiveness of this dataset by evaluating different machine learning strategies to exploit its multifaceted structure, as well as comparing it with the results we achieve when using either synthetic or weak labels sources.
Publications
- C. Zhou, M. Prabhushankar, and G. AlRegib, “On the Ramifications of Human Label Uncertainty,” in NeurIPS 2022 Workshop on Human in the Loop Learning, Oct. 27, 2022, [PDF], [Code].
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Perceptual Quality-Based Model Training Under Annotator Label Uncertainty,” in International Meeting for Applied Geoscience & Energy (IMAGE) 2023, Houston, TX, Aug. 28-Sept. 1, 2023, [PDF], [Code].
- M. Prabhushankar, K. Kokilepersaud, J. Quesada, Y. Yarici, C. Zhou, M. Alotaibi, G. AlRegib, A. Mustafa, Y. Kumakov, “CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface Faults,” arXiv preprint, 2024, [PDF], [Code].
- J. Quesada, et al., “A Crowdsourced Machine Learning Benchmark for Fault Segmentation,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “A Unified View of Generative Modeling for Seismic Interpretation,” submitted to The Leading Edge (TLE), Oct. 16, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Generative Modeling of Disagreements for Expertise-based Seismic Fault Labels,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Uncertainty in Seismic Image Interpretation,” under review, 2024.
-
C. Zhou, J. Quesada, Y. Yarici, M. Prabhushankar, and G. AlRegib, “Expertise-based Label Fusion for Seismic Fault Delineation,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Abstract. In this work, we present an effective fusion framework that utilizes annotations across multiple levels of expertise to enhance the ML model’s performance on fault delineation. In another presentation, crowdsourced annotations are shown to be useful. Commonly, crowdsourced labels exhibit expertise-based discrepancies. The question is how to utilize labels from different expertise levels to enhance the ML model’s performance. Our intuition is that the labels from multiple expertise levels contain complementary information, which can be fused during pre-training to effectively approximate expert-level annotations. We validate our intuition on the CRACKS dataset. We pre-train a fault delineation model with fusion labels from two expertise levels, and then fine-tune it with lesser amount of expert-level labels. We then conduct a study on label fusion between multiple practitioners and novices in different weighting configurations. Our results show that 1) label fusion from different expertise during pre-training enhances fault delineation, and 2) better performance can be achieved with larger weight on higher expertise.
Publications
- C. Zhou, M. Prabhushankar, and G. AlRegib, “On the Ramifications of Human Label Uncertainty,” in NeurIPS 2022 Workshop on Human in the Loop Learning, Oct. 27, 2022, [PDF], [Code].
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Perceptual Quality-Based Model Training Under Annotator Label Uncertainty,” in International Meeting for Applied Geoscience & Energy (IMAGE) 2023, Houston, TX, Aug. 28-Sept. 1, 2023, [PDF], [Code].
- M. Prabhushankar, K. Kokilepersaud, J. Quesada, Y. Yarici, C. Zhou, M. Alotaibi, G. AlRegib, A. Mustafa, Y. Kumakov, “CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface Faults,” arXiv preprint, 2024, [PDF], [Code].
- J. Quesada, et al., “A Crowdsourced Machine Learning Benchmark for Fault Segmentation,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “A Unified View of Generative Modeling for Seismic Interpretation,” submitted to The Leading Edge (TLE), Oct. 16, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Generative Modeling of Disagreements for Expertise-based Seismic Fault Labels,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Uncertainty in Seismic Image Interpretation,” under review, 2024.
-
P. Chowdhury, M. Prabhushankar, and G. AlRegib, “Leveraging Uncertainty and Disagreement for Enhanced Annotation in Seismic Interpretation,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Data selection for deep learning in seismic interpretation is crucial, especially given the challenges of label scarcity and interpreter disagreement. Effective training relies on identifying the most informative samples, yet seismic datasets are often limited and subject to inconsistencies among interpreters.
To address these challenges, we propose a novel data selection framework that incorporates interpretation disagreement as a key factor. The framework:
- Models disagreement through representation shifts within neural networks.
- Enhances data selection by focusing on geologically significant regions.
- Integrates active learning to offer a comprehensive strategy for training set selection.
Experimental results demonstrate that our method consistently outperforms traditional active learning methods, achieving:
- Up to a 12% improvement in mean intersection-over-union.
- Better generalization of deep learning models in seismic interpretation.
These findings underscore the potential of incorporating uncertainty and disagreement to improve seismic interpretation.
Publications
- M. Prabhushankar, K. Kokilepersaud, J. Quesada, Y. Yarici, C. Zhou, M. Alotaibi, G. AlRegib, A. Mustafa, Y. Kumakov, “CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface Faults,” arXiv preprint, 2024. [PDF], [Code]
- J. Quesada, et al., “A Crowdsourced Machine Learning Benchmark for Fault Segmentation,” under review, 2024.
- P. Chowdhury, et al., “Leveraging Uncertainty & Disagreement for Enhanced Annotation in Seismic Interpretation,” submitted to TGRS, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “A Unified View of Generative Modeling for Seismic Interpretation,” submitted to The Leading Edge (TLE), Oct. 16, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Generative Modeling of Disagreements for Expertise-based Seismic Fault Labels,” under review, 2024.
-
C. Zhou, M. Prabhushankar, and G. AlRegib, “Disagreement-based Seismic Fault Labeling with Reduced Expert Annotations,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: In this work, we discuss the potential of leveraging labels from lay annotators to enhance seismic fault interpretation while reducing the need for expert labeling. Interpretations exhibit disagreement within and between different levels of expertise, e.g., a geophysicist expert and less experienced practitioners. Conventionally, this disagreement is viewed as disadvantageous for machine learning models that rely on gold standard labels for training.
We show that leveraging practitioner-labeled faults in the seismic sections that exhibit less expertise-based disagreements can reduce the need for expert labeling. Thus, it is important to characterize expertise-based disagreements. To this end, we develop a framework that:
- Identifies a small number of seismic sections with the highest degree of expertise-based disagreements for expert labeling.
- Uses practitioner annotations on the large remaining data to augment the training set.
Our findings demonstrate that:
- Augmenting with a large number of faults labeled by lay annotators achieves better fault interpretation than using only a small number of expert labels.
- This approach is more effective than using synthetic data for pre-training.
Publications
- C. Zhou, M. Prabhushankar, and G. AlRegib, “On the Ramifications of Human Label Uncertainty,” in NeurIPS 2022 Workshop on Human in the Loop Learning, Oct. 27, 2022, [PDF], [Code].
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Perceptual Quality-Based Model Training Under Annotator Label Uncertainty,” in International Meeting for Applied Geoscience & Energy (IMAGE) 2023, Houston, TX, Aug. 28-Sept. 1, 2023, [PDF], [Code].
- M. Prabhushankar, K. Kokilepersaud, J. Quesada, Y. Yarici, C. Zhou, M. Alotaibi, G. AlRegib, A. Mustafa, Y. Kumakov, “CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface Faults,” arXiv preprint, 2024, [PDF], [Code].
- J. Quesada, et al., “A Crowdsourced Machine Learning Benchmark for Fault Segmentation,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “A Unified View of Generative Modeling for Seismic Interpretation,” submitted to The Leading Edge (TLE), Oct. 16, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Generative Modeling of Disagreements for Expertise-based Seismic Fault Labels,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Uncertainty in Seismic Image Interpretation,” under review, 2024.
-
K. Kokilepersaud, M. Prabhushankar, and G. AlRegib, “SSL in Seismic Requires Additional Volumetric Spread,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Self-supervised learning (SSL) approaches are seeing increased popularity within annotation-scarce domains due to their focus on training without explicit access to labeled data. For this reason, these approaches have received widespread attention within the seismic community as obtaining quality labeled data is challenging within this application domain.
However, self-supervised algorithms were trained and tested within the domain of large natural image datasets. Consequently, it is unclear whether conventional self-supervised approaches are appropriately formulated for the unique challenges of the seismic domain. Specifically, traditional self-supervised approaches:
- Lack the capability to assess what features a quality seismic representation space should possess.
- Do not adequately integrate these optimal features into the representation space.
In this work, we demonstrate that:
- A quality self-supervised seismic representation space is one that is more distributed across the overall representation space.
- Using a novel volumetric-based loss function explicitly induces additional spread within the representation space.
We show visually and numerically that the resultant model is better able to rectify fine-grained structures within a seismic segmentation task.
Publications
- K. Kokilepersaud, S. Kim, M. Prabhushankar, G. AlRegib, ”HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms,” in 2025 Winter Applications of Computer Vision (WACV), Tuscon, Arizona, 2025. [PDF], [Code]
- K. Kokilepersaud, Yavuz Yarici, M. Prabhushankar, and G. AlRegib, “Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships Into the Contrastive Loss,” in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, UAE, Oct. 27-Oct. 30, 2024. [PDF], [Code]
- K. Kokilepersaud, M. Prabhushankar, Y. Yarici, G. AlRegib, and A. Parchami, “Exploiting the Distortion-Semantic Interaction in Fisheye Data,” in Open Journal of Signals Processing, Apr. 28, 2023. [PDF]
- K. Kokilepersaud, S. Trejo Corona, M. Prabhushankar, G. AlRegib, C. Wykoff, “Clinically Labeled Contrastive Learning for OCT Biomarker Classification,” in IEEE Journal of Biomedical and Health Informatics, May. 15, 2023. [PDF], [Code]
-
Y. Yarici, M. Prabhushankar, and G. AlRegib, “Assisting Experts with Probability Maps for Seismic Fault Detection,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Seismic fault detection is a critical task in geophysical exploration, often requiring extensive manual labeling by experts. This process can be labor-intensive and subjective due to the complex nature of seismic data. Various seismic methods exist to detect and segment faults in seismic images; however, both human labeling and machine learning model predictions can result in mislabels.
In this work, we present a framework to assist expert annotators by leveraging probability maps generated by a deep learning model for seismic fault detection. These probability maps:
- Indicate the likelihood of fault occurrences at different locations in the seismic data.
- Allow experts to focus on regions with high predicted fault likelihood.
These maps provide valuable insights to expert geoscientists, assisting them in:
- Refining their labeling tasks.
- Potentially reducing human error and bias.
Publications
- Mohit Prabhushankar, Kiran Kokilepersaud, Jorge Quesada, Yavuz Yarici, Chen Zhou, Mohammad Alotaibi, Ghassan AlRegib, Ahmad Mustafa, Yusufjon Kumakov. “CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface Faults.” (2024).
Session: Generative and Ensemble Learning Techniques
-
G. Kaviani, M. Prabhushankar, and G. AlRegib, “Learning from Multiview Multimodal Sparse Data,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Human daily activity data is inherently sparse unless trimmed and curated for training machine learning models. Within each activity pattern, certain data segments are more representative. Data collected from different sensors capture these activity patterns with varying levels of detail and specificity, resulting in differing degrees of sparsity across each signal.
For example:
- A sensor on the hand captures diverse hand interactions.
- An insole sensor records similar standing or sitting patterns during the same period.
A multimodal learning approach is essential for effectively detecting and segmenting these patterns, leveraging the complementary insights from multiple sensors to address the challenges posed by sparsity.
Publications
- G. Kaviani, Y. Yarici, M. Prabhushankar, and G. AlRegib, “Exploring Human Daily Activity Through a Hierarchical Multimodal Lens,” in Conference on Neural Information Processing Systems (NeurIPS-W), 2024.
- Ghazal Kaviani, Yavuz Yarici, Mohit Prabhushankar, Ghassan AlRegib, Mashhour Solh, Ameya Patil. (2024). “DARai: Daily Activity Recordings for AI and ML Applications.” IEEE. https://dx.doi.org/10.21227/ecnr-hy49.
-
S. Kim, M. Prabhushankar, and G. AlRegib, “Anticipation for Sparse Dataset,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: We aim to address the challenge of predicting key events in sparse data environments by leveraging hierarchical labeling and temporal pattern learning. Both human daily activity and seismic data share a common trait of sparsity: in human activity datasets, most frames lack meaningful cues related to action transitions, making it difficult to pinpoint crucial moments.
For example:
- In a one-hour video of daily activities, the critical clues required to predict the next action may only span a few seconds of actual behavioral change.
- In seismic datasets, structures of interest such as faults and salt domes appear sporadically, while less informative features like horizons dominate the data.
To tackle this, we propose a hierarchical labeling with temporal sequence models to accurately capture the essential patterns within sparse data. By refining the granularity of labels and focusing on key temporal points, our method:
- Improves next-action predictions in human anticipation tasks.
- Provides a robust framework that can be extended to other domains facing sparse data challenges.
Publications
- Kokilepersaud, K., Kim, S., Prabhushankar, M., & AlRegib, G. (2024). “HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms.” arXiv preprint arXiv:2410.23200.
- G. Kaviani, Y. Yarici, M. Prabhushankar, and G. AlRegib, “Exploring Human Daily Activity Through a Hierarchical Multimodal Lens,” in Conference on Neural Information Processing Systems (NeurIPS-W), 2024.
- Ghazal Kaviani, Yavuz Yarici, Mohit Prabhushankar, Ghassan AlRegib, Mashhour Solh, Ameya Patil. (2024). “DARai: Daily Activity Recordings for AI and ML Applications.” IEEE. https://dx.doi.org/10.21227/ecnr-hy49.
-
Z. Fowler, M. Prabhushankar, and G. AlRegib, “Tackling Generalization and Personalization in Federated Learning,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides TBP], [Recording]
Abstract: Statistical heterogeneity is a challenge in federated learning algorithms from both a local and global viewpoint, where the global model has difficulties generalizing to a broad variety of data and personalizing to each local client’s data. Furthermore, statistical heterogeneity increases catastrophic forgetting, where test samples previously learned become incorrect after a model update.
Prior work tends to focus on the generalization and personalization challenge separately, despite these issues being connected through catastrophic forgetting. In this work, we address both challenges together by:
- Establishing how catastrophic forgetting impacts generalization and personalization.
- Reducing catastrophic forgetting through modifications to the local training stage of each client.
- Improving the global model aggregation process.
Our results, demonstrated on medical and natural image datasets, provide valuable insights and suggest how these findings can be extended to the seismic domain.
Publications
- Z. Fowler, K. Kokilepersaud, M. Prabhushankar, and G. AlRegib, “Clinical Trial Active Learning,” in Proceedings of the 14th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics (ACM-BCB), Jun. 11, 2023.
Session: Learning and Domain Generalization in Seismic Interpretation & Processing
-
J. Quesada, Z. Fowler, M. Alotaibi, M. Prabhushankar, and G. AlRegib, “Indicative Features of Prompting Performance in Non-Natural Domains,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Foundation models constitute a paradigm shift in the way machine learning tasks are approached, moving now to prompting-based approaches. However, there is little understanding of what factors make a prompting strategy effective, particularly in the visual domain.
We present the PointPrompt dataset, the first visual segmentation prompting dataset across several image domains. Our benchmark tasks provide an array of opportunities to:
- Improve understanding of how human prompts differ from automated ones.
- Identify underlying factors that make for effective visual prompts.
Overall, our experiments:
- Showcase the differences between human prompts and automated methods.
- Highlight potential avenues to leverage these differences for improved visual prompt design.
Publications
- J. Quesada*, Z. Fowler*, M. Alotaibi, M. Prabhushankar, and G. AlRegib, “Benchmarking Human and Automated Prompting in the Segment Anything Model,” accepted in IEEE Conference on Big Data 2024, Washington DC, USA, Sept. 8, 2024. [PDF], [Code]
- J. Quesada, M. Alotaibi, M. Prabhushankar, and G. AlRegib, “PointPrompt: A Multi-modal Prompting Dataset for Segment Anything Model,” Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on Prompting in Vision, June 2024. [PDF]
-
P. Chowdhury, M. Prabhushankar, and G. AlRegib, “Optimizing Prompting for Foundation Models in Seismic Image Segmentation,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: The advent of large foundation models has transformed artificial intelligence by providing generalized frameworks for large-scale downstream tasks. Segment Anything (SAM) is a vision-based model that performs image segmentation using “inclusion” and “exclusion” point prompts. In geophysics and seismic image analysis, facies and fault segmentation are cost-intensive, and SAM’s prompt-based approach offers fast segmentation without the need for model training on labeled data, thus saving time and resources.
However, prompting is intuitive; too few prompts lead to errors, while over-prompting degrades performance. To optimize this process, our work:
- Identifies an ideal combination of prompts by measuring each prompt’s importance through its impact on segmentation quality.
- Uses Intersection over Union (IoU) as a metric for segmentation quality.
- Employs sufficiency to calculate prompt importance and guide users to stop prompting at the optimal level.
This approach not only minimizes manual efforts but also:
- Enables automated prompting for subsequent slices in facies or fault data.
- Works without requiring ground truth labels for automation.
Publications
- J. Quesada*, Z. Fowler*, M. Alotaibi, M. Prabhushankar, and G. AlRegib, “Benchmarking Human and Automated Prompting in the Segment Anything Model,” accepted in IEEE Conference on Big Data 2024, Washington DC, USA, Sept. 8, 2024. [PDF], [Code]
- J. Quesada, M. Alotaibi, M. Prabhushankar, and G. AlRegib, “PointPrompt: A Multi-modal Prompting Dataset for Segment Anything Model,” Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on Prompting in Vision, June 2024. [PDF]
- P. Chowdhury, M. Prabhushankar, and G. AlRegib, “Optimizing Prompting for Foundation Models in Seismic Image Segmentation,” at International Meeting for Applied Geoscience and Energy (IMAGE), Houston, TX, Aug. 26-Aug. 30, 2024.
- P. Chowdhury, et al., “HIPPO: Human Initiated ProPmt Optimization for Segment Anything Model,” submitted, 2024.
-
M. Alotaibi, M. Prabhushankar, K. Kokilepersaud, and G. AlRegib, “Redefining Prompting for Seismic,” at ML4SEISMIC Partners Meeting, Nov. 13-Nov. 15, 2024. [Slides], [Recording]
Abstract: Machine-learning (ML) algorithms have emerged as a tool for seismic interpretation. However, they lack the expertise that human experts bring to the interpretation process. We hypothesize that cooperating ML methods with domain expertise can address ML limitations. In particular, interactive prompting-based models like ChatGPT and the Segment Anything Model (SAM) enable this interactivity between AI and experts.
To verify our hypotheses, we analyze the interaction between SAM and different users tasked with a seismic labeling problem. Our findings include:
- Users achieved an mIoU of 0.9 but struggled to influence SAM to segment the desired area.
- Users initially assumed that more prompts would lead to better segmentation and that only accurate prompting ensures accurate segmentation.
- Users adapted their approaches upon realizing these assumptions were incorrect.
- M. Alotaibi, M. Prabhushankar, and G. AlRegib, “LineSAM: Using Lines to Prompt on Segment Anything Model”, 2024 (submitted).
- M. Alotaibi, M. Prabhushankar, K. Kokilepersaud, and G. AlRegib, “Improving Seismic Interpretation Accuracy and Efficiency With Human-Machine Collaboration,” at International Meeting for Applied Geoscience and Energy (IMAGE), Houston, TX, Aug. 26-Aug. 30, 2024.
- J. Quesada, Z. Fowler, M. Alotaibi, M. Prabhushankar, and G. AlRegib, ‘Benchmarking Human and Automated Prompting in the Segment Anything Model’, arXiv [cs.CV], 2024.
Related Publications
Tutorials
Robust Neural Networks: Towards Explainability, Uncertainty, and Intervenability
Presenter: Mohit Prabhushankar [Slides]
Over the course of the last year, we delivered 11 tutorials and short course at major conferences in diverse fields including CVPR’24, AAAI’24, WACV’24, and IEEE BigData’23 among others. All tutorials and their slides are accessible here . These tutorials are based on our work on inferential machine learning where we provide robustness, uncertainty, and intervenability plug-ins for trained ML models. The videos of some of these tutorials are available on our Youtube channel. Specifically, we refer to timestamps 58:23, 2:46:56, and 3:42:08 for understanding and implementing our inferential techniques.
Publications
- G. AlRegib and M. Prabhushankar, “Explanatory Paradigms in Neural Networks: Towards Relevant and Contextual Explanations,” in IEEE Signal Processing Magazine, Special Issue on Explainability in Data Science, Feb. 18, 2022. [PDF], [Code]
- J. Lee, C. Lehman, M. Prabhushankar, and G. AlRegib, “Probing the Purview of Neural Networks via Gradient Analysis,” in IEEE Access, Mar. 21, 2023. [PDF]
- M. Prabhushankar and G. AlRegib, “VOICE: Variance of Induced Contrastive Explanations to Quantify Uncertainty in Neural Network Interpretability,” in Journal of Selected Topics in Signal Processing (J-STSP) Special Series on AI in Signal & Data Science, May. 23, 2024. [PDF], [Code]
- M. Prabhushankar and G. AlRegib, “Introspective Learning: A Two-Stage Approach for Inference in Neural Networks,” in Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA, Nov. 29 – Dec. 1, 2022. [PDF], [Code]
- G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, “Backpropagated Gradient Representations for Anomaly Detection,” in Proceedings of the European Conference on Computer Vision (ECCV), SEC, Glasgow, Aug. 23-28, 2020. [PDF], [Code], [Link]
- M. Prabhushankar and G. AlRegib, “Counterfactual Gradients-based Quantification of Prediction Trust in Neural Networks,” in IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, Aug. 7-9, 2024 (Invited Paper). [PDF], [Code]
- J. Quesada*, Z. Fowler*, M. Alotaibi, M. Prabhushankar, and G. AlRegib, “Benchmarking Human and Automated Prompting in the Segment Anything Model,” submitted to IEEE Conference on Big Data 2024, Washington DC, USA, Sept. 8, 2024. [PDF], [Code]
Human-Initiated Prompt Optimization (HIPPO)
Presenter: Prithwijit Chowdhury
Automated prompting techniques for instance discovery and segmentation often lack the precision and adaptability of humans. To address this limitation, we propose Human-Initiated Prompt Optimization (HIPPO), a framework that combines human expertise with automated optimization to enhance prompt generation and propagation. The process begins with a single human-annotated image, where initial point-based prompts are defined. These prompts are refined through a sufficiency-based optimization method to identify the most effective combination from the human-defined set and to avoid chances of overprompting. The optimized prompts are then used as exemplars for spatial and temporal propagation, leveraging an embedding-based feature matching scheme. This enables the automatic discovery and segmentation of instances of the same object in different locations outperforming current State of the Art automated prompting techniques.
Related Publications
- J. Quesada, Z. Fowler, M. Alotaibi, M. Prabhushankar, and G. AlRegib, ‘Benchmarking Human and Automated Prompting in the Segment Anything Model’, arXiv [cs.CV], 2024.
- P. Chowdhury, M. Prabhushankar, and G. AlRegib, “Optimizing Prompting for Foundation Models in Seismic Image Segmentation,” at International Meeting for Applied Geoscience and Energy (IMAGE), Houston, TX, Aug. 26-Aug. 30, 2024.
- P. Chowdhury, M. Prabhushankar, and G. AlRegib, “Human-Initiated PromPt Optimization (HIPPO) for Segment Anything Model”, 2024, (submitted).
LineSAM: Segmenting Facies by Line Prompting
Presenter: Mohammad Alotaibi
Prompting by points has inherent conceptual limitations, particularly for intricate tasks that requires expert precision, such as seismic interpretation. This is due to the fact that a point is associated with a large special area not just a few pixels. As a result, a user can’t interact with SAM in a very low level. To address this limitation of current prompting methods, we propose a new method to prompt that enables the user to interact with SAM and delineate very fine-grained details. Our new concept of prompting introduces the use of lines as a interaction form, which offers many seismic-driven prompting like horizons and reflections.
Related Publications
- M. Alotaibi, M. Prabhushankar, and G. AlRegib, “LineSAM: Using Lines to Prompt on Segment Anything Model”, 2024 (submitted).
- M. Alotaibi, M. Prabhushankar, K. Kokilepersaud, and G. AlRegib, “Improving Seismic Interpretation Accuracy and Efficiency With Human-Machine Collaboration,” at International Meeting for Applied Geoscience and Energy (IMAGE), Houston, TX, Aug. 26-Aug. 30, 2024.
- J. Quesada, Z. Fowler, M. Alotaibi, M. Prabhushankar, and G. AlRegib, ‘Benchmarking Human and Automated Prompting in the Segment Anything Model’, arXiv [cs.CV], 2024.
Fault Label Annotation and Disagreement Visualization
Presenter: Jorge Quesada Pacora
Segmenting faults is of paramount importance in the seismic interpretation pipeline, albeit involving both costly and labor-intensive expert annotation. Alternatives to expertly labeled data imply often relying on synthetic data or weakly labeled data. In this work, we present the CRACKS dataset, a comprehensive fault segmentation dataset spanning labels across multiple levels of expertise and confidence. We benchmark the effectiveness of this dataset by evaluating different machine learning strategies to exploit its multifaceted structure, as well as comparing it with the results we achieve when using either synthetic or weak labels sources.
Publications
- C. Zhou, M. Prabhushankar, and G. AlRegib, “On the Ramifications of Human Label Uncertainty,” in NeurIPS 2022 Workshop on Human in the Loop Learning, Oct. 27, 2022, [PDF], [Code].
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Perceptual Quality-Based Model Training Under Annotator Label Uncertainty,” in International Meeting for Applied Geoscience & Energy (IMAGE) 2023, Houston, TX, Aug. 28-Sept. 1, 2023, [PDF], [Code].
- M. Prabhushankar, K. Kokilepersaud, J. Quesada, Y. Yarici, C. Zhou, M. Alotaibi, G. AlRegib, A. Mustafa, Y. Kumakov, “CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface Faults,” arXiv preprint, 2024, [PDF], [Code].
- J. Quesada, et al., “A Crowdsourced Machine Learning Benchmark for Fault Segmentation,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “A Unified View of Generative Modeling for Seismic Interpretation,” submitted to The Leading Edge (TLE), Oct. 16, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Generative Modeling of Disagreements for Expertise-based Seismic Fault Labels,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Uncertainty in Seismic Image Interpretation,” under review, 2024.
Probabilistic Modelling of Seismic Interpretation
Presenter: Chen Zhou
Conventional machine learning-assisted interpretation frameworks rely on deterministic labels and typically assume a gold standard. However, the gold standard label assumption is invalid for interpreting geophysical data. In subsurface fault interpretation, the disagreement between multiple label sources can be prominent. Disagreement labels provide diverse interpretation hypotheses with informative geophysical insights to aid Geophysicists. Thus, instead of predicting a single label, we aim to generate multiple plausible interpretations. In this tutorial, we present a generative approach that models the disagreement labels using different sources of pseudo and/or weak labels, including but not limited to predictions from a pre-trained network, attribute-based labels, to model the seismic interpretation workflow probabilistically.
Publications
- C. Zhou, M. Prabhushankar, and G. AlRegib, “On the Ramifications of Human Label Uncertainty,” in NeurIPS 2022 Workshop on Human in the Loop Learning, Oct. 27, 2022, [PDF], [Code].
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Perceptual Quality-Based Model Training Under Annotator Label Uncertainty,” in International Meeting for Applied Geoscience & Energy (IMAGE) 2023, Houston, TX, Aug. 28-Sept. 1, 2023, [PDF], [Code].
- M. Prabhushankar, K. Kokilepersaud, J. Quesada, Y. Yarici, C. Zhou, M. Alotaibi, G. AlRegib, A. Mustafa, Y. Kumakov, “CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface Faults,” arXiv preprint, 2024, [PDF], [Code].
- J. Quesada, et al., “A Crowdsourced Machine Learning Benchmark for Fault Segmentation,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “A Unified View of Generative Modeling for Seismic Interpretation,” submitted to The Leading Edge (TLE), Oct. 16, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Generative Modeling of Disagreements for Expertise-based Seismic Fault Labels,” under review, 2024.
- C. Zhou, M. Prabhushankar, and G. AlRegib, “Uncertainty in Seismic Image Interpretation,” under review, 2024.
Georgia Tech’s AI Makerspace
Presenter: Ghassan AlRegib
An exciting first-of-its-kind initiative, that has been in the making at Georgia Tech. (Article)
The first three usage cases are:
-
Senior-level FunML (Fundamentals of Machine Learning)
Starts from the basics of regression and ends with transformers and deep networks. The class covers theory and hands-on assignments. It is a ride and the textbook will soon see the light. This class was the reason behind the proposal to create the AI Makerspace. -
Sophomore-level
This is too a first-of-its-kind class. It provides students foundational entry into AI and the AI_Makerapce. It contains a number of hands-on studios. -
VIP Team AI Makerspace Nexus
That is building tools and services to truly democratize access to the AI Makerapce for all Georgia Institute of Technology faculty, staff, and students. These tools will soon see the light and exciting things are ahead.. Learn more about the team here.
Conferences and Workshops of Interest To Seismic Interpretation Community
- IEEE BigData 2024: Conference on Various Applications Related to ML and Policy-driven ML
- CAS@BigData’24: Workshop on Digital Archiving and Big Data Technologies
- MMAI@BigData’24: Workshop on Multimodal AI
- AI for Climate Change workshop at BigData’24
- BSD@BigData’24: Workshop on Big Spatial Data at BigData’24
- WACV’25: Application-driven CV Conference
- ASTAD@WACV’25: Workshop on Automated Spatial and Temporal Anomaly Detection at WACV’25
- Workshop on Image Quality in Computer Vision and GenAI: Workshop at WACV’25
- CV4EO@WACV’25: Workshop on Computer Vision for Earth Observation at WACV’25
- GeoCV@WACV’25: Workshop on Geospatial Image Analysis at WACV’25
- Multimodal AI in Healthcare Workshop inn WACV’25
- AAAI’25: Conference on AI in Applications in Various Fields
- AI4TS@AAAI’25: Workshop on AI for Time series Analysis at AAAI’25
- AICT@AAAI’25: Workshop on Causal Theories in AI at AAAI’25
- FLUID@AAAI’25: Workshop on Federated Learning at AAAI’s5