P. Chowdhury, M. Prabhushankar, and G. AlRegib
The advent of large foundation models has revolutionized the field of artificial intelligence, by providing generalized frameworks to tackle large scale downstream tasks. Segment Anything (SAM) is a vision-based foundation model which relies on points to guide their task of image segmentation. Users can prompt the model to focus on a region of interest using an “inclusion” point and ignore a specific area using the “exclusion” point as shown in Fig 1 with the green and red points, respectively. The usual technique of manual prompting is to start with a single point prompt and then keep adding inclusion or deletion points till the desired segmentation is achieved. With the proper prompting strategy SAM can produce state-of-the-art results on tasks with no requirement of fine tuning of the base model. In geophysics and seismic image analysis, facies and fault segmentation are cost intensive tasks. This prompt assisted SAM model can allow fast segmentation without the need for any model training on previously labelled data, thus saving on time and resources.
However, this prompting is a very intuitive task. While too few prompts can give erroneous results, over-prompting a model can lead to a degraded performance. The ideal combination of inclusion and exclusion prompts leading to the optimal segmentation of the area of interest is very difficult to achieve. This becomes more labor intensive when the user must label hundreds or thousands of image slices in a set. Hence it is important to optimize this task of prompting so that the user knows when ideal segmentation is achieved and thus prompting can be stopped
In our work we aim to find this optimal prompting solution for a task by measuring the level of importance of each prompting point and finally come up with an overall objective rating of the entire strategy (Fig 2). We consider prompts as causes leading to the model’s segmentation and thus, employee the definition of sufficiency to calculate the importance measure of each prompt point as a function of the segmentation Intersection over Union (IoU). As more prompts are added and better segmentation is approached the overall score of the strategy increases, however this score starts to decrease once the optimal level is crossed, and the user starts to over-prompt the model. That’s when our algorithm will intimate the user to stop prompting. This will significantly reduce the amount of time a user spends to manually prompt SAM to generate a segmentation mask. Moreover, once the optimal prompting strategy is found, the user can automate the prompting tasks for upcoming slices in their facies or fault data in an interpolative fashion without the need for ground truth labels.
