* Indicates equal contribution
* Indicates equal contribution
For each new segmentation task, existing strategies (left) need to train a new model before making predictions. UniverSeg (right) employs a single global model that can make predictions for images from a new segmentation task with a few labeled examples as input, without additional fine-tuning. (Click anywhere on the page and mouseover the figure below for animation)
While deep learning models have become the predominant method for medical image segmentation, they are typically not capable of generalizing to unseen segmentation tasks involving new anatomies, image modalities, or labels.
Given a new segmentation task, researchers have to train or fine-tune models, which is time-consuming and poses a substantial barrier for clinical researchers, who often lack the resources and expertise to train neural networks. In this paper, we present UniverSeg, a method for solving unseen medical segmentation tasks without additional training.
Given a query image and support set of image-label pairs that define a new segmentation task, UniverSeg employs a novel CrossBlock mechanism to produce accurate segmentations without the need for additional training. To achieve strong generalization to new tasks, we have gathered, standardized, and trained on a collection of 53 open-access medical segmentation datasets with over 22,000 scans, which we refer to as MegaMedical. We used this collection to train UniverSeg on a diverse set of anatomies and imaging modalities. We demonstrate that UniverSeg substantially outperforms several related methods on unseen tasks, and thoroughly analyze and draw insights about important aspects of our system.
A UniverSeg network (left) takes as input a query image and a support set of image and label-maps (pairwise concatenated in the channel dimension) and employs multi-scale CrossBlock features. A CrossBlock (right) takes as input representations of the query u and support set V, and interacts u with each support entry to produce u' and V'.
We assess the segmentation performance of UniverSeg models compared to few-shot medical segmentation baselines and individually trained fully supervised networks on several held-out medical datasets.
If you find our work or any of our materials useful, please cite our paper: