HKUST launches AI "Semantic-SAM" for image segmentation, which provides semantic tags for entities

CTOnews.com July 17, the team of Hong Kong University of Science and Technology developed an image segmentation AI model called Semantic-SAM. Compared with the previously released SAM model of Meta, Semantic-SAM has stronger granularity and semantic function, which can segment and identify objects at different granularity levels, and provide semantic tags for segmented entities.

It is reported that Semantic-SAM is developed based on the Mask DINO framework, and its model structure is mainly improved in the decoder part, while supporting general segmentation and interactive segmentation.

The research team uses decoupled object classification and component classification methods to learn the semantic information of objects and components, thus realizing the optimization of multi-granularity segmentation tasks and interactive segmentation tasks. The experimental results show that Semantic-SAM is better than Meta's SAM model in segmentation quality and granularity controllability.

The project has been published in GitHub, and the paper has also been uploaded to ArXiv. Interested CTOnews.com friends can check it out.