真实世界超分辨率—语义分割联合框架研究

刘晓; 王正勇; 何小海; 任超

doi:10.3969/j.issn.2096-6091.2023.24.002

您当前的位置：

首页 >

文章列表页 >

真实世界超分辨率—语义分割联合框架研究

研究论文 | 更新时间：2024-10-08

- 真实世界超分辨率—语义分割联合框架研究
- Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation
- 新一代信息技术 2023年6卷第24期页码：6-11
- 作者机构：
  
  四川大学电子信息学院，四川成都 610065
- 作者简介：
  
  [ "刘晓（1998—），男，山西朔州，硕士研究生，主要研究方向为图像处理；" ]
  [ "王正勇（1969—），女，四川成都，博士，副教授，主要研究方向为图像处理、智能系统设计；" ]
  [ "何小海（1964—），男，四川成都，博士，教授，主要研究方向为图像处理与网络通信；" ]
  [ "任　超（1988—），男，四川成都，博士，副教授，主要研究方向为图像处理、计算机视觉、人工智能、多媒体通信与信息系统等。" ]
- 基金信息：
  
  国家自然科学基金项目(62171304);四川大学达州市校地合作项目(2022CDDZ-09)
- DOI：10.3969/j.issn.2096-6091.2023.24.002
  中图分类号： TN911.73
- 纸质出版日期：2023-12-31，
- 稿件说明：
移动端阅览
刘晓, 王正勇, 何小海, 等. 真实世界超分辨率—语义分割联合框架研究[J]. 新一代信息技术, 2023, 6(24): 06-11

LIU Xiao, WANG Zheng-yong, HE Xiao-hai, et al. A Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation[J]. New Generation of Information Technology, 2023, 6(24): 06-11
刘晓, 王正勇, 何小海, 等. 真实世界超分辨率—语义分割联合框架研究[J]. 新一代信息技术, 2023, 6(24): 06-11 DOI： 10.3969/j.issn.2096-6091.2023.24.002.

LIU Xiao, WANG Zheng-yong, HE Xiao-hai, et al. A Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation[J]. New Generation of Information Technology, 2023, 6(24): 06-11 DOI： 10.3969/j.issn.2096-6091.2023.24.002.

摘要

现有的语义分割方法在干净的图像上可以产生较好的结果，但是在干净图像上训练的分割模型应用到真实世界的图像上时则会出现性能下降，这是因为训练域和测试域之间存在域间隙，从而降低了分割的准确性。针对真实世界语义分割的问题，本文提出了一种超分辨率—语义分割联合框架，用于提升语义分割准确性。具体来说，所提出的框架嵌入了一个两分支网络，其中包括超分辨率分支、语义分割分支和一个特征共享模块。超分辨率任务鼓励网络找到对不同分辨率特征鲁棒的表示，从而分割头部可以使用恢复的“干净”特征进行更好的预测。其中超分辨率分支仅配置在训练过程中，在推理阶段可以丢弃。基于构建的伪真实配对数据集CityDeg进行监督训练，提出的框架联合现有先进的语义分割方法能够在不引入额外计算成本的情况下有效提高低分辨率场景语义分割性能。

Abstract

Existing semantic segmentation methods produce better results on clean images

but segmentation models trained on clean images applied to real-world images experience performance degradation because of the domain gap between the training and testing domains

which reduces the segmentation accuracy. To address the problem of real-world semantic segmentation

this paper proposes a joint super-resolution-semantic segmentation framework for improving semantic segmentation accuracy. Specifically

the proposed framework embeds a two-branch network that includes a super-resolution branch

a semantic segmentation branch

and a feature sharing module. The super-resolution task encourages the network to find a robust representation of features with different resolutions

so that the segmentation head can use the recovered “clean" features for better prediction. The super-resolution branch is configured only during training and can be discarded during the inference phase. Based on the constructed pseudo-real pairwise dataset CityDeg for supervised training

the proposed framework

together with the existing state-of-the-art semantic segmentation methods

is able to effectively improve the performance of semantic segmentation for low-resolution scenes without introducing additional computational cost.

关键词

超分辨率语义分割联合框架深度学习

Keywords

super-resolutionsemantic segmentationjoint frameworkdeep learning

references

YU C Q, GAO C X, WANG J B, et al. BiSeNet V2: Bilateral network with guided aggregation for real-time semantic segmentation[J]. International Journal of Computer Vision, 2021, 129(11): 3051-3068.

FAN M Y, LAI S Q, HUANG J S, et al. Rethinking BiSeNet for real-time semantic segmentation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 9716-9725.

XU J C, XIONG Z X, BHATTACHARYYA S P. PIDNet: A real-time semantic segmentation network inspired by PID controllers[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 19529-19539.

LIU X, SHI X Y, CHEN L F, et al. Efficient parallel multi-scale detail and semantic encoding network for lightweight semantic segmentation[C]//Proceedings of the 31st ACM International Conference on Multimedia. New York: ACM, 2023: 2544-2552.

胡杰, 昌敏杰, 徐博远, 等. ConvFormer: 基于Transformer的视觉主干网络[J]. 电子学报, 2024, 52(1): 46-57.

WEI Y Y, ZHANG Z, ZHENG H, et al. SGINet: Toward sufficient interaction between single image deraining and semantic segmentation[C]//Proceedings of the 30th ACM International Conference on Multimedia. New York: ACM, 2022: 6202-6210.

CHEN W T, CHEN I H, YEH C Y, et al. SJDL-vehicle: Semi-supervised joint defogging learning for foggy vehicle re-identification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(1): 347-355.

LI Y, CHANG Y, YU C F, et al. Close the loop: A unified bottom-up and top-down paradigm for joint image deraining and segmentation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(2): 1438-1446.

HASHMI K A, KALLEMPUDI G, STRICKER D, et al. FeatEnHancer: Enhancing hierarchical features for object detection and beyond under low-light vision[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2023: 6725-6735.

HONG Y, WEI K, CHEN L, et al. Crafting object detection in very low light[J]//Proceedings of the British Machine Vision Conference, 2021, 1(2): 3.

WANG X T, XIE L B, DONG C, et al. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Piscataway: IEEE, 2021: 1905-1914.

LIU X, LIAO X, SHI X, et al. Efficient information modulation network for image super resolution[C]//2023 European Conference on Artificial Intelligence (ECAI). Amsterdam: IOS Press,2023: 1544-1551.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

深度学习下的小样本玉米叶片病害识别研究

基于GoogLeNet的乳腺癌超声图像分类

基于轻量化VGG16和注意力机制的骨龄预测研究

文本辅助图像信息的行人重识别方法