浏览全部资源
扫码关注微信
四川大学电子信息学院,四川成都 610065
[ "刘 晓 (1998—),男,山西朔州,硕士研究生,主要研究方向为图像处理;" ]
[ "王正勇 (1969—),女,四川成都,博士,副教授,主要研究方向为图像处理、智能系统设计;" ]
[ "何小海 (1964—),男,四川成都,博士,教授,主要研究方向为图像处理与网络通信;" ]
[ "任 超 (1988—),男,四川成都,博士,副教授,主要研究方向为图像处理、计算机视觉、人工智能、多媒体通信与信息系统等。" ]
纸质出版日期:2023-12-31
移动端阅览
刘晓, 王正勇, 何小海, 等. 真实世界超分辨率—语义分割联合框架研究[J]. 新一代信息技术, 2023, 6(24): 06-11
LIU Xiao, WANG Zheng-yong, HE Xiao-hai, et al. A Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation[J]. New Generation of Information Technology, 2023, 6(24): 06-11
刘晓, 王正勇, 何小海, 等. 真实世界超分辨率—语义分割联合框架研究[J]. 新一代信息技术, 2023, 6(24): 06-11 DOI: 10.3969/j.issn.2096-6091.2023.24.002.
LIU Xiao, WANG Zheng-yong, HE Xiao-hai, et al. A Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation[J]. New Generation of Information Technology, 2023, 6(24): 06-11 DOI: 10.3969/j.issn.2096-6091.2023.24.002.
现有的语义分割方法在干净的图像上可以产生较好的结果,但是在干净图像上训练的分割模型应用到真实世界的图像上时则会出现性能下降,这是因为训练域和测试域之间存在域间隙,从而降低了分割的准确性。针对真实世界语义分割的问题,本文提出了一种超分辨率—语义分割联合框架,用于提升语义分割准确性。具体来说,所提出的框架嵌入了一个两分支网络,其中包括超分辨率分支、语义分割分支和一个特征共享模块。超分辨率任务鼓励网络找到对不同分辨率特征鲁棒的表示,从而分割头部可以使用恢复的“干净”特征进行更好的预测。其中超分辨率分支仅配置在训练过程中,在推理阶段可以丢弃。基于构建的伪真实配对数据集CityDeg进行监督训练,提出的框架联合现有先进的语义分割方法能够在不引入额外计算成本的情况下有效提高低分辨率场景语义分割性能。
Existing semantic segmentation methods produce better results on clean images
but segmentation models trained on clean images applied to real-world images experience performance degradation because of the domain gap between the training and testing domains
which reduces the segmentation accuracy. To address the problem of real-world semantic segmentation
this paper proposes a joint super-resolution-semantic segmentation framework for improving semantic segmentation accuracy. Specifically
the proposed framework embeds a two-branch network that includes a super-resolution branch
a semantic segmentation branch
and a feature sharing module. The super-resolution task encourages the network to find a robust representation of features with different resolutions
so that the segmentation head can use the recovered “clean" features for better prediction. The super-resolution branch is configured only during training and can be discarded during the inference phase. Based on the constructed pseudo-real pairwise dataset CityDeg for supervised training
the proposed framework
together with the existing state-of-the-art semantic segmentation methods
is able to effectively improve the performance of semantic segmentation for low-resolution scenes without introducing additional computational cost.
YU C Q , GAO C X , WANG J B , et al . BiSeNet V2: Bilateral network with guided aggregation for real-time semantic segmentation [J ] . International Journal of Computer Vision , 2021 , 129 ( 11 ): 3051 - 3068 .
FAN M Y , LAI S Q , HUANG J S , et al . Rethinking BiSeNet for real-time semantic segmentation [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 9716 - 9725 .
XU J C , XIONG Z X , BHATTACHARYYA S P . PIDNet: A real-time semantic segmentation network inspired by PID controllers [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2023 : 19529 - 19539 .
LIU X , SHI X Y , CHEN L F , et al . Efficient parallel multi-scale detail and semantic encoding network for lightweight semantic segmentation [C ] // Proceedings of the 31st ACM International Conference on Multimedia . New York : ACM , 2023 : 2544 - 2552 .
胡杰 , 昌敏杰 , 徐博远 , 等 . ConvFormer: 基于Transformer的视觉主干网络 [J ] . 电子学报 , 2024 , 52 ( 1 ): 46 - 57 .
WEI Y Y , ZHANG Z , ZHENG H , et al . SGINet: Toward sufficient interaction between single image deraining and semantic segmentation [C ] // Proceedings of the 30th ACM International Conference on Multimedia . New York : ACM , 2022 : 6202 - 6210 .
CHEN W T , CHEN I H , YEH C Y , et al . SJDL-vehicle: Semi-supervised joint defogging learning for foggy vehicle re-identification [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 1 ): 347 - 355 .
LI Y , CHANG Y , YU C F , et al . Close the loop: A unified bottom-up and top-down paradigm for joint image deraining and segmentation [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 2 ): 1438 - 1446 .
HASHMI K A , KALLEMPUDI G , STRICKER D , et al . FeatEnHancer: Enhancing hierarchical features for object detection and beyond under low-light vision [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 6725 - 6735 .
HONG Y , WEI K , CHEN L , et al . Crafting object detection in very low light [J ] // Proceedings of the British Machine Vision Conference , 2021 , 1 ( 2 ): 3 .
WANG X T , XIE L B , DONG C , et al . Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data [C ] // 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) . Piscataway : IEEE , 2021 : 1905 - 1914 .
LIU X , LIAO X , SHI X , et al . Efficient information modulation network for image super resolution [C ] // 2023 European Conference on Artificial Intelligence (ECAI) . Amsterdam : IOS Press , 2023 : 1544 - 1551 .
0
浏览量
453
下载量
0
CSCD
关联资源
相关文章
相关作者
相关机构