Abstract

This paper aims to restore original background images in watermarked videos, overcoming challenges posed by traditional approaches that fail to handle the temporal dynamics and diverse watermark characteristics effectively. Our method introduces a unique framework that first “decouples” the extraction of prior knowledge—such as common-sense knowledge and residual background details—from the temporal modeling process, allowing for independent handling of background restoration and temporal consistency. Subsequently, it “couples” these extracted features by integrating them into the temporal modeling backbone of a video inpainting (VI) framework. This integration is facilitated by a specialized module, which includes an intrinsic background image prediction sub-module and a dual-branch frame embedding module, designed to reduce watermark interference and enhance the application of prior knowledge. Moreover, a frame-adaptive feature selection module dynamically adjusts the extraction of prior features based on the corruption level of each frame, ensuring their effective incorporation into the temporal processing. Extensive experiments on YouTube-VOS and DAVIS datasets validate our method’s efficiency in watermark removal and background restoration, showing significant improvement over state-of-the-art techniques in visible image watermark removal, video restoration, and video inpainting.

Framework

Experiment

Conclusion

In this paper, we propose a novel framework named DECO to address the challenge of visible video watermark removal. DECO effectively extracts frame-wise prior features by leveraging common-sense knowledge and residual background information, while accurately modeling the temporal dependencies necessary for video content restoration. Extensive experiments and comprehensive ablation studies demonstrated the superiority and generality of DECO. Our method not only overcomes the limitations of existing video restoration techniques in handling complex watermark noise but also significantly improves the quality of the recovered image content. This innovation and advancement have the potential to evaluate the robustness and security of watermarks, offering important practical application value. Nonetheless, our study has certain limitations. For example, watermarks consisting of nested patterns with varying opacities have not been addressed. These limitations indicate areas for improvement and further investigation, which we plan to pursue in the future.