Researchers Rongji Xun, Junjie Yuan, and Zhongjie Wang from the School of Computer Science and Technology at the University of Science and Technology of China have developed a new framework for film restoration that could have significant implications for the energy industry’s archival and monitoring systems.
The team’s work, titled “HaineiFRDM: Explore Diffusion to Restore Defects in Fast-Movement Films,” addresses the limitations of current open-source film restoration methods. These methods often fall short compared to commercial alternatives due to their reliance on low-quality synthetic data and noisy optical flows. Moreover, they have not been optimized for high-resolution films.
The researchers propose a novel solution called HaineiFRDM, which leverages the powerful content-understanding capabilities of diffusion models to assist human experts in restoring film defects. To make high-resolution film restoration feasible on a single 24GB-VRAM GPU, they employ a patch-wise training and testing strategy. Additionally, they introduce several innovative modules: a position-aware Global Prompt and Frame Fusion Modules, and a global-local frequency module to ensure consistent textures across different patches. The framework also mitigates blocky artifacts caused by the patching process by first restoring a low-resolution version of the film and using it as a global residual.
To support their work, the researchers constructed a film restoration dataset containing both restored real-degraded films and realistic synthetic data. Comprehensive experiments demonstrated that HaineiFRDM outperforms existing open-source methods in defect restoration ability. The code and dataset will be released to facilitate further research.
For the energy industry, this technology could be particularly useful in restoring and analyzing archival footage from surveillance cameras, drones, and other monitoring systems. High-quality film restoration can help in assessing equipment performance, identifying maintenance issues, and ensuring safety compliance. Additionally, the ability to restore high-resolution films can provide more detailed insights into energy infrastructure and operations, supporting better decision-making and planning.
The research was published in the prestigious journal, arXiv.
This article is based on research available at arXiv.