Date of Award

Fall 2025

Document Type

Open Access Dissertation

Department

Computer Science and Engineering

First Advisor

Song Wang

Abstract

Image reconstruction seeks to restore corrupted images and recover visual content that has been lost or degraded. Such degradation may result from low resolution, occlusion, masking, or shadow interference. This problem has become an increasingly significant research topic, as visual information plays a central role in almost every aspect of modern life. Neural network based approaches have recently emerged as highly effective solutions for this task. In particular, convolutional neural networks and transformer based architectures have demonstrated remarkable success in producing visually convincing reconstructions. However, these models remain constrained in several important ways, one of the most critical being that they typically operate on inputs and outputs of fixed dimensions. This restriction limits their flexibility and reduces their applicability in real world scenarios.

Recently, implicit neural representation (INR) has been proposed for a wide range of image processing tasks. INR provide a powerful framework for mapping discrete data into continuous representations, enabling improved flexibility and generalization. INR based approaches have achieved significant progress in tasks such as image super resolution, generation, and semantic segmentation. A key advantage of these methods is their ability to achieve super resolution for images of arbitrary sizes. Despite these advancements, current INR based techniques face several challenges. First, they primarily focus on intact images, which makes them less effective in scenarios involving damaged or incomplete regions. Second, many methods emphasize the continuous representation of pixel color while neglecting the rich semantic information embedded within images. Finally, training these models often requires large amounts of paired data, which are particularly difficult to obtain for tasks such as image deshadowing due to the scarcity of labeled examples.

In this dissertation, we propose three novel approaches to address these limitations. To fully exploit the potential of implicit neural representation for processing damaged images, we introduce a new task called SuperInpaint, which reconstructs missing regions in low resolution images while generating complete outputs at arbitrary higher resolutions. To address the second limitation, we develop the Semantic Aware Implicit Representation (SAIR) framework. This approach augments the implicit representation of each pixel by jointly encoding appearance and semantic information, thereby enhancing the model’s ability to capture both fine grained details and global contextual structures. Building on the proven success of INR in image reconstruction, we further extend their applicability to the image deshadowing task. To this end, we design a specialized method that removes shadows while faithfully preserving the underlying image content. A key challenge in existing INR based methods lies in their dependence on large training datasets, which are particularly scarce for de shadowing due to the limited availability of labeled data. To overcome this issue, we adopt a pretraining strategy in which the model is initially trained on large scale image inpainting datasets. This enables the network to acquire strong reconstruction priors that are subsequently transferred and fine tuned for the shadow removal task.

The proposed methods provide a comprehensive and effective response to the identified challenges, establishing a robust framework applicable to a diverse range of image processing tasks. By systematically addressing the key limitations of existing approaches, this work expands both the flexibility and scalability of implicit neural representation, enabling them to manage complex visual scenarios with greater accuracy, robustness, and efficiency.

Rights

©2025, Canyu Zhang

Download

Share

COinS