RePaint: Inpainting using Denoising Diffusion Probabilistic Models 论文阅读
date
Dec 7, 2022
Last edited time
Mar 27, 2023 08:40 AM
status
Published
slug
RePaint论文阅读
tags
DL
DDPM
summary
type
Post
Field
Plat
Abstract
- Problem
Free-form inpainting is the task of adding new content to an image in the regions specified by an arbitrary binary mask. Most existing approaches train for a certain distribution of masks, which limits their generalization capabilities to unseen mask types.
- Method
We employ a pretrained unconditional DDPM as the generative prior. To condition the generation process, we only alter the reverse diffusion iterations by sampling the unmasked regions using the given image information.
We do not modify or condition the original DDPM network itself.
Method
The goal of inpainting is to predict missing pixels of an image using a mask region as a condition.
is sampled using the known pixels in the given image , while is sampled from the model, given the previous iteration .
Resampling
When directly applying the method described before, we observe that only the content type matches with the known regions.
The inpainted region matches the texture of the neighboring region, it is semantically incorrect.
Thus, we diffuse the output back to as .
Some information incorporated in the generated region is still preserved in . It leads to a new which is both more harmonized with and contains conditional information from it.
Since this operation can only harmonize one step, it might not be able to incorporate the semantic information over the entire denoising process. To overcome this problem, we denote the time horizon of this operation as jump length, which is for the previous case.
没看懂这里的 jump length 指的是什么.
Experiments
For our final approach, we use timesteps, and applied times resampling with jumpy size .
