From 9ca21e1b8bcaa6b987f03ea8c9a86354ce854201 Mon Sep 17 00:00:00 2001 From: zsyOAOA Date: Fri, 13 Dec 2024 15:22:49 +0800 Subject: [PATCH] add huggingface and colab demo --- README.md | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 4cc7c6d..18a030b 100644 --- a/README.md +++ b/README.md @@ -2,18 +2,19 @@ [Zongsheng Yue](https://zsyoaoa.github.io/), [Kang Liao](https://kangliao929.github.io/), [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/) - +[![arXiv](https://img.shields.io/badge/arXiv%20paper-2412.09013-b31b1b.svg)](https://arxiv.org/abs/2412.09013) [![Hugging Face](https://img.shields.io/badge/Demo-%F0%9F%A4%97%20Hugging%20Face-blue)](https://huggingface.co/spaces/OAOA/InvSR) google colab logo ![visitors](https://visitor-badge.laobi.icu/badge?page_id=zsyOAOA/InvSR) - + -:star: If InvSR is helpful to your researches or projects, please help star this repo. Thanks! :hugs: +:star: If you've found InvSR useful for your research or projects, please show your support by starring this repo. Thanks! :hugs: --- >This study presents a new image super-resolution (SR) technique based on diffusion inversion, aiming at harnessing the rich image priors encapsulated in large pre-trained diffusion models to improve SR performance. We design a \textit{Partial noise Prediction} strategy to construct an intermediate state of the diffusion model, which serves as the starting sampling point. Central to our approach is a deep noise predictor to estimate the optimal noise maps for the forward diffusion process. Once trained, this noise predictor can be used to initialize the sampling process partially along the diffusion trajectory, generating the desirable high-resolution result. Compared to existing approaches, our method offers a flexible and efficient sampling mechanism that supports an arbitrary number of sampling steps, ranging from one to five. Even with a single sampling step, our method demonstrates superior or comparable performance to recent state-of-the-art approaches. > --- ## Update +- **2024.12.13**: Add [![Hugging Face](https://img.shields.io/badge/Demo-%F0%9F%A4%97%20Hugging%20Face-blue)](https://huggingface.co/spaces/OAOA/InvSR) and google colab logo. - **2024.12.11**: Create this repo. ## Requirements @@ -35,28 +36,32 @@ pip install -r requirements.txt [](https://imgsli.com/MzI2MTU5) [](https://imgsli.com/MzI2MTUx) [](https://imgsli.com/MzI2MTUy) [](https://imgsli.com/MzI2MTU0) [](https://imgsli.com/MzI2MTU3) [](https://imgsli.com/MzI2MTU1) -### :point_right: General Image Inhancement +### :point_right: General Image Enhancement [](https://imgsli.com/MzI2MTYw) [](https://imgsli.com/MzI2MTYy) -[](https://imgsli.com/MzI2MjAx) [](https://imgsli.com/MzI2MjAz) [](https://imgsli.com/MzI2MjA0) +[](https://imgsli.com/MzI2MjAx) [](https://imgsli.com/MzI2NTk1) [](https://imgsli.com/MzI2MjA0) -### :point_right: AIGC Image Inhancement +### :point_right: AIGC Image Enhancement [](https://imgsli.com/MzI2MjQy) [](https://imgsli.com/MzI2MjQ1) [](https://imgsli.com/MzI2MjQ3) [](https://imgsli.com/MzI2MjQ5) [](https://imgsli.com/MzI2MjUw) [](https://imgsli.com/MzI2MjUx) - - - - - ## Inference ### :rocket: Fast testing ``` python inference_invsr.py -i [image folder/image path] -o [result folder] --num_steps 1 ``` -1. This script will automatically download the pre-trained [noise predictor](https://huggingface.co/OAOA/InvSR/tree/main) and [SD-Turbo](https://huggingface.co/stabilityai/sd-turbo/tree/main). If you have pre-downloaded them manually, please include them via ``--started_ckpt_path`` and ``--sd_path``. -2. To deal with large images, e.g., 1k---->4k, we recommend adding the option ``--chopping_size 256``. -3. You can freely adjust the sampling steps via ``--num_steps``. +1. **To deal with large images, e.g., 1k---->4k, we recommend adding the option** ``--chopping_size 256``. +2. Other options: + + Specify the pre-downloaded [SD Turbo](https://huggingface.co/stabilityai/sd-turbo) Model: ``--sd_path``. + + Specify the pre-downloaded noise predictor: ``--started_ckpt_path``. + + The number of sampling steps: ``--num_steps``. + + If your GPU memory is limited, please add the option ``--chopping_bs 1``. + +### :railway_car: Online Demo +You can try our method through an online demo: +``` +python app.py +``` ### :airplane: Reproducing our paper results + Synthetic dataset of ImageNet-Test: [Google Drive](https://drive.google.com/file/d/1PRGrujx3OFilgJ7I6nW7ETIR00wlAl2m/view?usp=sharing).