aljaz/generative-models
1
0
Fork 0
mirror of https://github.com/Stability-AI/generative-models.git synced 2025-12-19 22:34:22 +01:00
Code Issues Packages Projects Releases Wiki Activity

Fixes azimuth, adds simple instruction (#307)

Browse Source 
* Fixes azimuth, adds simple instruction

* Adds assertts

---------

Co-authored-by: Vikram Voleti <vikram@ip-26-0-153-234.us-west-2.compute.internal>
This commit is contained in:
Vikram Voleti
2024-03-19 02:30:38 +05:30
committed by GitHub
parent 2a532db0e8
commit bdbae9948f
2 changed files with 13 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
README.md
View File

@@ -16,9 +16,10 @@ To run SV3D_u on a single image:
- Download `sv3d_u.safetensors` from https://huggingface.co/stabilityai/sv3d to `checkpoints/sv3d_u.safetensors`
- Run `python scripts/sampling/simple_video_sample.py --input_path <path/to/image.png> --version sv3d_u`
Additionally for SV3D_p,
- Specify sequences of 21 elevations and 21 azimuths (in degrees) to `elevations_deg` ([-90, 90]), and `azimuths_deg` [0, 360] in sorted order from 0 to 360. For example:
`python scripts/sampling/simple_video_sample.py --input_path <path/to/image.png> --version sv3d_p --elevations_deg [<list of 21 elevations in degrees>] --azimuths_deg [<list of 21 azimuths in degrees>]`
To run SV3D_u on a single image:
- Download `sv3d_p.safetensors` from https://huggingface.co/stabilityai/sv3d to `checkpoints/sv3d_p.safetensors`
1. Generate static orbit at a specified elevation eg. 10 : `python scripts/sampling/simple_video_sample.py --input_path <path/to/image.png> --version sv3d_p --elevations_deg 10.0`
2. Generate dynamic orbit at a specified elevations and azimuths: specify sequences of 21 elevations (in degrees) to `elevations_deg` ([-90, 90]), and 21 azimuths (in degrees) to `azimuths_deg` [0, 360] in sorted order from 0 to 360. For example: `python scripts/sampling/simple_video_sample.py --input_path <path/to/image.png> --version sv3d_p --elevations_deg [<list of 21 elevations in degrees>] --azimuths_deg [<list of 21 azimuths in degrees>]`
To run SVD or SV3D on a streamlit server:
`streamlit run scripts/demo/video_sampling.py`

11
scripts/sampling/simple_video_sample.py
View File

@@ -34,7 +34,7 @@ def sample(
device: str = "cuda",
output_folder: Optional[str] = None,
elevations_deg: Optional[float | List[float]] = 10.0, # For SV3D
azimuths_deg: Optional[float | List[float]] = None, # For SV3D
azimuths_deg: Optional[List[float]] = None, # For SV3D
image_frame_ratio: Optional[float] = None,
verbose: Optional[bool] = False,
):
@@ -81,10 +81,17 @@ def sample(
cond_aug = 1e-5
if isinstance(elevations_deg, float) or isinstance(elevations_deg, int):
elevations_deg = [elevations_deg] * num_frames
assert (
len(elevations_deg) == num_frames
), f"Please provide 1 value, or a list of {num_frames} values for elevations_deg! Given {len(elevations_deg)}"
polars_rad = [np.deg2rad(90 - e) for e in elevations_deg]
if azimuths_deg is None:
azimuths_deg = np.linspace(0, 360, num_frames + 1)[1:] % 360
azimuths_rad = [np.deg2rad(a) for a in azimuths_deg]
assert (
len(azimuths_deg) == num_frames
), f"Please provide a list of {num_frames} values for azimuths_deg! Given {len(azimuths_deg)}"
azimuths_rad = [np.deg2rad((a - azimuths_deg[-1]) % 360) for a in azimuths_deg]
azimuths_rad[:-1].sort()
else:
raise ValueError(f"Version {version} does not exist.")