zion
387c5bf774
quick patch for new prior loader ( #123 )
2022-05-29 16:25:53 -07:00
Phil Wang
501a8c7c46
small cleanup
2022-05-22 15:39:38 -07:00
Phil Wang
8997f178d6
small cleanup with timer
2022-05-20 20:05:01 -07:00
Phil Wang
bb151ca6b1
unet_number on decoder trainer only needs to be passed in if there is greater than 1 unet, so that unconditional training of a single ddpm is seamless (experiment in progress locally)
2022-05-16 09:17:17 -07:00
zion
4a59dea4cf
Migrate to text-conditioned prior training ( #95 )
...
* migrate to conditioned prior
* unify reader logic with a wrapper (#1 )
* separate out reader logic
* support both training methods
* Update train prior to use embedding wrapper (#3 )
* Support Both Methods
* bug fixes
* small bug fixes
* embedding only wrapper bug
* use smaller val perc
* final bug fix for embedding-only
Co-authored-by: nousr <>
2022-05-15 20:16:38 -07:00
Phil Wang
8eaacf1ac1
remove indirection
2022-05-15 12:05:45 -07:00
Phil Wang
74f222596a
remove todo
2022-05-15 11:01:35 -07:00
Phil Wang
aa6772dcff
make sure optimizer and scaler is reloaded on resume for training diffusion prior script, move argparse to click
2022-05-15 10:48:10 -07:00
Phil Wang
71d0c4edae
cleanup to use diffusion prior trainer
2022-05-15 10:16:05 -07:00
Phil Wang
89de5af63e
experiment tracker agnostic
2022-05-15 09:56:40 -07:00
z
10b905b445
smol typo ( #81 )
2022-05-10 09:52:50 -07:00
Phil Wang
64f7be1926
some cleanup
2022-05-09 16:50:21 -07:00
z
cb07b37970
Ensure Eval Mode In Metric Functions ( #79 )
...
* add eval/train toggles
* train/eval flags
* shift train toggle
Co-authored-by: nousr <z@localhost.com >
2022-05-09 16:05:40 -07:00
Phil Wang
a774bfefe2
add attention and feedforward dropouts to train_diffusion_prior script
2022-05-09 13:57:15 -07:00
Phil Wang
2ae57f0cf5
cleanup
2022-05-09 13:51:26 -07:00
Kumar R
8647cb5e76
Val loss changes, with quite a few other changes. This is in place of the earlier PR( https://github.com/lucidrains/DALLE2-pytorch/pull/67 ) ( #77 )
...
* Val_loss changes - no rebased with lucidrains' master.
* Val Loss changes - now rebased with lucidrains' master
* train_diffusion_prior.py updates
* dalle2_pytorch.py updates
* __init__.py changes
* Update train_diffusion_prior.py
* Update dalle2_pytorch.py
* Update train_diffusion_prior.py
* Update train_diffusion_prior.py
* Update dalle2_pytorch.py
* Update train_diffusion_prior.py
* Update train_diffusion_prior.py
* Update train_diffusion_prior.py
* Update train_diffusion_prior.py
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
* Update README.md
2022-05-09 08:53:29 -07:00
Nasir Khalid
2eac7996fa
Additional image_embed metric ( #75 )
...
Added metric to track image_embed vs predicted_image_embed
2022-05-07 14:32:33 -07:00
z
cd5f2c1de4
simulate unrelated captions as a training metric ( #66 )
...
* add unrelated embedding metric
* change to torch.roll
Co-authored-by: nousr <z@localhost.com >
Co-authored-by: nousr <>
2022-05-07 05:34:59 -07:00
Phil Wang
63029f7388
remove l2norm output from train_diffusion_prior.py
2022-05-05 19:07:58 -07:00
Kumar R
2d9963d30e
Reporting metrics - Cosine similarity. ( #55 )
...
* Update train_diffusion_prior.py
* Delete train_diffusion_prior.py
* Cosine similarity logging.
* Update train_diffusion_prior.py
* Report Cosine metrics every N steps.
2022-05-04 08:04:36 -07:00
Kumar R
72c16b496e
Update train_diffusion_prior.py ( #53 )
2022-05-02 22:44:57 -07:00
z
81d83dd7f2
defaults align with paper ( #52 )
...
Co-authored-by: nousr <>
2022-05-02 13:52:11 -07:00
Phil Wang
aa8d135245
allow laion to experiment with normformer in diffusion prior
2022-05-02 11:35:00 -07:00
Romain Beaumont
2d25c89f35
Fix passing of l2norm_output to DiffusionPriorNetwork ( #51 )
2022-05-02 10:48:16 -07:00
Phil Wang
3fe96c208a
add ability to train diffusion prior with l2norm on output image embed
2022-05-02 09:53:20 -07:00
Phil Wang
7ee0ecc388
mixed precision for training diffusion prior + save optimizer and scaler states
2022-05-02 09:31:04 -07:00
Phil Wang
f7df3caaf3
address not calculating average eval / test loss when training diffusion prior https://github.com/lucidrains/DALLE2-pytorch/issues/49
2022-05-02 08:51:41 -07:00
Phil Wang
d991b8c39c
just clip the diffusion prior network parameters
2022-05-01 12:01:08 -07:00
Phil Wang
35cd63982d
add gradient clipping, make sure weight decay is configurable, make sure learning rate is actually passed into get_optimizer, make sure model is set to training mode at beginning of each epoch
2022-05-01 11:55:38 -07:00
Kumar R
53ce6dfdf6
All changes implemented, current run happening. Link to wandb run in comments. ( #43 )
...
* Train DiffusionPrior with pre-computed embeddings
This is in response to https://github.com/lucidrains/DALLE2-pytorch/issues/29 - more metrics will get added.
2022-05-01 11:46:59 -07:00
