146 Commits

Author	SHA1	Message	Date
Phil Wang	a35c309b5f	add sparse attention layers in between convnext blocks in unet (grid like attention, used in mobilevit, maxvit [bytedance ai], as well as a growing number of attention-based GANs)	2022-04-19 09:49:03 -07:00
Phil Wang	a54e309269	prioritize todos, play project management	2022-04-18 13:28:01 -07:00
Phil Wang	c6bfd7fdc8	readme	2022-04-18 12:43:10 -07:00
Phil Wang	960a79857b	use some magic just this once to remove the need for researchers to think	2022-04-18 12:40:43 -07:00
Phil Wang	7214df472d	todo	2022-04-18 12:18:19 -07:00
Phil Wang	00ae50999b	make kernel size and sigma for gaussian blur for cascading DDPM overridable at forward. also make sure unets are wrapped in a modulelist so that at sample time, blurring does not happen	2022-04-18 12:04:31 -07:00
Phil Wang	6cddefad26	readme	2022-04-18 11:52:25 -07:00
Phil Wang	0332eaa6ff	complete first pass at full cascading DDPM setup in Decoder, flexible enough to support one unet for testing	2022-04-18 11:44:56 -07:00
Phil Wang	1e939153fb	link to AssemblyAI explanation	2022-04-15 12:58:57 -07:00
Phil Wang	1abeb8918e	personal project management for next week	2022-04-15 08:04:01 -07:00
Phil Wang	b423855483	commit to jax version	2022-04-15 07:16:25 -07:00
Phil Wang	5b4ee09625	ideation	2022-04-14 13:48:01 -07:00
Phil Wang	9f55c24db6	allow for decoder conditioning with the text encodings from CLIP, if it is passed in. use lazy linear to avoid researchers having to worry about text encoding dimensions, but remove later if it does not work well	2022-04-14 11:46:45 -07:00
Phil Wang	69e822b7f8	"project management"	2022-04-14 10:20:37 -07:00
Phil Wang	68e9883f59	use cross attention for conditioning unet based on image embedding tokens (which opens up the door on conditioning on text encodings as well	2022-04-14 10:10:04 -07:00
Phil Wang	e1b0c140f1	cleanup readme	2022-04-14 08:51:22 -07:00
Phil Wang	5989569a44	link to OpenCLIP effort	2022-04-14 08:31:15 -07:00
Phil Wang	7fb3f695d5	offer continuously parameterized time embedding for diffusion prior network, remove a hyperparameter that may trip up people, if not set correctly	2022-04-14 08:28:11 -07:00
Phil Wang	7e93b9d3c8	make sure classifier free guidance condition scaling is exposed on DALLE2 forward function	2022-04-13 20:14:28 -07:00
Phil Wang	4c827ba94f	typo	2022-04-13 19:01:03 -07:00
Phil Wang	cb3923a90f	readme tweak	2022-04-13 18:43:34 -07:00
Phil Wang	cc30676a3f	lengthen todo	2022-04-13 18:34:09 -07:00
Phil Wang	c7fb327618	link to x-clip	2022-04-13 18:26:30 -07:00
Phil Wang	14ddbc159c	cleanup	2022-04-13 18:24:32 -07:00
Phil Wang	0692f1699f	favorite quote	2022-04-13 18:17:59 -07:00
Phil Wang	26c4534bc3	readme	2022-04-13 18:11:55 -07:00
Phil Wang	a1a8a78f21	fix everything and make sure it runs end to end, document everything in readme for public	2022-04-13 18:05:25 -07:00
Phil Wang	2a424b6a28	readme	2022-04-13 10:58:06 -07:00
Phil Wang	3aa6f91e7a	be transparent	2022-04-13 10:32:11 -07:00
Phil Wang	9f1fe6c7ae	update todo	2022-04-13 10:09:08 -07:00
Phil Wang	6d4e9c97bf	todo	2022-04-12 20:50:29 -07:00
Phil Wang	40140b54d6	put on project manager hat	2022-04-12 17:51:23 -07:00
Phil Wang	83aabd42ca	move epsilon inside of square root for further stability in rmsnorm ... improvise and use rmsnorm in convnext blocks too	2022-04-12 11:18:36 -07:00
Phil Wang	cf22affcbb	bring in modified unet using convnext blocks https://arxiv.org/abs/2201.03545	2022-04-12 10:58:44 -07:00
Phil Wang	604765b563	readme	2022-04-12 10:35:56 -07:00
Phil Wang	de75a8af76	link to yannic, since he is the best	2022-04-12 10:27:01 -07:00
Phil Wang	24b428bdfc	readme	2022-04-12 10:12:42 -07:00
Phil Wang	0070547e3b	add a link to laion discord	2022-04-10 19:03:31 -07:00
Phil Wang	2dc8717bbe	readme	2022-04-09 10:47:49 -07:00
Phil Wang	7b54195da4	explain to public	2022-04-07 09:53:56 -07:00
Phil Wang	0754a694ba	cite katherine, as she was the true genesis of CLIP + diffusion (and now latent diffusion)	2022-04-07 09:26:28 -07:00
Phil Wang	c5d49db762	intent	2022-04-07 09:14:08 -07:00
Phil Wang	f283bf25be	scaffold	2022-04-07 07:29:34 -07:00
Phil Wang	25fb133c83	diagram	2022-04-07 05:08:11 +00:00
Phil Wang	32b584d6c0	readme	2022-04-06 21:17:16 -07:00
Phil Wang	cfba049416	Initial commit	2022-04-06 21:14:09 -07:00