11 Commits

Author	SHA1	Message	Date
Phil Wang	1924c7cc3d	fix issue with mixed precision and gradient clipping	2022-05-02 09:20:19 -07:00
Phil Wang	ebe01749ed	DecoderTrainer sample method uses the exponentially moving averaged	2022-04-30 14:55:34 -07:00
Phil Wang	63195cc2cb	allow for division of loss prior to scaling, for gradient accumulation purposes	2022-04-30 12:56:47 -07:00
Phil Wang	a2ef69af66	take care of mixed precision, and make gradient accumulation do-able externally	2022-04-30 12:27:24 -07:00
Phil Wang	5fff22834e	be able to finely customize learning parameters for each unet, take care of gradient clipping	2022-04-30 11:56:05 -07:00
Phil Wang	a9421f49ec	simplify Decoder training for the public	2022-04-30 11:45:18 -07:00
Phil Wang	5063d192b6	now completely OpenAI CLIP compatible for training ... just take care of the logic for AdamW and transformers used namedtuples for clip adapter embedding outputs	2022-04-29 13:05:01 -07:00
Phil Wang	f4a54e475e	add some training fns	2022-04-29 09:44:55 -07:00
Phil Wang	6edb1c5dd0	fix issue with ema class	2022-04-27 16:40:02 -07:00
Phil Wang	45262a4bb7	bring in the exponential moving average wrapper, to get ready for training	2022-04-25 19:24:13 -07:00
Phil Wang	5e03b7f932	get ready for all the training related classes and functions	2022-04-12 09:54:50 -07:00