11 Feb
2023
11 Feb
'23
11:55 a.m.
big inhib! the nice thing with t5 is there are mt5 and t0pp and flan-t5 pretrained models that have been trained on a lot of tokens, not sure if they are in adapterhub. i found the newer hf scripts can train decoder-only models on pure text with linebreaks whereas t5 models may take jsonlines of the format {“translation”:{“key1”:”text1”,”key2”:”text2”}} where key1 and key2 are specified at launch time