Skip to content

Added small fix to split by eos_token_id before decoding#2512

Open
EtashGuha wants to merge 1 commit intoEleutherAI:mainfrom EtashGuha:etashg/tokenize_fix
+1

Commits