Show HN: Byte-Pair Encoding tokenizer for training LLMs on large datasets

5 pointsposted a year ago
by yu3zhou4
(github.com)

No comments yet