Login
Roast topics
Find topics
Find it!
From:
mlops.systems
(Uncensored)
subscribe
Alex Strick van Linschoten - The What, Why, and How of Tokenisation in Machine Learning
https://mlops.systems/posts/2023-06-01-why-tokenisation.html
links
backlinks
Tagged with:
nlp
tokenisation
balochi-language-model
Roast topics
Find topics
Roast it!
The basics around the tokenisation process: why we do it, the spectrum of choices when you get to choose how to do it, and the family of algorithms most commonly used at the moment.
