One trillion tokens per day. Is that a lot? “And when we look narrowly at just the number of tokens served by Foundry APIs, we processed over 100t tokens this quarter, up 5x year over year, including a record 50t tokens last month alone.” In April, Microsoft shared a statistic, revealing their Foundry product is processing about 1.7t tokens per month. Yesterday, Vipul shared Together.ai is processing 2t of open-source inference daily. In July, Google announced a staggering number : “At ...
