Login
From:
Tim Dettmers
(Uncensored)
subscribe
TPUs vs GPUs for Transformers (BERT) — Tim Dettmers
https://timdettmers.com/2018/10/17/tpus-vs-gpus-for-transformers-bert/
links
backlinks
Tagged with:
hardware
gpu
accelerators
matrix multiplication
Roast topics
Find topics
Find it!
Here I develop a theoretical model of TPUs vs GPUs for transformers as used by BERT and show that current GPUs are about 32% to 54% slower for this task.
