Login
From:
Giles' Blog
(Uncensored)
subscribe
Writing an LLM from scratch, part 13 -- the 'why' of attention, or: attention heads are dumb :: Giles' blog
https://www.gilesthomas.com/2025/05/llm-from-scratch-13-taking-stock-part-1-attention-heads-are-dumb
links
backlinks
Tagged with:
ai
llm from scratch
til deep dives
A pause to take stock: realising that attention heads are simpler than I thought explained why we do the calculations we do.
Roast topics
Find topics
Find it!
