Roast topics
Find topics
Roast it!
Roast topics
Find topics
Find it!
Login
From:
Neel Nanda
(Uncensored)
subscribe
Tiny Mech Interp Projects: Emergent Positional Embeddings of Words — Neel Nanda
https://www.neelnanda.io/mechanistic-interpretability/emergent-pos
links
backlinks
A rough post exploring the emergent positional embedding hypothesis - rather than representing "this is the token in position 5" models may represent eg "this token is the second name in the sentence"
