Login
From:
Giles' Blog
(Uncensored)
subscribe
The fixed length bottleneck and the feed forward network :: Giles' blog
https://www.gilesthomas.com/2025/08/the-fixed-length-bottleneck-and-the-feed-forward-network
links
backlinks
Tagged with:
musings
python
ai
The feed-forward network in an LLM processes context vectors one at a time. This feels like it would cause similar issues to the old fixed-length bottleneck, even though it almost certainly does not.
Roast topics
Find topics
Find it!
