A few weeks back, I implemented GPT-2 using WebGL and shaders (Github Repo) which made the front page of Hacker News (discussion). This is a short write-up over what I learned about old-school general-purpose GPU programming over the course of this project. The Origins of General-Purpose GPU Programming# --- In the early 2000s, NVIDIA introduced programmable shaders with the GeForce 3 (2001) and GeForce FX (2003). Instead of being limited to predetermined transformations and effects of earlie...
