Still thinking about that METR study and I saw John Whiles write this: The developers could provide chunks of that mental model to their AI tools - but doing so is a slow and lossy process that will never truly capture the theory of the program that exists in their minds. By offloading their software development work to an LLM they hampered their unique ability to work on their codebases effectively. This feels to me like a good take on why AI assistants slowed down developers in that study. ...
