We explore why most ChatGPT Plugins and LLM agent projects feel underwhelming, and uncover some challenges developers face in building LLM agents. The challenges are mainly in four categories: 1) latency, 2) reliability, 3) UX, and 4) trust / security. We also argue that many LLM agent projects are doomed to fail because natural language is an inferior user interface to GUIs in many situations.