I want to get back into writing more regularly this year, so in light of that, here’s my last year in review. Evaluating LLMs Like many of us in tech, I spent a large portion of 2024 thinking about and working with LLMs, but I was lucky enough to do it for work. I spent the year designing, building, open-sourcing, (and naming! 🐊) an application to evaluate LLMs, Lumigator. In support of that work, I did open-source work in the LLM ecosystem and learned a ton of stuff along the way. Just ...
