Grug-brained evals
Big brain spend months building perfect quality metrics. Grug brain no trust, and just want dumb labels from coworkers 👍/👎.| Doug Turnbull's Blog
Big brain spend months building perfect quality metrics. Grug brain no trust, and just want dumb labels from coworkers 👍/👎.| Doug Turnbull's Blog
Leading a new effort at work meant to transform the company? May you last long enough to become the villain.| Doug Turnbull's Blog
RAG apps have a big blindspot - using actual user engagement to drive improvement. But its a hard problem. Let’s discuss!| Doug Turnbull's Blog
Agents need tools they understand, like simple keyword search. They can reason about these tools, evaluate the results, refine, and iterate to deliver rather...| Doug Turnbull's Blog
Quick experiment few shotting vs using rules with little LLMs| Doug Turnbull's Blog
We need to evaluate agents from a completely new angle: as researchers not reasoners.| Doug Turnbull's Blog
AI coding requires a tradeoff between human and AI context, and your job is now to decide where to best spend your limited attention vs to let the AI do work| Doug Turnbull's Blog
You can get started improving search relevance without labels and judgments. Which is an imperfect model anyway.| Doug Turnbull's Blog
Elasticsearch knn query is both a joy and a headache - here is where you’ll get stuck and the hacks I’ve used to overcome them.| Doug Turnbull's Blog
Reciprocal Rank Fusion, while a useful tool, doesnt magically make hybrid search relevant| Doug Turnbull's Blog
