Closing the Loop: Evaluating and Improving Replit Agent at Scale
How evaluation at Replit went from launch check to improvement loop — learning from production traces to ship a better Replit Agent, week over week.
Intellectual musings.
Need to start writing more...
3 Takeaways From Building AI Agents in Freight Tech
I spent a couple of years building email and voice agents for freight carrier sales. Three takeaways stuck with me.
Evaluating LLMs with Chat Templates
Do chat templates matter at inference time? We test the theory with an instruction-following benchmark and the best open-source LLMs of early 2024.