Frequently Asked Questions (And Answers) About AI Evals - “the projects we’ve worked on, we’ve spent 60-80% of our development time on error analysis and evaluation. Expect most of your effort to go toward understanding failures (i.e. looking at data) rather than building automated checks."
A consumption basket approach to measuring AI progress - “In contrast, actual human users typically deploy AIs to help them with relatively easy problems. They use AIs for (standard) legal advice, to help with the homework, to plot travel plans, to help modify a recipe, as a therapist or advisor, and so on. You could say that is the actual consumption basket for LLM use, circa 2025.” // Also, interesting aside that maybe AI is as good as it’ll get (in a good way).
Air France-KLM to increase intelligence of bots that have saved 200,000 hours - Four or five AI uses cases - maybe even agentic! - from the airline I fly.
Untitled - “AI-generated art is abundant and cheap to produce. Yet they are both priced the same by Adobe."
American - “I know the nationalities of my fellow oath-takers because of the next stage of the ceremony. This was the Roll Call of Nations. I did not know this was going to happen. Every country of origin represented was announced in turn. As your country was named, you were asked to stand up, and remain standing. Afghanistan came first. Then Algeria. The last person to stand, immediately to my left, was from the United Kingdom. There were twenty seven countries in all, out of only fifty or so people. For me this part in particular was enormously, irresistibly moving. It perfectly expressed the principle, the claim, the myth–as you please–that America is an idea. That it does not matter where you are from. That, in fact, America will in this moment explicitly and proudly acknowledge the sheer variety of places you are all from. That built in to the heart of the United States is the republican ideal not just that anyone can become an American, but that this possibility is what makes the country what it is."
AI Cleanup/AI catchphrases, Wikipedia - List of words and writing styles commonly found in AI output. I’m really interested in how AI-language leaks (is leaking?) into human language. Also, these would be words and styles you’d tell the AI to NOT do, if you cared.
How to Fix Your Context - Lots to digest in here.
AI agents wrong ~70% of time: Carnegie Mellon study - ”Gartner still expects that by 2028 about 15 percent of daily work decisions will be made autonomously by AI agents, up from 0 percent last year. Also, the firm sees 33 percent of enterprise software applications including agentic AI by that time.”