Ship the Paper takes one research result into a production decision. Generative AI is for everyone else who has to use the models this week.
Ship the PaperEvaluation
LatestA new paper lets an AI judge keep only the verdicts it is sure of, with a stated cap on how many can be wrong. I rebuilt it on a laptop to find where that cap stops holding.
Generative AIModels
I fine-tuned the small model that reads my stock-exchange filings, and named it Argus. Claude refused to teach it, so an open model did, and Claude graded the exam.
Generative AIModels
TypeSafe's Jev answers with a probability instead of a paragraph. I wanted to know when you can let that probability make the call, so I built a small test.
Ship the PaperEfficiency & Serving
FinCacheServe lets a financial assistant reuse finished answers until a filing changes. I rebuilt its reuse check to see whether an up-to-date answer is also the right one.
Generative AIProduct
Meta's Muse went to the top of the App Store by doing errands instead of chatting. I rebuilt its approval guard in miniature to see which bad errands still get through.
Ship the PaperLong-Horizon Planning & Reliability
Dream-RSI scores a new exploration policy by replaying a finished discovery run instead of paying for another one. I rebuilt the replay to see what it can pick, and what it never sees.
Generative AIOperations (LLMOps)
I built FIRE, an AI advisor for Indian investors, then let it promote its own daily ranking. One lucky night was enough to go live.
Generative AIData
Three models rebuilt a windmill in Blender from an empty scene. Two passed every machine check. The third I could only see after I added a camera.
Generative AIModels
From cap sets to olympiad gold to fluids: a walk through the objects language models have left in mathematics, and the ones still on the wall.
Generative AIApplications
A phone went missing. I had WhatsApp voice notes from that night, and I used to think dropping them in a chatbot was the same as listening.
Generative AISafety & Governance
Two labs said their eval agents got out because a prompt lied and a path was open. I built the smallest version of that path to see which wrapper refuses a lookalike host.
Generative AIProduct
Claude's chat memory now rides into Cowork. I packed one months-old chat store into three agent drafts, and a brainstorm left in the speaker email as a confirmed sponsor.
Ship the PaperLong-Horizon Planning & Reliability
A new paper scores a context summary by the blocked or repeated tool calls it causes next. I rebuilt the scorer to find out how much of a summary's damage ever makes that noise.
Generative AIInfrastructure
I ran one scripted model inside four apps and asked each the same questions about one expense sheet. The app decided what came back right, and whether a wrong answer sounded wrong.
Generative AIApplications
A colleague wanted the launch email out of a Custom GPT by Friday. I built it four ways from the same messy brief and watched the price the notes said to keep quiet walk into the draft.
Generative AIData
A chatbot with the whole HR folder uploaded gave last year's leave allowance. I packed the same ten files into a prompt five ways to see which number each one hands back.
Generative AIModels
Three frontier chatbots sit two points apart on this month's leaderboards. I gave them the same two pieces of everyday work to see whether that gap shows up in anything I would send.
Ship the PaperSafety & Security
Ehghaghi and colleagues price a jailbreak by the compute it burns instead of the queries it sends. I rebuilt their two summary metrics to find out which one a nightly build could trust.
Ship the PaperAgents
Swap the harness around a coding agent and its score moves nearly as far as swapping the model. I rebuilt the step behind most of that gap: how the agent's finished patch gets collected.
Ship the PaperAgents
FCPAgent makes a browser agent write down, before each step, what would prove that step wrong. I rebuilt the checking half and found the one failure no such statement can see.
Ship the PaperAgents
AgentOccam shrinks what a browser agent reads and what it is allowed to do. I rebuilt the shrinking step and it held up; the question it never asks is whether the action is allowed.