Ashley Raiteri

A paper came across my feed this week: FormFactory, an interactive benchmarking suite from NUS and friends that tests how well multimodal LLMs can fill out web forms. Their headline finding stopped me cold: every model they tested — GPT-4o, Gemini 2.5 Pro, Claude 3.7 Sonnet — scored near 0% click accuracy on form fields. Zero percent. On forms. The models could often predict the right values. They knew what should go where. But they couldn’t reliably click the right elements to put the data there. The gap between knowing and doing was absolute. ...

I’ve been automating browser workflows with AI agents for a while now, and I’ve landed on a pattern that works unreasonably well. It’s three sessions, three tools, three mindsets. Each one does exactly one thing, and trying to collapse them into a single session is how you burn tokens and get garbage. Session 1: Discovery with Claude-in-Chrome The first session is pure reconnaissance. You open the app in Chrome, start a Claude Code session with the claude-in-chrome MCP extension connected1, and say: tell me everything about this page. ...

There are probably a million reasons why I shouldn’t have built these. If you know any, I’d love to hear them. I use AI coding agents constantly. Claude, mostly. And I am profoundly, constitutionally lazy about the manual parts. The agent does something genuinely smart — writes a query, generates a function, debugs a failing test — and then I have to drag a screenshot, or carefully select text around markdown fences, or copy-paste between windows like it’s 2005. ...

There’s a moment in every engineer’s career when you stumble onto an algorithm so elegant it feels like cheating. For me, that moment came in a windowless lab in Tucson, Arizona, staring at three separate navigation systems that couldn’t agree on where a missile was. I was a guidance software engineer at Raytheon, working on the Tomahawk cruise missile. My job was sensor fusion — taking the outputs of GPS, an inertial measurement unit (accelerometers and gyroscopes), and DSMAC (Digital Scene Matching Area Correlator, which is a fancy way of saying “a camera that compares what it sees to stored photos of the ground”) and turning them into a single, confident answer to the question every guidance system needs answered at every moment: where am I, and where am I going? ...