Post

Conversation

Dwarkesh Patel
@dwarkesh_sp
Here's a question I find confusing and interesting and which actually tells us a lot about the nature of current AI progress: Why has progress on computer use been so slow? Computer use is so clearly verifiable. I think the answer is that it is not enough for a domain to be verifiable. It also has to be very grindable—in the sense that you can run lots of parallel rollouts against a deterministic and replayable simulator. If you’re trying to make a model better at coding, you can create an environment that has a software repo with some missing feature that you’ve tasked the AIs with creating, and then you have a thousand parallel agents just go at the problem, each with their identical copy of the container. But this doesn’t work with computer use—at least not trivially. You can’t have a thousand agents go try the same checkout flow on Amazon. Because Andy Jassy will find and detect your bots and shut your ass down. How would we train an AI to build a business? How would you make an AI that’s really good at winning court cases? Or having a profitable day trading in the markets? Or helping a candidate win an election? What is the RL environment to make an AI as good at politics as Lyndon Johnson, or as good at building a space launch business as Elon Musk? The rollout requires interacting with the world and cannot be recreated simply within the datacenter. And the outer loop verification may take months or years of real world actions to elicit, and cannot be re-observed by perturbing the model’s actions thousands of times in parallel so that you can isolate what exactly the model did that actually worked.
Quote
Dwarkesh Patel
@dwarkesh_sp
What does the next training paradigm look like? 0:00:00 – The big research bet the labs are making 0:02:12 – Grindability is just as important as verifiability 0:06:10 – Will RLVR alone generalize? 0:08:41 – Getting the learning back to the weights 0:15:22 – Dreaming 0:17:23 –
0:13 / 19:53
561.5K
Views
View quotes
Amitav Krishna
Post your reply

Chetan
@chetanankola
THIS music was made with computer use - GPT 5.6 Astra learnt how to use Maschine from native instruments and made music = yeah i know its not great - but i make music and this seems very cool coz it made it.. this is not Suno which suffers in terms of music quality - these are
Quote
Chetan
@chetanankola
ok this is insane - i asked Astra to use Maschine from @NI_News (native instruments) to make up a neo soul music using computer use ! it made something which isnt that bad at all!!!! am gonna teach it a lot of things to make music in my style of music! this is what it made..
0:04 / 0:36
600
sensho
@sensho
5.5 / 5.6 computer use is pretty good on codex but your broader point for unverifiable domains just sounds like a compute problem? if we had 10x or 100x the compute we have today then the rollouts on more long horizon simulators would be trivial
3.2K
alth0u🧶
@alth0u
its bc of this
Quote
alth0u🧶
@alth0u
if you look at the plot of (1-sim2real issues) on the ordinate with domains on the abscissa, you see power law with math most leftward code in a sharp drop afterward and then a very long tail and we are at the code drop right now everything beyond this shoulder we will have
1.4K
ar0cket1
@ar0cket1
llms aren't built to do computer use natively nearly as well as code writing, I don't think the amazon thing is really a big bottleneck but just an LLM skill issue.
3.7K
Lazarz
@Laz4rz
Computer-use and it will be solved fast, as I’ve worked on it quite a lot:
Quote
Lazarz
@Laz4rz
This has been very clear for a long time for anybody working near computer-use. The frontier labs just didn’t care. A short history on this is: until second half of last year only Google out of all big labs had a model capable of using a computer how a human would (understanding x.com/dwarkesh_sp/st…
1.1K
Tianbao Xie
@TianbaoX
The first step is to have a valid and solid signal, for this we created OSWorld series; and provide ways of scaling the envs and tasks (CUA-gym)
406
James
@Darpinian
Learned world models make any domain "grindable". Instead of training agents on real world experience directly we will train them inside a world model. Real data will only be used to improve the world model. This is the path to human level sample efficiency.
913
callum
@SharrockCallum
does this not pass the bottleneck back to sample efficiency (or realistic sims)? since real world feedback is sparse and, in the case of eg building a company, costly if wrong?
279
Anam Hira
@anamhira
You could create a clone of Amazon and horizontally scale reward training on that. Most of the infrastructure is nascent though, it will be solved
86
Square profile picture
Center for AI Safety
@CAIS
> You can’t have a thousand agents go try the same checkout flow on Amazon. You can make simulated Amazon environments, but that doesn't make computer use trivial either. A simpler answer is that they don't have good enough vision skills.
439
Leo Linsky
@leo_linsky
You are on to something here -- we've found that scalable, *challenging* environments are key to generalizing intelligent behavior. We've solved this with dynamic multi-agent game environments at gertlabs.com/spectate We auto-generate high quality, verifiable environments at
Image
81
Ofek Shaked | AI Engineer
@VibeCoderOfek
Coding improved quickly because cheap deterministic sandboxes exist. Browser and desktop agents do not have that luxury, which is why the outer loop stays slow no matter how good the model gets at single steps.
521
Tarun Vedula
@TarunVedula
If a thousand agents are trying to do the same checkout flow you’d probably wanna look into an api over asking an agent to act like a human no?
74
Square profile picture
Phi Browser
@phibrowser
From the inside it's worse than the bot-bans: the site itself isn't stationary. The checkout flow I ran yesterday is a different DOM today, so you can't even replay one clean trajectory, let alone a thousand parallel ones. The simulator keeps rewriting itself.
431
Alexey Moiseenkov
@Darkolorin
I don’t think we should consider computer use as general whatever purpose task. There are clearly lots of verifiable feature to be built. But everybody focus on mouse clicking control via text input.
117
Ankit Maloo
@ankit2119
problem is not access to amazon website or clone of the same. i believe the clones are already used by model labs. even with unlimited rollouts, right now models are not good at search because the branching factor is enormous. you need prohibitive amount of search per task to
117
vibebuilder
@vibebuild
Progress on Computer Use was slow a year ago, have you tried GPT 5.5 Computer Use? It's really really good. Not perfect, but very good. Main bottleneck is vision and mouse control. Writes too much code to operate a computer (we don't write code to operate).
791
Lon Lundgren
@Lon
I’ve been working on a computer use paradigm that blows everything today out of the water. The demo is almost pants shittingly good and getting better every day. Busy integrating it into my workflow vs trying to create a standalone product. Not really the biz I want to be in.
474
Luis Lozano
@luislozanog86
I was working on computer use for most of 2025, I even built a engine based on AX trees that worked relatively fine, the issue for me is the UI is made for humans not for AI, so yes you can have Codex click here and there, but is the best way if a bash can solve it easier and in
531
Jason
@snellchapo
It's a simple answer, really. The interface was designed for humans. Why would you want the AI agent to open the calculator app to do math, the way a human would? It's a computer, it computes, it can execute code. This is precisely why it's "expensive". They're doing it wrong.
397
Milind S
@milindlabs
A library of verified, replayable task demos is worth way more in computer-use than in coding which is why im building something towards that
Quote
Milind S
@milindlabs
I made a claude co-work style workflow recorder But for your mac You can now record your most boring, repetitive workflows and hand them over to your AI You can copy all the steps as markdown context and any AI will understand what you are doing. Its very simple but a very
204
Aleksandar Stanic
@aleks_stanic
> I think the answer is that it is not enough for a domain to be verifiable. Or because MMU/vision requires some fundamental breakthrough(s). in particular in how we map images -> discrete tokens (that you get for free in purely text-based problems).
27
James Gish
@jamesgish_
Google Deepmind is partnering with EVE Online to train within its sandbox universe. Its economy is a decent first-order approximation to the real economy, so there may be good isomorphisms between the real world and that one...
454
Tim Tyler
@tim_tyler
Encoding a video of your screen takes a lot of tokens. It's an expensive thing to do. For most applications there are other approaches.
194
Rahul Janagouda
@rahuljanagouda
I was having a late night chat with claude last month asking the exact same question. Its an architectural limitation, but wanted to understand what model said when i asked, "Why cant you (claude) update your weights based the learnings from every sessions.?"
Image
508
Nivge
@Nivgeron
I thought it was a context window issue. Effective context is also improving slowly, and computer use requires a lot of context to be good (many many images).
84
Justus August
@justusfaugust
what a bs take, this is all trivial with more compute, and computer use in codex is already great (even with the publicly available models), also what kind of fake problem is getting bot-detected, you could simple reconstruct such pages and get rid of that problem easily.
110
Dipankar Sarkar
@dipankarsarkar
the uncomfortable part: grindability and real world value are almost inversely correlated. the domains easy to simulate (code, games, math) are easy because the environment is cheap to copy. the ones worth automating are valuable because they can't be replayed. RL eats the
261
Thomas
@selectsand
Computer use on Linux is great ootb because so much is text based Which I know is sorta bypassing the longer term agi question but also We put door handles where we can reach them. We could shift the built environment to match the agents, more CLIs everywhere, more text based
235
Buddy Williams
@BuddyIterate
Re: “rollout requires interacting with the world” - why aren’t simulations an option, say for politics? Personas can be RL trained on real human data, conversations. Events can be modeled. These can then be used as RL environments. See RL environments as nested rather than flat
700
Harshad
@harshad1313
Imo it hasn't been so slow The increase in llm performance with extreme scale has made some forget how hard and slow it is to actually build things If we still don't have great computet use in 2 years I'll start to complain a little
12
Florian Buetow
@bytetweets
The main reason is that a computer use mimics how humans use a computer. So while it might be compelling to try to replicate that with AI, it's the equivalent of building a robot to drive a car made for humans. You don't need the robot to interface with your car, you just need to
70
Taylor Luchini
@TaylorLuchini
I wouldn't be surprised if companies like Amazon eventually build interfaces specifically for AI rather than having agents navigate websites designed for people.
14
D@RWIN
@DarwinSantosNYC
Perhaps, you are confounding solving all issues through computer use vs computer use being useful beyond what a human can do. The latter is already the case. I know because I wouldn't use a computer without Codex or a harness that allows me to do all like never before:
Quote
D@RWIN
@DarwinSantosNYC
Replying to @TheRealAdamG and @dwarkesh_sp
Agree. I've used Computer Use daily. I believe computer use as such is a mostly solved issue (clicks, key events, OS and apps control). Use Mac's codex long enough, and you'd come to the same realization. Same for Claude Code from the terminal, and Google's Antigravity. They
133
Element Dong
@elementdsj
this reads as a question about computer use pace. what it's actually surfacing is that verifiability and grindability are orthogonal — most benchmarks conflate them and then act surprised.
388
Phelix Juma
@jumaphelix
The answer lies in skill acquisition. AI currently does not have skill acquisition capability (memory and claude skills are not "skills" in the correct definition of the word). AI uses computer via reasoning, expert humans use procedural memory
342
noob trader
@ishwors46091536
but why does it have to be amazon or a web application that can be rate limited or blocked? why not using adobe, using mspaint, using desktop application?
343
Nicholas Pipitone
@npip99
If it can play Factorio it can buy groceries for you. Often, the main adoption problem is that the UI, search bar, recommendation system, are already optimized for the consumer anyway. And if it's B2B/Infra rather than consumer-facing, API is always better.
27
Mandeep
@themandeepc
That’s true. But it’s also less verifiable than you may think. See my tweet:
Quote
Mandeep
@themandeepc
I think folks are being misled by "high performance" on browser use "benchmarks". It's not appreciated enough just how different they are to LLM benchmarks, and why they're difficult to do right and currently extremely flawed. LLM benchmarks are "closed world": the model
758
Art Seabra 🫆
@ifthis
Quote
Art Seabra 🫆
@ifthis
Article cover image
The User Said Yes. That Wasn’t Enough.
A field note on false agreement, the reconstruction cost of agreement, and human–AI co-generation loops. Observed and written by Art Seabra @ifthis. TL;DR This text explores the critical...
135
Em Dash
@NoemiTitarenco
Its not about amazon shutting you down. LLMs are error machines. Coding has always been very tolerant of errors. We deal with errors all the time it's part of the workflow. Most domains are not like this.
764
Ferbin
@Ferbin08
it's distribution shift. websites change, buttons move, pages load janky. training is always cleaner than reality. robotics teams spend years on this problem alone.
108
Mochael
@MichaelMotorcy9
It’s true, really hard for RL environments to correctly represent all the issues on systems people are trying to automate like old ass ERPs/EHRs. I think computer use at least in the browser has gotten good, where coding agents can lean on playwright and using internal network
385

Trending now

What’s happening

Politics · Trending
25th Amendment
Politics · Trending
Obamacare
Politics · Trending
Carney
Trending
John Brown
Show more