The README is geared towards technical people (complete with the HN screenshot) but when I see a tool like this all I can think of is how helpful this would be for my mom when I'm trying to tell her how to download and print a document over the phone
Interesting. When I read the headline I imagined this would be a sort of thinking trace booster -- letting the agent focus its own attention on different parts of the screen. But this is cool in a different way. I bet agentic harnesses would find it useful for communicating with other agents / themselves as well.
What terminal / agentic workflows spawn the demonstrated dialogue boxes that require the User to click/reject the action? Aren't most such flows actually inline UIs? And finally, if the core issue is that these confirmation boxes are tied to the terminal that triggered them, which does not autofocus, what's the point of said "big arrow" that is also lurking behind without focus?
If the said "big arrow" automatically gains focus, isn't the real fix here to just make the dialogue boxes themselves gain focus automatically? Both are similarly disruptive anyway.
The first example (of HN) is the one that feels like it has the most potential to me.
"Teach me to use this app myself" kinda stuff. Guiding agent rather than doing agent.
Honestly, @franze, if you're reading this, maybe update your screenshots with that bent (showing us an agent in tutorial mode on some complicated app)?
This could be great for documentation. Screenshots in docs are frequently useless because they show me a screen and say "click X" where I still have to search X visually. And I could just to dhat in the other tab I have open.
Claude refuses to do certain actions (enter passwords, change security settings, create new accounts on external services) even in Yolo mode running as sudo. (I tested it all on its own mac machine)
What a time to be a radical centrist - the AI haters seem out of touch, the AI thought leaders can't stop huffing their farts and being condescending, and somehow this is on the top of HN. What a silly time.
- rainbow dripping arrows
- angrily pointing arrows
- flame-surrounded text boxes with particle effects
- the ability for the agent to play airhorn.wav at max volume, overriding existing volume or mute settings
What terminal / agentic workflows spawn the demonstrated dialogue boxes that require the User to click/reject the action? Aren't most such flows actually inline UIs? And finally, if the core issue is that these confirmation boxes are tied to the terminal that triggered them, which does not autofocus, what's the point of said "big arrow" that is also lurking behind without focus?
If the said "big arrow" automatically gains focus, isn't the real fix here to just make the dialogue boxes themselves gain focus automatically? Both are similarly disruptive anyway.
"Teach me to use this app myself" kinda stuff. Guiding agent rather than doing agent.
Honestly, @franze, if you're reading this, maybe update your screenshots with that bent (showing us an agent in tutorial mode on some complicated app)?
Does one need 4 programming languages to draw something on a mac?
https://donhopkins.com/home/archive/psiber/cyber/pointer.ps
https://youtu.be/_fqCeuue5Ac?t=213
https://medium.com/@donhopkins/the-shape-of-psiber-space-oct...
Grim. If you're just there to click sudo buttons for the bot, you might as well give it root access and be done.
~guywithnopowertodisallowit