Computer-use-preview model never sends a "back" action

ido · March 23, 2025, 10:21pm

I am using computer-use-preview-2025-03-11 to run a browser agent, both with:

The official Python example project: openai/openai-cua-sample-app
My own Node.js version: loadmill/node-playwright-cua

No matter how I try to instruct it to use the browser’s back function, it never sends a back action.

I can see the back action is defined in the official Python example (here), but I have never been able to make it trigger.

Besides the official Python example, I couldn’t find a list of supported actions, and the documentation page seems to be broken:
https://platform.openai.com/docs/assistants/tools-computer-use

Has anyone successfully triggered the back action? Is there a workaround or an updated list of supported actions?

JUAN_AREVALO · May 8, 2025, 4:49pm

did you managed to solve it? I think I’m having a similar issue, in my case I’m using the agents sdk and AsyncComputer implementation

ido · May 11, 2025, 4:38pm

Sort of. I was able to prompt around it and hack it.
In my system prompt, I instructed the model to use a keyboard shortcut when it intends to use the back action and mapped that shortcut to the actual browser back action.

See here:

github.com/loadmill/tiny-cua

index.js

master


      
          If unsure, take a screenshot once before proceeding.  
          Do not repeat actions that have no visible effect.  
          
          Available browser actions:
          click, double_click, move, drag, scroll, type, keypress, wait, goto, back, forward, screenshot.
          
          **Keyboard Shortcuts:**
          This web browser is running on ${osName}.
          Use OS-specific shortcuts. For example:
          On macOS, "CMD" should be used where applicable.
          On macOS if the user asks you to go back (to the previous page), use a combination of Cmd + [ keys.
          `;
          
          let previousResponseId;
          let messages = [{ role: "system", content: initialSystemText }];
          
          while (true) {
            
            let userInput;
            if (instructions.length > 0) {
              userInput = instructions.shift();

github.com/loadmill/tiny-cua

actions.js

master


      
          case "keypress":
            console.log(`Pressing key: ${keys}`);
            const mappedKeys = keys.map(key => keyMap[key.toUpperCase()] || key);
            const modifiers = mappedKeys.filter(key => modifierKeys.has(key));
            const normalKeys = mappedKeys.filter(key => !modifierKeys.has(key));
          
            if (
              (mappedKeys[0] === "Meta" && mappedKeys[1] === "[") ||
              (mappedKeys[0] === "Alt" && mappedKeys[1] === "ArrowLeft")
            ) {
              await page.goBack();
              break;
            }
          
            // Hold down modifier keys
            for (const key of modifiers) {
              await page.keyboard.down(key);
            }
          
            // Press normal keys
            for (const key of normalKeys) {

JUAN_AREVALO · May 14, 2025, 3:02pm

That’s great, thanks for sharing!!

I’ve share my solution in a different post, using function tools. I don’t know why I can attach links, just look for the thread “Extending AsyncComputer methods in Agents SDK”. I’m attaching a picture of the response as well

Hope it helps!

Topic		Replies	Views
Extending AsyncComputer methods in Agents SDK API agents , agents-sdk	1	104	May 14, 2025
New tools for building agents: Responses API, web search, file search, computer use, and Agents SDK Announcements	64	10903	July 8, 2025
I need help using assitant in python (required_action infinite loop) API python , assistants-api	12	11030	January 3, 2025
Issue with Generated but non-clickable Links in Assistant Output Plugins / Actions builders actions	9	2813	March 7, 2024
Incorrect API docs for computer use preview Documentation api , agents	2	222	March 31, 2025

Computer-use-preview model never sends a "back" action

Related topics