Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stagehand.act interacts with stagehand-nav components #301

Open
dncnbuck opened this issue Dec 13, 2024 · 1 comment
Open

stagehand.act interacts with stagehand-nav components #301

dncnbuck opened this issue Dec 13, 2024 · 1 comment
Assignees
Labels
bug Something isn't working
Milestone

Comments

@dncnbuck
Copy link

dncnbuck commented Dec 13, 2024

I updated from stagehand version 1.3 to 1.6 so I could remove the debug dom elements without having to run manual cleanup

Note: they still are present in my session with debugDom: false
image

It seems like the system prompts update (https://github.com/browserbase/stagehand/blob/main/lib/prompt.ts#L22)

For me this has caused the agent to now attempt to resolve actions by clicking these debugDom elements. Which ends up in an endless loop where it clicks the button and clicks it again. (Note I downgraded to 1.3 and this behaviour is not seen)
Heres a small snippet of the log for a acting on a url https://zip.co/us with action=close any popups, banners, terms and conditions that are present when you navigate to the homepage.

[stagehand:action] received response from LLM {"response":{"value":"{\"method\":\"click\",\"element\":0,\"args\":[],\"step\":\"Clicked on the 'Next' button to potentially close a popup or navigate through a carousel.\",\"why\":\"This button might be part of a popup or carousel that needs to be closed or navigated through to access the main content.\",\"completed\":false}","type":"object"}}
[stagehand:action] executing method {"method":{"value":"click","type":"string"},"elementId":{"value":"0","type":"integer"},"xpaths":{"value":"[\"/html/body[1]/button[2]\",\"//html/body/button[2]\"]","type":"object"},"args":{"value":"[]","type":"object"}}
[stagehand:action] page URL before action {"url":{"value":"https://zip.co/us","type":"string"}}

I quick fix is to tell the agent to never click on selectors with classname = 'stagehand-nav', but I think a change in the system prompt might be a more long term change, for example

Note 3: A step action to accomplish the user's goal will never be successful if stagehand-nav elements  are clicked
@kamath kamath self-assigned this Dec 13, 2024
@kamath kamath added the bug Something isn't working label Dec 13, 2024
@kamath kamath added this to Stagehand Dec 13, 2024
@kamath kamath added this to the Foundation milestone Dec 13, 2024
@kamath
Copy link
Contributor

kamath commented Dec 13, 2024

Thanks so much for making this a GitHub issue! @seanmcguire12 pointed this out internally when running evals. Appreciate the detailed write-up, but idk if a system-prompt is going to help here. We should probably just deprecate/remove stagehand nav; I don't personally find it very useful, and it's clearly causing some bugs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: No status
Development

No branches or pull requests

2 participants