Skip to content

[Bug]: Hitting Timeouts error almost systematically #57

@optimass

Description

@optimass

Browsergym Version

0.13.3

Playwright Version

1.49.0

Operating System Type

Ubuntu

Operating System Version

Ubuntu 22.04 LTS

Affected Browsers

Chromium

What happened?

Hi, I'm running a WorkArena evaluation on L1, and hitting Timeouts errors almost systematically (rate of ~80%).

Reproduction Steps

  1. Installed dependencies
  2. created a Washington instance
  3. ran workarena-install,
  4. launched some evaluations on L1 using AgentLab

Relevant Logs

Traceback (most recent call last):
  File "path/browsergym/experiments/src/browsergym/experiments/loop.py", line 246, in run
    step_info.from_reset(
  File "path/browsergym/experiments/src/browsergym/experiments/loop.py", line 437, in from_reset
    self.obs, env_info = env.reset(seed=seed)
                         ^^^^^^^^^^^^^^^^^^^^
  File "path/gymnasium/wrappers/common.py", line 146, in reset
    return super().reset(seed=seed, options=options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/gymnasium/core.py", line 328, in reset
    return self.env.reset(seed=seed, options=options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/gymnasium/wrappers/common.py", line 400, in reset
    return super().reset(seed=seed, options=options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/gymnasium/core.py", line 328, in reset
    return self.env.reset(seed=seed, options=options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/browsergym/core/src/browsergym/core/env.py", line 303, in reset
    task_goal, task_info = self.task.setup(page=self.page)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/browsergym/workarena/tasks/base.py", line 153, in setup
    self.start(page)
  File "path/browsergym/workarena/tasks/list.py", line 265, in start
    self._wait_for_ready(page)
  File "path/browsergym/workarena/tasks/list.py", line 189, in _wait_for_ready
    page.wait_for_function(
  File "path/playwright/sync_api/_generated.py", line 11541, in wait_for_function
    self._sync(
  File "path/playwright/_impl/_sync_base.py", line 115, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "path/playwright/_impl/_page.py", line 1082, in wait_for_function
    return await self._main_frame.wait_for_function(**locals_to_params(locals()))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/playwright/_impl/_frame.py", line 771, in wait_for_function
    return from_channel(await self._channel.send("waitForFunction", params))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path/playwright/_impl/_connection.py", line 528, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.TimeoutError: Page.wait_for_function: Timeout 30000ms exceeded.
2025-01-06 21:42:34,299 - 72112 - browsergym.experiments.loop - ERROR - Error while closing the environment in the finally block: Page.wait_for_function: Timeout 30000ms exceeded.

Additional Context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions