Skip to content

Setting cookies will request a page twice #63

@n-studio

Description

@n-studio

When I set cookies, the spider will request the page twice instead of once. Is it an expected behavior or is it a bug? It also sleeps twice.

Without cookies

DEBUG -- : BrowserBuilder (selenium_chrome): created browser instance
DEBUG -- : BrowserBuilder (selenium_chrome): enabled before_request.delay
DEBUG -- : Browser: sleep 2.93 seconds before request...
DEBUG -- : BrowserBuilder (selenium_chrome): enabled custom user_agent
DEBUG -- : BrowserBuilder (selenium_chrome): enabled native headless_mode
INFO -- : Browser: started get request to: https://www.google.com
[DEPRECATION] :driver_path is deprecated. Use :service with an instance of Selenium::WebDriver::Service instead.
INFO -- : Browser: finished get request to: https://www.google.com
INFO -- : Info: visits: requests: 1, responses: 1

With cookies

DEBUG -- : BrowserBuilder (selenium_chrome): created browser instance
DEBUG -- : BrowserBuilder (selenium_chrome): enabled custom cookies
DEBUG -- : BrowserBuilder (selenium_chrome): enabled before_request.delay
DEBUG -- : Browser: sleep 1.63 seconds before request...
DEBUG -- : BrowserBuilder (selenium_chrome): enabled custom user_agent
DEBUG -- : BrowserBuilder (selenium_chrome): enabled native headless_mode
DEBUG -- : Browser: sleep 3.03 seconds before request... // <-- sleeps a second time
INFO -- : Browser: started get request to: https://www.google.com
[DEPRECATION] :driver_path is deprecated. Use :service with an instance of Selenium::WebDriver::Service instead.
INFO -- : Browser: finished get request to: https://www.google.com
INFO -- : Info: visits: requests: 1, responses: 1
DEBUG -- : Browser: driver.current_memory: 333616
INFO -- : Browser: started get request to: https://www.google.com // <-- makes another request
INFO -- : Browser: finished get request to: https://www.google.com
INFO -- : Info: visits: requests: 2, responses: 2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions