Merge pull request #4072 from seleniumbase/cdp-mode-patch-71

CDP Mode: Patch 71

Author: mdmintz

README.md

@@ -54,15 +54,15 @@
 <br />
 </p>
 
-<p>SeleniumBase is a browser automation framework for the modern web. Both new and experienced Python users alike can easily get started. With special stealth features like UC Mode and CDP Mode, you'll be evading bot-detection and bypassing CAPTCHAs in minutes.</p>
+<p>SeleniumBase is a browser automation framework that empowers software teams to innovate faster and handle modern web challenges with ease. With stealth options like CDP Mode, you'll avoid the usual restrictions imposed by websites deploying bot-detection services.</p>
 
 --------
 
 📚 Learn from [**over 200 examples** in the **SeleniumBase/examples/** folder](https://github.com/seleniumbase/SeleniumBase/tree/master/examples).
 
-🐙 Stealth modes: <a translate="no" href="https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/uc_mode.md"><b>UC Mode</b></a> and <a translate="no" href="https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/ReadMe.md"><b>CDP Mode</b></a> help you evade bot-detection.
+🐙 Stealth modes: <a translate="no" href="https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/uc_mode.md"><b>UC Mode</b></a> and <a translate="no" href="https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/ReadMe.md"><b>CDP Mode</b></a> can bypass bot-detection, solve CAPTCHAs, and call advanced methods from the <a href="https://chromedevtools.github.io/devtools-protocol/" translate="no">Chrome Devtools Protocol</a>.
 
-ℹ️ Most scripts run with raw <code translate="no"><b>python</b></code>, although some scripts use <a href="https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md">Syntax Formats</a> that expect <a href="https://docs.pytest.org/en/latest/how-to/usage.html" translate="no"><b>pytest</b></a> (a Python unit-testing framework included with SeleniumBase that can discover, collect, and run tests automatically).
+ℹ️ Many examples run with raw <code translate="no"><b>python</b></code>, although some use <a href="https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/syntax_formats.md">Syntax Formats</a> that expect <a href="https://docs.pytest.org/en/latest/how-to/usage.html" translate="no"><b>pytest</b></a> (a Python unit-testing framework included with SeleniumBase that can discover, collect, and run tests automatically).
 
 --------
 

examples/cdp_mode/ReadMe.md

@@ -2,7 +2,7 @@
 
 ## [<img src="https://seleniumbase.github.io/img/logo6.png" title="SeleniumBase" width="32">](https://github.com/seleniumbase/SeleniumBase/) CDP Mode 🐙
 
-🐙 <b translate="no">SeleniumBase</b> <b translate="no">CDP Mode</b> is a stealth mode of SeleniumBase that uses the <a href="https://chromedevtools.github.io/devtools-protocol/" translate="no"><span translate="no">Chrome Devtools Protocol</span></a> (via <a href="https://github.com/mdmintz/MyCDP" translate="no"><span translate="no">MyCDP</span></a>) to control the web browser. <b translate="no">CDP Mode</b> can be used either as a subset of <b><a href="https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/uc_mode.md" translate="no"><span translate="no">SeleniumBase UC Mode</span></a></b>, or via <b><a href="#Pure_CDP_Mode" translate="no">Pure CDP Mode</a></b> (<code>sb_cdp</code>), which doesn't use WebDriver at all, and has a slightly different setup.
+🐙 <b translate="no">SeleniumBase</b> <b translate="no">CDP Mode</b> is a stealth mode of SeleniumBase that uses the <a href="https://chromedevtools.github.io/devtools-protocol/" translate="no">Chrome Devtools Protocol</a> (via <a href="https://github.com/mdmintz/MyCDP" translate="no"><span translate="no">MyCDP</span></a>) to control the web browser. <b translate="no">CDP Mode</b> can be used either as a subset of <b><a href="https://github.com/seleniumbase/SeleniumBase/blob/master/help_docs/uc_mode.md" translate="no"><span translate="no">SeleniumBase UC Mode</span></a></b>, or via <b><a href="#Pure_CDP_Mode" translate="no">Pure CDP Mode</a></b> (<code>sb_cdp</code>), which doesn't use WebDriver at all, and has a slightly different setup.
 
 --------
 
@@ -55,6 +55,7 @@ with SB(uc=True, test=True, locale="en") as sb:
     sb.activate_cdp_mode(url)
     sb.sleep(2)
     sb.solve_captcha()
+    sb.sleep(2)
 ```
 
 <img src="https://seleniumbase.github.io/other/cf_sec.jpg" title="SeleniumBase" width="332"> <img src="https://seleniumbase.github.io/other/gitlab_bypass.png" title="SeleniumBase" width="288">
@@ -63,7 +64,7 @@ with SB(uc=True, test=True, locale="en") as sb:
 
 --------
 
-`sb.cdp.gui_click_element(selector)` lets you click on elements using `PyAutoGUI`. Example:
+You can also use `PyAutoGUI` to click on elements with the mouse by calling `sb.cdp.gui_click_element(selector)`. Example:
 
 ```python
 from seleniumbase import SB
@@ -86,7 +87,7 @@ Eg. `sb.cdp.gui_click_element("#turnstile-widget div")`
 
 In most cases, `sb.solve_captcha()` is good enough for CF Turnstiles without needing `sb.cdp.gui_click_element(selector)`. (See [SeleniumBase/examples/cdp_mode/raw_planetmc.py](https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/raw_planetmc.py))
 
-ℹ️ Note that `PyAutoGUI` is an optional dependency. If calling a method that uses it when not already installed, then `SeleniumBase` installs `PyAutoGUI` at run-time.
+ℹ️ Note that `PyAutoGUI` is an optional dependency. If calling a method that uses it when not already installed, then `SeleniumBase` installs `PyAutoGUI` at runtime.
 
 --------
 

examples/cdp_mode/raw_albertsons.py

@@ -5,24 +5,24 @@
     sb.activate_cdp_mode(url)
     sb.sleep(2.5)
     sb.remove_element("div > div > article")
-    sb.cdp.scroll_into_view('input[type="search"]')
+    sb.scroll_into_view('input[type="search"]')
     close_btn = ".notification-alert-wrapper__close-button"
-    sb.cdp.click_if_visible(close_btn)
-    sb.cdp.click("input#search-suggestion-input")
+    sb.click_if_visible(close_btn)
+    sb.click("input#search-suggestion-input")
     sb.sleep(0.2)
     search = "Avocado Smoked Salmon"
     required_text = "Salmon"
-    sb.cdp.press_keys("input#search-suggestion-input", search)
+    sb.press_keys("input#search-suggestion-input", search)
     sb.sleep(0.8)
-    sb.cdp.click("#suggestion-0 a span")
+    sb.click("#suggestion-0 a span")
     sb.sleep(0.8)
-    sb.cdp.click_if_visible(close_btn)
-    sb.sleep(2.8)
+    sb.click_if_visible(close_btn)
+    sb.sleep(3.2)
     print('*** Albertsons Search for "%s":' % search)
     print('    (Results must contain "%s".)' % required_text)
     unique_item_text = []
     item_selector = 'a[href*="/meal-plans-recipes/shop/"]'
-    items = sb.cdp.find_elements(item_selector)
+    items = sb.find_elements(item_selector)
     for item in items:
         sb.sleep(0.06)
         if required_text in item.text:

examples/cdp_mode/raw_cdp_copilot.py

@@ -1,7 +1,7 @@
 from seleniumbase import sb_cdp
 
 url = "https://copilot.microsoft.com/"
-sb = sb_cdp.Chrome(url, locale="en", guest=True)
+sb = sb_cdp.Chrome(url, locale="en")
 textarea = "textarea#userInput"
 sb.wait_for_element(textarea)
 sb.sleep(1.5)

examples/cdp_mode/raw_cf_clearance.py

@@ -12,7 +12,7 @@ def get_cf_clearance_cookie(sb):
 url = "https://gitlab.com/users/sign_in"
 sb = sb_cdp.Chrome(url, incognito=True)
 sb.sleep(2.2)  # Wait for CAPTCHA to load
-sb.gui_click_captcha()  # (Only if found)
+sb.solve_captcha()  # (Only if found)
 sb.sleep(2.2)  # Wait for CAPTCHA success
 cf_cookie = get_cf_clearance_cookie(sb)
 if cf_cookie:

examples/cdp_mode/raw_copilot.py

@@ -1,6 +1,6 @@
 from seleniumbase import SB
 
-with SB(uc=True, test=True, guest=True) as sb:
+with SB(uc=True, test=True, locale="en") as sb:
     url = "https://copilot.microsoft.com/"
     sb.activate_cdp_mode(url)
     textarea = "textarea#userInput"

examples/cdp_mode/raw_driver.py

@@ -6,6 +6,6 @@
 url = "www.planetminecraft.com/account"
 driver.uc_activate_cdp_mode(url)
 driver.sleep(1)
-driver.cdp.solve_captcha()
+driver.solve_captcha()
 driver.wait_for_element_absent("input[disabled]")
 driver.sleep(2)

examples/cdp_mode/raw_elal.py

@@ -6,7 +6,7 @@
     url = "www.elal.com/flight-deals/en-us/flights-from-boston-to-tel-aviv"
     sb.activate_cdp_mode(url)
     sb.sleep(3)
-    sb.cdp.click('label:contains("Departure date")')
+    sb.click('label:contains("Departure date")')
     sb.sleep(1)
     today = datetime.date.today()
     days_ahead = (4 - today.weekday() + 7) % 7
@@ -25,21 +25,21 @@
     sb.sleep(1)
     sb.cdp.gui_press_keys("\b" * 10 + formatted_date + "\n")
     sb.sleep(1)
-    sb.cdp.click('button[data-att="done"]')
+    sb.click('button[data-att="done"]')
     sb.sleep(1)
-    sb.cdp.click('button[data-att="search"]')
+    sb.click('button[data-att="search"]')
     sb.sleep(5)
-    sb.cdp.click_if_visible("#onetrust-close-btn-container button")
+    sb.click_if_visible("#onetrust-close-btn-container button")
     sb.sleep(1)
     view_other_dates = 'button[aria-label*="viewOtherDates.cta"]'
-    if sb.cdp.is_element_visible(view_other_dates):
-        sb.cdp.click(view_other_dates)
+    if sb.is_element_visible(view_other_dates):
+        sb.click(view_other_dates)
         sb.sleep(5)
     if sb.is_element_visible("flexible-search-calendar"):
         print("*** Flight Calendar for El Al (Boston to Tel Aviv): ***")
-        print(sb.cdp.get_text("flexible-search-calendar"))
+        print(sb.get_text("flexible-search-calendar"))
         prices = []
-        elements = sb.cdp.find_elements("span.matric-cell__content__price")
+        elements = sb.find_elements("span.matric-cell__content__price")
         if elements:
             print("*** Prices List: ***")
             for element in elements:
@@ -50,32 +50,32 @@
             print("*** Lowest Price: ***")
             lowest_price = prices[0]
             print(lowest_price)
-            sb.cdp.scroll_down(12)
+            sb.scroll_down(12)
             sb.sleep(1)
-            sb.cdp.find_element_by_text(lowest_price).click()
+            sb.find_element_by_text(lowest_price).click()
             sb.sleep(2)
             search_cell = 'button[aria-label*="Search.cell.buttonTitle"]'
-            sb.cdp.scroll_into_view(search_cell)
+            sb.scroll_into_view(search_cell)
             sb.sleep(1)
-            sb.cdp.click(search_cell)
+            sb.click(search_cell)
             sb.sleep(5)
     else:
         print("*** Lowest Prices: ***")
         departure_prices = "#uiFlightPanel0 div.ui-bound__price__value"
         return_prices = "#uiFlightPanel1 div.ui-bound__price__value"
-        elements = sb.cdp.find_elements(departure_prices)
+        elements = sb.find_elements(departure_prices)
         for element in elements:
             if "lowest price" in element.text:
                 print("Departure Flight:")
                 print(element.text)
                 break
-        elements = sb.cdp.find_elements(return_prices)
+        elements = sb.find_elements(return_prices)
         for element in elements:
             if "lowest price" in element.text:
                 print("Return Flight:")
                 print(element.text)
                 break
-    dates = sb.cdp.find_elements('div[class*="flight-date"]')
+    dates = sb.find_elements('div[class*="flight-date"]')
     if len(dates) == 2:
         print("*** Departure Date: ***")
         print(dates[0].text)

examples/cdp_mode/raw_nordstrom.py

@@ -4,17 +4,17 @@
     url = "https://www.nordstrom.com/"
     sb.activate_cdp_mode(url)
     sb.sleep(2.2)
-    sb.cdp.click("input#keyword-search-input")
+    sb.click("input#keyword-search-input")
     sb.sleep(0.8)
     search = "cocktail dresses for women teal"
-    sb.cdp.press_keys("input#keyword-search-input", search + "\n")
+    sb.press_keys("input#keyword-search-input", search + "\n")
     sb.sleep(2.2)
     for i in range(17):
-        sb.cdp.scroll_down(16)
+        sb.scroll_down(16)
         sb.sleep(0.14)
     print('*** Nordstrom Search for "%s":' % search)
     unique_item_text = []
-    items = sb.cdp.find_elements("article")
+    items = sb.find_elements("article")
     for item in items:
         description = item.querySelector("article h3")
         if description and description.text not in unique_item_text: