Merge remote-tracking branch 'origin/master' into crawler-tests

Author: Pijukatel

.github/workflows/pre_release.yaml

@@ -20,13 +20,13 @@ jobs:
       version_number: ${{ steps.release_metadata.outputs.version_number }}
       tag_name: ${{ steps.release_metadata.outputs.tag_name }}
       changelog: ${{ steps.release_metadata.outputs.changelog }}
-      existing_changelog_path: CHANGELOG.md
     steps:
       - uses: apify/workflows/git-cliff-release@main
         id: release_metadata
         name: Prepare release metadata
         with:
           release_type: prerelease
+          existing_changelog_path: CHANGELOG.md
 
   lint_check:
     name: Lint check

.github/workflows/run_code_checks.yaml

@@ -6,6 +6,16 @@ on:
   # step required for PRs from forks. This prevents their potential exposure.
   pull_request:
 
+  # Pushing to the master branch triggers code checks
+  push:
+    branches:
+      - master
+    tags-ignore:
+      - "**" # Ignore all tags to prevent duplicate checks when tags are pushed.
+
+  # It should also be possible to trigger checks manually
+  workflow_dispatch:
+
 jobs:
   lint_check:
     name: Lint check

CHANGELOG.md

@@ -150,19 +150,19 @@ All notable changes to this project will be documented in this file.
 - [**breaking**] Drop support for Python 3.8
 
 
-## [1.7.2](../../releases/tag/v1.7.2) - 2024-07-08
+## [1.7.2](https://github.com/apify/apify-sdk-python/releases/tag/v1.7.2) (2024-07-08)
 
 - Add Actor Standby port
 
 
-## [1.7.1](../../releases/tag/v1.7.1) - 2024-05-23
+## [1.7.1](https://github.com/apify/apify-sdk-python/releases/tag/v1.7.1) (2024-05-23)
 
 ### 🐛 Bug Fixes
 
 - Set a timeout for Actor cleanup
 
 
-## [1.7.0](../../releases/tag/v1.7.0) - 2024-03-12
+## [1.7.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.7.0) (2024-03-12)
 
 ### 🚀 Features
 
@@ -174,7 +174,7 @@ All notable changes to this project will be documented in this file.
 - Scrapy's `Request.dont_filter` works.
 
 
-## [1.6.0](../../releases/tag/v1.6.0) - 2024-02-23
+## [1.6.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.6.0) (2024-02-23)
 
 ### 🐛 Bug Fixes
 
@@ -185,28 +185,28 @@ All notable changes to this project will be documented in this file.
 - Remove `ApifyRetryMiddleware` and stay with the Scrapy's default one
 
 
-## [1.5.5](../../releases/tag/v1.5.5) - 2024-02-01
+## [1.5.5](https://github.com/apify/apify-sdk-python/releases/tag/v1.5.5) (2024-02-01)
 
 ### 🐛 Bug Fixes
 
 - Fix conversion of `headers` fields in Apify <--> Scrapy request translation
 
 
-## [1.5.4](../../releases/tag/v1.5.4) - 2024-01-24
+## [1.5.4](https://github.com/apify/apify-sdk-python/releases/tag/v1.5.4) (2024-01-24)
 
 ### 🐛 Bug Fixes
 
 - Fix conversion of `userData` and `headers` fields in Apify <--> Scrapy request translation
 
 
-## [1.5.3](../../releases/tag/v1.5.3) - 2024-01-23
+## [1.5.3](https://github.com/apify/apify-sdk-python/releases/tag/v1.5.3) (2024-01-23)
 
 ### 🚀 Features
 
 - Add `apply_apify_settings` function to Scrapy subpackage
 
 
-## [1.5.2](../../releases/tag/v1.5.2) - 2024-01-19
+## [1.5.2](https://github.com/apify/apify-sdk-python/releases/tag/v1.5.2) (2024-01-19)
 
 ### 🐛 Bug Fixes
 
@@ -220,22 +220,22 @@ All notable changes to this project will be documented in this file.
 - Replace asserts with custom checks in Scrapy subpackage
 
 
-## [1.5.1](../../releases/tag/v1.5.1) - 2024-01-10
+## [1.5.1](https://github.com/apify/apify-sdk-python/releases/tag/v1.5.1) (2024-01-10)
 
 ### Chore
 
 - Allowed running integration tests from PRs from forks, after maintainer approval
 - Do not close `nested_event_loop` in the `Scheduler.__del__`
 
 
-## [1.5.0](../../releases/tag/v1.5.0) - 2024-01-03
+## [1.5.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.5.0) (2024-01-03)
 
 ### 🚀 Features
 
 - Add `ApifyHttpProxyMiddleware`
 
 
-## [1.4.1](../../releases/tag/v1.4.1) - 2023-12-21
+## [1.4.1](https://github.com/apify/apify-sdk-python/releases/tag/v1.4.1) (2023-12-21)
 
 ### 🐛 Bug Fixes
 
@@ -246,21 +246,21 @@ All notable changes to this project will be documented in this file.
 - Fix type hint problems for resource clients
 
 
-## [1.4.0](../../releases/tag/v1.4.0) - 2023-12-05
+## [1.4.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.4.0) (2023-12-05)
 
 ### Chore
 
 - Migrate from Autopep8 and Flake8 to Ruff
 
 
-## [1.3.0](../../releases/tag/v1.3.0) - 2023-11-15
+## [1.3.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.3.0) (2023-11-15)
 
 ### 🚀 Features
 
 - Add `scrapy` extra
 
 
-## [1.2.0](../../releases/tag/v1.2.0) - 2023-10-23
+## [1.2.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.2.0) (2023-10-23)
 
 ### 🚀 Features
 
@@ -271,7 +271,7 @@ All notable changes to this project will be documented in this file.
 - Fix lint error (E721) in unit tests (for instance checks use `isinstance()`)
 
 
-## [1.1.5](../../releases/tag/v1.1.5) - 2023-10-03
+## [1.1.5](https://github.com/apify/apify-sdk-python/releases/tag/v1.1.5) (2023-10-03)
 
 ### 🚀 Features
 
@@ -283,7 +283,7 @@ All notable changes to this project will be documented in this file.
 - Remove PR Toolkit workflow
 
 
-## [1.1.4](../../releases/tag/v1.1.4) - 2023-09-06
+## [1.1.4](https://github.com/apify/apify-sdk-python/releases/tag/v1.1.4) (2023-09-06)
 
 ### 🐛 Bug Fixes
 
@@ -296,15 +296,15 @@ All notable changes to this project will be documented in this file.
 - Unpin `apify-client` and `apify-shared` to improve compatibility with their newer versions
 
 
-## [1.1.3](../../releases/tag/v1.1.3) - 2023-08-25
+## [1.1.3](https://github.com/apify/apify-sdk-python/releases/tag/v1.1.3) (2023-08-25)
 
 ### Chore
 
 - Unify indentation in configuration files
 - Update the `Actor.reboot` method to use the new reboot endpoint
 
 
-## [1.1.2](../../releases/tag/v1.1.2) - 2023-08-02
+## [1.1.2](https://github.com/apify/apify-sdk-python/releases/tag/v1.1.2) (2023-08-02)
 
 ### Chore
 
@@ -314,14 +314,14 @@ All notable changes to this project will be documented in this file.
 - Pin `apify-client` and `apify-shared` to prevent their implicit updates from breaking SDK
 
 
-## [1.1.1](../../releases/tag/v1.1.1) - 2023-05-23
+## [1.1.1](https://github.com/apify/apify-sdk-python/releases/tag/v1.1.1) (2023-05-23)
 
 ### 🐛 Bug Fixes
 
 - Relax dependency requirements to improve compatibility with other libraries
 
 
-## [1.1.0](../../releases/tag/v1.1.0) - 2023-05-23
+## [1.1.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.1.0) (2023-05-23)
 
 ### 🚀 Features
 
@@ -339,7 +339,7 @@ All notable changes to this project will be documented in this file.
 - Switch from `setup.py` to `pyproject.toml` for specifying project setup
 
 
-## [1.0.0](../../releases/tag/v1.0.0) - 2023-03-13
+## [1.0.0](https://github.com/apify/apify-sdk-python/releases/tag/v1.0.0) (2023-03-13)
 
 ### 🐛 Bug Fixes
 

pyproject.toml

@@ -34,8 +34,8 @@ keywords = [
     "scraping",
 ]
 dependencies = [
-    "apify-client>=1.11.0",
-    "apify-shared>=1.3.0",
+    "apify-client<2.0.0",
+    "apify-shared<2.0.0",
     "crawlee[parsel]~=0.6.0",
     "cryptography>=42.0.0",
     "httpx>=0.27.0",
@@ -64,7 +64,7 @@ scrapy = ["scrapy>=2.11.0"]
 dev = [
     "build~=1.2.0",
     "dycw-pytest-only>=2.1.1",
-    "griffe~=1.8.0",
+    "griffe~=1.9.0",
     "mypy~=1.17.0",
     "pre-commit~=4.2.0",
     "pydoc-markdown~=4.8.0",

tests/integration/README.md

@@ -126,3 +126,26 @@ async def test_something(
 
     assert actor_run.status == 'SUCCEEDED'
 ```
+
+### Asserts
+
+Since test Actors are not executed as standard pytest tests, we don't get introspection of assertion expressions. In case of failure, only a bare `AssertionError` is shown, without the left and right values. This means, we must include explicit assertion messages to aid potential debugging.
+
+```python
+async def test_add_and_fetch_requests(
+    make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
+) -> None:
+    """Test basic functionality of adding and fetching requests."""
+
+    async def main() -> None:
+        async with Actor:
+            rq = await Actor.open_request_queue()
+            await rq.add_request(f'https://apify.com/')
+            assert is_finished is False, f'is_finished={is_finished}'
+
+    actor = await make_actor(label='rq-test', main_func=main)
+    run_result = await run_actor(actor)
+
+    assert run_result.status == 'SUCCEEDED'
+```

tests/integration/test_actor_api_helpers.py

@@ -400,12 +400,12 @@ async def main_server() -> None:
         async with Actor:
 
             class WebhookHandler(BaseHTTPRequestHandler):
-                def do_GET(self) -> None:
+                def do_GET(self) -> None:  # noqa: N802
                     self.send_response(200)
                     self.end_headers()
                     self.wfile.write(bytes('Hello, world!', encoding='utf-8'))
 
-                def do_POST(self) -> None:
+                def do_POST(self) -> None:  # noqa: N802
                     nonlocal webhook_body
                     content_length = self.headers.get('content-length')
                     length = int(content_length) if content_length else 0