[Core] Fixed Pickle file load error

[oiadebayo]

User description

Description

What - Handles a deserialization error when loading an object from a pickle file during resync operations

Why -

How - I updated the deserialization logic to handle possible exceptions rather than interrupt resync

Type of change

Please leave one option from the following and delete the rest:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• New Integration (non-breaking change which adds a new integration)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Non-breaking change (fix of existing functionality that will not change current behavior)
• Documentation (added/updated documentation)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

• Integration able to create all default resources from scratch
• Resync finishes successfully
• Resync able to create entities
• Resync able to update entities
• Resync able to detect and delete entities
• Scheduled resync able to abort existing resync and start a new one
• Tested with at least 2 integrations from scratch
• Tested with Kafka and Polling event listeners
• Tested deletion of entities that don't pass the selector

Integration testing checklist

• Integration able to create all default resources from scratch
• Resync able to create entities
• Resync able to update entities
• Resync able to detect and delete entities
• Resync finishes successfully
• If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
• If resource kind is updated, run the integration with the example data and check if the expected result is achieved
• If new resource kind is added or updated, validate that live-events for that resource are working as expected
• Docs PR link here

Preflight checklist

• Handled rate limiting
• Handled pagination
• Implemented the code in async
• Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.

---

PR Type

Bug fix

---

Description

• Fixed pickle file deserialization errors in multiprocess mode

• Added comprehensive error handling for corrupted pickle files

• Enhanced IPC load method with fallback to default values

• Added extensive test coverage for error scenarios

---

Changes diagram

flowchart LR
  A["FileIPC.load()"] --> B["File exists?"]
  B -->|No| C["Return default"]
  B -->|Yes| D["Try pickle.load()"]
  D -->|Success| E["Return data"]
  D -->|Exception| F["Log warning"]
  F --> G["Return default"]

Loading

---

Changes walkthrough 📝

	Relevant files
Bug fix	ipc.py Enhanced pickle deserialization with error handling            port_ocean/utils/ipc.py Added comprehensive exception handling for pickle deserialization Wrapped pickle.load() in try-catch block for multiple error types Added logging for failed load operations with fallback to default Imported loguru.logger for error logging +17/-2
Tests	test_ipc.py Added comprehensive FileIPC error handling tests                  port_ocean/tests/utils/test_ipc.py Added comprehensive test suite for FileIPC error handling Tests for corrupted, truncated, and missing pickle files Tests for TypeError, AttributeError, and ImportError scenarios Validates proper fallback to default values and logging +146/-0
Documentation	CHANGELOG.md Updated changelog for version 0.25.4                                          CHANGELOG.md Added version 0.25.4 entry Documented pickle error fix for multiprocess mode +5/-1
Configuration changes	pyproject.toml Version bump to 0.25.4                                                                      pyproject.toml Bumped version from 0.25.3 to 0.25.4 +1/-1

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Exception Handling The exception handling catches a broad range of exceptions including TypeError, AttributeError, and ImportError. While this prevents crashes, it might mask legitimate programming errors that should be addressed rather than silently falling back to defaults. except ( pickle.PickleError, EOFError, OSError, TypeError, AttributeError, ImportError, ) as e: Test Coverage The tests mock pickle.load to simulate errors but don't test actual pickle file corruption scenarios. Consider adding tests with real corrupted pickle files to ensure the error handling works in practice. # Mock pickle.load to raise TypeError (simulating constructor signature mismatch) with patch( "pickle.load", side_effect=TypeError( "KindNotImplementedException.__init__() missing 1 required positional argument: 'available_kinds'" ), ):

[qodo-merge-pro]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Separate file and deserialization errors The exception handling is too broad and may mask legitimate file I/O errors that should be propagated. Consider separating file access errors from pickle deserialization errors to handle them appropriately. port_ocean/utils/ipc.py [27-41] try: with open(self.file_path, "rb") as f: - return pickle.load(f) -except ( - pickle.PickleError, - EOFError, - OSError, - TypeError, - AttributeError, - ImportError, -) as e: - logger.warning( - f"Failed to load IPC data from {self.file_path}: {str(e)}. Returning default value." - ) - return self.default_return + try: + return pickle.load(f) + except ( + pickle.PickleError, + EOFError, + TypeError, + AttributeError, + ImportError, + ) as e: + logger.warning( + f"Failed to load IPC data from {self.file_path}: {str(e)}. Returning default value." + ) + return self.default_return +except OSError as e: + logger.error(f"Failed to open IPC file {self.file_path}: {str(e)}") + raise Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies that catching OSError with other deserialization errors can hide file system problems, and proposes a more robust, nested error handling approach.	Medium
Update

[github-actions]

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/16412632315/artifacts/3594599322

Code Coverage Total Percentage: 83.51%

[Copilot]

Pull Request Overview

This PR fixes pickle file deserialization errors that occur during resync operations in multiprocess mode by adding comprehensive error handling to the FileIPC class.

• Added robust exception handling for various pickle loading failures
• Enhanced error logging with fallback to default values when deserialization fails
• Added comprehensive test coverage for all error scenarios

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
port_ocean/utils/ipc.py	Enhanced pickle deserialization with comprehensive error handling and logging
port_ocean/tests/utils/test_ipc.py	Added extensive test suite covering all error scenarios for FileIPC
pyproject.toml	Version bump from 0.27.0 to 0.27.1
CHANGELOG.md	Updated changelog to document the pickle error fix

[Copilot]

The version number in pyproject.toml (0.27.1) is inconsistent with the version mentioned in the PR description and CHANGELOG.md (0.25.4). This version mismatch could cause confusion and deployment issues.

Suggested change

	version = "0.27.1"
	version = "0.25.4"

[Copilot]

The version number in CHANGELOG.md (0.27.1) is inconsistent with the version mentioned in the PR description (0.25.4). This version mismatch could cause confusion and should be aligned with the actual version being released.

Suggested change

	## 0.27.1 (2025-08-06)
	## 0.25.4 (2025-08-06)

[github-actions]

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/16777035921/artifacts/3700704135

Code Coverage Total Percentage: 84.53%

[matan84]

requested change

[matan84]

Let's raise a more indicative error and not return the self.default_return here