Dataset adapter backend check #21905
Conversation
Summary of ChangesHello @Shekar-77, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly improves the robustness of the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This PR introduces backend compatibility checks for data adapters, which is a valuable addition. My review focuses on a critical placement issue of one of the checks, code duplication, and improving the clarity of error messages to align with the repository's style guide. Addressing these points will make the implementation more robust and maintainable.
| backend_name = backend.backend() | ||
|
|
||
| if backend_name not in ("tensorflow", "numpy", "torch", "jax"): | ||
| raise ValueError( | ||
| f"Incompatible backend '{backend_name}'" | ||
| "Supported backends TensorFlow , numpy , torch , jax backend." | ||
| ) |
There was a problem hiding this comment.
This backend compatibility check is placed in the PyDataset class body, which means it will be executed at import time. This is a critical issue because it can prevent the module from being imported if an unsupported backend is configured, even if PyDataset is not used with an adapter. This check should be removed from here and moved into the __init__ method of the PyDatasetAdapter class, which is the component with backend-specific dependencies.
Additionally, the error message is poorly formatted and inconsistent. A better error message would be:
f"Unsupported backend: '{backend.backend()}'. PyDatasetAdapter only supports the following backends: ['tensorflow', 'numpy', 'torch', 'jax']."
This validation logic is also duplicated in other adapters and should ideally be centralized in a utility function.
References
- Error messages should be contextual, informative, and actionable, clearly stating what happened, what was expected, and how to fix it. The current error message is poorly formatted and lacks clarity. (link)
| backend_name = backend.backend() | ||
| if backend_name not in ("tensorflow", "numpy", "torch", "jax"): | ||
| raise ValueError( | ||
| f"Incompatible backend '{backend_name}'" | ||
| "Supported backends TensorFlow , numpy , torch , jax backend." | ||
| ) |
There was a problem hiding this comment.
The error message can be improved for better readability and to be more informative, as per the style guide (lines 140-143). The current message has formatting issues (missing space, inconsistent casing) and could be clearer.
Additionally, this validation logic is duplicated across py_dataset_adapter.py and torch_data_loader_adapter.py. It would be better to extract this into a shared utility function to avoid code duplication and ensure consistency.
backend_name = backend.backend()
supported_backends = ("tensorflow", "numpy", "torch", "jax")
if backend_name not in supported_backends:
raise ValueError(
f"Unsupported backend: '{backend_name}'. `TFDatasetAdapter` "
f"only supports the following backends: {list(supported_backends)}."
)References
- Error messages should be contextual, informative, and actionable, clearly stating what happened, what was expected, and how to fix it. The current error message is poorly formatted and lacks clarity. (link)
| backend_name = backend.backend() | ||
|
|
||
| if backend_name not in ("tensorflow", "numpy", "torch", "jax"): | ||
| raise ValueError( | ||
| f"Incompatible backend '{backend_name}'" | ||
| "Supported backends TensorFlow , numpy , torch , jax backend." | ||
| ) |
There was a problem hiding this comment.
The error message can be improved for better readability and to be more informative, as per the style guide (lines 140-143). The current message has formatting issues (missing space, inconsistent casing) and could be clearer.
Additionally, this validation logic is duplicated across py_dataset_adapter.py and tf_dataset_adapter.py. It would be better to extract this into a shared utility function to avoid code duplication and ensure consistency.
| backend_name = backend.backend() | |
| if backend_name not in ("tensorflow", "numpy", "torch", "jax"): | |
| raise ValueError( | |
| f"Incompatible backend '{backend_name}'" | |
| "Supported backends TensorFlow , numpy , torch , jax backend." | |
| ) | |
| backend_name = backend.backend() | |
| supported_backends = ("tensorflow", "numpy", "torch", "jax") | |
| if backend_name not in supported_backends: | |
| raise ValueError( | |
| f"Unsupported backend: '{backend_name}'. `TorchDataLoaderAdapter` " | |
| f"only supports the following backends: {list(supported_backends)}." | |
| ) |
References
- Error messages should be contextual, informative, and actionable, clearly stating what happened, what was expected, and how to fix it. The current error message is poorly formatted and lacks clarity. (link)
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #21905 +/- ##
==========================================
- Coverage 76.30% 76.29% -0.01%
==========================================
Files 580 580
Lines 60029 60037 +8
Branches 9432 9434 +2
==========================================
+ Hits 45803 45807 +4
- Misses 11750 11752 +2
- Partials 2476 2478 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
3 participants
This PR improves backend compatibility checks for DatasetAdapter.
It raises clear errors when a unsupported backend (other than numpy, torch, tf, jax) is used.
Changes
Added backend-type validation to DatasetAdapter.init in:
-tf_dataset_adapter.py
-py_dataset_adapter.py
-torch_data_loader_adapter.py