Skip to content

Conversation

@dkrako
Copy link
Member

@dkrako dkrako commented Mar 26, 2025

Description

This PR adds DatasetDefinition.has_resources as a getter property that is inferred dynamically on each call.

The has_resources property functions both as bool:

>>> definition = pm.DatasetLibrary.get('ToyDataset')
>>> definition.has_resources
True

and as an indexable object:

>>> definition.has_resources['gaze']
True

This is achieved by a helper class _HasResourcesIndexer that is forwarded by the getter property.

The contribution has two shortcomings that should be improved on in follow-up PRs:

  • Resources have to be reset on each call of has_resources. This could be resolved by implementing a DatasetResources class that would be the type of DatasetDefintion.resources. DatasetDefintion.has_resources would then simply call DatasetDefintion.resources.has_resources.
  • It doesn't fully replace DatasetDefinition.has_files, as its usage in dataset_files.py has different semantics: it does not refer to downloadable resources, but to local resources (which could have been downloaded before). Moving resource-dependent attributes to each resource would make a full replacement possible.

Both solutions to the shortcomings are addressed in #1120.

Implemented changes

  • implement _HasResourcesIndexer to make has_resources indexable
  • implement has_resources() property and forward _HasResourceIndexer
  • change has_files to has_resources in public_dataset_processing_test.py

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change is or requires a documentation update

How Has This Been Tested?

  • tested various cases for boolean context
  • tested various cases for indexable context

Context

related issues:

requires:

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

@github-actions github-actions bot added the enhancement New feature or request label Mar 26, 2025
@codecov
Copy link

codecov bot commented Mar 26, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (d8a26c4) to head (01c90a9).
⚠️ Report is 54 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main     #1064   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           90        91    +1     
  Lines         3986      4028   +42     
  Branches       708       712    +4     
=========================================
+ Hits          3986      4028   +42     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

# Conflicts:
#	src/pymovements/dataset/dataset_definition.py
@dkrako dkrako changed the title feat: add dynamic has_resources field to DatasetDefinition feat: add dynamic has_resources property to DatasetDefinition Mar 26, 2025
@dkrako dkrako mentioned this pull request Apr 12, 2025
14 tasks
@dkrako dkrako marked this pull request as ready for review April 26, 2025 16:16
Copy link
Contributor

@SiQube SiQube left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple comments/questions.

@dkrako dkrako enabled auto-merge (squash) April 28, 2025 12:17
pre-commit-ci bot and others added 2 commits April 28, 2025 12:20
# Conflicts:
#	src/pymovements/dataset/_utils/_resources.py
@dkrako dkrako requested a review from SiQube April 28, 2025 12:27
@SiQube
Copy link
Contributor

SiQube commented May 20, 2025

ah I was confused about | not changed but I guess 3.9 ****

@dkrako dkrako merged commit 37f0bcd into main May 20, 2025
26 checks passed
@dkrako dkrako deleted the feat/definition-has-resources branch May 20, 2025 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants