WIP: Test lowest versions of all required and optional dependencies #3639
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,18 +33,18 @@ dependencies = [ | |
| "numpy>=1.24", | ||
| "pandas>=2.0", | ||
| "xarray>=2023.04", | ||
| "netCDF4>=1.7", | ||
|
There was a problem hiding this comment. We discussed making NetCDF an optional dependency before in #429. Maybe we could revisit that discussion? There was a problem hiding this comment. That's a good point. NetCDF4 was needed by xarray when reading and writing netCDF files. As mentioned in #239 (comment), previously, our After we refactor our I've opened PR #3643 to see how the PyGMT relies on the netCDF package. Edit: There are only 14 failures in the Python 3.11 + Ubuntu CI job after removing netCDF entirely (see https://github.com/GenericMappingTools/pygmt/actions/runs/11968416461/job/33367210857?pr=3643) and most of them are caused by There was a problem hiding this comment. There is another error that popped up on Ubuntu-22.04-ARM64 at https://github.com/GenericMappingTools/pygmt/actions/runs/14457975280/job/40545076044?pr=3639#step:7:1079, related to ==================================== ERRORS ====================================
__________________ ERROR at setup of test_clib_read_data_grid __________________
self = CachingFileManager(<class 'netCDF4._netCDF4.Dataset'>, '/home/runner/.gmt/cache/static_earth_relief.nc', mode='r', kwa...r': True, 'diskless': False, 'persist': False, 'format': 'NETCDF4'}, manager_id='26bc1ee9-cb26-4f6f-bcea-ac669a2faa62')
needs_lock = True
def _acquire_with_cache_info(self, needs_lock=True):
"""Acquire a file, returning the file and whether it was cached."""
with self._optional_lock(needs_lock):
try:
> file = self._cache[self._key]
../.venv/lib/python3.11/site-packages/xarray/backends/file_manager.py:210:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <xarray.backends.lru_cache.LRUCache object at 0xff802d7a2b80>
key = [<class 'netCDF4._netCDF4.Dataset'>, ('/home/runner/.gmt/cache/static_earth_relief.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), '26bc1ee9-cb26-4f6f-bcea-ac669a2faa62']
def __getitem__(self, key: K) -> V:
# record recent use of the key by moving it to the front of the list
with self._lock:
> value = self._cache[key]
E KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('/home/runner/.gmt/cache/static_earth_relief.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), '26bc1ee9-cb26-4f6f-bcea-ac669a2faa62']
../.venv/lib/python3.11/site-packages/xarray/backends/lru_cache.py:56: KeyError
During handling of the above exception, another exception occurred:
@pytest.fixture(scope="module", name="expected_xrgrid")
def fixture_expected_xrgrid():
"""
The expected xr.DataArray object for the static_earth_relief.nc file.
"""
> return load_dataarray(which("@static_earth_relief.nc"))
../pygmt/tests/test_clib_read_data.py:30:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../pygmt/io.py:44: in load_dataarray
with xr.open_dataarray(filename_or_obj, **kwargs) as dataarray:
../.venv/lib/python3.11/site-packages/xarray/backends/api.py:686: in open_dataarray
dataset = open_dataset(
../.venv/lib/python3.11/site-packages/xarray/backends/api.py:525: in open_dataset
backend_ds = backend.open_dataset(
../.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:588: in open_dataset
store = NetCDF4DataStore.open(
../.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:389: in open
return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
../.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:336: in __init__
self.format = self.ds.data_model
../.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:398: in ds
return self._acquire()
../.venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py:392: in _acquire
with self._manager.acquire_context(needs_lock) as root:
../../../../.local/share/uv/python/cpython-3.11.12-linux-aarch64-gnu/lib/python3.11/contextlib.py:137: in __enter__
return next(self.gen)
../.venv/lib/python3.11/site-packages/xarray/backends/file_manager.py:198: in acquire_context
file, cached = self._acquire_with_cache_info(needs_lock)
../.venv/lib/python3.11/site-packages/xarray/backends/file_manager.py:216: in _acquire_with_cache_info
file = self._opener(*self._args, **kwargs)
src/netCDF4/_netCDF4.pyx:2489: in netCDF4._netCDF4.Dataset.__init__
???
src/netCDF4/_netCDF4.pyx:2092: in netCDF4._netCDF4._get_vars
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E RuntimeError: only endian='native' allowed for NETCDF3 files, got 'big' (variable 'lon', group '/')
src/netCDF4/_netCDF4.pyx:4062: RuntimeErrorThis seems to be fixed in Unidata/netcdf4-python#1355, released in netcdf4=1.7.2. So we can either set the minimum version to Edit: netCDF4 is no longer a required dependency of PyGMT since #3643, so no need to worry about this anymore. |
||
| "packaging>=22.0", | ||
| ] | ||
| dynamic = ["version"] | ||
|
|
||
| [project.optional-dependencies] | ||
| all = [ | ||
| "contextily>=1.2", | ||
|
There was a problem hiding this comment. Need to pin to pygmt/pygmt/datasets/tile_map.py Lines 12 to 19 in 97185e8 Xref geopandas/contextily#183, where There was a problem hiding this comment. Good to know that we require contexily>=1.2 |
||
| "geopandas>=0.14", | ||
| "IPython>=8", # 'ipython' is not the correct module name. | ||
| "pyarrow>=14", | ||
| "rioxarray>=0.14", | ||
|
There was a problem hiding this comment. Getting this test failure on windows-2022 at https://github.com/GenericMappingTools/pygmt/actions/runs/15524973929/job/43703740279?pr=3639#step:7:1095: _______________ [doctest] pygmt.datasets.tile_map.load_tile_map _______________
118 >>> raster.sizes
119 Frozen({'band': 3, 'y': 256, 'x': 512})
120 >>> raster.coords # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
121 Coordinates:
122 * band (band) uint8... 1 2 3
123 * y (y) float64... -7.081e-10 -7.858e+04 ... -1.996e+07 -2.004e+07
124 * x (x) float64... -2.004e+07 -1.996e+07 ... 1.996e+07 2.004e+07
125 spatial_ref int... 0
126 >>> # CRS is set only if rioxarray is available
127 >>> if hasattr(raster, "rio"):
Expected:
'EPSG:3857'
Got:
'PROJCS["WGS 84 / Pseudo-Mercator",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Mercator_1SP"],PARAMETER["central_meridian",0],PARAMETER["scale_factor",1],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1],AXIS["Easting",EAST],AXIS["Northing",NORTH],EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0 +k=1 +units=m +nadgrids=@null +wktext +no_defs"],AUTHORITY["EPSG","3857"]]'
D:\a\pygmt\pygmt\pygmt\datasets\tile_map.py:127: DocTestFailureSimilar situation to the one reported at #3575. We could perhaps pin to a rioxarray version that requires Suggestions on what to do? There was a problem hiding this comment. The actual error messages are Some related discussions are in corteva/rioxarray#447. I'm unsure if pin rioxarray>=0.19.0 can solve the issue, but please give it a try first. There was a problem hiding this comment. Hmm you're right, got the same There was a problem hiding this comment. The dependency tree GMT->GDAL->PROJ installs PROJ 9.4.0, while uv/pypi installs pyproj 3.7.1 which contains PROJ 9.5.1. Perhaps we can install pyproj via micromamba instead? There was a problem hiding this comment.
Tried at commit 4540a5d, same error 😢 There was a problem hiding this comment. What happens if setting env xref: https://pyproj4.github.io/pyproj/dev/installation.html#setup-pyproj There was a problem hiding this comment.
|
||
| ] | ||
|
|
||
| [project.urls] | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using
--resolution lowest-directfor now, because--resolution lowestgrabs many other transitive dependencies that may be hard to install (e.g. missing wheels for newer Python versions, so requires compilation from source). We could switch to--resolution lowestonce the Python ecosystem does lower bound pins a bit more thoroughly (which may take years).