-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
What happened?
When using the h5netcdf backend for NetCDF files, it seems that unlimited variables are stored in ds.encoding["unlimited_dims"]
. If you then change the dimension name (in my case I called ds.swap_dims
, but I assume any method of renaming or deleting the dimension will reproduce this), and call ds.to_netcdf
with engine='h5netcdf'
, an error is produced.
What did you expect to happen?
The file should have saved successfully.
Minimal Complete Verifiable Example
import h5netcdf
import xarray as xr
with h5netcdf.File('minimal.nc', 'w') as f:
f.dimensions['test'] = None
f.create_variable('test_var', ('test',), float)
ds = xr.open_dataset('minimal.nc', engine='h5netcdf')
ds = ds.swap_dims({'test': 'something_else'})
ds.to_netcdf('output.nc', engine='h5netcdf')
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
- Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Relevant log output
$ python repro.py
Traceback (most recent call last):
File "h5py/utils.pyx", line 105, in h5py.utils.convert_tuple
TypeError: an integer is required
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ".../repro.py", line 17, in <module>
ds.to_netcdf('output.nc', engine='h5netcdf')
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../python/lib/python3.13/site-packages/xarray/core/dataset.py", li
ne 2029, in to_netcdf
return to_netcdf( # type: ignore[return-value] # mypy cannot resolve the overloads:(
self,
...<10 lines>...
auto_complex=auto_complex,
)
File ".../python/lib/python3.13/site-packages/xarray/backends/api.py", li
ne 1984, in to_netcdf
dump_to_store(
~~~~~~~~~~~~~^
dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File ".../python/lib/python3.13/site-packages/xarray/backends/api.py", li
ne 2031, in dump_to_store
store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../python/lib/python3.13/site-packages/xarray/backends/common.py",
line 473, in store
self.set_dimensions(variables, unlimited_dims=unlimited_dims)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../python/lib/python3.13/site-packages/xarray/backends/common.py",
line 550, in set_dimensions
self.set_dimension(dim, length, is_unlimited)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../python/lib/python3.13/site-packages/xarray/backends/h5netcdf_.p
self.ds.resize_dimension(name, length)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File ".../python/lib/python3.13/site-packages/h5netcdf/core.py", line 139
self._dimensions[dim]._resize(size)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File ".../python/lib/python3.13/site-packages/h5netcdf/dimensions.py", li
self._h5ds.resize((size,))
~~~~~~~~~~~~~~~~~^^^^^^^^^
File ".../python/lib/python3.13/site-packages/h5py/_hl/dataset.py", line
self.id.set_extent(size)
~~~~~~~~~~~~~~~~~~^^^^^^
File "h5py/_objects.pyx", line 56, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 57, in h5py._objects.with_phil.wrapper
File "h5py/h5d.pyx", line 359, in h5py.h5d.DatasetID.set_extent
File "h5py/utils.pyx", line 107, in h5py.utils.convert_tuple
TypeError: Can't convert element 0 (None) to hsize_t
Anything else we need to know?
I can get this to happen by manually removing or renaming the entry in the ds.encoding["unlimited_dims"]
set.
Environment
INSTALLED VERSIONS
commit: None
python: 3.13.5 (main, Jun 11 2025, 15:36:57) [Clang 17.0.0 (clang-1700.0.13.3)]
python-bits: 64
OS: Darwin
OS-release: 24.5.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: (None, 'UTF-8')
libhdf5: 1.14.6
libnetcdf: None
xarray: 2025.7.1
pandas: 2.3.1
numpy: 2.3.1
scipy: None
netCDF4: None
pydap: None
h5netcdf: 1.6.3
h5py: 3.14.0
zarr: None
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: None
pip: 25.1.1
conda: None
pytest: None
mypy: None
IPython: None
sphinx: None