Skip to content

Issue with swap_dims and netCDF unlimited dimensions #10549

@rilwal

Description

@rilwal

What happened?

When using the h5netcdf backend for NetCDF files, it seems that unlimited variables are stored in ds.encoding["unlimited_dims"]. If you then change the dimension name (in my case I called ds.swap_dims, but I assume any method of renaming or deleting the dimension will reproduce this), and call ds.to_netcdf with engine='h5netcdf', an error is produced.

What did you expect to happen?

The file should have saved successfully.

Minimal Complete Verifiable Example

import h5netcdf
import xarray as xr

with h5netcdf.File('minimal.nc', 'w') as f:
    f.dimensions['test'] = None
    f.create_variable('test_var', ('test',), float)

ds = xr.open_dataset('minimal.nc', engine='h5netcdf')

ds = ds.swap_dims({'test': 'something_else'})
ds.to_netcdf('output.nc', engine='h5netcdf')

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

$ python repro.py
Traceback (most recent call last):
  File "h5py/utils.pyx", line 105, in h5py.utils.convert_tuple
TypeError: an integer is required

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../repro.py", line 17, in <module>
    ds.to_netcdf('output.nc', engine='h5netcdf')
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../python/lib/python3.13/site-packages/xarray/core/dataset.py", li
ne 2029, in to_netcdf
    return to_netcdf(  # type: ignore[return-value]  # mypy cannot resolve the overloads:(
        self,
    ...<10 lines>...
        auto_complex=auto_complex,
    )
  File ".../python/lib/python3.13/site-packages/xarray/backends/api.py", li
ne 1984, in to_netcdf
    dump_to_store(
    ~~~~~~~~~~~~~^
        dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File ".../python/lib/python3.13/site-packages/xarray/backends/api.py", li
ne 2031, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../python/lib/python3.13/site-packages/xarray/backends/common.py",
 line 473, in store
    self.set_dimensions(variables, unlimited_dims=unlimited_dims)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../python/lib/python3.13/site-packages/xarray/backends/common.py",
 line 550, in set_dimensions
    self.set_dimension(dim, length, is_unlimited)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../python/lib/python3.13/site-packages/xarray/backends/h5netcdf_.p
    self.ds.resize_dimension(name, length)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File ".../python/lib/python3.13/site-packages/h5netcdf/core.py", line 139
    self._dimensions[dim]._resize(size)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File ".../python/lib/python3.13/site-packages/h5netcdf/dimensions.py", li
    self._h5ds.resize((size,))
    ~~~~~~~~~~~~~~~~~^^^^^^^^^
  File ".../python/lib/python3.13/site-packages/h5py/_hl/dataset.py", line 
    self.id.set_extent(size)
    ~~~~~~~~~~~~~~~~~~^^^^^^
  File "h5py/_objects.pyx", line 56, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 57, in h5py._objects.with_phil.wrapper
  File "h5py/h5d.pyx", line 359, in h5py.h5d.DatasetID.set_extent
  File "h5py/utils.pyx", line 107, in h5py.utils.convert_tuple
TypeError: Can't convert element 0 (None) to hsize_t

Anything else we need to know?

I can get this to happen by manually removing or renaming the entry in the ds.encoding["unlimited_dims"] set.

Environment

INSTALLED VERSIONS

commit: None
python: 3.13.5 (main, Jun 11 2025, 15:36:57) [Clang 17.0.0 (clang-1700.0.13.3)]
python-bits: 64
OS: Darwin
OS-release: 24.5.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: (None, 'UTF-8')
libhdf5: 1.14.6
libnetcdf: None

xarray: 2025.7.1
pandas: 2.3.1
numpy: 2.3.1
scipy: None
netCDF4: None
pydap: None
h5netcdf: 1.6.3
h5py: 3.14.0
zarr: None
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: None
pip: 25.1.1
conda: None
pytest: None
mypy: None
IPython: None
sphinx: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugneeds triageIssue that has not been reviewed by xarray team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions