"Package would be ignored" warnings for data subdirectories inside package

[mgorny]

We're working on modernizing setuptools usage in translate-toolkit. As part of that, @nijel moved data files into translate/share directory and started using packages = find:.

The layout for the package resembles:

translate
├── [...]
├── __init__.py
├── share
│   ├── langmodels
│   │   ├── afrikaans.lm
│   │   ├── [...]
│   └── stoplist-en
├── [...]

Notably, translate and its "normal" subpackages all include __init__.py but share and share/langmodels don't.

When building via python -m build, we get the following warnings:

/tmp/build-env-s7o1om1t/lib/python3.11/site-packages/setuptools/command/build_py.py:153: SetuptoolsDeprecationWarning:     !!


    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'translate.share' as an importable package, however it is
    included in the distribution as "data".
    This behavior is likely to change in future versions of setuptools (and
    therefore is considered deprecated).

    Please make sure that 'translate.share' is included as a package by using
    setuptools' `packages` configuration field or the proper discovery methods.

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.


!!

  check.warn(importable)
/tmp/build-env-s7o1om1t/lib/python3.11/site-packages/setuptools/command/build_py.py:153: SetuptoolsDeprecationWarning:     !!


    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'translate.share.langmodels' as an importable package, however it is
    included in the distribution as "data".
    This behavior is likely to change in future versions of setuptools (and
    therefore is considered deprecated).

    Please make sure that 'translate.share.langmodels' is included as a package by using
    setuptools' `packages` configuration field or the proper discovery methods.

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.


!!

  check.warn(importable)

To be honest, I've dug through setuptools documentation and I've only been able to found two solutions that make the warning disappear:

• either adding __init__.py to data directories which seems suboptimal
• or using automatic discovery which is documented as experimental

What is the recommended way to install these files?

[abravalheri]

Hi @mgorny, would it be possible for you to use packages = find_namespace:?

[utkonos]

@abravalheri Thanks for this. Sorry for not understanding it the first few tries.

Should the packaging documentation be adjusted from packages = find: to packages = find_namespace: in tutorial here:
https://packaging.python.org/en/latest/tutorials/packaging-projects/#configuring-metadata

[abravalheri]

As a personal opinion, I don't see why not using packages = find_namespace: if the project uses the src-layout. However there are mixed feelings in the community about PEP 420 and people have different opinions about tooling supporting implicit namespaces by default.

I don't know if the maintainers of pypa/packaging.python.org prefer to leave packages = find: as it can be safer for people that are partially following the tutorial but are trying to avoid the src-layout.

[abravalheri]

In the latest version, I added a note to the warning advising about replacing find: with find_namespaces:

setuptools/setuptools/command/build_py.py

Lines 261 to 275 in 499c468

	############################
	# Package would be ignored #
	############################
	Python recognizes {importable!r} as an importable package, however it is
	included in the distribution as "data".
	This behavior is likely to change in future versions of setuptools (and
	therefore is considered deprecated).
	Please make sure that {importable!r} is included as a package by using
	setuptools' `packages` configuration field or the proper discovery methods
	(for example by using `find_namespace_packages(...)`/`find_namespace:`
	instead of `find_packages(...)`/`find:`).
	You can read more about "package discovery" and "data files" on setuptools
	documentation page.

[abravalheri]

We are working on documentation updates for setuptools in #3335. Later this week I should propose something different for the namespaces.

[vkottler]

This seems kind of insane to me. We are explicitly specifying data files to setuptools and it is complaining right now.

Why should some kind of "magic" like an opaque package discoverer matter at all. setuptools is being told exactly what to do.

The deprecation warning is also confusing because it states that these data directories are importable packages, but they don't have __init__.py files so why would that be the case?

We want to be able to tell setuptools what the package's data files are so it can manifest them. That's currently how it works. Why is this changing?