From a81a315fb4d7b2a94d78bcfd42c95fc2aedd6f23 Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Sun, 4 Aug 2024 11:13:42 -0400 Subject: [PATCH 1/7] Add instructions about splitting header and runtime + run_export --- docs/maintainer/knowledge_base.md | 33 +++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/maintainer/knowledge_base.md b/docs/maintainer/knowledge_base.md index bf21ff503e..6ee3ba08e1 100644 --- a/docs/maintainer/knowledge_base.md +++ b/docs/maintainer/knowledge_base.md @@ -1695,6 +1695,8 @@ add some information on r packages which make heavy use of `noarch: generic` - [importlib_metadata and importlib-metadata](https://github.com/conda-forge/importlib_metadata-feedstock/blob/main/recipe/meta.yaml) - [typing_extensions and typing-extensions](https://github.com/conda-forge/typing_extensions-feedstock/blob/main/recipe/meta.yaml) + + ### Common pitfalls with `outputs` This is a non-exhaustive list of common pitfalls when using `outputs`. @@ -1704,6 +1706,37 @@ This is a non-exhaustive list of common pitfalls when using `outputs`. - The `outputs[].script` field can only be set to a script name. If you prefer passing shell commands, you have to use `outputs[].build.script`. Compare [geopandas-feedstock](https://github.com/conda-forge/geopandas-feedstock/blob/8b985635a8538af1ee213900bd563085e3cdbd92/recipe/meta.yaml#L17) to [gym-feedstock](https://github.com/conda-forge/gym-feedstock/blob/2b47e0479923b7d49a39e9860ba30a28e263480b/recipe/meta.yaml#L31), respectively. - Some `PIP_*` environment variables that are usually set for the top-level scripts are not automatically set for the outputs. If you are invoking `pip` in an output, you may need to pass additional flags. See [napari-feedstock](https://github.com/conda-forge/napari-feedstock/blob/32a4eb04ca7b6ccd2c4e146bde204f1dd5425a17/recipe/meta.yaml#L26). This issue is tracked in [conda/conda-build#3993](https://github.com/conda/conda-build/issues/3993). +- Pin as much as possible each output to the exact version of the subpackages. Pinning to the same minor version is often not enough since it may allow for different build numbers of packages built from the same recipe to be co-installed. + + ```yaml + requirements: + run: + - {{ pin_subpackage('subpackage_name', exact=True) }} + ``` + +- Often multi-output recipes are used when maintainers wish to split the runtime files from the build time files. The runtime package will often include shared library files and any runtime configuration files, while the development package will include compilation time headers, and other compilation files. The intention is to make it necessary to have both the development and runtime package installed at compilation time, but at runtime, only the runtime package would be installed. We wish that maintainers of downstream packages depend on the "developement" package at build time in the host section. Thus the development package should have a run_export on the runtime package. This is done by adding the following to the development package's meta.yaml: + + ```yaml + outputs: + - name: runtime_package_name + # no run_export should be added here. This package is missing the + # development headers and other compilation files. + - name: development_package_name + build: + run_exports: + # the max_pin should be adjusted + - {{ pin_subpackage('runtime_package_name', max_pin='x.x') }} + requirements: + run: + - runtime_package_name + - other + - required + - compilation + - dependencies + ``` + + Once the package is created, it the development package should be added to the list of global pinnings for conda forge in the [`conda_build_config.yaml`](https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml) file. + ## Build matrices From a0a280a5c671077ebd1b42a62c3665c2d3944e78 Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Fri, 9 Aug 2024 06:50:28 -0400 Subject: [PATCH 2/7] Update docs/maintainer/knowledge_base.md Co-authored-by: jaimergp --- docs/maintainer/knowledge_base.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/maintainer/knowledge_base.md b/docs/maintainer/knowledge_base.md index 6ee3ba08e1..e3637160a0 100644 --- a/docs/maintainer/knowledge_base.md +++ b/docs/maintainer/knowledge_base.md @@ -1728,7 +1728,7 @@ This is a non-exhaustive list of common pitfalls when using `outputs`. - {{ pin_subpackage('runtime_package_name', max_pin='x.x') }} requirements: run: - - runtime_package_name + - {{ pin_subpackage('runtime_package_name', exact=True) }} - other - required - compilation From b0e2bbb4e81af14b4f352a23e6e55e94e390ee4a Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Fri, 9 Aug 2024 18:14:09 -0400 Subject: [PATCH 3/7] Update docs/maintainer/knowledge_base.md Co-authored-by: Uwe L. Korn --- docs/maintainer/knowledge_base.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/maintainer/knowledge_base.md b/docs/maintainer/knowledge_base.md index e3637160a0..69d16815a3 100644 --- a/docs/maintainer/knowledge_base.md +++ b/docs/maintainer/knowledge_base.md @@ -1721,6 +1721,7 @@ This is a non-exhaustive list of common pitfalls when using `outputs`. - name: runtime_package_name # no run_export should be added here. This package is missing the # development headers and other compilation files. + # Typically this package is suffixed with `-devel` - name: development_package_name build: run_exports: From cc3048b3aaaef5c800ef09ea3ea138ba17091e94 Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Fri, 9 Aug 2024 18:14:14 -0400 Subject: [PATCH 4/7] Update docs/maintainer/knowledge_base.md Co-authored-by: Uwe L. Korn --- docs/maintainer/knowledge_base.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/maintainer/knowledge_base.md b/docs/maintainer/knowledge_base.md index 69d16815a3..1f54afbe96 100644 --- a/docs/maintainer/knowledge_base.md +++ b/docs/maintainer/knowledge_base.md @@ -1728,6 +1728,9 @@ This is a non-exhaustive list of common pitfalls when using `outputs`. # the max_pin should be adjusted - {{ pin_subpackage('runtime_package_name', max_pin='x.x') }} requirements: + # Add the runtime package as a host dependency to ensure that files are deduplicated between the packages. + host: + - {{ pin_subpackage('runtime_package_name', exact=True) }} run: - {{ pin_subpackage('runtime_package_name', exact=True) }} - other From 67bd9f73ffadaabcf6f54df4e54afce2b800e15f Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 15 Aug 2024 07:36:39 +0000 Subject: [PATCH 5/7] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/maintainer/knowledge_base.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/maintainer/knowledge_base.md b/docs/maintainer/knowledge_base.md index 1f54afbe96..10e339e9ad 100644 --- a/docs/maintainer/knowledge_base.md +++ b/docs/maintainer/knowledge_base.md @@ -1695,8 +1695,6 @@ add some information on r packages which make heavy use of `noarch: generic` - [importlib_metadata and importlib-metadata](https://github.com/conda-forge/importlib_metadata-feedstock/blob/main/recipe/meta.yaml) - [typing_extensions and typing-extensions](https://github.com/conda-forge/typing_extensions-feedstock/blob/main/recipe/meta.yaml) - - ### Common pitfalls with `outputs` This is a non-exhaustive list of common pitfalls when using `outputs`. From 35fe8cf609476b662c6b99849629a50633739db6 Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Thu, 15 Aug 2024 08:22:13 -0400 Subject: [PATCH 6/7] Update docs/maintainer/knowledge_base.md Co-authored-by: jaimergp --- docs/maintainer/knowledge_base.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/maintainer/knowledge_base.md b/docs/maintainer/knowledge_base.md index 10e339e9ad..18da93facf 100644 --- a/docs/maintainer/knowledge_base.md +++ b/docs/maintainer/knowledge_base.md @@ -1712,7 +1712,7 @@ This is a non-exhaustive list of common pitfalls when using `outputs`. - {{ pin_subpackage('subpackage_name', exact=True) }} ``` -- Often multi-output recipes are used when maintainers wish to split the runtime files from the build time files. The runtime package will often include shared library files and any runtime configuration files, while the development package will include compilation time headers, and other compilation files. The intention is to make it necessary to have both the development and runtime package installed at compilation time, but at runtime, only the runtime package would be installed. We wish that maintainers of downstream packages depend on the "developement" package at build time in the host section. Thus the development package should have a run_export on the runtime package. This is done by adding the following to the development package's meta.yaml: +- Often multi-output recipes are used when maintainers wish to split the runtime files from the build time files. The runtime package will often include shared library files and any runtime configuration files, while the development package will include compilation time headers, and other compilation files. The intention is to make it necessary to have both the development and runtime package installed at compilation time, but at runtime, only the runtime package would be installed. We wish that maintainers of downstream packages depend on the "development" package at build time in the host section. Thus the development package should have a `run_export` on the runtime package. This is done by adding the following to the development package's `meta.yaml`: ```yaml outputs: From c1eac53de7e39a3a2c3515e1365b67a12892c93d Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Thu, 15 Aug 2024 08:22:21 -0400 Subject: [PATCH 7/7] Update docs/maintainer/knowledge_base.md Co-authored-by: jaimergp --- docs/maintainer/knowledge_base.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/maintainer/knowledge_base.md b/docs/maintainer/knowledge_base.md index 18da93facf..c6d2eac2a8 100644 --- a/docs/maintainer/knowledge_base.md +++ b/docs/maintainer/knowledge_base.md @@ -1737,7 +1737,7 @@ This is a non-exhaustive list of common pitfalls when using `outputs`. - dependencies ``` - Once the package is created, it the development package should be added to the list of global pinnings for conda forge in the [`conda_build_config.yaml`](https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml) file. +Once the package is created, the development package should be added to the list of pinnings for conda-forge in the [global `conda_build_config.yaml`](https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml) file.