From 532e91bcfef5c20d05bd440dcda2aaa510b0c645 Mon Sep 17 00:00:00 2001 From: Leah Wasser Date: Thu, 15 Dec 2022 10:32:07 -0700 Subject: [PATCH 1/5] Fix: update TOC --- index.md | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-) diff --git a/index.md b/index.md index 5c531ed5a..50b980bde 100644 --- a/index.md +++ b/index.md @@ -123,26 +123,12 @@ Contributing & license files :caption: Package structure & code style Intro -package-structure-code/code-structure-style +Code Style & Format package-structure-code/release package-structure-code/overview package-structure-code/collaboration ``` -```{toctree} -:hidden: -:caption: Code Style & Structure - -code-style-structure/index -``` - -```{toctree} -:hidden: -:caption: Test your code -testing-infrastructure/test-code -testing-infrastructure/continuous-integration -``` - From 76bdfad316e62e43dc2590ea3baa1f392976e900 Mon Sep 17 00:00:00 2001 From: Leah Wasser Date: Thu, 15 Dec 2022 10:32:22 -0700 Subject: [PATCH 2/5] Add: package code style section --- code-style-structure/intro.md | 4 - .../code-structure-style.md | 250 ++++++++++++++++++ package-structure-code/collaboration.md | 55 ++++ package-structure-code/intro.md | 44 +++ package-structure-code/release.md | 73 +++++ package-structure-code/testing-ci.md | 83 ++++++ 6 files changed, 505 insertions(+), 4 deletions(-) delete mode 100644 code-style-structure/intro.md create mode 100644 package-structure-code/code-structure-style.md create mode 100644 package-structure-code/collaboration.md create mode 100644 package-structure-code/intro.md create mode 100644 package-structure-code/release.md create mode 100644 package-structure-code/testing-ci.md diff --git a/code-style-structure/intro.md b/code-style-structure/intro.md deleted file mode 100644 index 7f540b302..000000000 --- a/code-style-structure/intro.md +++ /dev/null @@ -1,4 +0,0 @@ -# Code style and structure - - -Under development \ No newline at end of file diff --git a/package-structure-code/code-structure-style.md b/package-structure-code/code-structure-style.md new file mode 100644 index 000000000..555bcef1d --- /dev/null +++ b/package-structure-code/code-structure-style.md @@ -0,0 +1,250 @@ +# Python Package Code Style & Linting + +Consistent code format and style is useful to both your package +and across the scientific python ecosystem because using similar +formats makes code easier to read. + +For instance, if you saw a sentence like this one without any +spaces it would take your brain longer to process it. + +``` +forinstanceifyousawasentencelikethisonewithoutany... +``` + +pyOpenSci requires you to follow standard Python PEP 8 format +rules. But we also suggest that you use a code formatter too. +If more packages start using formatter, we will slowly gain consistency +across the scientific ecosystem. Code across packages will then +be consistent and as such easier to scan, understand and contribute to. + +### pyOpenSci requirements for code style and format + +pyOpenSci doesn't require you to use a specific code format tool. However we do look for +consistency and readbility in code style. pyOpenSci also requires that you +follow the [python PEP8 style guide](https://peps.python.org/pep-0008/) for decisions +related to styling your open source Python packages code. We also require that your code adheres +to the PEP8 style guidelines. + +We do strongly recommend that you implement a code styler tool suite in your +package as it will save you and your maintainer team time. It will also ensure +that format and style is consistent accross all of your code. + + +Below you will find a list of commonly used Python code formatters. +Currently, [black](https://github.com/psf/black) is the most popular code styler for Python. + +## Code format and style +We suggest that you use a code formatter such as [black](https://black.readthedocs.io/en/stable/) or +[blue](https://blue.readthedocs.io/en/latest/) to ensure your +code is consistently formatted throughout your package. Using an existing, +widely used tool will avoid lengthy discussions with contributors and other +maintainers about personalized code format preferences during reviews. + +## Code linters and formatters +A code linter is a tool that will review your code and identify errors or +issues. A linter typically does not actually modify the code for you. It will +just tell you what the error is and on what line it was discovered. + +### flake8 is a preferred code linting tool for Python packages + +To adhere to Python `pep8` format standards, we suggest that you use +[flake8](https://flake8.pycqa.org/en/latest/). + +Below you can see the output of running +`flake8 filename.py` at the command line for a python file within a package +called `stravalib`. The line length standard for PEP8 is 79 characters. flake8 +will: +* flags every line in your code that extends beyond 79 characters +* spacing issues that conflict with pep8 such as missing spaces after commas + +It also flags unused imports in your modules. + +Notice below that you run flak8 in the command line and it returns a list of +each error it finds in a file. All of this output is at the command line. The +python file is not modified. + +```bash +(stravalib-dev) username@computer stravalib % flake8 stravalib/model.py +stravalib/model.py:8:1: F401 'os' imported but unused +stravalib/model.py:29:80: E501 line too long (90 > 79 characters) +stravalib/model.py:34:80: E501 line too long (95 > 79 characters) +stravalib/model.py:442:80: E501 line too long (82 > 79 characters) +stravalib/model.py:443:39: E231 missing whitespace after ',' +stravalib/model.py:493:20: E225 missing whitespace around operator +stravalib/model.py:496:80: E501 line too long (82 > 79 characters) +``` + +### Black and Blue +While `flake8` is a linter that identifies issues with your code, [black](https://black.readthedocs.io/en/stable/) and also +[blue](https://blue.readthedocs.io/en/latest/) (which wraps around black) are code +formatters. Both black and blue will manually (and unapologetically) +fix spacing issues and ensure format is consistent throughout your package. + +These formatters leave you more time to work on code function rather than +format. Black and blue do +generally adhere to pep8 style guidelines with some exceptions. + +* Black defaults to a line width of 88 (79 + 10%) rather than the 79 character `pep8` specification (but this setting can be manually overwritten in your black setup) +* Black and blue will not adjust line length in your comments or docstrings. +* Neither tool will review and fix import order (you need isort to do that). + +Blue adresses a few format decisions in black that some maintainers do not like. +[You can compare the differences here](https://blue.readthedocs.io/en/latest/#so-what-s-different) and decide which tool suites your preferences! + +```{tip} +If you are interested in seeing how black will format your code, you can +use the [black playground](https://black.vercel.app/?version=stable&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4ARsAnNdAD2IimZxl1N_WlkPinBFoXIfdFTaTVkGVeHShArYj9yPlDvwBA7LhGo8BvRQqDilPtgsfdKl-ha7EFp0Ma6lY_06IceKiVsJ3BpoICJM9wU1VJLD7l3qd5xTmo78LqThf9uibGWcWCD16LBOn0JK8rhhx_Gf2ClySDJtvm7zQJ1Z-Ipmv9D7I_zhjztfi2UTVsJp7917XToHBm2EoNZqyE8homtGskFIiif5EZthHQvvOj8S2gJx8_t_UpWp1ScpIsD_Xq83LX-B956I_EBIeNoGwZZPFC5zAIoMeiaC1jU-sdOHVucLJM_x-jkzMvK8Utdfvp9MMvKyTfb_BZoe0-FAc2ZVlXEpwYgJVAGdCXv3lQT4bpTXyBwDrDVrUeJDivSSwOvT8tlnuMrXoD1Sk2NZB5SHyNmZsfyAEqLALbUnhkX8hbt5U2yNQRDf1LQhuUIOii6k6H9wnDNRnBiQHUfzKfW1CLiThnuVFjlCxQhJ60u67n3EK38XxHkQdOocJXpBNO51E4-f9z2hj0EDTu_ScuqOiC9cI8qJ4grSZIOnnQLv9WPvmCzx5zib3JacesIxMVvZNQiljq_gL7udm1yeXQjENOrBWbfBEkv1P4izWeAysoJgZUhtZFwKFdoCGt2TXe3xQ-wVZFS5KoMPhGFDZGPKzpK15caQOnWobOHLKaL8eFA-qI44qZrMQ7sSLn04bYeenNR2Vxz7hvK0lJhkgKrpVfUnZrtF-e-ubeeUCThWus4jZbKlFBe2Kroz90Elij_UZBMFCcFo0CfIx5mGlrINrTJLhERszRMMDd39XsBDzpZIYV4TcG7HoMS_IF8aMAAAxI-5uTWXbUQAAY8F7QgAAP01Vc6xxGf7AgAAAAAEWVo=) +``` + + +### Isort + +Python imports refer to the python packages that a module in your package require to run. Following conventional standards, +all Python imports should be called at the top of your code (.py)files. + +In addition to these imports all being at the top of each code +file (or each module) [PEP 8 has specific standards for the order of these imports](https://peps.python.org/pep-0008/#imports). These standards are listed below: + + +> Imports should be grouped in the following order: +> +> * Standard library imports. +> * Related third party imports. +> * Local application/library specific imports. + +While flake8 will identify unused imports in your code, it won't +fix or identify issues with the order of those imports. + +`isort` will identify where imports in your code are out of +order. It will then modify your module, automatically reordering +all imports. This leaves you with one less +thing to think about when cleaning up your code. Further, if +setup as a precommit hook, isort and a continuous integration +check, using isort will ensure that new contributors, who may +be less familiar with pep 8 standards, follow +the pep 8 import order standards as well. + +### Example application of isort + +Code imports before `isort` is run: + +Below, the exc module is a part of starvalib which is a +third party package. `abc` and `logging` are core `Python` packages +distributed with `Python`. Also notice that there are extra +spaces in the imports listed below. + +```python +from stravalib import exc +import abc +import logging + +from collections.abc import Sequence + + +from stravalib import unithelper as uh +``` +Run: +`isort stravalib/model.py` + +Python module after `isort` has been run + +```python +import abc +import logging +from collections.abc import Sequence + +from stravalib import exc +from stravalib import unithelper as uh +``` + + +## How to setup a code formatter to support your + +### Linters, code formatters and your favorite coding tools +Linters can be run as a command-line tool as shown above. They also can +be run within your favorite coding tool (e.g. VScode, pycharm, etc). Tools like +`flake8` and `black` can often be integrated with your favorite code editors to +run automatically when you save a file. + +### Linters, code formatters and pre-commit hooks +Some maintainers setup a `pre-commit hook` in their repository. + +This section will make the most sense if you are familiar with git. +You can skip it if you are not yet familiar with git commit! + +A pre-commit hook cna be best described as a trigger that happens making a +tool run when you type and run +`git commit -m "message here` at the command line. + +For instance, if you setup the black code formatter as a pre-commit hook, +when you go to commit changes to a code file and hit enter, black will run +on your the code files in your commit. It will update any files to match +black format standards. You can then retype the commit and push files to +GitHub that have been formatted by black. + +Using pre-commit hooks is helpful to ensure code consistency for edits to your +code files. + +```{note} +If you already have a Python package and you want to apply a code formatter, you +may need to go through the initial effort of formatting all of your code files. + +Running a tool like black is normally quick. However implementing fixes found by +flake8 can take a bit of time. Make sure that you allocate a bit of time to +format your code. Also make sure that other maintainers are not actively +updating your code base as an initial commit with black format changes will +likely change a significant amount of your code. This is a recipe for merge +conflicts! +``` + +## Setting up a `git` pre-commit hook + +Git pre-commit hook is a useful tool that checks your code automatically when you run a `git commit` and, +if it fails, the `git commit` is canceled. This is often used to make sure +that the changes to your code match a particular style, or that there are no +code linting errors. + +For example, if you want that `git commit` checks if your code matches the PEP8 specification, +you can configure a git flake8 pre-commit hook: + +```yaml +# file: .pre-commit-config.yaml +repos: + - repo: https://gitlab.com/pycqa/flake8 + rev: '3.7.9' + hooks: + - id: flake8 +``` + +```{note} +See [the flake8 hooks explanation](https://flake8.pycqa.org/en/latest/user/using-hooks.html) for more details. +``` + +This file specifies a hook that will be triggered automatically before each `git commit`, +in this case, it specifies a `flake8` using version `3.7.9`. + +Before you can see any change, first you should install `pre-commit` library. +One way to do it is using `pip` or `conda`: + +```sh +pip install pre-commit + +# or + +conda install -c conda-forge pre-commit +``` + +Now, you can install your pre-commit hook using `pre-commit install`, and it will create the hook based on +the file `.pre-commit-config.yaml`. + +Before each `git commit` command, in this case, git will run `flake8` and, if it fails, the `commit` will be canceled. + +## Summary +pyOpenSci suggests setting up a linter and a code styler for +your package, regardless of whether you use pre-commit hooks, CI +or other infrastructure to manage code format. Setting up these +tools will give you automatic feedback about your code's +structure as you (or a contributor) write it. And using tools +like black or blue that format code for you, reduce effort that +you need to make surrounding decisions around code format and +style. \ No newline at end of file diff --git a/package-structure-code/collaboration.md b/package-structure-code/collaboration.md new file mode 100644 index 000000000..c2cee89fe --- /dev/null +++ b/package-structure-code/collaboration.md @@ -0,0 +1,55 @@ +# Collaboration Guide + +🚧 UNDER CONSTRUCTION - THIS CONTENT SHOULD BE FURTHER DEVELOPED BY THE END OF 2022! KEEP CHECKING BACK TO UPDATES AS THEY ARE IN PROGRESS🚧 + + +Note: pyOpenSci is still building out this section of our guidebook. Many of the examples come from rOpenSci. If you have suggestions for changes, check out the [GitHub repo](https://github.com/pyOpenSci/contributing-guide). + +## Make your repo contribution and collaboration friendly + +### Code of conduct + +We require that you use a code of conduct such as the [Contributor Covenant](https://contributor-covenant.org/) in developing your package. You should document your code of conduct in a `CODE_OF_CONDUCT` file in the package root directory, and link to this file from the `README` file. + +### Contributing guide + +October 2022 NOTE: we are in the process of updating all of this and will likely deprecate +this cookie cutter. Proceed at your own risk :) + +We have a [template for contributing guidelines](https://github.com/pyOpenSci/cookiecutter-pyopensci/blob/master/%7B%7Bcookiecutter.project_slug%7D%7D/CONTRIBUTING.rst) in our cookiecutter repository. Even if you're not using cookiecutter to build your project (see our [packaging guide](../authoring/overview#project-template) for info), you can download our CONTRIBUTING.rst as a starting point. + +You can tweak it a bit depending on your workflow and package. For example, make sure contributors have instructions in your CONTRIBUTING file for running local tests if not trivial. CONTRIBUTING can also contain some details about how you acknowledge contributions (see [this section](#attributions)) and the roadmap of your package (cf [this example for R](https://github.com/ecohealthalliance/fasterize/blob/master/CONTRIBUTING.md)). + +### Issue labelling + +You can use labels such as "help wanted", "good first issue", or "beginner" to help potential collaborators, including newbies, find your repo. For more info, see this [GitHub article](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/encouraging-helpful-contributions-to-your-project-with-labels). + +## Working with collaborators + +### Onboarding collaborators + +There's no general pyOpenSci rule as to how you should onboard collaborators. You should increase their rights to the repo as you gain trust, and you should definitely acknowledge contributions (see [this section](#attributions)). + +You can ask a new collaborator to make PRs (see following section for assessing a PR locally, i.e. beyond CI checks) to dev/main and assess them before merging, and after a while let them push to main, although you might want to keep a system of PR reviews... even for yourself once you have team mates! + +A possible model for onboarding collaborators is provided by Jim Hester in [his `lintr` repo](https://github.com/jimhester/lintr/issues/318). + +If your problem is _recruiting_ collaborators, you can post an open call like Jim Hester's [on Twitter](https://twitter.com/jimhester_/status/997109466674819074), ([GitHub](https://github.com/jimhester/lintr/issues/318)). As an pyOpenSci package author, you can also ask for help from pyOpenSci. + +### Working with collaborators (including yourself) + +You could implement the "[gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)" philosophy as explained by Amanda Dobbyn in [this rOpenSci blog post](https://ropensci.org/blog/2018/04/20/monkeydo/). + +One particular aspect of working with collaborators is reviewing pull requests. Even if not adopting gitflow it might make sense for repo collaborators to make PRs and have them reviewed, and in general PRs by external developers will need to be assessed. Sometimes you'll be fine just reading the changes and trusting [Continuous integration](../authoring/overview#continuous-integration). Sometimes you'll need to dive deeper, e.g. by downloading a copy of the branch that the PR was created from. + +(attributions)= +### Be generous with attributions + +If someone contributes to your repository consider adding them in CONTRIBUTORS, as contributor for small contributions, author for bigger contributions. Also consider adding their name near the feature/bug fix line in HISTORY We recommend your being generous with such acknowledgements. + +If your package was reviewed by pyOpenSci and you feel that your reviewers have made a substantial contribution to the development of your package, you may list them in CONTRIBUTORS as a reviewer. Only include reviewers after asking for their consent. + +```{note} +Please do not list pyOpenSci editors and reviewers as contributors to your project. +Your participation in and contribution to pyOpenSci is thanks enough! +``` diff --git a/package-structure-code/intro.md b/package-structure-code/intro.md new file mode 100644 index 000000000..4d76e48be --- /dev/null +++ b/package-structure-code/intro.md @@ -0,0 +1,44 @@ +# Python package structure information + +```{tip} +### Python packaging resources that we love + +We think the resources below are excellent but each have particular opinions +that you may or may not find in our packaging guide. For instance, the PyPA +guide encourages users to store their package in a `src/package-name` directory. +While we accept that approach many of our community members prefer to not use +the `src` directory. + +* [Python packaging for research software engineers](https://merely-useful.tech/py-rse/) +* [PyPA packaging guide](https://packaging.python.org/en/latest/) +``` + +If you plan to submit a package for review to pyOpenSci and are looking for +some guidance on package structure, code formats and style, then this section is for you. + + +```{note} +If you are considering submitting a package to pyOpenSci, or if you are just +getting started with building a package, you may want to have a look at the +bare-minimum [editor checks.](open-source-software-submissions/editor-in-chief-guide.html#editor-checklist-copy-template-below-to-use-in-the-issue) that pyOpenSci +performs before a review even begins. + +In general these are basic items that should be in any open software repository. +``` + +## Packaging Guide + + +PyPI also has a [short tutorial](https://packaging.python.org/tutorials/packaging-projects/) +on how to package a Python project for easy installation. + + + + + diff --git a/package-structure-code/release.md b/package-structure-code/release.md new file mode 100644 index 000000000..b13ec9b06 --- /dev/null +++ b/package-structure-code/release.md @@ -0,0 +1,73 @@ +# Releasing a package + +🚧 UNDER CONSTRUCTION - THIS CONTENT SHOULD BE FURTHER DEVELOPED BY THE END OF 2022! KEEP CHECKING BACK TO UPDATES AS THEY ARE IN PROGRESS🚧 + + +This section covers releasing your package to PyPI as well as releasing future versions. Your package should have different versions over time: snapshots of a state of the package that you can release to PyPI for instance. These versions should be properly _numbered_, _released_ and _described in a NEWS file_. More details below. + + +## Original Release + +After the review process has finished, you're ready to release your package to PyPI so others can find and use your awesome package! Releasing to PyPI is easy. Once your package is ready, register it on PyPI by running: + +``` +python setup.py register +``` + +## Releasing Updated Versions + +When you update your package, you release a new version to PyPI. Fortunately, this is easy! First, we'll talk about the metadata you'll need to update for each version. Then we'll cover how to release your updated version to PyPI [manually via the command line](manual-release) or [automatically via Travis CI](travis-release). + +### Version Naming + +* We recommend that pyOpenSci packages use [semantic versioning](https://www.python.org/dev/peps/pep-0440/#semantic-versioning). In addition to the [PEP section on it](https://www.python.org/dev/peps/pep-0440/#semantic-versioning), [the semver website has more detail](https://semver.org/). + +* Versioning can be done using [bumpversion](https://github.com/peritus/bumpversion), e.g. for a minor update: + +``` +bumpversion minor +``` + +"minor" can be replaced with "major" or "patch" depending on the level of update. + +(history)= +### History/News/Changelog file + +A HISTORY (or NEWS or CHANGELOG) file describing changes associated with each version makes it easier for users to see what's changing in the package and how it might impact their workflow. You must add one for your package, and make it easy to read. + +* It is mandatory to use a `HISTORY` file in the root of your package. It can also be called NEWS or CHANGELOG We recommend using `[NAME].md` or `[NAME].rst` to make the file more browsing-friendly on GitHub (GitHub renders certain file types, including Markdown and reStructured text). + +* Update the history file before every PyPI release, with a section with the package name, version and date of release. + +``` +foobar 0.2.0 (2016-04-01) +========================= +``` + +* Under that header, put in sections as needed, including: `NEW FEATURES`, `MINOR IMPROVEMENTS`, `BUG FIXES`, `DEPRECATED AND DEFUNCT`, `DOCUMENTATION FIXES` and any special heading grouping a large number of changes. Under each header, list items as needed. For each item give a description of the new feature, improvement, bug fix, or deprecated function/feature. Link to any related GitHub issue like `(#12)`. The `(#12)` will resolve on GitHub in Releases to a link to that issue in the repo. + +* After you have added a `git tag` and pushed up to GitHub, add the news items for that tagged version to the Release notes of a release in your GitHub repo with a title like `pkgname v0.1.0`. See [GitHub docs about creating a release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository). + +(manual-release)= +### Releasing Versions: Manual + +To manually upload a new package version to PyPI, follow these steps: + +1. Update your HISTORY file as described [above](#history) +2. Open `setup.py` and change the version, e.g., version='1.0.3' +3. If you added new, non-Python files to the project that need to be distributed as well, e.g., configuration files, add them to `MANIFEST.in`. This does not need to be done for Python code files ending in .py. +4. Open a terminal and go into the parent of the project's root dir +5. `python setup.py sdist` +6. Check the resulting files, especially the egg file: are all the files contained? +7. If everything is ok, upload the new version to PyPI: `python setup.py sdist upload` + +That's it! + +(travis-release)= +### Releasing via Travis CI +Instead of manually uploading new package versions, Travis can be configured to automatically upload new versions. If you use this, each time you tag a new release and push it to GitHub, Travis will release it to PyPI (assuming it passes testing). + +* The pyOpenSci cookiecutter comes with this option mostly set up. For details on how to finish the set up, see [this guide](https://cookiecutter-pyopensci.readthedocs.io/en/latest/travis_pypi_setup.html). + +* Also, be sure to check out this [PyPI release checklist](https://cookiecutter-pyopensci.readthedocs.io/en/latest/pypi_release_checklist.html). + diff --git a/package-structure-code/testing-ci.md b/package-structure-code/testing-ci.md new file mode 100644 index 000000000..b323b82cd --- /dev/null +++ b/package-structure-code/testing-ci.md @@ -0,0 +1,83 @@ +# Testing and CI?? + + + + + + +### Testing +- All packages should have a test suite that covers major functionality of the package. The tests should also cover the behavior of the package in case of errors. +- It is good practice to write unit tests for all functions, and all package code in general, ensuring key functionality is covered. Test coverage below 75% will likely require additional tests or explanation before being sent for review. +- We recommend using pytest for writing tests, but you can use other tools. Strive to write tests as you write each new function. This serves the obvious need to have proper testing for the package, but allows you to think about various ways in which a function can fail, and to defensively code against those. +- Consider using tox to test your package with multiple versions of Python 2 and 3. +- If you set up CI with code coverage, use your package's code coverage report to identify untested lines, and to add further tests. + +**Good/Better/Best:** +- **Good:** A test suite that covers major functionality of the package. +- **Better:** The above, with high code coverage. +- **Best:** All of the above, plus using tox to test multiple versions of Python. + +### Continuous Integration +All pyOpenSci packages must use some form of continuous integration. + +- For Linux and Mac OSX, we suggest GitHub Actions, Circle CI or Travis CI. +- For Windows, we suggest GitHub Actions or AppVeyor CI. +- In many cases, you will want CI for all platforms. Different continuous integration services will support builds on different operating systems. Packages should have CI for all platforms when they contain: + - Compiled code + - Java dependencies + - Dependencies on other languages + - Packages with system calls + - Text munging such as getting people’s names (in order to find encoding issues). + - Anything with file system / path calls + - In case of any doubt regarding the applicability of these criteria to your package, it’s better to add CI for all platforms, and most often not too much hassle. + +**Good/Better/Best:** +- **Good:** Some sort of CI service with status badge in your README. +- **Better:** The above plus integrated code coverage and linting. +- **Best:** Continuous integration for all platforms: Linux, Mac OSX, and Windows. + + + + + +This section provides guidelines and tips for creating a Python package to submit for peer-review. + +pyOpenSci packages must: +- Have a clear README _including_ installation instructions. +- Contain full documentation for any user-facing functions. +- Have a test suite that covers the major functionality of the package. +- Use continuous integration. +- Use an OSI approved software license. + + + + + +### License +pyOpenSci projects should use an open source software license that is approved by the Open Software Initiative (OSI). OSI's website has a [list of popular licenses](https://opensource.org/licenses), and GitHub has a [handy tool](https://choosealicense.com/) for choosing a license. + +**Good/Better/Best:** +- **Good:** Include a open source software license with your package. +- **Better/Best:** Choose a license based on your needs and future use of package, plus explain your choice in your submission for review. + +## Other recommendations +### Python version support +You should always be explicit about which versions of Python your package supports. +Keeping compatibility with old Python versions can be difficult as functionality changes. +A good rule of thumb is that the package should support, at least, +the latest three Python versions (e.g., 3.8, 3.7, 3.6). + +### Code Style +pyOpenSci encourages authors to consult [PEP 8](https://www.python.org/dev/peps/pep-0008/) for information on how to style your code. + +### Linting +An automatic linter (e.g. flake8) can help ensure your code is clean and free of syntax errors. These can be integrated with your CI. + +### Badges + +Badges are a useful way to draw attention to the quality of your project and to +assure users that it is well-designed, tested, and maintained. +It is common to provide a collection of badges in a table for others +to quickly browse. + +[See this example of a badge table](https://github.com/ropensci/drake). Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.) From d226f0ec76504591969192cce1387ac3ed5e0925 Mon Sep 17 00:00:00 2001 From: Leah Wasser Date: Tue, 27 Dec 2022 13:19:19 -0700 Subject: [PATCH 3/5] more updates --- index.md | 3 +- .../{release.md => publish-python-package.md} | 16 +- .../python-dependency-versions.md | 8 + .../python-package-structure.md | 213 ++++++++++++++++++ .../python-package-versions.md | 152 +++++++++++++ 5 files changed, 378 insertions(+), 14 deletions(-) rename package-structure-code/{release.md => publish-python-package.md} (77%) create mode 100644 package-structure-code/python-dependency-versions.md create mode 100644 package-structure-code/python-package-structure.md create mode 100644 package-structure-code/python-package-versions.md diff --git a/index.md b/index.md index 50b980bde..636e07236 100644 --- a/index.md +++ b/index.md @@ -124,7 +124,8 @@ Contributing & license files Intro Code Style & Format -package-structure-code/release +Python package structure +Package Versions package-structure-code/overview package-structure-code/collaboration ``` diff --git a/package-structure-code/release.md b/package-structure-code/publish-python-package.md similarity index 77% rename from package-structure-code/release.md rename to package-structure-code/publish-python-package.md index b13ec9b06..637d2940d 100644 --- a/package-structure-code/release.md +++ b/package-structure-code/publish-python-package.md @@ -1,9 +1,9 @@ -# Releasing a package +# Publish Your Python Package in a Community Channel -🚧 UNDER CONSTRUCTION - THIS CONTENT SHOULD BE FURTHER DEVELOPED BY THE END OF 2022! KEEP CHECKING BACK TO UPDATES AS THEY ARE IN PROGRESS🚧 -This section covers releasing your package to PyPI as well as releasing future versions. Your package should have different versions over time: snapshots of a state of the package that you can release to PyPI for instance. These versions should be properly _numbered_, _released_ and _described in a NEWS file_. More details below. + +## maybe also have a page in creating an installable package ## Original Release @@ -18,17 +18,7 @@ python setup.py register When you update your package, you release a new version to PyPI. Fortunately, this is easy! First, we'll talk about the metadata you'll need to update for each version. Then we'll cover how to release your updated version to PyPI [manually via the command line](manual-release) or [automatically via Travis CI](travis-release). -### Version Naming - -* We recommend that pyOpenSci packages use [semantic versioning](https://www.python.org/dev/peps/pep-0440/#semantic-versioning). In addition to the [PEP section on it](https://www.python.org/dev/peps/pep-0440/#semantic-versioning), [the semver website has more detail](https://semver.org/). - -* Versioning can be done using [bumpversion](https://github.com/peritus/bumpversion), e.g. for a minor update: - -``` -bumpversion minor -``` -"minor" can be replaced with "major" or "patch" depending on the level of update. (history)= ### History/News/Changelog file diff --git a/package-structure-code/python-dependency-versions.md b/package-structure-code/python-dependency-versions.md new file mode 100644 index 000000000..2c5fcc472 --- /dev/null +++ b/package-structure-code/python-dependency-versions.md @@ -0,0 +1,8 @@ +# Dep versions + +https://scientific-python.org/specs/spec-0000/ + +https://endoflife.date/python +https://numpy.org/neps/nep-0029-deprecation_policy.html + +36 month drop python version diff --git a/package-structure-code/python-package-structure.md b/package-structure-code/python-package-structure.md new file mode 100644 index 000000000..9141bb975 --- /dev/null +++ b/package-structure-code/python-package-structure.md @@ -0,0 +1,213 @@ +# Python Package Structure + +```{important} +Things to add: +* using pyproject toml and peps that support it +* moving way from setup py +* setup.cfg +``` +## Package publication + +pyOpenSci requires that your package can be installed from a public community +repository such as `PyPI` or a `conda` channel such as `bioconda` or `conda-forge`. + +## What should the structure of your package look like? + +Below you can see the recommended structure of a (scientific) Python package. + +``` +myPackage/ +β”œβ”€β”€ CHANGELOG.md ┐ +β”œβ”€β”€ CODE_OF_CONDUCT.md β”‚ +β”œβ”€β”€ CONTRIBUTING.md β”‚ +β”œβ”€β”€ docs/ β”‚ Package documentation +β”‚ └── ... β”‚ +β”œβ”€β”€ LICENSE β”‚ +β”œβ”€β”€ README.md β”˜ +β”œβ”€β”€ pyproject.toml ┐ +| myPackage/ β”‚ Package source code, metadata, +β”‚ β”œβ”€β”€ __init__.py β”‚ and build instructions +β”‚ β”œβ”€β”€ moduleA.py β”‚ +β”‚ └── moduleB.py β”˜ + tests/ ┐ + └── test-file1.py | + └── .... β”˜ Package tests +``` +pyOpenSci suggests this package structure as it aligns with the structures of +core scientific Python ecosystem including: + +* [numpy](https://github.com/numpy/numpy) +* [scipy](https://github.com/scipy/scipy) +* [pandas](https://github.com/pandas-dev/pandas) +* [xarray](https://github.com/pydata/xarray) +* [Jupyter-core](https://github.com/jupyter/jupyter_core) +* [Jupyter notebook](https://github.com/jupyter/notebook) +* [scikit-learn](https://github.com/scikit-learn/scikit-learn) + + +## Core file requirements for a Python package + +In the above example you can see both the core documentation files +that pyOpenSci requires all live in the root of your project +directory. These files include: + + +* A documentation website +* CHANGELOG.md +* CODE_OF_CONDUCT.md +* CONTRIBUTING.md +* LICENSE +* README.md + +And the tests/ directory containing your test suite. + +```{button-link} https://www.pyopensci.org/python-package guide/documentation +:color: primary +:class: sd-rounded-pill float-left +Click here to read about our packaging documentation requirements. +``` + +## Scientific Python community standards + + +The discussion below focuses on the recommended structure for Scientific +Python packages. + + +```{note} + +### Src vs. flat layouts for Python packaging +While you will see other structures, [such as the `src/` +layout](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/) recommended by the [Python packaging authority](https://py-pkgs.org/04-package-structure) and others, all scientific +Python packages have used a flat layout. This structure has been +prevalent for almost a decade. + +Because a core goal shared by both pyOpenSci and the Scientific Python +project, is to see increased standardization in Python packaging, we +are advocating for a structure that is used by existing packages in the +ecosystem. + +Many of our core scientific Python packages wrap around other compiled languages, such as C++. They thus have source code that requires compilation. It would be challenging and potentially time-consuming for all of those packages to modify their current +structure. And further it would not provide them with any core benefit. + +pyOpenSci however will never require a specific package structure for its +peer review process. The overview on this page presents recommendations. +``` + + +### The flat layout for Python packages + +The flat structure has been historically, the most common package approach used in the scientific packaging +ecosystem. It is a structure used by the entire scientific Python ecosystem. + +The flat layout's primary characteristics are: + +* The source code for your package lives in a directory with your package's name in the root of your directory +* Often the `tests/` directory also lives within that same `package-name` directory. + +#### Benefits of using this structure + +* This structure has historically been the most common so you'd be following a convention that many packages use already. +* This structure is simpler to setup compared to the `src/` approach listed below +* You can directly install the package directly from the root directory +* Tests are shipped with the packages. This allows developers to invoke tests on various machines to troubleshoot installations if needed. + + +```bash +$ cd package-name +$ pip install -e .` +``` + +## setup.py, setup.cfg and pyproject.toml file + +In recent years, Python package standards have begun to move away from +including both package metadata and python code needed to setup a package installation in the same setup.py file. The current recommendations are +to [include all project based metadata, and to specify the build system that you want to use in a `pyproject.toml` file.](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/) + + +Below is an example build configuration for a Python project. This setup + +requires: + * **setuptools** to create the package structure, + * **wheel** which is used by `setuptools` to create the whl (wheel) file. + * It also uses `setuptools_scm` to handle version updates + +The build back end is + +setuptools.build: + + + into setuptools +``` +[build-system] +requires = ["setuptools>=45", "wheel", "setuptools_scm[toml]>=6.2"] +build-backend = "setuptools.build_meta" +``` + + +If you use `build` to create your package's wheel and tarball, you can now +store dependencies in either the setup.cfg file or the pyproject.toml file. + + +```{note} +It is important to note, that in some packages requiring complete configuration, a setup.py file might still be required. +``` + +[PEP518 describes the move away from setup.py to the pyproject.toml file.](https://peps.python.org/pep-0518/). + + +```{tip} +PEPs stand for Python Enhancement Protocols. They provide guidelines for standardizing +Python code and packaging. +``` + +Example of setup.cfg - verde: https://github.com/fatiando/verde/blob/main/setup.cfg +Build system for this - https://github.com/fatiando/verde/blob/main/pyproject.toml + + + diff --git a/package-structure-code/python-package-versions.md b/package-structure-code/python-package-versions.md new file mode 100644 index 000000000..53c9019c5 --- /dev/null +++ b/package-structure-code/python-package-versions.md @@ -0,0 +1,152 @@ +# Creating New Versions of Your Python Package + +## Section about releases +* mention release should be incremetal +* rep changes in the code that are either patches, minor fixes, major updates + +pyOpenSci recommends that you follow the [Python PEP440](https://peps.python.org/pep-0440) which recommends using +[semantic versioning guidelines](https://www.python.org/dev/peps/pep-0440/#semantic-versioning) +when assigning release values to new package versions. + +[Semantic versioning, discussed in detail at semver.org,](https://semver.org/) uses the following approach to bumping your +package's versions: + +> Given a version number MAJOR.MINOR.PATCH, increment the: +> +> * **MAJOR version** when you make incompatible API changes +> * **MINOR version** when you add functionality in a backwards compatible manner +> * **PATCH version** when you make backwards compatible bug fixes +> Additional labels for pre-release and build metadata are +> available as extensions to the MAJOR.MINOR.PATCH format. + + +```{tip} +Resources: + +* Poetry bump version - go discussion here - https://py-pkgs.org/07-releasing-versioning#manual-version-bumping + +``` + + +```{important} +pyOpenSci requires that your package has an installable distribution that can be installed from a public community repository such as PyPI or a conda channel. +``` + +## Tools to manage versions for your Python package + +PyOpenSci doesn't require any specific tools to support +creating an installable distribution for (packaging) and releasing your package. HOwever there are a handful that +have been broadly adopted by the scientific python community, +and we provide an overview of each of those below. + +If you are on the fence about what tool to use, we suggest +that you use `setuptools-scm`. + +### Tools for bumping Python package versions +There are many tools available to manage "bumping" and naming versions for your Python +package. + +```{note} +Bumping a package version refers to the step of increasing the package +version after a set number of changes have been made to it. For example, +you might bump from version 0.8 to 0.9 of a package. or from 0.9 to 1.0. + +Using semantic versioning, there are three main "levels" +of versions that you might consider: + +Major, minor and patch. These are described in more detail below. +``` + +Below we discuss some of the tools that are most common +in the Python scientific package ecosystem. We do not require +you to use a specific tool. We simply recommend tools based on +what the community is currently using. + +The list below are the tools discussed on this page: + +* setuptools-scm +* bump2version (last commit mar 2 - is it maintained?) - https://github.com/c4urself/bump2version +* versioneer +* python-semantic-version + +### Tool 1: setuptools-scm a light weight version option + +[`Setuptools-scm`](https://github.com/pypa/setuptools_scm/) is one of the more popular tools see in our packages +both within the [pyOpenSci package ecosystem](https://www.pyopensci.org/python-packages) and across the Python Scientific ecosystem. + +We like `setuptools-scm` because: + +**Pros** +* It creates a single-source file that contains your package version. +* You never manually update the package version +* You can automate writing the version anywhere in your package including your documentation! +* It supports a purely GitHub based release workflow. This simplifies maintenance workflows. +* Version number is updated in your package via a hidden `_version.py` file. There is no manual configuration updates required. +* While we like detailed commit messages (See Python Semantic Version below), we know that sometimes when maintaining a package specific guidelines around commit messages can be hard to apply and manage. + +**Cons** +* In a CI workflow you will end up manually entering or creating the version number via a TAG on GitHub. +* Not well documented (we hope to fix that!) + +```{important} +pyOpenSci will be creating tutorials on working with `setuptools-scm` and GitHub releases to +update versions of your package and push to PyPI. These should be published sometime +during the spring/summer 2023. In the meantime [here is a high quality blog post +that will help you get started with using setuptools-scm](https://www.moritzkoerber.com/posts/versioning-with-setuptools_scm/) +``` + +### Tool 2: [Python semantic release](https://python-semantic-release.readthedocs.io/en/latest/) + + +**Pros** +* Follows semver versioning closely +* Enforces maintainers using descriptive commit messages which can simplify troubleshooting + +**Cons** +* requires very specific commit language to work. In practice some maintainers may not be able to maintain that level of specificity in commit messages (altho there are bots to help with commit checks) +* Release happens at the command line. This makes is harder to implement a GitHub based release workflow as the wrong commit message could trigger a release. +* The version number is manually updated in a configuration file such as `pyproject.toml` vs. in the package itself. + +As the name implies, Python Semantic Release follows python +semantic version release rules. +Python Semantic Release, similar to **setuptools-scm**, also helps you automate version release workflows. + +However, this tool differs +from **setuptools-scm**. With **setuptools-scm**, a version +control tag, is used to trigger a version update. +With Python semantic release, versions are triggered using +specific language found in a repository commit message. + +For example, the words `fix(attribute_warning):` trigger Python Semantic Release to implement a patch version bump. +So for instance if your package was at version 1.1.0 and you +made the commit below with the words fix(text-here), Python Semantic Release would bump your package to version 1.1.1. + +```bash +$ git commit -m "fix(mod_plotting): fix for warnings returned for some athlete attributes" +``` + +Similarly a feature (`feat()`) triggers a minor version bump. +For example from version 1.1 to version 1.2 + +```bash +git commit -m "feature(add_conversions): add value conversions to activity date using pint" +``` + +```{tip} +You can find a thoughtful discussion of python semantic version [in this Python package guide.](https://py-pkgs.org/07-releasing-versioning#automatic-version-bumping). Note that the guide hasn't been updated since 2020 and as such some of the commands are now dated in it. +``` + +### Tool 3: Versioneer + +I haven't used this tool before... +Ask martin??? + +### Tool 4: Bump version + +* Versioning can be done using [bumpversion](https://github.com/peritus/bumpversion), e.g. for a minor update: + +``` +bumpversion minor +``` + +"minor" can be replaced with "major" or "patch" depending on the level of update. \ No newline at end of file From 05f8e0fd8694db62e8f286be448ac5690d20d4bb Mon Sep 17 00:00:00 2001 From: Leah Wasser Date: Thu, 29 Dec 2022 16:29:31 -0700 Subject: [PATCH 4/5] package structure updates --- index.md | 4 +- package-structure-code/intro.md | 54 ++-- .../publish-your-python-package-pypi-conda.md | 10 + .../python-package-build-tools.md | 175 ++++++++++++ .../python-package-structure.md | 248 +++++++++--------- 5 files changed, 343 insertions(+), 148 deletions(-) create mode 100644 package-structure-code/publish-your-python-package-pypi-conda.md create mode 100644 package-structure-code/python-package-build-tools.md diff --git a/index.md b/index.md index 636e07236..32b8e8c29 100644 --- a/index.md +++ b/index.md @@ -123,11 +123,13 @@ Contributing & license files :caption: Package structure & code style Intro -Code Style & Format + Python package structure +Package Build Tools Package Versions package-structure-code/overview package-structure-code/collaboration +Code Style & Format ``` diff --git a/package-structure-code/intro.md b/package-structure-code/intro.md index 4d76e48be..5c3833fce 100644 --- a/package-structure-code/intro.md +++ b/package-structure-code/intro.md @@ -1,37 +1,37 @@ # Python package structure information -```{tip} -### Python packaging resources that we love +If you plan to submit a package for review to pyOpenSci and are looking for +some guidance on package structure, code formats and style, then this section is for you. -We think the resources below are excellent but each have particular opinions -that you may or may not find in our packaging guide. For instance, the PyPA -guide encourages users to store their package in a `src/package-name` directory. -While we accept that approach many of our community members prefer to not use -the `src` directory. +## Guidelines for pyOpenSci's packaging recommendations -* [Python packaging for research software engineers](https://merely-useful.tech/py-rse/) -* [PyPA packaging guide](https://packaging.python.org/en/latest/) -``` + -If you plan to submit a package for review to pyOpenSci and are looking for -some guidance on package structure, code formats and style, then this section is for you. +There are some differing opinions on what Python package structure should +look like and what tools to use across the Python ecosystem. + +In this guide, we have made decisions around suggested standards and required +standards, based upon the commonly used approaches in the scientific Python +community. Our goal is to help standardize packaging across this ecosystem. + +In some cases: +* The suggestions here may diverse from those in the non-scientific parts of the Python ecosystem. +* The suggestions here are made with the intent of being helpful, but are not specific requirements for your package to be reviewed and accepted into our ecosystem! + +In all cases, we try to align our suggestions with the most current, accepted +[PEP's (Python Enhancement Protocols)](https://peps.python.org/pep-0000/) and the [scientific-python community specs](https://scientific-python.org/specs/). ```{note} -If you are considering submitting a package to pyOpenSci, or if you are just -getting started with building a package, you may want to have a look at the +Have a look at the bare-minimum [editor checks.](open-source-software-submissions/editor-in-chief-guide.html#editor-checklist-copy-template-below-to-use-in-the-issue) that pyOpenSci -performs before a review even begins. +performs before a review begins. These checks are useful to explore +for both authors planning to submit a package to us for review and for +anyone who is just getting started with creating a Python package. In general these are basic items that should be in any open software repository. ``` -## Packaging Guide - - -PyPI also has a [short tutorial](https://packaging.python.org/tutorials/packaging-projects/) -on how to package a Python project for easy installation. - + \ No newline at end of file diff --git a/package-structure-code/publish-your-python-package-pypi-conda.md b/package-structure-code/publish-your-python-package-pypi-conda.md new file mode 100644 index 000000000..984fda6e1 --- /dev/null +++ b/package-structure-code/publish-your-python-package-pypi-conda.md @@ -0,0 +1,10 @@ +## Package publication + +pyOpenSci requires that your package can be installed from a public community +repository such as `PyPI` or a `conda` channel such as `bioconda` or `conda-forge`. + +### What is PyPi + +### What is Conda + +### How are they different? \ No newline at end of file diff --git a/package-structure-code/python-package-build-tools.md b/package-structure-code/python-package-build-tools.md new file mode 100644 index 000000000..483e1e128 --- /dev/null +++ b/package-structure-code/python-package-build-tools.md @@ -0,0 +1,175 @@ +# Python Package Build Tools + +Where i'm leaving off here + +* setuptools is the OG clearly a lot of ppl use it. but the code base seems +really messy and it's built on top of disutils that may be sunsetted i python 3.12 ?? so does it make sense for us all to use it or should we consider +an example using hatch which seems really nice. extensible and has vcs built +in as far as i can tell + +Below talk about each tool and the potential drawbacks. +post on slack about setuptools (next week) and also maybe discord. + +I think this page should be it's own separate PR as i really want eyes on it. +So more eyes == better. +I can then have another Pr that has package structure... + + + +## Python package distribution files + +There are two types of distribution files that you will create to support +publishing your Python package: + +* SDist additional +* a wheel + +### What is a SDist file? + +SDist, short for **S**ource **D**istribution file is a packaged file in `.tar.gz` +format (often called a "tarball") that has all of the files needed to build your +package. In the SDist format, your package's files are not included in a built +format. Thus, anyone using this file, will have to build the files first before +they can be installed. + +```{tip} +When you make a release on GitHub, it creates a SDist file (`.tar.gz`) that +anyone can download and use. +``` + + + + +### Wheel (.whl files): + +A wheel or `.whl` file, is a zipped file that has +the extension `.whl`. The wheel does not contain any of your packages +configuration files such as **setup.cfg** or **pyproject.toml**. This distribution +is a pre-build, ready-to-install format. + +Because it is prebuilt, the wheel file will be faster to install for pure Python +projects and can lead to consistent installs across machines. + +[Read more about the wheel format here](https://pythonwheels.com/) + +## Tools to build python packages + +There are a suite of build tools that you can use for creating package distributions for your Python package. Before discussing some of the +more popular toole, it's important to clarify the difference between a +build tool front-end and build backend. + +## Build front-end vs build backend + +Each tool discussed below has both a front end interface that you can use to +perform different types of Python packaging tasks. Each also has a back-end +build tool that actually builds the package and creates associated distribution +files. The backend build tool for each package, runs the steps needed to +create a package wheel and tarball (zipped up version of all of +the package files). + +Some of these tools provide additional front-end functionality. For instance, +you can use Flit to both build (it has a build-backend) and to publish directly +to PyPI (or test PyPI) using the front-end tool. + +`flit publish --repository testpypi` + +This means that you don't need to remember another tool (like `twine`) to publish +to PyPi. But Flit also offers a build backend too. This allows you to run: + +`flit build` + +To build your package. Flit thus makes your workflow commands consistent and simple. + +If you are using `setuptools`, then you will need to use `twine` to publish to PyPI. + +## An ecosystem of Python build tools + +Below we introduce several of these tools that support publication on +PyPI. Each tool has various features that might make you chose to use it +or not use it. There is no right or wrong tool to use as far as pyOpenSci is +concerned. We are just trying to help you find the tool that works best for +your workflow. + +The tools that we review below include: + +* setuptools +* flit +* hatch +* pdm + + +These tools were some of the most popular tools listed in hte PyPA packaging +survey that was run in 2022 + +```{note} +Note that we are intentionally not including poetry in this list. because + +* add reasons here `^` for pinning deps +* look at how it creates a pyproject toml file too... + +TRANSLATE THIS into something easier to understand: Upper limits in dependencies: ^3.6 should be changed to >=3.6. Poetry will force you to add an upper limit if a package you include does this, though, so the bad practice percolates. +``` + + +## How to chose a build development tool + +When deciding what tools you wish to use, there are a few basic criteria that +you can use to help guide your decision: + +If your package is a pure Python package and it doesn't have any additional build steps (such as compiling code, etc) you can use any of the tools discussed on +this page. + +* If your package is not pure python, or it has complex build steps (or build +steps that you need to customize), then you should consider using: +setuptools, or hatch. + +Both of these tools allow you to customize your workflow with build steps needed +to compile code. + +setuptools is the "OG" tool that has been around for a long time. +hatch is the newer tool - flit like but supports customization. + +## Tools with built in versioning + +* setuptools_scm +* hatch_vcs + +TODO - which tools have VSC compatible - hatch-vcs does pdm have that? +i still don't understand why you'd use hatch vs pdm . + +A summary of what each of the tools offers can be found in the table below: + +REMOVE POETRY BELOW: +| | Setuptools | Flit | Hatch | Poetry | PDM | +| ------------------------------------------------------- | ------------------------------------------------------- | ---------------------------- | --------- | ------------ | ------------ | +| Build backend | setuptools.build\_meta | flit\_core | hatchling | poetry\_core | | +| Supports projects that aren't pure python | yes | no | yes | no | yes | +| Supports custom build steps (before creating wheel) | yes | no | yes | no | yes | +| Has built in dependency management | no | no | no | yes | no | +| Can be used to publish directly to pypi | no (use twine) | yes | yes | yes? | ? | +| Has versioning tooling | setuptools\_scm | | | yes | | +| Supports automated GitHub only releases (no CLI needed) | yes (GitHub only) | | | no | | +| Used by core scientific Python packages | Xarray, geopandas, pandas | | | no? | scipy, numpy | +| One tool does everything | No (need scm for versioning but it's very light weight) | Yes???? might need flit\_vcs | ? | Yes | + + +## Setuptools + + +## Flit + + +many thanks to the resources here: https://henryiii.github.io/level-up-your-python/notebooks/2.7%20Creating%20Packages.html# + diff --git a/package-structure-code/python-package-structure.md b/package-structure-code/python-package-structure.md index 9141bb975..8e7305b7d 100644 --- a/package-structure-code/python-package-structure.md +++ b/package-structure-code/python-package-structure.md @@ -1,19 +1,46 @@ -# Python Package Structure +# Python Package Structure for Scientific Python Projects +There are two different general layouts that you will commonly see +within the Python packaging ecosystem: [src and flat layouts (click here to lear more).](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/). While we believe that both layouts have advantages, +and we know that the Python packaging authority does advocate for the [src/ layout](https://py-pkgs.org/04-package-structure), we suggest that scientific packages adhere to the **flat-layout** given: -```{important} -Things to add: -* using pyproject toml and peps that support it -* moving way from setup py -* setup.cfg -``` -## Package publication +* All of the core scientific Python packages use it (see list below) +* It's the most commonly found layout with the scientific Python ecosystem +* Many Python tools depend upon tools in other language and / or complex builds with compilation steps. Many developers thus appreciate features of the flat layout such as tests being included with the package when they are installed) to support troubleshooting across installations. + +```{note} +Advocating for the flat-layout is the only recommendation where we diverge some +from the Python Packaging +authority in an effort to support the broader scientific Python community. This +effort is core to the pyOpenSci mission. +``` + +```{tip} + +## Core scientific Python packages that use the flat layout + +* [numpy](https://github.com/numpy/numpy) +* [scipy](https://github.com/scipy/scipy) +* [pandas](https://github.com/pandas-dev/pandas) +* [xarray](https://github.com/pydata/xarray) +* [Jupyter-core](https://github.com/jupyter/jupyter_core) +* [Jupyter notebook](https://github.com/jupyter/notebook) +* [scikit-learn](https://github.com/scikit-learn/scikit-learn) + +It would be a significant maintenance cost and burden to move all of these +packages to a different layout. The potential benefits of the source layout +for these tools is not worth the maintenance investment. To avoid dividing the scientific Python community, +pyOpenSci supports maintainers using a flat layout for scientific packages. +``` + +## What does the flat layout structure look like? -pyOpenSci requires that your package can be installed from a public community -repository such as `PyPI` or a `conda` channel such as `bioconda` or `conda-forge`. +The flat layout's primary characteristics are: -## What should the structure of your package look like? +* The source code for your package lives in a directory with your package's name in the root of your directory +* Often the `tests/` directory also lives within that same `package-name` directory. -Below you can see the recommended structure of a (scientific) Python package. +Below you can see the recommended structure of a scientific Python package +using the flat layout. ``` myPackage/ @@ -24,42 +51,55 @@ myPackage/ β”‚ └── ... β”‚ β”œβ”€β”€ LICENSE β”‚ β”œβ”€β”€ README.md β”˜ -β”œβ”€β”€ pyproject.toml ┐ -| myPackage/ β”‚ Package source code, metadata, -β”‚ β”œβ”€β”€ __init__.py β”‚ and build instructions +β”œβ”€β”€ pyproject.toml ] Package metadata and build configuration +| myPackage/ ┐ +β”‚ β”œβ”€β”€ __init__.py β”‚ Package source code β”‚ β”œβ”€β”€ moduleA.py β”‚ β”‚ └── moduleB.py β”˜ tests/ ┐ - └── test-file1.py | - └── .... β”˜ Package tests + └── test-file1.py | Package tests + └── .... β”˜ ``` -pyOpenSci suggests this package structure as it aligns with the structures of -core scientific Python ecosystem including: -* [numpy](https://github.com/numpy/numpy) -* [scipy](https://github.com/scipy/scipy) -* [pandas](https://github.com/pandas-dev/pandas) -* [xarray](https://github.com/pydata/xarray) -* [Jupyter-core](https://github.com/jupyter/jupyter_core) -* [Jupyter notebook](https://github.com/jupyter/notebook) -* [scikit-learn](https://github.com/scikit-learn/scikit-learn) + +```{note} +If you look at [the `matplotlib` repository on GitHub](https://github.com/matplotlib/matplotlib), you will notice it too has a src/ directory, however +that structure is there because `matplotlib` has uncompiled source code in that +directory. it is not adhering to the **src/** layout explicitly. +``` + +### Benefits of using the flat layout in your Python package + +There are numerous benefits to the scientific community in using this layout. + +* This structure has historically been used across the ecosystem and packages using it are unlikely to change. Thus, you'd be following a convention that many packages use already. +* This structure is simpler to setup to support packaging compared to the `src/` approach given the package source code is in the root directory. +* You can directly install the package directly from the root directory +* Tests are shipped with your package. This allows developers to invoke tests on various machines to troubleshoot installations if needed. + +To install your package in editable mode use: + +```bash +$ cd package-name +$ pip install -e .` +``` ## Core file requirements for a Python package -In the above example you can see both the core documentation files -that pyOpenSci requires all live in the root of your project -directory. These files include: +In the above example, notice that all of the core documentation files that +pyOpenSci requires live in the root of your project directory. These files +include: -* A documentation website * CHANGELOG.md * CODE_OF_CONDUCT.md * CONTRIBUTING.md -* LICENSE +* LICENSE.txt * README.md -And the tests/ directory containing your test suite. +Also note that there is a **docs/** directory at the root where your user-facing +documentation website content lives. ```{button-link} https://www.pyopensci.org/python-package guide/documentation :color: primary @@ -67,15 +107,21 @@ And the tests/ directory containing your test suite. Click here to read about our packaging documentation requirements. ``` -## Scientific Python community standards - +Finally, notice that the **tests/** directory containing your test suite is +located within the **packageName/** directory. -The discussion below focuses on the recommended structure for Scientific -Python packages. + +```{important} +If your package tests require data, we suggest that you NOT include that +data within your package structure. We will discuss this in more detail in a +tutorial. +``` + +## Use a pyproject.toml file for your package configuration & metadata -### The flat layout for Python packages - -The flat structure has been historically, the most common package approach used in the scientific packaging -ecosystem. It is a structure used by the entire scientific Python ecosystem. +We recommend that you follow current recommendations and [include all project based metadata, and build system specifications in a `pyproject.toml` file.](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/). -The flat layout's primary characteristics are: +The [TOML (Tom's Obvious, Minimal Language) format](https://toml.io/en/), is an easy-to-read structure that is founded on key / value sections. -* The source code for your package lives in a directory with your package's name in the root of your directory -* Often the `tests/` directory also lives within that same `package-name` directory. - -#### Benefits of using this structure - -* This structure has historically been the most common so you'd be following a convention that many packages use already. -* This structure is simpler to setup compared to the `src/` approach listed below -* You can directly install the package directly from the root directory -* Tests are shipped with the packages. This allows developers to invoke tests on various machines to troubleshoot installations if needed. +Each section in the file contains a `[key]` and below it values associated with that section. +```{note} +[PEP518 describes the move away from setup.py to the pyproject.toml file.](https://peps.python.org/pep-0518/) +Python package standards are moving away from +including both package metadata and python code needed to setup a package installation in the same **setup.py** file. Instead we are moving towards using +a **proproject.toml** file sometimes combined with a **setup.cfg** file. -```bash -$ cd package-name -$ pip install -e .` +In some cases, +where a build is particularly complex, a setup.py file may still be required. ``` -## setup.py, setup.cfg and pyproject.toml file - -In recent years, Python package standards have begun to move away from -including both package metadata and python code needed to setup a package installation in the same setup.py file. The current recommendations are -to [include all project based metadata, and to specify the build system that you want to use in a `pyproject.toml` file.](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/) - +## Example pyproject.toml Below is an example build configuration for a Python project. This setup - requires: - * **setuptools** to create the package structure, - * **wheel** which is used by `setuptools` to create the whl (wheel) file. - * It also uses `setuptools_scm` to handle version updates - -The build back end is -setuptools.build: +* **setuptools** to create the package structure, +* **wheel** which is used by `setuptools` to create the [**.whl** (wheel) file](https://realpython.com/python-wheels/). +* **setuptools build** to "build" the package +* **setuptools_scm** to manage package version updates - - into setuptools ``` [build-system] requires = ["setuptools>=45", "wheel", "setuptools_scm[toml]>=6.2"] build-backend = "setuptools.build_meta" -``` +[project] +name = "examplePy" +authors = [ + {name = "Some Maintainer", email = "some-email@pyopensci.org"} +] +maintainers = [{name = "All the contributors"}] +license = {text = "BSD 3-Clause"} +description = "An example Python package used to support Python packaging tutorials" +keywords = ["pyOpenSci", "python packaging"] +readme = "README.md" +``` -If you use `build` to create your package's wheel and tarball, you can now -store dependencies in either the setup.cfg file or the pyproject.toml file. +Above also notice that all of your package's metadata can be stored in the +**pyproject.toml** file (rather than the setup.cfg file or the setup.py file). +You also can specify dependencies in this file. -```{note} -It is important to note, that in some packages requiring complete configuration, a setup.py file might still be required. -``` - -[PEP518 describes the move away from setup.py to the pyproject.toml file.](https://peps.python.org/pep-0518/). + +We discuss build tools for python package more here. + ```{tip} PEPs stand for Python Enhancement Protocols. They provide guidelines for standardizing Python code and packaging. ``` -Example of setup.cfg - verde: https://github.com/fatiando/verde/blob/main/setup.cfg -Build system for this - https://github.com/fatiando/verde/blob/main/pyproject.toml - - + \ No newline at end of file From 44f031e6878bd3c61e08848edbf23577e1fc8cf0 Mon Sep 17 00:00:00 2001 From: Leah Wasser Date: Tue, 3 Jan 2023 16:56:17 -0700 Subject: [PATCH 5/5] Add: src and flat + build tools --- .../python-package-tools-2022-survey-pypa.png | Bin 0 -> 18655 bytes package-structure-code/intro.md | 11 +- .../python-package-build-tools.md | 261 +++++++++++++----- .../python-package-structure.md | 154 +++++++++-- 4 files changed, 329 insertions(+), 97 deletions(-) create mode 100644 images/python-package-tools-2022-survey-pypa.png diff --git a/images/python-package-tools-2022-survey-pypa.png b/images/python-package-tools-2022-survey-pypa.png new file mode 100644 index 0000000000000000000000000000000000000000..c3d7d4f8a9542048d92a62cef1c7ad4d2b4f3521 GIT binary patch literal 18655 zcmdUXc{r5&-@kLtSEoa0u~fFBl45Y$2;r0@gvc`1(a}P(FB4WHFTO+m6CXU;N-a% z^V`j~D94^TZJlg-TVYSk$v+;sZ`o!&G`y^ZvA{NFTnP={oiWJQSmavkzBadLZ{c0Z zoE;8kZ7G!j&4{>`+*NfWXhL;xKUgj{~6P z0>4?7fxo}aas5U>;BROC$G^aNM9ahs#qB)wf;%FHIqOo{zLLpbmFMf$^Ld=6;tzT) zG4Ngn->y9h5GWURDRV(HZTKx`++N2>G!-KQXz56H5T8Th>x9_&vW77vOo^;c z@IG9B>oA|kdcoQzKg=0oQD>`%8R4rf2!e>T#y+1bSUjdr^Rl@I&dyX7ZuZzvviE%1 zWLtWzW1ELTK=#37thDd$iJV!AI+Egy zT&45qBwh=8WhA=kN$NEQV%hBRg?@T(m$2=HOAic>iI->&hdn>w7ANgf*Ety2ZC0y_ z!24fbm^}H{J8a9&#OG0m4uC}FR2aT3b1~$qfSD4JOQR`QgYZ)kp#}YF}=vC@5&#sgLzYTX{eH_4id_B*cL+5{l$aH4N;S;wqU?7-l1uCvoMC=FNnSQI zO&=59$xTv*%VXA_k>Xq|CM25{-ZvZS@0ic*WeuU)Q+BuF_i?uz)G5{RrXI%(ka$d7 zQ}6C{Qc_W_s}1}4{mX+DT2eEdZ|_@}*_OG4Oh}M^>r!W08J>|AEG=L%V-LBMEeXT= zw|cr2$JLwMR0 zM1}hUQVCJWWA^|&CL@;IM+hn=rA>s;!TWHT{gqj1Ux#A$PXNCevxz&nPO& zTX*a~_epT_;rQi8MS-sZnio<}3!1hZ3nX>tn=NX2eD?$vWo-ZaJ`5K}?rpEC zZ_ivCR|+4)h2(I$7w_&9CRce4P{qPOeTUI$O*xy~XnV3djLa>9Pt%yceJ}S(znCz5 zsUap@l)1Ka;_ACcBS&P8T)E57*<;nTR%-1gsM6AYDmlos6@PzN>esMq&^J^iRW z+!(3HP};=n1XK7btu37=6wJhFiwl1&h>Fr&e(C1ED2)8U(x$1I0KCi#Kj~_}_ zN0$~a+V8P!u);QxdQ6*A6PF{Vn1s1{P7JyGjLxcVKGDVUw;SURjMmyMPUo4mFu7;S z>-JkVbPZ8fxMwh1js}+pl41K=Ja=4PdL!*4o;*!+MBS=zdTl)_XsUQnd6ROHpS_$T z?SaQ*6S$N0qcK5xJBIf>v%9;%LOm7JpOZt;bJ_I#-SC^Ewq@mOuuJyUJ!!(L=x^Q@ z>{o&JAMl9GMpX3w^lExA%7Hdrkhv@^fR>ejaVtwnPb(eu@HEpDn|BotxgOXGcBt{1G5c2YXVz5Tbg;s+(|I?C{-?a$GiV9aqtmc(HDkYveG z(-bu({f()Mb(sr>%Y2_YT|T)lRCZKazO?vyLY~Sj*l9O4#{Dei`W^S!BNCswPM6uG z#em(MB0Wh{;-86CA}lFv*$uV@Lj9*%Q?JN9j;9+z}Zk$lESWcw&%wD7Opq8 zv{)Pj`aj<&&uTGAe}_xtR!XTT@3f!vach<+f$0m|las6h)W%tYKW3(uxEj)2rg5HL zJ4Z>^ZFW|ii#kcltq8X+t9aHiv4Xhm*MDOsl$)I})sYbV+Vf?eh<(_cfm8pc+K29L zk2RLoK;*DrF?`GnDUc{x39`%*vMjT{vUKPhupte^`7Wa<#q0=)VU(pgIc;BPYqo)6 z=O*2dseMpyPcJwfo5-2yluIbV8C#c)Mwjhl%LT!)X}V3Plta8KkR~ZXHHC*P^*S@2 zTIIp4n%wAOiq2$(hFmK_73GA9H_57wYlQ!*TOM-{PfKq~k5*tuVM!kY-=7Lpi^RqY zCG!Ze!+OT79Qs)FvB1M#6?Ug7({)?DZfBEz+Up_x(JDJGPc&=lP)?TaH`+bf^s#ci zIe$UZ=y6V zZZpdsd+U`-x>g4s`TwZ*V_3&&coEilI0MNWf(GbTzdRN?l%Bp>)<~BKcB?84R?H zvY-9fIGmiK*J#p`BNAHfYFr@|Xn|47PMk=LK9Hl^H6M{j>~_cx*@bat9w~n5-Ps;S zUSCC~>(NeAi*^cw5LhU*7fL>@bZbVPw4&o_A^}tFrb$mrJS36zhMfO*{rB89?|{uVbiJL+JkWMGt9>}qWNw^7iNOR`>92ufZC(U)b%0W^czMYtO zJ~LgS)f6q6lKfhqv8M&ByAA2BDuP*{&-JJl`R$=HJe%}siIG2bvu{euX5oJ7ly72q zgJ=BhTj3Ar>h0sTm4S|2MI7Q4VR&5YpdXmHhbEawFmXSO^t(=L75e5yt&#Yf!|*ju_N;M z(sL=KwLBC+1k{{(y&3R_s+1WvibIXXX|C~~uyW+NZ@J@2`hFzttd9!88BZS#UmKI> z@DCeJj6fLsiH4U6w)}~S_YuGt_+hLhzC43M;#Kl9q;u+Q-UtYk*)rFYBod$yET7-A zgempd%YC)Oq6Rq>(&*EJ;d6#LbQjd@F-%Bb`Q^(#l=ne?1QIO+)#gR5=`?$M__glh zyCY3D!QJn@$XI8(=_4$sJCkP}=5SKE3Lap6k1laoU#nLGvlFQ ztR%khLIGP>8$ia0fqHZ;$(()~CA)-cJ_~YDus(mao$F z@1C_5s)#-t-B{6?Y==Pi;y66cTrHDM;lwSzh!W-oDSvn5Y8#B|lX~mcCPcrr_!2J0 zGEK;=JkWS8*mwdPo1!Rdb(g^bzN#|OZp(DT--Rtld9sI~95=DA`n&L_tsxid*6?_ItDQl2b)J;{uMz@O*Q3SJXgKf255!T0kJx zE;*HVjQp&d zFIwvHU1r$z+1{>&+?CmcW@o9SJa-H8hmp&S>2gulo#j7D;Lox@dYwjH4J5X)x|nP> zpqOqW>>4ZFg=EU93FFtssAB6b>q+ZN`~DpL>8l3%d$*xU^R(>QtPMt;)uo=W82dQDjN;j937dcA3~Njgosp~&C83RqxXXc-QZa#_ zFZ|$AmThpT|JO5fhr5R&R~Km53LFqzD71C{>Gp7Bx3CLZx;*9L>xl%iOW$@+{dDC3%Tth5j%t!6ke-4`t+6m|2L{XOO$W#FE| zW-2LTdi=G;sELJW{j~;~jQ)dqDqBF{XwnBk;gP5CsWGADW1Jb5puo>JBYGn|aE@WP zzWQGf+}iaN!66fmi?IYL~63S;h%1{N~mVMAne0+NmQ7 zDCu$U=#O4E-=#++8Sod>`3X)puY4L9=4pUW-TdfvzgD)qdfQ>7kgcd?+4Y-gH{Ra* zPrq2`N<1E_9NFOd)AEQVLg!^E)^T)qFvG-qQ^)B8a2s<$`Yjjk&MliFpNYjELbnAx zzje;Hc;XEVD=*KK3Rq00@BR)I_rn z&OM2E?FznC5*g7Ob^vY$k4RmpKL`g6X?UxB0N$#^5~PD^35eVz;;$RJPi{S`6T?fg|?k=`dr_`fS$Tz#L)&Tje+Y2NMuqBJ!@9sXfBxjNJ<$qFBp{Mry<* zD@vZJ6ao&EF)dO(cDU?fq$7w`>V~q~fu9gK$LhA>{H};fAfeW#rtapyrywvybt~ln z5fPC>78)oAwcO=__BdG`vMRjnk71XTIq>usXuERra@S+Y2D|@dM4hByN&AvgSBBCr zw@<{GB^nJ> z`>;|HS;UaVD-?XdRb=ZiigM&aEKb;ix(fmK&nyC1p1#L{j9&3EncQM%k|LCEoYcpglrd^&<$*` z^(L`(3ns`<4d2(2OC4)>Se)pTx7{I;HjY2v*|BQUshuT6HdQB%&v7PF5X+!Lx}Ibh zhTEK^BHVJRHT78d>c_t4Dh@_qBArAhKA%X#z@fzY$IsjK1~PYI4cq{xEASksDn;;S zigA)65|YNrKX%Gtbq~Vh^?5fewM>Rsi|zVzI3NmIUVJ=Np6oeiqewqn?rLZ|!|U|7 z=`zL>ErPimmbybQ+5YORZJAo_$MW~P+eCI9ekB{&aWX2PEP+)TkY` z->B;@XYEB$`XP=gAzX9Dg@U%F|o@U)1VA+{8qQQ zPU{>#<~l%$P)$MB|E&V!*Y8Aqfw~?zB73vkZ|n?m^6^!6`&+{5YmV%HQAH2+$PgA`r<*w-C2+h6%f3;w{0r3y4+oc@+VT0oDb5k6Wj^r^3N z9(|wKHjU4>3snlHq6}BI4*4ux^&6}QGuLr1Fmqypo?`1EFY-ZwWjF1F9YW^_e%H@C zvRrMn);Hcqj@!F}iEHHhKX}_X*Tm^H=YTsBu-U^`e%iU^O});H=hd3tAXBH8ZO%=t z0;!up2z@=g>(1m3|4{|$^6Ha};I~6(-ck?3wN5vbZ~MP+@s$$($j-aT+AB-*Kb`WP zvc%uVUnujx&5ym`;9Aksz|WLA^HxMxS?aH(2sW>&+_B8$5_bRNCWQ0bAIERWB?&*) z5mbpj0D^l?o|vrKs|;a$meItiN%Mwg1!!2^rz0aOk^gvl>K`?JV@{3tWD{c}p?GjN zS?>TGrS$hF75@P3;xmB@XTmH7TWw3OD7xp zbrdw0o;tmy7Gqs!NqinQKmSc@x(>w|$hq|$Php0PdX6^;gd995;}8UeC0BxiR`2OB z4wM8;=cTX{a`!A#1KROB%#SxP8G{2}@5>rWU5NcDRid)mD>zE2qul{>Wj6u?K&mWg zQ#%0)3)RnDYKEaOtlk~89^%fJ%8Qo}claB`H%GlA&Imz~KQ`Gpu+uQyg1kICFgV%z z#2Ivc@EFDELPgtp(7#dmGjaZnbUh4!BD^nGXRdB6wzH>@zY>*j;`C055=S3mW!X74 zlTPC?5&TNu)sp9J3#ga74WQ(&iauYqZ2sp$U=^vrUq*|mJ1hgDEH2th{PaSYM~@Pg zk}Q3GWBwOX7;kMgTs^Vgr8d~BQ`<I9&yewkql zcXU4d`W^M9n7&hm`9Wx>c6NO)=s9w<-@O~t*o)@$zWpAhl# z`a_xHatH0o0`*HBtCwjvAQ8jZ#d-VJR@wAV)K_g9tvOy0~02;(#he4H0`p_>o zb;j=-tZDxh;IjjY+fFQ=L!^4wuTGW=np(fFXO{XXN)=>(xy4bv5-^FP8;JT~xqCxs*%*SVqggn!f{W!H) z2+=sOe|Mctzb_qXt#*lFV^g*)vHsNrBeTp!C-m&Qah%$vt$*$xhzqRWv}$~0(b8L@ zu!CAe@RcsQCjQU+%J6<}8-TN_UOkjH-(q@l&8_s>1QOx?$$tjNLf5R-sQ(L;;j7f9 za@@jyI7R=p*7z#}jY~d})Y<#Qxb;`yzb_X*KeL33$~kHEO71H{Q1|#OKzSQ^aPoJ5h1L0z$xE# z0`cXm6>loTjm1mWgg^gggD@Gu%)#5+_J&zb1CpRK@$dpQ+b}kU{8fq&$n7+epT*)Q zcO$#aH1Qd_D1!GO0S`)|?|}gLtgHh2~nxjM&_5ZdA_q>w)9 zvGDD9Jv6(fDN#wvIWw1rZc5VKW2!!B^6ct0R$fO6}aTcdvF z%u=@nlFMeQIGIb9m}Fu=WRzA=k}Llwfz`dyKmc)f@Z|ThqCm7sPLngFz{t-T85~>yd>TJm7^5 zqLb@y_JT65RYA*q#Co{C4A3A1Z~@?_0tVLH3Z%U<^8`?tX@`=_9Fkh&$9rZJFXs$b zI^9q-zFLsAVbV?tlR-s3?6=2y=+KLOZRt9LQlu3_M6QO<&_|Y#$DDOq^WEjq>mVf} zY#X{VaL1F9aOhc29^Gr83eOmenV1)4b z(bwUtYpaXg%MvAX;jCIi`c%0n^Um@u4{kwbIKe&Lr0t^ibZ%m>%6Y%MwuV$qpA1Z$ zyE+w=cZngqQyWlUq-4ntJNDOr%Kd&RsduTzTBKr=RO}>G5znv0A@qPQjnOJ~^}tVc zWRnr)myA2LnX+~L89=&9_pggxq z%VbfMEJ7)lRVi|69wn|sCfP9@J!>ChDk+}+GU)xqX#CYoG`ZBdmJD%f`OtR`Wr1>! z2+e9dDz3&YUbm_+b(*4rRRuDNuq^bA9HS) zYSpB*)L|e@-&g1wC!^Jcsk`Z2ple4M1`>uis2XUX zZIk63QBC!W>2oWM?N(*~Rg{Rw9%mI5vAiWr3Tqiuc(4yS#%2w^Paj?7p5M*2aG92;2De z4P*T$#rR?Z{`W7I=ry4cyobvs8`VO4?t0!l0>86i1Qm)S@+=oUnu>7gkHbuC6gDK? zw}6&U`CdFG^m+OcFWLWz?C66((sul1&`eKLdgx4HcpoU_%x&7T^E{jevK7% z%N@sOE4rb4%gDGf`a?AY)M55zn?z-%)M`FNyqNGCdxHnLLsBhes$< zy&Jx2We@57baLsFT7bT}6u^2GD0vbkpxi?m0>Hp2T2yw80`9An*a-=?ocS`@ANvjD zhg#solTtNRfo~JtRu%Snx=fGg{o4UJC;!Wdxc04h3Hri@$9*G2u0Qx?D^vf-5HEp3 zOd?g~P%jye@DqwrWZx zr)*3f|E*p7RmI_wec3g38V%?c`!lU z7qUqGH1d+mU7GH01XYATpaMDr*cW{v0UbpGdc#KH?DA7r5n1gD_dY`@=W&*X*Kqyd z+;3JPgtdj1SaPRJ$)yIuXpG7)p2t~v>gXo)h%n+nzjS`%4A8)@mv((6l@xCQqMla_ zYV?lM+E4+cT$jS1y})n*?v^S|Jn_A$py$$5=d)vJI^dEsDV!UV>>(?A?fRCljfQ2B zD{-mIwBDKs+T#s;3_QA`X|j0fb4@{0y2jTgxcPuhnH*-#4^d=`b($UurK|EQ+lpgy zTxXwrxq?7TOIp-wOWgbJD;Q8AxZmE;)uBKfzcUT&+j@eY@*5V)<+d@r z{1LvcMmc)p^UB&FBX&yMt|MdSL7Kv*x}uN75* z7rK;A1(acl%MLI^{;uu1b9w3$(;X`FqCsGVj#BoDBa1ryIJSu2x*?rL>(J(zV(M zOQMt^N`AX}Eq{^oYu6MWuTYHnN}3HmX;u#0SF}@CLgL3m^v8lY*^Zf=sT4bTg1d(csuXa3+UJ zbIR01Q^%VUxeNcQ->%ZDya#8Xpan%iDEgUeO^8>!cV@pp+p^#1Z2&?nP^J7{#`jmK zM)*VriLC`SlXZ<1G+mf}V)?xr#D1|=Rped@rFP_HG-uB-prb)20c>$Dvrt^s4xIWx za;f%CCk;kPxlD;_PyMKQxmmp0wc%StenpBWI2$3MAG~|R3UdKgqnkU-4>&ACqzM9U zQ7uwVzQ5H@WycDo*c64b7_(4i=tcRs!V9S2w_O9xUV2duN}&XGU^9WFW(0QpS@%~v z)wK9KRGUPC5{ZPWDMV1Izkc(bP)ed89Wt>CdIB{e>M~S>sRpVfpP8Xowc$CS?0qL; z5k$~JU;8NccaIM7sKr1`mTIe1wkz9z?$L*SAduELnJC7me=Vr(KBxtW3emE`k5B*# z4gzY&y)&zQk?7QrdekF9{UXFpFWOuE*I#!g1RhM&l+Xgx5GzDpnIGdCGadlu$c{MX z_U7)3!<|~TJCw|q5OxdizZz{GBm{t!Q)?iq^~x+=ySEi6#OZ+N)t^X5FZV?S1*RuY zkI5vfBN!>^pg4vFas`h7bZbJEK{VUoE{?dIN~%w2H2+!#ihG!L%#&Qh<8o?&a=~dY z?izy461CU)+SEAnNc)Lysi#kT>ig6O>O!9LS8f68`66%e`4Zy~JC~3=-zO$=IvZ0d zZYCnh?kk%G5+uJ6d?1xFOXwf8Y;dpP2B3EScEqvCZvSJRZ~sOb&eQh0(+CRxsQ;}GTXr4;zXuJDb0OOk}erCONU zLgKKqtJucqc=_OZ+$3%c#$QF=|K>&3bVM%^g=7)u(Cs2}C*DK4AgF!zI-{Arz~JZQ z{p$8M9vc!JL0!EAI_nfQFiXXEgM&qF%$eSnN}u7ObiX&d7(lv#Iq|_k9Q?`1O?Wxo z(3OEqu=u?}Jc^=E0MCOTnzIa5RRfIVLUNxR^|0z>0_nb@b~4qI3SqU-<&V>IG?4m( zK`@6o;5Jg1`gY(oO9O?X-qt6Y?SIz-nO^~NzIM(*HY33)E}@0=uDn=v<91zEn$F10 z9gt=xJYkd)uQj<7!j5)!b~I2QJC5N(PJSlPT&f3rNL(t2$AjPmjgy&K94QQ(ND`Mb z5Hy`IGc;obfRkT%aDb_19n+X^478w?B|x^xcJE86Us7a6sfRVpzJHceb{*(N;Xa9V zP!a6rKZ{^3?SQilg2=wcoY%E#NmgA%H9@g{U6xD+vLqE!IHBC)qpl1uNb?7t1Q1LE z5S)HWOmfmj(#BeiI0oe4?8}f|sD9Yb!6=$D4h)Sf!E0dj4e2s|ueL=Iof_p(B|0E} z3_d(~feMOyh>9V|IpIU{9eI&G0i923p|xkusTaGVS+&Ee4o1KXJE1P6`c(!^)fY-8 zOQfUn_E=w=^37PA^}z%{aYqMKdz>LK4vrm7i3v7C%$%TNhlNo|;JI?_HN0 zvs@=EzT9_sAnAcZSZv4N3t?7ST^U7*=Gsjug>UIN62k}hJ7qBUY(O!W`vcQY`+L(C zO|SlO^$TSS8%UayUqb>oenml$b+%N}rlD#V>Z$ebWE^5nOavUKYvRe?RIgvj1{Kal zDj*h9dMh8k!*Kfh9WYjFAgQ1gx=DFMhG!d^Q@#X-{NnrZd@zk4 zI=RyxI49`CBoe0jcfGceif;2oRQ)J}$a2B_Qs<5NyDecVRLGY1dE8*339hrnUm+k+ zx%}VNvdYCvQuSzfsN!DxhulUeTvYKHT>kDt>*pZrFE#Ex?%BT@#D)ZyfBtfw0N0mH z-B3!I*!mUF_n&z2d|u{w#^q0ib8#DWh@vI_o_B{=Fv8Ls<9Cm^plQxzcY+PDgRJ_l zU^XTPGn;jOpZ9rvkL4s$0zzm{{8^}g1z4?{r?@E=@xvhP&tjIT-tu44+*U_7BnXK~ zI^c9jVk~ka!U(YW7cK6MkgY%WFG21N`MonZK@@f!tgVf2&jlZ~3Xm+>xAE>3Rrnl{ zl_7o9{XV@jCmIshbb!H*8Y7`1MDh8{rIV6op016*V6OFA)uH04FBcqpV*m?)T}>Jc z4wZZW2KNTuCZ2x<55#Aub#?ElWsXpFY&Y7)ze$mi9joRK5bARKBJflN1HP?ZlCwSV z&r%k|8e;V_l~Vw}47EgSkWwbh!nI9kd8}i+u#u|J8jX)r1 zBdLCk_#q;S^X)zPQ@1GSpO^&&MMRKD2MwOt(!x!czhN2Hh@yhhJ~ zl$EvateAT|(~%X)4+ZwZDCTHjCLxoKt!>jcnq^vfNb80MUQFLANUs|34glLeSaUEi z5hEjE>YVRd>Gb&`AodB7p&XHsW1P5O0*pn~JELlwBLzKf5gkEQx7_7vouVpj1JYz476&qmMHvwLv_43CBh1}A#`uFnoq zEF?>C7%ttAc9{syv?&Uc6BJF?m{1TV)qxieKWA95q>UP`^?e?gP|qbxZiD*AljGo& zC}hoCQ#d87Z9TmL9N7uQQu|C}n>(uOnW~s6$lJrv8D2I2>uom0ws>$-pz^1{J*dKn zynr`L)S<9NGtBDkz4zW1M*yM8XO+{P@$9&eZOa#B>nEVxhHEueN*`~r9$$Cw8O*UA zK~0Mv3Vi>WgKxQR>fMm#0*vf-{U$DCJ5(xY^vQI>fb7BB$MO%%gbwHczfC~vF^I4N z1E0Cx>Lm(riDU=0Y12z$gp*^^hIF5vl0foSX&1^I%05ptL)QgOX|~fHzc=3*I#%$~ z3moOl&<|My!~Th$2AE0xjdOs$s5;ej0M)ApLU_dGeor3zl815lxyYFNdbT6b$yQRD z6M|K|9VE)5)nTg(FBWR1oNp4;buE$(%ifaF3BG<&R&@Zw=MDnfDUE^$3fUUh)}d^C zdBBGi59)V2G4h&J-$>HAhHLmQCq@EhS=gB&PaMO6!W{!-ya3=kkD_aWTCG?+nzxvV zgjhKhoU)k#2eOl1Fq^s(z_C6BmpSd8Q0*;`oe-4FkjcmO6@y`mtx*$xg!Tt}jo9(G zI~IE~REV*@AkhOt)8h}0 z1Fv2;nf<$RPFBA1Kc%N={#R3h=f5}r<*{h9uO!K2(TOOT&!ndj{^!9NvXH;l7yY(% z1AOtn{T0RcKk=gb!gy1Yq5CF~MO^&1rT*|w_r|~X8x<`bjq;d}kf97H0W|#qVBEH8 zvazBVJMy@5m|@6l2ein1UwL5d3kK!r;1U00RxSYDr>`$sNb|^^$n(Ez>Zs(Ug=@Gf z7rt^4(l_uxli=KS37u9XolvcI9@4{tJ~?a{)FU7Ex2YjuDfdTjy}srcC>z$&6RC1;e)$_p77{K}zSIsqM?$7+S_2;~(YN4vO^OER!3oKrJz#U_Bh|Jm4!(Xx}kwCKgJ zR?vs9~zob6}E>%8<9R8ehQpU1`B z4>jC7eq@#Upj$BMe?6`Vjf6@A`iuM}3;wum34A0aZ$fIl<>mal8dcR)|5!)hHT>7s z?2^3)TqC9Yy^LYr>@c+}&y>6HSLa+j7~(JkZES52-ImV;B~KyDU?VgQC$gm+)!D#U zZYrmawQY*%h2Kkn8on8Nm=f0F^6VV&I5)N zO3nhxRjzMWn#Kuj?349DdkfJoa8_ZQWDXj2<+DcVpI9AF!~zeGH~6WMsLTIo7l*NQ zMBYngcZ;0;K5qc14{>(TG_Jq@V3A+T zW789hbFqn4PJ!9Zd>)s32bplf#ip!2`;Lp)#p)Xy?zOfvM5*UvTy;EsT?q4{a-Dc6Yq2jTQX^Z)<= literal 0 HcmV?d00001 diff --git a/package-structure-code/intro.md b/package-structure-code/intro.md index 5c3833fce..1481ac311 100644 --- a/package-structure-code/intro.md +++ b/package-structure-code/intro.md @@ -14,17 +14,20 @@ In this guide, we have made decisions around suggested standards and required standards, based upon the commonly used approaches in the scientific Python community. Our goal is to help standardize packaging across this ecosystem. -In some cases: +In some cases the suggestions here may diverge from those in the non-scientific parts of the Python ecosystem. -* The suggestions here may diverse from those in the non-scientific parts of the Python ecosystem. -* The suggestions here are made with the intent of being helpful, but are not specific requirements for your package to be reviewed and accepted into our ecosystem! +```{note} +The suggestions for package layout in this section are made with the +intent of being helpful; they are not specific requirements for your +package to be reviewed and accepted into our ecosystem. +``` In all cases, we try to align our suggestions with the most current, accepted [PEP's (Python Enhancement Protocols)](https://peps.python.org/pep-0000/) and the [scientific-python community specs](https://scientific-python.org/specs/). ```{note} Have a look at the -bare-minimum [editor checks.](open-source-software-submissions/editor-in-chief-guide.html#editor-checklist-copy-template-below-to-use-in-the-issue) that pyOpenSci +bare-minimum [editor checks](https://www.pyopensci.org/peer-review-guide/software-peer-review-guide/editor-in-chief-guide.html#editor-checklist-template) that pyOpenSci performs before a review begins. These checks are useful to explore for both authors planning to submit a package to us for review and for anyone who is just getting started with creating a Python package. diff --git a/package-structure-code/python-package-build-tools.md b/package-structure-code/python-package-build-tools.md index 483e1e128..910d1731e 100644 --- a/package-structure-code/python-package-build-tools.md +++ b/package-structure-code/python-package-build-tools.md @@ -1,5 +1,11 @@ # Python Package Build Tools +Below, we discuss some of the tools that are commonly used +to build Python packages. This page is intended to help +maintainers select a build tool to use. + + + ## Python package distribution files +Before we dive into specific build tools, it's important +to review the pieces of a "built" Python package. + +### Build +To understand the two distributions below, it is important to understand two different types of files: + +**Source files:** source files are the unbuilt files needed to build your package. An example of a build step would be compiling uncompiled code. +**Binary files:** binary files are the files needed to install your package. These files are pre-build. As such any code that need to be compiled has been compiled / built in a binary distribution. + There are two types of distribution files that you will create to support -publishing your Python package: +publishing your Python package (on PyPI): + +1. SDIST and +1. Wheel + + + +```{note} +If your package is a pure python package with no additional +build / compilation steps then the SDIST and Wheel files will have the same content. +``` + -* SDist additional -* a wheel ### What is a SDist file? @@ -37,15 +66,12 @@ When you make a release on GitHub, it creates a SDist file (`.tar.gz`) that anyone can download and use. ``` - + +## Hatch + +* Build Backend: **hatchling** +* Version control based versioning: Yes - **hatch-vcs** +* Push to PyPI: + +[**Hatch**](https://hatch.pypa.io/latest/) can be compared to **Flit** in that it simplifies your +package's build workflow using consistent commands. However, it adds: + +* A fully customizable build back-end +* Full flexibility for each step of the build process. +* Matrix environment creation to support testing across Python versions +* [Nox / MAKEFILE like functionality](https://hatch.pypa.io/latest/environment/#selection) +where you can create workflows in the pyproject.toml configuration to do things +like serve docs locally and cleaning your package build directory. + +Hatch does about everything that you might need to support +package development. It is ideal for packages that have compiled code and thus +require custom build steps. + +While hatch does have some "opinions" about how parts +of the packaging build run, you can customize any aspect +of it's build making if fully flexible. + +## PDM + +[PDM is a Python packaging and dependency management tool.](https://pdm.fming.dev/latest/). +It is designed to support pure python projects. + +Benefits of using PDM: -many thanks to the resources here: https://henryiii.github.io/level-up-your-python/notebooks/2.7%20Creating%20Packages.html# +* Dependency management +HELP! Can someone help me understand why you'd pick pdm please? my guess if is you want dependency management diff --git a/package-structure-code/python-package-structure.md b/package-structure-code/python-package-structure.md index 8e7305b7d..d1af26fa1 100644 --- a/package-structure-code/python-package-structure.md +++ b/package-structure-code/python-package-structure.md @@ -1,18 +1,29 @@ # Python Package Structure for Scientific Python Projects There are two different general layouts that you will commonly see -within the Python packaging ecosystem: [src and flat layouts (click here to lear more).](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/). While we believe that both layouts have advantages, -and we know that the Python packaging authority does advocate for the [src/ layout](https://py-pkgs.org/04-package-structure), we suggest that scientific packages adhere to the **flat-layout** given: +within the Python packaging ecosystem: +[src and flat layouts (click here to learn more).](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/) +We believe that both layouts have advantages. +The Python packaging authority advocates for the [**src/** layout](https://py-pkgs.org/04-package-structure), +in their tutorials. + +```{important} +pyOpenSci however will never require a specific package structure for its +peer review process. The overview on this page presents recommendations. +``` + +Below you will learn about the pros and cons of both layouts. + +Currently most scientific packages use the **flat-layout** given: -* All of the core scientific Python packages use it (see list below) * It's the most commonly found layout with the scientific Python ecosystem * Many Python tools depend upon tools in other language and / or complex builds with compilation steps. Many developers thus appreciate features of the flat layout such as tests being included with the package when they are installed) to support troubleshooting across installations. -```{note} -Advocating for the flat-layout is the only recommendation where we diverge some -from the Python Packaging -authority in an effort to support the broader scientific Python community. This -effort is core to the pyOpenSci mission. -``` + In the end, the advantages of using the **src/** layout for scientific packages + that already use this approach do not + outweigh the maintenance cost that a large cultural change in + package structure would bring to existing maintainers the scientific + Python ecosystem. + ```{tip} @@ -42,7 +53,7 @@ The flat layout's primary characteristics are: Below you can see the recommended structure of a scientific Python package using the flat layout. -``` +```bash myPackage/ β”œβ”€β”€ CHANGELOG.md ┐ β”œβ”€β”€ CODE_OF_CONDUCT.md β”‚ @@ -63,20 +74,83 @@ myPackage/ ```{note} -If you look at [the `matplotlib` repository on GitHub](https://github.com/matplotlib/matplotlib), you will notice it too has a src/ directory, however +If you look at +[the `matplotlib` repository on GitHub](https://github.com/matplotlib/matplotlib), +you will notice it too has a src/ directory, however that structure is there because `matplotlib` has uncompiled source code in that directory. it is not adhering to the **src/** layout explicitly. ``` ### Benefits of using the flat layout in your Python package -There are numerous benefits to the scientific community in using this layout. +There are some benefits to the scientific community in using the flat layout. + +* This structure has historically been used across the ecosystem and packages +using it are unlikely to change. Thus, you'd be following a convention that many +packages use already. +* This structure is simpler to setup to support packaging compared to the `src/` +approach given the package source code is in the root directory. This means that +your package will be "found" by build back-ends automagically. +* You can directly install the package directly from the root directory. +* Tests are shipped with your package. This allows developers to invoke tests on +various machines to troubleshoot installations if needed. (This could easily +be implemented using a `src/` layout too!) + +### The `src/` layout for Python packages + +The `src/` layout is another option to structure you Python package. This layout +is advocated for by the [PyPA](https://packaging.python.org/en/latest/tutorials/packaging-projects/) +The key characteristic of this layout is that your package +uses a `src/package-name` directory structure. Tests are +often located in a directory outside of the package (but they don't have to be). + +#### Pros of the src/ layout + +The benefits of the `src/` layout approach include: + +* It ensures that tests are always running on the installed version of your +package rather than on the flat files +* If `tests/` are outside of the `src/` directory, they aren't delivered to the user +installing your package. This makes the package size slightly smaller. +* This layout is semantically more clear. Code is always found in the +`src/package-name` directory, `tests/` are in the root and with docs/ + +#### Cons of the src/ layout +* Can be slightly trickier to configure. examples: + * Tools like build won't automatically find the package directory if it's + within `src/package-name` without being configured. +* Setting up continuous integration is more complex as you will have to account +for the `src/` directory when installing the package +* While this layout is common in the broader Python ecosystem, it does not +necessarily support the needs of the scientific Python ecosystem which often has +tools that wrap around other compiled languages such as C++. Often times code +that needs to be compiled is stored in `src/`. + +An example of the `src/` layout structure can be seen below: + +``` +myPackage +β”œβ”€β”€ CHANGELOG.md ┐ +β”œβ”€β”€ CODE_OF_CONDUCT.md β”‚ +β”œβ”€β”€ CONTRIBUTING.md β”‚ +β”œβ”€β”€ docs β”‚ Package documentation +β”‚ └── index.md +β”‚ └── ... β”‚ +β”œβ”€β”€ LICENSE β”‚ +β”œβ”€β”€ README.md β”˜ +β”œβ”€β”€ pyproject.toml ┐ +β”œβ”€β”€ src β”‚ +β”‚ └── myPackage β”‚ Package source code, metadata, +β”‚ β”œβ”€β”€ __init__.py β”‚ and build instructions +β”‚ β”œβ”€β”€ moduleA.py β”‚ +β”‚ └── moduleB.py β”˜ +└── tests ┐ + └── ... β”˜ Package tests +``` -* This structure has historically been used across the ecosystem and packages using it are unlikely to change. Thus, you'd be following a convention that many packages use already. -* This structure is simpler to setup to support packaging compared to the `src/` approach given the package source code is in the root directory. -* You can directly install the package directly from the root directory -* Tests are shipped with your package. This allows developers to invoke tests on various machines to troubleshoot installations if needed. + + To install your package in editable mode use: ```bash @@ -84,7 +158,6 @@ $ cd package-name $ pip install -e .` ``` - ## Core file requirements for a Python package In the above example, notice that all of the core documentation files that @@ -104,7 +177,7 @@ documentation website content lives. ```{button-link} https://www.pyopensci.org/python-package guide/documentation :color: primary :class: sd-rounded-pill float-left -Click here to read about our packaging documentation requirements. +Click here to read about our packaging documentation requirements. <> ``` Finally, notice that the **tests/** directory containing your test suite is @@ -136,18 +209,28 @@ ecosystem. Many of our core scientific Python packages wrap around other compiled languages, such as C++. They thus have source code that requires compilation. It would be challenging and potentially time-consuming for all of those packages to modify their current structure. And further it would not provide them with any core benefit. -pyOpenSci however will never require a specific package structure for its -peer review process. The overview on this page presents recommendations. + ``` --> ## Use a pyproject.toml file for your package configuration & metadata -We recommend that you follow current recommendations and [include all project based metadata, and build system specifications in a `pyproject.toml` file.](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/). +We strongly recommend that you [include all project based metadata and build system specifications in a `pyproject.toml` file.](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/) [Using setup.py to manage both package setup and hold metadata can present numerous risks.](https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html) + + +```{note} +Moving away from the **setup.py** file provides two benefits: + +1. Because setup.py has a mixture of code and metadata, it opens the user to a potential bad code injection on their computer when the package is installed. +1. Including your package's metadata in a separate human-readable `pyproject.toml` format also allows someone to view the project's metadata without +running any Python code. +``` -The [TOML (Tom's Obvious, Minimal Language) format](https://toml.io/en/), is an easy-to-read structure that is founded on key / value sections. +The [TOML (Tom's Obvious, Minimal Language) format](https://toml.io/en/), is an easy-to-read structure that is founded on key: value pairs. -Each section in the file contains a `[key]` and below it values associated with that section. +Each section in the file contains a `[table identifier]`. +Below that table identifier are key value pairs that +support configuration for that particular table. ```{note} [PEP518 describes the move away from setup.py to the pyproject.toml file.](https://peps.python.org/pep-0518/) @@ -155,8 +238,8 @@ Python package standards are moving away from including both package metadata and python code needed to setup a package installation in the same **setup.py** file. Instead we are moving towards using a **proproject.toml** file sometimes combined with a **setup.cfg** file. -In some cases, -where a build is particularly complex, a setup.py file may still be required. +In some cases +where a build is particularly complex, a **setup.py** file may still be required. ``` ## Example pyproject.toml @@ -169,6 +252,12 @@ requires: * **setuptools build** to "build" the package * **setuptools_scm** to manage package version updates +In the example below `[build-system]` is the first table +of values. It has two keys that specify the build front end and backend for a package: + +1. `requires =` +1. `build-backend =` + ``` [build-system] requires = ["setuptools>=45", "wheel", "setuptools_scm[toml]>=6.2"] @@ -184,17 +273,22 @@ license = {text = "BSD 3-Clause"} description = "An example Python package used to support Python packaging tutorials" keywords = ["pyOpenSci", "python packaging"] readme = "README.md" + +dependencies = [ + "dependency-package-name-1", + "dependency-package-name-2", +] ``` -Above also notice that all of your package's metadata can be stored in the -**pyproject.toml** file (rather than the setup.cfg file or the setup.py file). -You also can specify dependencies in this file. + +Notice that you also can specify dependencies in this file. -We discuss build tools for python package more here. +turn this into button: +We discuss build tools for python package more here. +--> ```{tip}