dx: Add pre-commit as a central linter, formatter and checker

[cdecker]

We have quite the variety of different formatters, checkers and linters:

• Python formatting, code style and linting
• shellcheck run via make
• check-amount-access grepping for access to the u64 inside of the amount_msat and amount_sat structs.
• JSON-Schema formatting
• discouraged-fundctions
• check-gen-updated
• check-includes
  • check-src-includes
  • check-hdr-includes
• check-hsm-versions
• check-manpages
• check-spelling
• check-whitespace

For many of these we have custom scripts that require us to maintain
them as well, despite there being off-the-shelf tooling that allows us
to check and enfore these.

pre-commit standardizes the execution of these checkers, running
them against the proposed changes in a PR. It allows for checking and
fixing as separate steps, so CI can just check, and show the diff of
changes that would have been applied.

The --from-ref and --to-ref params help us to limit the scope of
the checks to the proposed changes, allowing us to incrementally adopt
these standards as we visit the files.

If you adopt a new file, i.e., modify a file that has not been checked
before, you might want to just quickly run pre-commit run --from-ref=master --to-ref=HEAD after the first modification, and fix
up any lints or failing checks before moving on to actually
implementing your changes. This keeps the lint cleanup separate from
logic changes that are to be reviewed.

[s373nZ]

I see flake8 checks in the Makefile as well. Is the intention here as well to replace flake8 with ruff?

[cdecker]

Whoops? What is flake8 doing with the Makefile? Presumably it fails to parse it as python?

[s373nZ]

Whoops? What is flake8 doing with the Makefile? Presumably it fails to parse it as python?

I should have phrased my comment more clearly. I see the Makefile is utilizing flake8 to run checks on the Python code using the check-python-flake8 target. My question is if the intention is to replace flake8 checks with ruff or if we want to add the flake8 existing checks in an additional hook.

[cdecker]

Oh, I see. The intention is to use one l'Inter per language. Otherwise we might get into a territory where they disagree causing unresolvable lint warnings.

Let's remove flake8 altogether.

[s373nZ]

For others reviewing this PR, here is another demo PR against the original branch containing some initial precommit experiments: cdecker#7

[rustyrussell]

Good start! Let's merge this so we can build on it!