Rust maintenance docs

docs/conf.py

@@ -280,7 +280,7 @@
 ]
 
 # Excludes files or directories from processing
-# exclude_patterns = []
+exclude_patterns = ["maintainers/NPM/common"]
 
 
 # Adds custom CSS files, located under 'html_static_path'

docs/maintainers/NPM/common/empty-directories-notice.md

@@ -0,0 +1,3 @@
+:::{attention}
+Some empty directories won't be included in the Git commit. This is a [known issue](https://pad.lv/1917877) not unique to `rustc`. Unfortunately, this means that you'll have to re-extract and overlay every time you clone the Git repo to a new place, run `git clean`, switch to a branch without that vendored dependency, etc.
+:::

docs/maintainers/NPM/common/lintian-checks.md

@@ -0,0 +1,46 @@
+### Lintian Checks
+
+{term}`Lintian` ({manpage}`lintian(1)`) checks your source package for bugs and [Debian policy](https://www.debian.org/doc/debian-policy/) violations.
+
+(updating-rust-lintian-prep)=
+
+[Clean up previous build artifacts](updating-rust-clean-build) then build the source package:
+
+(updating-rust-build-source-package)=
+
+```none
+$ dpkg-buildpackage -S -I -i -nc -d -sa
+```
+
+First, check the Lintian output with just the warnings and errors:
+
+```none
+lintian -i --tag-display-limit 0 2>&1 | tee <path_to_log_file>
+```
+
+#### Addressing warnings and errors
+
+You must address all of these in one way or another. They must either be fixed or added to `debian/source/lintian-overrides{,.in}`, with a few notable exceptions:
+
+- `E: rustc-1.86 source: field-too-long Vendored-Sources-Rust`
+  - This is simply the length of the field. While we _would_ like to change this in the future in `dh-cargo`, there's nothing that can (or should) be done about this for now.
+- `E: rustc-1.86 source: unknown-file-in-debian-source [debian/source/lintian-overrides.in]`
+  - This is just the file used to generate the Lintian overrides for a given Rust version. It's completely harmless to have in the source tree.
+- `E: rustc-1.86 source: version-substvar-for-external-package Depends ${binary:Version} cargo-<X.Y> -> rustc [debian/control:*]`
+  - This is just a fallback for a non-versioned `rustc` package. While it's unlikely to ever be used, it's not a typo, so you don't need to worry about it.
+- `W: rustc-1.86 source: unknown-field Vendored-Sources-Rust`
+  - This is a custom field, not a typo.
+
+As for any other warnings or errors, you must figure out whether the lint should be ignored or remedied. Don't be afraid to ask for help from more experienced package maintainers, or consult the existing Lintian overrides for precedence.
+
+#### Extra lints
+
+Now you can run Lintian with all the pedantic, experimental, and informational lints enabled. It isn't typically necessary to fix most of the extra lints, but it's a good idea to check everything and see if there are some ways to improve the package based on these lints.
+
+```none
+lintian -i -I -E --pedantic
+```
+
+:::{important}
+Don't forget to [clean and rebuild the source package](updating-rust-lintian-prep) before re-running Lintian, otherwise your changes to the package won't apply!
+:::

docs/maintainers/NPM/common/local-build.md

@@ -0,0 +1,77 @@
+### Local Build and Bugfixing
+
+You're now ready to try to build `rustc` using {manpage}`sbuild(1)`.
+
+First, make sure that all previous build artifacts have been cleaned from your upper-level directory:
+
+(updating-rust-clean-build)=
+
+```none
+$ rm -vf ../*.{debian.tar.xz,dsc,buildinfo,changes,ppa.upload}
+$ rm -vf debian/files
+$ rm -rf .pc
+```
+
+Then, run the build! Depending on your computer, a full build tends to take about 1-3 hours.
+
+```none
+$ sbuild -Ad <release>
+```
+
+#### Using another PPA to bootstrap
+
+Not all `rustc` releases are necessarily in the archive. Perhaps you're waiting on a previous version to be upload, or you're creating a backport which isn't needed by the subsequent Ubuntu release.
+
+If this applies to you, you must add your PPA as an extra repository to your `sbuild` command:
+
+```none
+$ sbuild -Ad <release> \
+    --extra-repository="deb [trusted=yes] http://ppa.launchpadcontent.net/<lpuser>/<ppa_name>/ubuntu/ <release> main"
+```
+
+#### Fixing bugs
+
+If the build fails, it's up to you to figure out why. This will require problem-solving skills and attention to detail.
+
+First, try to find any upstream issues related to the problem on the [Rust GitHub page](https://github.com/rust-lang/rust/issues?q=is%3Aissue). It's quite common for non-packaging-related problems to be already known upstream, and you can often find a patch from there.
+
+#### Searching for failing tests within the build log
+
+`sbuild` saves the build logs to your computer. You can easily jump to the standard output of each failing test by searching for the following within the log:
+
+```none
+stdout ----
+```
+
+#### Running individual tests
+
+If the build fails, then `sbuild` will place you in an interactive shell for debugging. This is extremely useful, as you can change the source code and retry tests without rebuilding the whole thing.
+
+For example, here's how to re-run all the bootstrap tests:
+
+```none
+$ debian/rules override_dh_auto_test-arch RUSTBUILD_TEST_FLAGS="src/bootstrap/"
+```
+
+Here's how to re-run _just_ the `alias_and_path_for_library` bootstrap test:
+
+```none
+$ debian/rules override_dh_auto_test-arch RUSTBUILD_TEST_FLAGS="src/bootstrap/ --test-args alias_and_path_for_library"
+```
+
+#### Proper patch header format
+
+In order to fix certain bugs, it's likely you'll need to create your own patch at some point. It's important that this patch contains enough information for other people to understand _what_ it's doing and _why_ it's doing it.
+
+First, ensure that [Debian](https://salsa.debian.org/rust-team/rust/-/tree/debian/experimental?ref_type=heads) has not already created an equivalent patch. If so, you can simply use their patch directly. If you need to modify the patch in any way, make sure to add `Origin: backport, <Debian VCS patch URL>` to the patch header.
+
+Otherwise, you must create your own patch. A template {term}`DEP-3 <DEP 3>` header can be generated using the following command:
+
+```none
+$ quilt header -e --dep3 <path/to/patch>
+```
+
+For the most part, you can follow the [Debian DEP-3 patch guidelines](https://dep-team.pages.debian.net/deps/dep3/). However, there are a few extra things you must do:
+
+- Debian developers typically don't use the `This patch header follows DEP-3 [...]` line added by `quilt`. Delete this line.
+- If this patch isn't something needed to get the new Rust version to build, and you're instead updating an existing source package, add a `Bug-Ubuntu:` line linking to the Launchpad bug.

docs/maintainers/NPM/common/local-repo-setup.md

@@ -0,0 +1,42 @@
+## Setting up the Repository Locally
+
+This only needs to be done once when setting up a machine for Rust toolchain maintenance for the first time.
+
+### Project directory structure
+
+Since the Debian build tools generate files in the parent directory of your package source directory, it's recommended to keep things organized by placing the cloned repository inside of a fresh directory of its own.
+
+Clone the repository inside an existing `rustc` directory so your file structure looks like the following:
+
+```none
+rustc
+├── rustc
+│   ├── [...]
+│   ├── Cargo.lock
+│   ├── Cargo.toml
+│   ├── debian
+│   └── [...]
+└── rustc-<...>.orig.tar.xz
+```
+
+Naturally, your higher-level `rustc` directory won't have any {term}`.orig.tar.xz <orig tarball>` files yet, but they will be stored there once you start working on the package.
+
+### Cloning the Git repository
+
+The main repository for _all_ versioned Rust toolchain packages is the Foundations [Launchpad Git repository](https://git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc). A branch exists for every single upstream release and {term}`backport` and serves as a central place to store _all_ Rust toolchain code, regardless of which versioned package a particular branch belongs to.
+
+Clone the Foundations Git repository within your existing parent directory:
+
+```none
+$ git clone git+ssh://<lpuser>@git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc
+```
+
+Then, create your own personal Git repository on Launchpad:
+
+```none
+$ git remote add <lpuser> git+ssh://<lpuser>@git.launchpad.net/~<lpuser>/ubuntu/+source/rustc
+```
+
+Generally, it's recommended to use your personal Git repository as a remote backup throughout the process — the update procedure involves multiple rebases, so it's best to wait pushing to the Foundations repository until you're done.
+
+---

docs/maintainers/NPM/common/obtaining-upload-info.md

@@ -0,0 +1,25 @@
+It's recommended to create a local draft for your upload comment, so you can make sure everything is correct beforehand.
+
+You need both the [Lintian](updating-rust-lintian-checks) and the [autopkgtest](updating-rust-autopkgtests) results for your package.
+
+Retrieve links to all your passing autopkgtest build logs provided by the [autopkgtest command you used earlier](updating-rust-autopkgtests-url-command).
+
+Get the default Lintian output for the source package by [cleaning up all build artifacts](updating-rust-clean-build), using `dpkg-buildpackage` to [build the source package](updating-rust-build-source-package), then simply running the default Lintian command, redirecting the output to a file for later use:
+
+```none
+$ lintian > <path_to_saved_lintian_output_file>
+```
+
+Finally, you can push your `merge-<X.Y>` branch to the Foundations `rustc` Launchpad Git remote:
+
+```none
+$ git push <foundations> merge-<X.Y>
+```
+
+Compile the following information:
+
+- A link to your successfully-built PPA packages
+- A link to your `merge-<X.Y>` branch in the Foundations `rustc` Launchpad Git repository
+- A list of any notable packaging changes (_not_ upstream changes). These are changes _you_ made to the package, such as the removal or addition of patches. Nothing routine (such as patch refreshes or vendored dependency pruning) should be added here.
+- The output of `lintian` you just got
+- The links to the build logs of all PPA autopkgtests you just got

docs/maintainers/NPM/common/ppa-build.md

@@ -0,0 +1,86 @@
+### PPA Build
+
+Once everything builds on your local machine and Lintian is satisfied, it's time to test the package on all architectures by uploading it to a {term}`PPA`.
+
+#### Creating a new PPA
+
+If this is your first PPA upload for this Rust version, you must create a new PPA using the [`ppa-dev-tools` snap](https://snapcraft.io/ppa-dev-tools). The PPA name depends on whether you are updating Rust, backporting Rust, or patching Rust.
+
+New versioned Rust package:
+
+```none
+$ ppa create rustc-<X.Y>-merge
+```
+
+Rust backport:
+
+```none
+$ ppa create rustc-<X.Y>-release
+```
+
+Rust patch:
+
+```none
+$ ppa create rustc-<X.Y>-lp<lp_bug_number>
+```
+
+The command should return a URL leading to the PPA. You must go to that Launchpad URL and do two things:
+
+1. "Change Details" -> Enable all "Processors" (Make sure RISC-V is enabled!)
+1. "Edit PPA Dependencies" -> Set Ubuntu dependencies to "Proposed"
+
+If you are using another PPA to bootstrap, then you must explicitly add this PPA as a dependency in the "Edit PPA Dependencies" menu.
+
+#### PPA changelog entry
+
+Next, add a temporary changelog entry, appending `~ppa<N>` to your version number so the PPA version isn't used in favour of the actual version in the archive:
+
+:::{note}
+`<N>` is just the number of the upload. You may have to fix something and re-upload to this PPA, so you should use `~ppa1` for your first PPA upload, `~ppa2` for your second, etc.
+:::
+
+```none
+$ dch -bv <X.Y.Z>+dfsg0ubuntu1-0ubuntu1\~ppa<N> \
+    --distribution "<release>" \
+    "PPA upload"
+```
+
+#### Uploading the source package
+
+[Make sure that your source directory is clean](updating-rust-clean-build) (especially `debian/files`), then build the source package:
+
+```none
+$ dpkg-buildpackage -S -I -i -nc -d -sa
+```
+
+Finally, upload the newly-created source package:
+
+:::{note}
+You can get the `source-changes-file` script [here](https://github.com/canonical/foundations-sandbox/blob/master/maxgmr/source-changes-file).
+:::
+
+New versioned Rust package:
+
+```none
+$ dput ppa:<lpname>/rustc-<X.Y>-merge $(source-changes-file)
+```
+
+Rust backport:
+
+```none
+$ dput ppa:<lpname>/rustc-<X.Y>-<release> $(source-changes-file)
+```
+
+Rust patch:
+
+```none
+$ dput ppa:<lpname>/rustc-<X.Y>-lp<lp_bug_number> $(source-changes-file)
+```
+
+The PPA will then build the Rust package for all architectures supported by Ubuntu. These builds will highlight any architecture-specific build failures.
+
+#### Handling early PPA build failures
+
+Sometimes, a PPA build on a specific architecture will fail in under 15 minutes with no build log provided. If this happens, there was a Launchpad issue, and you can simply retry the build without consequence.
+
+If the build failed and there _is_ a build log provided, then there was indeed a build failure which you must address.

docs/maintainers/NPM/common/running-autopkgtests.md

@@ -0,0 +1,16 @@
+To run the {term}`autopkgtests <autopkgtest>` for real, run the following command provided by the [`ppa-dev-tools` snap](https://snapcraft.io/ppa-dev-tools) to get links to all the autopkgtests:
+
+(updating-rust-autopkgtests-url-command)=
+
+```none
+$ ppa tests \
+    ppa:<lpuser>/rustc-<X.Y>-merge \
+    --release <release> \
+    --show-url
+```
+
+Click all of the links except i386 to trigger the autopkgtests for each target architecture.
+
+Re-run the same `ppa tests ...` command to check the status of the autopkgtests themselves.
+
+The infrastructure can be a little flaky at times. If you get a "BAD" reponse (instead of a "PASS" or "FAIL"), then you just need to retry it.

docs/maintainers/NPM/common/substitution-terms.md

@@ -0,0 +1,21 @@
+## Substitution Terms
+
+From now on, the documentation will contain certain terms within angle brackets which must be replaced with the actual value that applies to your situation.
+
+As an example, let's assume you are updating `rustc-1.84` (upstream version `1.84.0`) to `rustc-1.85` (upstream version `1.85.1`) for Noble Numbat:
+
+- `<X.Y>`: The short Rust version you're updating _to_.
+  - Example: `1.85`
+- `<X.Y.Z>`: The long Rust version you're updating _to_.
+  - Example: `1.85.1`
+- `<X.Y_old>`: The short Rust version you're updating _from_.
+  - Example: `1.84`
+- `<X.Y.Z_old>`: The long Rust version you're updating _from_.
+  - Example: `1.84.0`
+- `<release>`: The target {term}`Ubuntu release` adjective.
+  - Example: `noble`
+- `<lpuser>`: Your {term}`Launchpad` username. This is also used to refer to your personal Launchpad Git {term}`repository's <repository>` remote name.
+- `<foundations>`: Your local Git remote name for the Foundations [`rustc` Git repository](https://git.launchpad.net/~canonical-foundations/ubuntu/+source/rustc).
+- `<lp_bug_number>`: The number of the {term}`Launchpad` {term}`bug` associated with this upload.
+
+---

docs/maintainers/NPM/common/toolchain-availability.md

@@ -0,0 +1 @@
+After uploading a new Rust version, you must also update the [Rust Toolchain Availability Page](https://documentation.ubuntu.com/ubuntu-for-developers/reference/availability/rust/) with the version you have added.

docs/maintainers/NPM/common/uscan.md

@@ -0,0 +1,13 @@
+We can use {manpage}`uscan(1)` (from the [`devscripts`](https://launchpad.net/ubuntu/+source/devscripts) package) to get the new source code and generate the new {term}`orig tarball`.
+
+The log of this should be saved somewhere because `uscan` will warn you if any files you've excluded from `debian/copyright` aren't actually in the original source. You must consult the upstream Rust changes and see what happened to that file, updating `debian/copyright` accordingly depending on if it was removed, renamed, or refactored.
+
+Download the orig tarball from the upstream Rust source, yanking out all excluded files:
+
+```none
+$ uscan --download-version <X.Y.Z> -v 2>&1 | tee <path_to_log_output>
+```
+
+This process can take a while. Once it is complete, you will find a file with an `.orig.tar.xz` suffix in your parent `rustc` directory. That is your orig tarball. It contains the new upstream source code for the new Rust version.
+
+You must then rename the orig tarball to match the first part of your package version number, i.e., `rustc-<X.Y>_<X.Y.Z>+dfsg0ubuntu0`.