New version of the ImageJ Updater

[ctrueden]

Work has begun on a new version of the ImageJ Updater. This issue is a central place where plan and progress can be tracked and discussed.

Requirements and wishes

• Secure (HTTPS) delivery of updates, including the files themselves, as well as the central list of update sites.
• Isolate future breaking changes to the update mechanism from old installations. It needs to be possible to terminate update sequences that work via a particular mechanism, with subsequent updates served via a different mechanism.
• Ability to downgrade an ImageJ installation to a particular point in time from the past.
• Ability to repair a broken ImageJ installation by launching the Updater in standalone/repair mode; ideally, we would detect when the installation is broken upon startup, and automatically launch the repair tool.
• A nicer user interface for the graphical Updater.

Design decisions

• Versioned update sites. Each major version of an update site lives at its own URL prefix (e.g. https://update.imagej.net/v2/). Each major version of the updater communicates with that prefix. If a new major version of the updater is created, the final version of the Updater served from .../v2 will know how to switch to the .../v3 site, but will not contain other enhancements and changes that come with the new V3 Updater. As such, multiple restarts of ImageJ might be necessary to fully update across multiple major updater versions, but the migration path will be there. Not yet clear whether the SemVer major digit of imagej-updater will correspond exactly to these update site major versions; my tentative vote would be no.

• The list of update sites now lives on GitHub Pages, to ensure all changes are vetted by reviewers via GitHub's Pull Request mechanism. The list of update sites has become a YAML document. See this thread from forum.image.sc for further details.

• The Updater changes will be tested against local dev versions of the core update sites from update.imagej.net, update.fiji.sc and sites.imagej.net. Note that there is an rsync daemon running on the server that serves the production version of these sites; try rsync -n imagej.net:: to verify this.

Terminology

• The currently existing version of the Updater codebase will henceforth be called the "V0 Updater."

• The next-generation version of the Updater that knows how to access and interpret the forthcoming https://imagej.github.io/list-of-update-sites/v2/sites.yml as well as v2 update sites will henceforth be called the "V2 Updater."

• The stopgap version of the Updater based on the current codebase, but knowing how to prime an ImageJ installation for switch-over to the V2 update mechanism will henceforth be called the "V1 Updater."

Remaining action items for V1 and V2 Updater work

• @frauzufall recalls having an issue with uploading via WebDAV over HTTPS. We need to test whether this works.

• http: is hardcoded in imagej-updater in a couple of places. This must be changed to accept either http or https.

• The updater does not follow redirects in all cases. See http to https redirect leads to `Not in GZIP format' #65.

• V1: If user's java is too old, user needs to be warned up front, and http: used as a fallback. The idea is to try connecting to https://imagej.net/api.php?... to access the list of update sites, and if that fails with a certificate error, fall back to http://imagej.net/api.php?... instead.

• V2: The fallback support for insecure HTTP will be discontinued.

• Double check the status of the current "remote updater" mechanism, and decide what to do about it.

[frauzufall]

So I merged #67 (following redirects) into #71 (V1, not pushed, only for testing) and @carandraug's update site worked well, but the ImageJ server HTTP links will not be redirected. I think @ctrueden made that happen on purpose (in case the updater is requesting the HTTP update sites from the server) for backwards compatibility.

So I am not sure how we will serve the HTTPS content to people with existing installations that can handle the certificate without updating the URLs in their local list of available update sites (#66). One awkward idea would be to attach a query string containing the information that HTTPS can be handled and using this to make the server return one or the other..