"pkg_comp auto" fails: pkg_add: A different version of ... is already installed

[jlmuir]

I ran the pkg_comp auto command on macOS Sierra 10.12.5:

# pkg_comp auto

and the pbulk build failed on the very first package (cwrappers):

Resolving...
Building...
Initialisation complete.
[1/417] Starting build of  cwrappers-20170611
[1/417] Failed to build    cwrappers-20170611
Building pkg_summary...
Building SHA512...
Build reports...
Sending report mail...
Uploading report...
Uploading packages...
pkg_comp: I: Destroying sandbox

The pbulk log shows that it built the package successfully, but the problem was in installing it; the following appears in bulklog/cwrappers-20170611/package.log:

=> Creating binary package /pkg_comp/work/pkg/pkgtools/cwrappers/work/.packages/cwrappers-20170611.tgz
pkg_add: A different version of cwrappers-20170611 is already installed: cwrappers-20170112
pkg_add: 1 package addition failed

[jlmuir]

Dropping into a sandbox shell has the same behavior when I do bmake install in pkgtools/cwrappers:

===> Building binary package for cwrappers-20170611
=> Creating binary package /pkg_comp/packages/pkg/All/cwrappers-20170611.tgz
===> Installing binary package of cwrappers-20170611
pkg_add: A different version of cwrappers-20170611 is already installed: cwrappers-20170112
pkg_add: 1 package addition failed
*** Error code 1

Stop.
bmake[2]: stopped in /pkg_comp/pkgsrc/pkgtools/cwrappers
*** Error code 1

Stop.
bmake[1]: stopped in /pkg_comp/pkgsrc/pkgtools/cwrappers
*** Error code 1

Stop.
bmake: stopped in /pkg_comp/pkgsrc/pkgtools/cwrappers

I also tried pkg_admin rebuild followed by bmake install, but the result is the same.

[jlmuir]

Hmm, in a sandbox shell, bmake update in pkgtools/cwrappers works:

# bmake update
===> Skipping vulnerability checks.
WARNING: No /opt/pkg/libdata/pkgdb/pkg-vulnerabilities file found.
WARNING: To fix run: `/opt/pkg/sbin/pkg_admin -K /opt/pkg/libdata/pkgdb fetch-pkg-vulnerabilities'.
===> Deinstalling for cwrappers-20170611
Running /opt/pkg/sbin/pkg_delete -K /opt/pkg/libdata/pkgdb -r cwrappers-20170112
===> Binary install for cwrappers-20170611
=> Installing cwrappers-20170611 from /pkg_comp/packages/pkg/All
cwrappers-20170611 successfully installed.
===> Cleaning for cwrappers-20170611

I'm not sure if there's a way to get pbulk to do this, though, or if there's a better way....

[jlmuir]

The issue seems to be that I need a new bootstrap that includes the updated cwrappers-20170611 package. I asked on the pkgsrc-users mailing list, and it was confirmed that the bootstrap must contain packages at versions that are the same as (or newer than?) the versions that would be built by pbulk.

This has implications for the pkg_comp-2.0-version-macos.pkg installer since it would need to include a bootstrap appropriate for what pkgsrc branch was to be built which is hard since you can't know which branch the user intends to build. I guess you could provide ones for a few recent quarterly branches and for -current, but that seems cumbersome.

In Jonathan Perkin's reply to my question on the pkgsrc-users mailing list, he said he builds a new bootstrap each time he runs his daily bulk builds. Maybe it would be good for pkg_comp to grow a new build-bootstrap (or create-, generate-, or new-bootstrap, or whatever) command that will create a new bootstrap? And then that command should be included in the set of commands run by the auto command so that it builds a new bootstrap before each build?

[jlmuir]

I looked at pkg_comp.sh and saw that it will create the bootstrap if it doesn't exist! So, I just blew away the bootstrap, and now it works!

# rm -rf /var/pkg_comp/{packages,pbulk-packages}

I didn't try it, but in theory I would only need to remove /var/pkg_comp/packages, right? The pbulk-packages tree is for the pbulk environment, and as long as I don't need a new version of pbulk or something, it should continue to work?

Anyway, it would probably be good to document this bootstrap behavior. The pkg_comp(8) man page does say the following in the bootstrap command section:

Any existing binaries within the PBULK_PACKAGES tree will be reused on subsequent bootstrap operations if found. If you want to start builds afresh (and you should if you experience strange build failures), you will have to remove the previous binary packages.

But I read that as referring to the binary packages, not the bootstrap.tgz file. Also, that paragraph refers to the PBULK_PACKAGES tree, but there is also the PACKAGES tree which is referenced in the paragraph just above it, and I think the PACKAGES tree is actually the more important of the two for overcoming build issues, right?

If you agree about a documentation change being good and already know what you'd write, great. But if it would be a help to add an FAQ with an entry for this, or if it would be a help to suggest a change to the pkg_comp(8) man page, I could try to come up with something.