Re: [PATCH v2 1/1] ci(osx): use new location of the `perforce` cask

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 6575 bytes --]

Hi Gábor,

On Fri, 18 Oct 2019, SZEDER Gábor wrote:

> On Thu, Oct 17, 2019 at 12:47:33PM +0000, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > The CI builds are failing for Mac OS X due to a change in the
>
> s/CI/Azure Pipelines/
>
> Our Travis CI builds are fine.

For the moment ;-)

> > location of the perforce cask. The command outputs the following
> > error:
> >
> >     + brew install caskroom/cask/perforce
> >     Error: caskroom/cask was moved. Tap homebrew/cask-cask instead.
> >
> > So let's try to call `brew cask install perforce` first (which is what
> > that error message suggests, in a most round-about way).
> >
> > The "caskroom" way was added in 672f51cb (travis-ci:
> > fix Perforce install on macOS, 2017-01-22) and the justification
> > is that the call "brew cask install perforce" can fail due to a checksum
> > mismatch: the recipe simply downloads the official Perforce distro, and
> > whenever that is updated, the recipe needs to be updated, too.
>
> This paragraph is wrong, it mixes up things too much.
>
> Prior to 672f51cb we used to install the 'perforce' _package_ with
> 'brew install perforce' (note: no 'cask' in there).  The justification
> for 672f51cb was that the command 'brew install perforce' simply
> stopped working, after Homebrew folks decided that it's better to move
> the 'perforce' package to a "cask".  It was _their_ justification for
> this move that 'brew install perforce' "can fail due to a checksum
> mismatch ...", and casks can be installed without checksum
> verification.  And indeed, both 'brew cask install perforce' and 'brew
> install caskroom/cask/perforce' printed something along the lines of:
>
>   ==> No checksum defined for Cask perforce, skipping verification
>
> It's unclear to me why 672f51cb used 'brew install
> caskroom/cask/perforce' instead of 'brew cask install perforce'.  It
> appears (by running both commands on old Travis CI macOS images) that
> both commands worked all the same already back then.
>
> Anyway, as the error message at the top of the log message shows,
> 'brew install caskroom/cask/perforce' has stopped working recently,
> but 'brew cask install perforce' still does, so let's use that.

If you don't mind, I am going to copy/edit these three paragraphs into
the commit message,

> Note that on Travis CI we explicitly specify which macOS image to use,
> and nowadays we don't run 'brew update' during the build process [1],
> so both commands work in our builds there.
>
> [1] f2f4715033 (ci: don't update Homebrew, 2019-07-03)
>
> > CI servers are typically fresh virtual machines, but not always. To
> > accommodate for that, let's try harder if `brew cask install perforce`
> > fails, by specifically pulling the latest `master` of the
> > `homebrew-cask` repository.
>
> Homebrew didn't record a checksum for Perforce versions r17.1, r17.2
> and r18.1, so installing those still works fine.  Our Travis CI images
> install r18.1.
>
> However, when Homebrew updated to Perforce r19.1, they included the
> checksum again for some reason (intentional or accidental; I didn't
> look why).  This worked fine for a while, until a couple of days ago
> Perforce updated the r19.1 binaries in place, breaking those
> checksums.
>
> If we were to still run 'brew update', then it would shortly fix the
> checksum mismatch.  But we don't run it, and we do not want to run it
> because it takes ages.  Falling back to pull from the 'homebrew-cask'
> repository could be a reasonable and quick workaround.

Okay, good.

> > This will still fail, of course, when `homebrew-cask` falls behind
> > Perforce's release schedule. But once it is updated, we can now simply
> > re-run the failed jobs and they will pick up that update.
>
> In our CI builds we don't at all care what the checksums of the
> Perforce binaries are, so I would really like to tell 'brew' to ignore
> any checksum mismatch when installing 'perforce'.  Alas, it appears
> that 'brew' has no public options to turn of or to ignore checksum
> verification.

Sad, yet true, that we indeed have no command-line option to say "you
know what, your checksum possibly mismatches, but we really don't care".

> Now, let's take a step back.
>
> All 'brew cask install perforce' really does is run 'curl' to download
> a tar.gz from the Perforce servers, verify its checksum, unpack it,
> and put the executables somewhere on $PATH.  That's not rocket
> science, we could easily do that ourselves; we don't even have to deal
> with a tar.gz, the 'p4' and 'p4d' binaries for mac are readily
> available for download at:
>
>   http://filehost.perforce.com/perforce/r19.1/bin.macosx1010x86_64/
>
> And, in fact, that's what we have been doing in some of our Linux jobs
> since the very beginning, so basically only the download URL has to be
> adjusted.

I'd rather not.

Just because there is no better way on Linux, and just because the
current `perforce` cask recipe happens to just download and unpack that
file does not mean that this won't change.

And if it changes, we will be a lot better off by using the provided
package.

As I wrote here:

> > As for updating `homebrew-cask`: the beginnings of automating this in
> > https://dev.azure.com/gitgitgadget/git/_build?definitionId=11&_a=summary
> > will be finished once the next Perforce upgrade comes around.

I am quite willing to do my share to keep the Homebrew recipe for
`perforce` up to date. We'll all be better off that way.

Ciao,
Dscho

> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> > ---
> >  ci/install-dependencies.sh | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> > index 85a9d6b15c..ce149ed39c 100755
> > --- a/ci/install-dependencies.sh
> > +++ b/ci/install-dependencies.sh
> > @@ -40,6 +40,11 @@ osx-clang|osx-gcc)
> >  	test -z "$BREW_INSTALL_PACKAGES" ||
> >  	brew install $BREW_INSTALL_PACKAGES
> >  	brew link --force gettext
> > +	brew cask install perforce || {
> > +		# Update the definitions and try again
> > +		git -C "$(brew --repository)"/Library/Taps/homebrew/homebrew-cask pull &&
> > +		brew cask install perforce
> > +	} ||
> >  	brew install caskroom/cask/perforce
> >  	case "$jobname" in
> >  	osx-gcc)
> > --
> > gitgitgadget
>