Re: [Buildroot] [PATCH] package/pkg-download: do not try to vendor _EXTRA_DOWNLOADS

[Arnout Vandecappelle <arnout@mind.be>]

On 06/05/2022 07:21, yann.morin@orange.com wrote:
> Arnout, All,
> 
> On 2022-05-05 22:37 +0200, Arnout Vandecappelle spake thusly:
>> On 05/05/2022 19:19, yann.morin@orange.com wrote:
>>> From: "Yann E. MORIN" <yann.morin@orange.com>
>>>
>>> For golang- or cargo-based packages, we apply a vendoring pass after the
>>> package's "main" download is done. Whether to vendor or not is based on
>>> the heuristic that a specific directory exists or not; for golang
>>> packages, we look for '/vendor', while for cargo, we look for '/VENDOR'.
>>>
>>> This is fine for the "main" (by lack of a better term) download, but
>>> this falls flat on its face for extra downloads. Indeed, so packages may
>>> need to download data sets, or assets, as _EXTRA_DOWNLOADS. Those are
>>> usually just data blobs, and are not actual golang or cargo packages; as
>>> such they do not need to be vendored, but worse, if we try to actually
>>> vendor them, this fails because the required files for vendoring are
>>> missing from the archives in such data sets.
>>>
>>> We fix that by decoupling the download for the extra download, from the
>>> download for the main archive. We pass the post-processing option only
>>> to the main download.
>>>
>>> This makes the hard assumption that extra downloads will never need to
>>> be post-processed for vendoring, of course; we hope this will always be
>>> correct in practice.
>>>
>>> Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
>>> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
>>
>>   Applied to master, thanks, with a few changes:
>>
>>       - no loop needed for MAIN_DOWNLOAD, it can have only one;
> 
> In fact, that was on purpose: it can have no main download...
> Indeed, a package can very well only declare extra downloads...
> I forgot to explain that in the commit log, sorry.

  OK. However, somehow this seems to work:

package/foo/foo.mk:
FOO_SOURCE =
FOO_VERSION = 1

FOO_INSTALL_TARGET_CMDS = echo '***foo!***'

$(eval $(generic-package))

bash 84467$ make foo
WARNING: no hash file for
 >>> foo 1 Extracting
 >>> foo 1 Patching
 >>> foo 1 Configuring
 >>> foo 1 Building
 >>> foo 1 Installing to target
echo '***foo!***'
***foo!***


  So I don't think it is needed after all? The dl-wrapper script apparently 
doesn't do anything if the filename is empty.

  Regards,
  Arnout

> 
> So, either we do the loop, which is a simple way to expand to nothing
> when there is no item but the reason is not obvious, or we add an
> explicit $(if ...) test, which is going to be a bit more verbose...
> 
> What's your preference?
> 
>>       - remove superfluous backslash in the definition of MAIN_DOWNLOAD;
> 
> Ah, yes I forgot to drop it... Good catch.
> 
>>       - introduce _ADDITIONAL_DOWNLOADS to avoid filter-out.
> 
> I did that initially, but I thought that yet another variable could be
> easily avoided with the filtering our. But OK, that's fine with me too.
> 
> Thanks!
> 
> Regards,
> Yann E. MORIN.
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot