[Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Peter Maydell]

Hi; we currently have an issue with the u-boot we're using where it
has a file and a directory that differ only in case (scripts/Kconfig
and scripts/kconfig/). This means that QEMU's release tarballs won't
unpack on a case-insensitive filesystem (OSX, Windows).

u-boot have now fixed this upstream:
http://git.denx.de/?p=u-boot.git;a=commitdiff;h=610eec7f0593574c034054ba54fc1c934755e208

but we need to decide how best to get this fix into QEMU so that
our tarballs will unpack, both for the upcoming QEMU 2.11 and
ideally for future point releases based on 2.10.

I can see a couple of options:
 (1) wait for next u-boot release (scheduled for Nov 13, 2017),
     and move to that
     [downsides: would be in the middle of QEMU's own release cycle,
      pretty late to fix any problems with the new version;
      rather a big change to put into stable]
 (2) move to u-boot current head-of-unstable
     [downsides: would mean running some random git commit version,
      also not really very suitable for stable]
 (3) backport the upstream fix to sit on top of the u-boot version
     we're currently using (I think the patch should apply as-is)
     [downsides: would need to figure out how to get that commit into
      the mirror of the u-boot repo that we use; would a build of it
      claim a misleading u-boot version number?]
 (4) suggest your better idea here!

Thoughts?

(Proposals for automated tests to ensure we don't run into the
case-sensitivity problem again in future also welcome.)

thanks
-- PMM

Re: [Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2469 bytes --]

On 10/10/2017 09:22 AM, Peter Maydell wrote:
> Hi; we currently have an issue with the u-boot we're using where it
> has a file and a directory that differ only in case (scripts/Kconfig
> and scripts/kconfig/). This means that QEMU's release tarballs won't
> unpack on a case-insensitive filesystem (OSX, Windows).
> 
> u-boot have now fixed this upstream:
> http://git.denx.de/?p=u-boot.git;a=commitdiff;h=610eec7f0593574c034054ba54fc1c934755e208
> 
> but we need to decide how best to get this fix into QEMU so that
> our tarballs will unpack, both for the upcoming QEMU 2.11 and
> ideally for future point releases based on 2.10.
> 
> I can see a couple of options:
>  (1) wait for next u-boot release (scheduled for Nov 13, 2017),
>      and move to that
>      [downsides: would be in the middle of QEMU's own release cycle,
>       pretty late to fix any problems with the new version;
>       rather a big change to put into stable]
>  (2) move to u-boot current head-of-unstable
>      [downsides: would mean running some random git commit version,
>       also not really very suitable for stable]
>  (3) backport the upstream fix to sit on top of the u-boot version
>      we're currently using (I think the patch should apply as-is)
>      [downsides: would need to figure out how to get that commit into
>       the mirror of the u-boot repo that we use; would a build of it
>       claim a misleading u-boot version number?]
>  (4) suggest your better idea here!
> 
> Thoughts?

half-and-half? Use option (1) for 2.11 (that is, wait for November's
release there), but option (3) for 2.10.2 (that is, backport just the
fix onto the u-boot version uses in 2.10.x right now)?  (I'm not sure
I'm a fan of the idea of split maintenance like that, but am throwing it
out as a possible (4) since you asked).

> 
> (Proposals for automated tests to ensure we don't run into the
> case-sensitivity problem again in future also welcome.)

Would it be as simple as using

LC_ALL=C
filelist=$(... )
diff <(printf '%s\n' $filelist | sort -f) \
     <(printf '%s\n' $filelist | sort -f -u)

where filelist is determined by 'tar tf tarball' or by some invocation
of 'git ls-files'?  Perhaps something we can even add to checkpatch.pl,
although my perl is not strong enough to write that.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Daniel P. Berrange]

On Tue, Oct 10, 2017 at 10:06:35AM -0500, Eric Blake wrote:
> On 10/10/2017 09:22 AM, Peter Maydell wrote:
> > Hi; we currently have an issue with the u-boot we're using where it
> > has a file and a directory that differ only in case (scripts/Kconfig
> > and scripts/kconfig/). This means that QEMU's release tarballs won't
> > unpack on a case-insensitive filesystem (OSX, Windows).
> > 
> > u-boot have now fixed this upstream:
> > http://git.denx.de/?p=u-boot.git;a=commitdiff;h=610eec7f0593574c034054ba54fc1c934755e208
> > 
> > but we need to decide how best to get this fix into QEMU so that
> > our tarballs will unpack, both for the upcoming QEMU 2.11 and
> > ideally for future point releases based on 2.10.
> > 
> > I can see a couple of options:
> >  (1) wait for next u-boot release (scheduled for Nov 13, 2017),
> >      and move to that
> >      [downsides: would be in the middle of QEMU's own release cycle,
> >       pretty late to fix any problems with the new version;
> >       rather a big change to put into stable]
> >  (2) move to u-boot current head-of-unstable
> >      [downsides: would mean running some random git commit version,
> >       also not really very suitable for stable]
> >  (3) backport the upstream fix to sit on top of the u-boot version
> >      we're currently using (I think the patch should apply as-is)
> >      [downsides: would need to figure out how to get that commit into
> >       the mirror of the u-boot repo that we use; would a build of it
> >       claim a misleading u-boot version number?]
> >  (4) suggest your better idea here!
> > 
> > Thoughts?
> 
> half-and-half? Use option (1) for 2.11 (that is, wait for November's
> release there), but option (3) for 2.10.2 (that is, backport just the
> fix onto the u-boot version uses in 2.10.x right now)?  (I'm not sure
> I'm a fan of the idea of split maintenance like that, but am throwing it
> out as a possible (4) since you asked).

There's always the "do nothing" option for the stable branch too.

It isn't nice but there is a workaround, which we could easily publicise
on the '/download/' page of the website until 2.11 comes out


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Richard Henderson]

On 10/10/2017 07:22 AM, Peter Maydell wrote:
> I can see a couple of options:
>  (1) wait for next u-boot release (scheduled for Nov 13, 2017),
>      and move to that
>      [downsides: would be in the middle of QEMU's own release cycle,
>       pretty late to fix any problems with the new version;
>       rather a big change to put into stable]
>  (2) move to u-boot current head-of-unstable
>      [downsides: would mean running some random git commit version,
>       also not really very suitable for stable]

It's always difficult to sync the release cycles of disjoint projects.

You had nothing particularly good to say about using a non-released u-boot, so
really you have two options:

  (a) Update u-boot for 2.12 and not 2.11,
  (b) Track u-boot during its release process, noticing and reporting problems,
      and updating to the final release when it appears.  If you go this route,
      you may well have to slip our release if theirs slips.

The second option is not particularly appealing.

Is there another possibility in which we split the packaging to cope with the
current u-boot?  Normally no one builds u-boot; we simply package it for
licensing.  If the packaging were in two parts, one could unpack the first part
only and not encounter the problem.


r~

Re: [Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Michael Roth]

Quoting Richard Henderson (2017-10-10 11:32:22)
> On 10/10/2017 07:22 AM, Peter Maydell wrote:
> > I can see a couple of options:
> >  (1) wait for next u-boot release (scheduled for Nov 13, 2017),
> >      and move to that
> >      [downsides: would be in the middle of QEMU's own release cycle,
> >       pretty late to fix any problems with the new version;
> >       rather a big change to put into stable]
> >  (2) move to u-boot current head-of-unstable
> >      [downsides: would mean running some random git commit version,
> >       also not really very suitable for stable]
> 
> It's always difficult to sync the release cycles of disjoint projects.
> 
> You had nothing particularly good to say about using a non-released u-boot, so
> really you have two options:
> 
>   (a) Update u-boot for 2.12 and not 2.11,
>   (b) Track u-boot during its release process, noticing and reporting problems,
>       and updating to the final release when it appears.  If you go this route,
>       you may well have to slip our release if theirs slips.
> 
> The second option is not particularly appealing.
> 
> Is there another possibility in which we split the packaging to cope with the
> current u-boot?  Normally no one builds u-boot; we simply package it for
> licensing.  If the packaging were in two parts, one could unpack the first part
> only and not encounter the problem.

Would something like this be sufficient?

diff --git a/scripts/make-release b/scripts/make-release
index fa6323fda8..96c21e7a74 100755
--- a/scripts/make-release
+++ b/scripts/make-release
@@ -20,6 +20,7 @@ git checkout "v${version}"
 git submodule update --init
 (cd roms/seabios && git describe --tags --long --dirty > .version)
 rm -rf .git roms/*/.git dtc/.git pixman/.git
+tar cfj roms/u-boot.tar.bz2 roms/u-boot && rm -rf roms/u-boot
 popd
 tar cfj ${destination}.tar.bz2 ${destination}
 rm -rf ${destination}

And then we can optionally just revert it once we move to a fixed
u-boot? Although we'd need to take care in that case to make sure
we don't use the wrong version of make-release for a particular
point release.

> 
> 
> r~
> 

Re: [Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Richard Henderson]

On 10/10/2017 10:43 AM, Michael Roth wrote:
> Quoting Richard Henderson (2017-10-10 11:32:22)
>> On 10/10/2017 07:22 AM, Peter Maydell wrote:
>>> I can see a couple of options:
>>>  (1) wait for next u-boot release (scheduled for Nov 13, 2017),
>>>      and move to that
>>>      [downsides: would be in the middle of QEMU's own release cycle,
>>>       pretty late to fix any problems with the new version;
>>>       rather a big change to put into stable]
>>>  (2) move to u-boot current head-of-unstable
>>>      [downsides: would mean running some random git commit version,
>>>       also not really very suitable for stable]
>>
>> It's always difficult to sync the release cycles of disjoint projects.
>>
>> You had nothing particularly good to say about using a non-released u-boot, so
>> really you have two options:
>>
>>   (a) Update u-boot for 2.12 and not 2.11,
>>   (b) Track u-boot during its release process, noticing and reporting problems,
>>       and updating to the final release when it appears.  If you go this route,
>>       you may well have to slip our release if theirs slips.
>>
>> The second option is not particularly appealing.
>>
>> Is there another possibility in which we split the packaging to cope with the
>> current u-boot?  Normally no one builds u-boot; we simply package it for
>> licensing.  If the packaging were in two parts, one could unpack the first part
>> only and not encounter the problem.
> 
> Would something like this be sufficient?
> 
> diff --git a/scripts/make-release b/scripts/make-release
> index fa6323fda8..96c21e7a74 100755
> --- a/scripts/make-release
> +++ b/scripts/make-release
> @@ -20,6 +20,7 @@ git checkout "v${version}"
>  git submodule update --init
>  (cd roms/seabios && git describe --tags --long --dirty > .version)
>  rm -rf .git roms/*/.git dtc/.git pixman/.git
> +tar cfj roms/u-boot.tar.bz2 roms/u-boot && rm -rf roms/u-boot
>  popd
>  tar cfj ${destination}.tar.bz2 ${destination}
>  rm -rf ${destination}
> 
> And then we can optionally just revert it once we move to a fixed
> u-boot? Although we'd need to take care in that case to make sure
> we don't use the wrong version of make-release for a particular
> point release.


That seems very reasonable to me.  It also avoids the problem that I would have
introduced wrt having two distribution tarballs.


r~

Re: [Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Thomas Huth]

On 10.10.2017 19:43, Michael Roth wrote:
> Quoting Richard Henderson (2017-10-10 11:32:22)
>> On 10/10/2017 07:22 AM, Peter Maydell wrote:
>>> I can see a couple of options:
>>>  (1) wait for next u-boot release (scheduled for Nov 13, 2017),
>>>      and move to that
>>>      [downsides: would be in the middle of QEMU's own release cycle,
>>>       pretty late to fix any problems with the new version;
>>>       rather a big change to put into stable]
>>>  (2) move to u-boot current head-of-unstable
>>>      [downsides: would mean running some random git commit version,
>>>       also not really very suitable for stable]
>>
>> It's always difficult to sync the release cycles of disjoint projects.
>>
>> You had nothing particularly good to say about using a non-released u-boot, so
>> really you have two options:
>>
>>   (a) Update u-boot for 2.12 and not 2.11,
>>   (b) Track u-boot during its release process, noticing and reporting problems,
>>       and updating to the final release when it appears.  If you go this route,
>>       you may well have to slip our release if theirs slips.
>>
>> The second option is not particularly appealing.
>>
>> Is there another possibility in which we split the packaging to cope with the
>> current u-boot?  Normally no one builds u-boot; we simply package it for
>> licensing.  If the packaging were in two parts, one could unpack the first part
>> only and not encounter the problem.
> 
> Would something like this be sufficient?
> 
> diff --git a/scripts/make-release b/scripts/make-release
> index fa6323fda8..96c21e7a74 100755
> --- a/scripts/make-release
> +++ b/scripts/make-release
> @@ -20,6 +20,7 @@ git checkout "v${version}"
>  git submodule update --init
>  (cd roms/seabios && git describe --tags --long --dirty > .version)
>  rm -rf .git roms/*/.git dtc/.git pixman/.git
> +tar cfj roms/u-boot.tar.bz2 roms/u-boot && rm -rf roms/u-boot

FWIW, that sounds like a good idea to me, too.

 Thomas

Re: [Qemu-devel] updating to a u-boot without the case-sensitive filename clash

[Alexander Graf]

On 11.10.17 15:11, Thomas Huth wrote:
> On 10.10.2017 19:43, Michael Roth wrote:
>> Quoting Richard Henderson (2017-10-10 11:32:22)
>>> On 10/10/2017 07:22 AM, Peter Maydell wrote:
>>>> I can see a couple of options:
>>>>  (1) wait for next u-boot release (scheduled for Nov 13, 2017),
>>>>      and move to that
>>>>      [downsides: would be in the middle of QEMU's own release cycle,
>>>>       pretty late to fix any problems with the new version;
>>>>       rather a big change to put into stable]
>>>>  (2) move to u-boot current head-of-unstable
>>>>      [downsides: would mean running some random git commit version,
>>>>       also not really very suitable for stable]
>>>
>>> It's always difficult to sync the release cycles of disjoint projects.
>>>
>>> You had nothing particularly good to say about using a non-released u-boot, so
>>> really you have two options:
>>>
>>>   (a) Update u-boot for 2.12 and not 2.11,
>>>   (b) Track u-boot during its release process, noticing and reporting problems,
>>>       and updating to the final release when it appears.  If you go this route,
>>>       you may well have to slip our release if theirs slips.
>>>
>>> The second option is not particularly appealing.
>>>
>>> Is there another possibility in which we split the packaging to cope with the
>>> current u-boot?  Normally no one builds u-boot; we simply package it for
>>> licensing.  If the packaging were in two parts, one could unpack the first part
>>> only and not encounter the problem.
>>
>> Would something like this be sufficient?
>>
>> diff --git a/scripts/make-release b/scripts/make-release
>> index fa6323fda8..96c21e7a74 100755
>> --- a/scripts/make-release
>> +++ b/scripts/make-release
>> @@ -20,6 +20,7 @@ git checkout "v${version}"
>>  git submodule update --init
>>  (cd roms/seabios && git describe --tags --long --dirty > .version)
>>  rm -rf .git roms/*/.git dtc/.git pixman/.git
>> +tar cfj roms/u-boot.tar.bz2 roms/u-boot && rm -rf roms/u-boot
> 
> FWIW, that sounds like a good idea to me, too.

I think it's the easiest path forward. And given that we test the e500
target in Travis, I doubt there's too much potential of breakage with
the next major U-Boot update if we still care enough to update for 2.12.


Alex