[Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Chad Joan]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]

Hello,

This is a one-line patch to the configure script that will allow QEMU to be
built on musl-libc based Linux systems.  This problem is only noticeable
when QEMU is built with --enable-curses.

Detailed reading material if you want to know where this came from:
https://bugs.gentoo.org/show_bug.cgi?id=609364

Hope that helps!
- Chad

[-- Attachment #2: qemu-2.8.0-musl-ncurses.patch --]
[-- Type: application/octet-stream, Size: 322 bytes --]

diff --git configure configure
index 1c9655e..9f83176 100755
--- configure
+++ configure
@@ -682,6 +682,7 @@ Haiku)
   vhost_net="yes"
   vhost_scsi="yes"
   vhost_vsock="yes"
+  QEMU_CFLAGS="-D_XOPEN_SOURCE=500 $QEMU_CFLAGS"
   QEMU_INCLUDES="-I\$(SRC_PATH)/linux-headers -I$(pwd)/linux-headers $QEMU_INCLUDES"
 ;;
 esac

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Paolo Bonzini]

On 16/02/2017 17:30, Chad Joan wrote:
> Hello,
> 
> This is a one-line patch to the configure script that will allow QEMU to be
> built on musl-libc based Linux systems.  This problem is only noticeable
> when QEMU is built with --enable-curses.
> 
> Detailed reading material if you want to know where this came from:
> https://bugs.gentoo.org/show_bug.cgi?id=609364

Hi,

can you explain exactly which function is missing without
-D_XOPEN_SOURCE=500?  If it is curses' wide-char functions, why does it
fail with musl but not with glibc?  Is _XOPEN_SOURCE always defined by
glibc if you have _D_GNU_SOURCE?

Paolo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Laszlo Ersek]

On 02/16/17 17:58, Paolo Bonzini wrote:
> 
> 
> On 16/02/2017 17:30, Chad Joan wrote:
>> Hello,
>>
>> This is a one-line patch to the configure script that will allow QEMU to be
>> built on musl-libc based Linux systems.  This problem is only noticeable
>> when QEMU is built with --enable-curses.
>>
>> Detailed reading material if you want to know where this came from:
>> https://bugs.gentoo.org/show_bug.cgi?id=609364
> 
> Hi,
> 
> can you explain exactly which function is missing without
> -D_XOPEN_SOURCE=500?  If it is curses' wide-char functions, why does it
> fail with musl but not with glibc?  Is _XOPEN_SOURCE always defined by
> glibc if you have _D_GNU_SOURCE?

It is not necessarily auto-defined, but the effect is "as if".

It is documented in the libc info:

https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fXOPEN_005fSOURCE

> Macro: _XOPEN_SOURCE
> Macro: _XOPEN_SOURCE_EXTENDED
>
>     If you define this macro, functionality described in the X/Open
>     Portability Guide is included. This is a superset of the POSIX.1
>     and POSIX.2 functionality and in fact _POSIX_SOURCE and
>     _POSIX_C_SOURCE are automatically defined.
>
>     As the unification of all Unices, functionality only available in
>     BSD and SVID is also included.
>
>     If the macro _XOPEN_SOURCE_EXTENDED is also defined, even more
>     functionality is available. The extra functions will make all
>     functions available which are necessary for the X/Open Unix brand.
>
>     If the macro _XOPEN_SOURCE has the value 500 this includes all
>     functionality described so far plus some new definitions from the
>     Single Unix Specification, version 2.

https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fGNU_005fSOURCE

> Macro: _GNU_SOURCE
>
>     If you define this macro, everything is included: ISO C89, ISO
>     C99, POSIX.1, POSIX.2, BSD, SVID, X/Open, LFS, and GNU extensions.
>     In the cases where POSIX.1 conflicts with BSD, the POSIX
>     definitions take precedence.

Curiously, I managed to recall this from years ago, when I was still
living and breathing the SUS (and looking frequently at the glibc info too).

Laszlo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Chad Joan]

I am glad others are chiming in and might provide better solutions.

Honestly, following the instructions at
http://wiki.qemu-project.org/Contribute/SubmitAPatch to-the-letter is quite
daunting to me, just to get one line of code changed.  It might help if
that page had some kind of dead-simple example for trivial patches;
something like:
$ cd <QEMU directory>
$ git format-patch blah blah blah
$ maybe-some-other-command
$ # Now copy the contents of file xyz.patch into your email client and send
to qemu-devel@nongnu.org and qemu-trivial@nongnu.org

Ask me to do signing/secure-signatures/etc (or any foray into gnupg
commands), setup and secure new email connectivity (ex: git send-email), or
install new software on my system... and I'm going to bounce ;)  Sorry.

I kind of expected that discerning maintainers might want to do this patch
differently.  I'm not familiar with the why's of how your configure script
is organized, so I really hope the QEMU devs can meet me in the middle and
take this the rest of the way.  I am perfectly willing to run commands on
my machine and echo results to you if it helps you learn the nature of this
problem.

Hope that helps.
- Chad

On Thu, Feb 16, 2017 at 12:23 PM, Laszlo Ersek <lersek@redhat.com> wrote:

> On 02/16/17 17:58, Paolo Bonzini wrote:
> >
> >
> > On 16/02/2017 17:30, Chad Joan wrote:
> >> Hello,
> >>
> >> This is a one-line patch to the configure script that will allow QEMU
> to be
> >> built on musl-libc based Linux systems.  This problem is only noticeable
> >> when QEMU is built with --enable-curses.
> >>
> >> Detailed reading material if you want to know where this came from:
> >> https://bugs.gentoo.org/show_bug.cgi?id=609364
> >
> > Hi,
> >
> > can you explain exactly which function is missing without
> > -D_XOPEN_SOURCE=500?  If it is curses' wide-char functions, why does it
> > fail with musl but not with glibc?  Is _XOPEN_SOURCE always defined by
> > glibc if you have _D_GNU_SOURCE?
>
> It is not necessarily auto-defined, but the effect is "as if".
>
> It is documented in the libc info:
>
> https://www.gnu.org/software/libc/manual/html_node/Feature-
> Test-Macros.html#index-_005fXOPEN_005fSOURCE
>
> > Macro: _XOPEN_SOURCE
> > Macro: _XOPEN_SOURCE_EXTENDED
> >
> >     If you define this macro, functionality described in the X/Open
> >     Portability Guide is included. This is a superset of the POSIX.1
> >     and POSIX.2 functionality and in fact _POSIX_SOURCE and
> >     _POSIX_C_SOURCE are automatically defined.
> >
> >     As the unification of all Unices, functionality only available in
> >     BSD and SVID is also included.
> >
> >     If the macro _XOPEN_SOURCE_EXTENDED is also defined, even more
> >     functionality is available. The extra functions will make all
> >     functions available which are necessary for the X/Open Unix brand.
> >
> >     If the macro _XOPEN_SOURCE has the value 500 this includes all
> >     functionality described so far plus some new definitions from the
> >     Single Unix Specification, version 2.
>
> https://www.gnu.org/software/libc/manual/html_node/Feature-
> Test-Macros.html#index-_005fGNU_005fSOURCE
>
> > Macro: _GNU_SOURCE
> >
> >     If you define this macro, everything is included: ISO C89, ISO
> >     C99, POSIX.1, POSIX.2, BSD, SVID, X/Open, LFS, and GNU extensions.
> >     In the cases where POSIX.1 conflicts with BSD, the POSIX
> >     definitions take precedence.
>
> Curiously, I managed to recall this from years ago, when I was still
> living and breathing the SUS (and looking frequently at the glibc info
> too).
>
> Laszlo
>

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Fam Zheng]

On Thu, 02/16 12:47, Chad Joan wrote:
> I am glad others are chiming in and might provide better solutions.
> 
> Honestly, following the instructions at
> http://wiki.qemu-project.org/Contribute/SubmitAPatch to-the-letter is quite
> daunting to me, just to get one line of code changed.  It might help if
> that page had some kind of dead-simple example for trivial patches;
> something like:
> $ cd <QEMU directory>
> $ git format-patch blah blah blah
> $ maybe-some-other-command
> $ # Now copy the contents of file xyz.patch into your email client and send
> to qemu-devel@nongnu.org and qemu-trivial@nongnu.org

Makes sense in general except for the sending part - email clients tend to
damage the patch when you copy and paste by wrapping long lines or messing up
other things. But your point is taken, we should make the first (or a one-shot)
contribution as easy as possible.

Fam

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Laszlo Ersek]

On 02/17/17 07:43, Fam Zheng wrote:
> On Thu, 02/16 12:47, Chad Joan wrote:
>> I am glad others are chiming in and might provide better solutions.
>>
>> Honestly, following the instructions at
>> http://wiki.qemu-project.org/Contribute/SubmitAPatch to-the-letter is quite
>> daunting to me, just to get one line of code changed.  It might help if
>> that page had some kind of dead-simple example for trivial patches;
>> something like:
>> $ cd <QEMU directory>
>> $ git format-patch blah blah blah
>> $ maybe-some-other-command
>> $ # Now copy the contents of file xyz.patch into your email client and send
>> to qemu-devel@nongnu.org and qemu-trivial@nongnu.org
> 
> Makes sense in general except for the sending part - email clients tend to
> damage the patch when you copy and paste by wrapping long lines or messing up
> other things. But your point is taken, we should make the first (or a one-shot)
> contribution as easy as possible.

I disagree (from the sidelines, that is; I'm not a QEMU maintainer --
I'm a co-maintainer elsewhere). The patch submission process exists for
a reason, the goal is to maximize the throughput of long-term
contributors and maintainers, because that's the best for the project's
overall health and progress.

It does not mean that one-off contributions are not welcome -- all
contributions are welcome that follow the process (and beyond that,
everyone is welcome to become a long-term contributor).

Just my two cents, of course; don't take this as an official standpoint
or whatever. (And, I'm saying this after having manually fixed up
garbled patches from one-off contributors.)

Laszlo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Fam Zheng]

On Fri, 02/17 10:23, Laszlo Ersek wrote:
> On 02/17/17 07:43, Fam Zheng wrote:
> > On Thu, 02/16 12:47, Chad Joan wrote:
> >> I am glad others are chiming in and might provide better solutions.
> >>
> >> Honestly, following the instructions at
> >> http://wiki.qemu-project.org/Contribute/SubmitAPatch to-the-letter is quite
> >> daunting to me, just to get one line of code changed.  It might help if
> >> that page had some kind of dead-simple example for trivial patches;
> >> something like:
> >> $ cd <QEMU directory>
> >> $ git format-patch blah blah blah
> >> $ maybe-some-other-command
> >> $ # Now copy the contents of file xyz.patch into your email client and send
> >> to qemu-devel@nongnu.org and qemu-trivial@nongnu.org
> > 
> > Makes sense in general except for the sending part - email clients tend to
> > damage the patch when you copy and paste by wrapping long lines or messing up
> > other things. But your point is taken, we should make the first (or a one-shot)
> > contribution as easy as possible.
> 
> I disagree (from the sidelines, that is; I'm not a QEMU maintainer --
> I'm a co-maintainer elsewhere). The patch submission process exists for
> a reason, the goal is to maximize the throughput of long-term
> contributors and maintainers, because that's the best for the project's
> overall health and progress.

Having a process and smoothly revealing it to new contributors are not in
conflict. The problem here is the long SubmitAPatch I think. It does include
more details than needed for the minimum of the very first submission; also it
is not in an easy-to-follow step-by-step form.

I believe people will happily learn the process once he feels appreciation on
his effort, once they get replies to his patch. :)

> 
> It does not mean that one-off contributions are not welcome -- all
> contributions are welcome that follow the process (and beyond that,
> everyone is welcome to become a long-term contributor).

IMHO one-off _fixes_ are also good, like this one. Like Peter noted, the only
recommendation for those who don't like formalities and have no intention to
contribute regularly, is to add a "signed-off-by" line to their patch.

> 
> Just my two cents, of course; don't take this as an official standpoint
> or whatever. (And, I'm saying this after having manually fixed up
> garbled patches from one-off contributors.)
> 
> Laszlo

Fam

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 17 February 2017 at 06:43, Fam Zheng <famz@redhat.com> wrote:
> But your point is taken, we should make the first (or a one-shot)
> contribution as easy as possible.

Yes; we could do with providing a "This page seems very long..."
introduction section. The absolute bare minimum requirements
for a submitter I think are:
 * Provide a Signed-off-by: line (this is a hard requirement
   because it's how you say "I'm legally OK to contribute this
   and am happy for it to go into QEMU")
 * send patch by email
 * read replies and act on them if you want your patch to go in

The larger your contribution is, the more important the other
requirements detailed on the page are; but personally I'm
happy to manually fix up patches from a first-time submitter,
and I think most other maintainers are too.

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

On 02/17/2017 03:28 AM, Peter Maydell wrote:
> On 17 February 2017 at 06:43, Fam Zheng <famz@redhat.com> wrote:
>> But your point is taken, we should make the first (or a one-shot)
>> contribution as easy as possible.
> 
> Yes; we could do with providing a "This page seems very long..."
> introduction section. The absolute bare minimum requirements
> for a submitter I think are:
>  * Provide a Signed-off-by: line (this is a hard requirement
>    because it's how you say "I'm legally OK to contribute this
>    and am happy for it to go into QEMU")
>  * send patch by email
>  * read replies and act on them if you want your patch to go in
> 
> The larger your contribution is, the more important the other
> requirements detailed on the page are; but personally I'm
> happy to manually fix up patches from a first-time submitter,
> and I think most other maintainers are too.

I've updated the wiki to put in that nice bullet list, prior to the
table of contents.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Chad Joan]

Wow, that is some quick turn-around.  Well done!

My thoughts:

   - I find this summary info very helpful!  On behalf of the N people
   trying to heal paper cuts: thank you!
   - I still recommend an example snippet for shell/bash interaction that
   demonstrates the workflow you expect from a first-time contributor.  It
   should be populated with commonly used values (ex: mailing list
   addresses).  I don't expect this to happen fast like the summary info; I
   suspect someone is going to have to pretend they are submitting a patch,
   write down what they do, and then think about how to present this.
   - Regarding the signature: IIRC, setting up certificates on a machine
   using gpg can be quite time consuming and learning-intensive if you've
   never needed to do it before.  Having that example will go a long way to
   help with this.  There is still a possible pain-point: you might write one
   line of git code in the example, and it is easy for you due to your
   workflow, but it could be hours of fiddling for someone who has never done
   it before.  If I'm wrong, show me (the hypothetical reader) how easy it is
   ;)  If I'm right (and that would be unfortunate, in this case), then it
   might be helpful if you politely ask the reader to spend time X amount of
   time on it (establish accurate expectations) and then provide a link to the
   most helpful how-to article you can find on the subject.
   - The email thing is a pain-point, but I wonder if QEMU can make it
   wonderful without sacrificing much--perhaps just a few words on the page.
   Presumably, not every email client mangles patches.  Can we have a
   whitelist?  So far the whitelist has one item: git send-email.  But git
   send-email is probably not even part of the majority population's workflow,
   so if we can test and approve even /some/ of the more popular mail clients
   (ex: gmail, thunderbird, outlook, etc) for use, it would help newbies A
   LOT.  Less importantly; a blacklist could be useful too, to prevent
   unnecessary "what about my mail client?" questions and unnecessary
   redundant testing in the future.
   - I should also mention that I found the rest of the document to be very
   well-written.  It's comprehensiveness became its weakness, but that's still
   important long-term, hence why I think an alternative path with a short
   example for trivial patches is plenty sufficient: from my perspective,
   there's no need to change the rest of the text; it is already good :).

Note that I'm bothering to stick around and provide feedback, despite other
pressing life obligations.  Providing advice on submit-a-patch usability
for QEMU isn't on my schedule, but I don't have the heart to bail on this,
especially when you all are kindly listening, having high quality
discussion, and sincerely trying to improve things.  If you read between
the lines, you see the truth: I am a yak shaver!

Oh man, when I hit a topic like "use git send-email", the hair started
flying: learning new git commands, two-factor auth on gmail, U2F keys,
governance for the no-mans-land of a server, and so on, a real yak-shaving
party.  After an hour or two, my safety triggered, and I thought, "man, I
am spending way too much time perfecting this workflow that I might never
do again" and I spent a few minutes writing a message in gmail.  I
certainly don't expect QEMU devs to fix garbled patches either: that also
seems like a huge waste of valuable time (and for talented and passionate
individuals, too).  There has to be a better way!  So I hope the whitelist
idea helps, or maybe it's enough to just call awareness to this potential
improvement.

Well, that ended up being very long.  I hope this is helpful and doesn't
spend too much of your time.

Thanks for listening!

On Fri, Feb 17, 2017 at 10:34 AM, Eric Blake <eblake@redhat.com> wrote:

> On 02/17/2017 03:28 AM, Peter Maydell wrote:
> > On 17 February 2017 at 06:43, Fam Zheng <famz@redhat.com> wrote:
> >> But your point is taken, we should make the first (or a one-shot)
> >> contribution as easy as possible.
> >
> > Yes; we could do with providing a "This page seems very long..."
> > introduction section. The absolute bare minimum requirements
> > for a submitter I think are:
> >  * Provide a Signed-off-by: line (this is a hard requirement
> >    because it's how you say "I'm legally OK to contribute this
> >    and am happy for it to go into QEMU")
> >  * send patch by email
> >  * read replies and act on them if you want your patch to go in
> >
> > The larger your contribution is, the more important the other
> > requirements detailed on the page are; but personally I'm
> > happy to manually fix up patches from a first-time submitter,
> > and I think most other maintainers are too.
>
> I've updated the wiki to put in that nice bullet list, prior to the
> table of contents.
>
> --
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
>

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 17 February 2017 at 16:54, Chad Joan <chadjoan@gmail.com> wrote:
> Regarding the signature: IIRC, setting up certificates on a machine using
> gpg can be quite time consuming and learning-intensive if you've never
> needed to do it before.

There is no requirement for gpg (except for maintainers submitting
pull requests). When we say "We need a signed-off-by" what we mean
is literally "your commit message must include a line of the form
Signed-off-by Some Person <some.person@wherever.com>".

I agree that gpg is terrible as a UI and we definitely don't
want it in our patch workflow for submitters ;-)

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Paolo Bonzini]

On 17/02/2017 17:54, Chad Joan wrote:
> Regarding the signature: IIRC, setting up certificates on a machine
> using gpg can be quite time consuming and learning-intensive if you've
> never needed to do it before.  Having that example will go a long way to
> help with this.  There is still a possible pain-point: you might write
> one line of git code in the example, and it is easy for you due to your
> workflow, but it could be hours of fiddling for someone who has never
> done it before.  If I'm wrong, show me (the hypothetical reader) how
> easy it is ;)  If I'm right (and that would be unfortunate, in this
> case), then it might be helpful if you politely ask the reader to spend
> time X amount of time on it (establish accurate expectations) and then
> provide a link to the most helpful how-to article you can find on the
> subject.

GPG signing is not needed.  All you need is "git commit -s".

Adding "Signed-off-by: Chad Joan <chadjoan@gmail.com>" basically is a
way to tell us "I understand I'm contributing this under the GNU GPL or
a compatible license".  It's not a cryptographic signature.

Paolo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Chad Joan]

How wonderful!  Problem solved.  Now I think that just having an example
could kill the misconception forever ;)

On Fri, Feb 17, 2017 at 11:57 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:

>
>
> On 17/02/2017 17:54, Chad Joan wrote:
> > Regarding the signature: IIRC, setting up certificates on a machine
> > using gpg can be quite time consuming and learning-intensive if you've
> > never needed to do it before.  Having that example will go a long way to
> > help with this.  There is still a possible pain-point: you might write
> > one line of git code in the example, and it is easy for you due to your
> > workflow, but it could be hours of fiddling for someone who has never
> > done it before.  If I'm wrong, show me (the hypothetical reader) how
> > easy it is ;)  If I'm right (and that would be unfortunate, in this
> > case), then it might be helpful if you politely ask the reader to spend
> > time X amount of time on it (establish accurate expectations) and then
> > provide a link to the most helpful how-to article you can find on the
> > subject.
>
> GPG signing is not needed.  All you need is "git commit -s".
>
> Adding "Signed-off-by: Chad Joan <chadjoan@gmail.com>" basically is a
> way to tell us "I understand I'm contributing this under the GNU GPL or
> a compatible license".  It's not a cryptographic signature.
>
> Paolo
>

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 17 February 2017 at 16:54, Chad Joan <chadjoan@gmail.com> wrote:
> so if we can test and
> approve even /some/ of the more popular mail clients (ex: gmail,
> thunderbird, outlook, etc) for use, it would help newbies A LOT.

Pretty sure we've seen mangled emails from all of those.
The problem is that most email clients will automatically
wrap long lines, which is fine for text but breaks an
inline patch. Changing trailing whitespace in patches is
also a common issue.

The documentation in "git help format-patch" has a section
"MUA-SPECIFIC HINTS" which says that gmail's web interface
is definitely no good, and Thunderbird defaults to the wrong
thing and requires a bunch of config changes which you'd then
have to switch back to your preferences for normal mail when
you're done sending patches. Outlook is so hopeless for
patch mail it isn't even listed :-)

The kernel docs have a longer list of mail clients with
notes about suitability:
https://kernel.org/doc/html/latest/process/email-clients.html
but the set of "just works" clients is very small.

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Chad Joan]

This is kinda depressing to read :(

But thanks for explaining.

I got a good laugh when it mentioned of Lotus Notes, "Run away from it."

Would it be reasonable to stick a link to that article in the submit a
patch document?  Something like, "If you can't use 'git send-email' for any
reason, then please review [link]this article[/link] to understand which
clients are capable of sending acceptable patches and how to configure
them."

I suspect I'm going to encounter this problem again as I try to make small
fixes for more projects, so it might be worth it for me to spend a small
amount of time at some point setting up a mail client that I can send git
patches with.  Or perhaps I can just move the patch(es) onto another
machine (ex: my personal laptop) and send it with 'git send-email' from
there, instead of needing to install a mail client that is outside of my
normal workflow.  I am not comfortable with putting smtp login information
onto the server that I am using for this work.  If anyone has any advice on
how to handle this situation, I'd appreciate it.  That kernel doc that
Peter linked is already pretty helpful, at least.

Thanks!

On Fri, Feb 17, 2017 at 12:15 PM, Peter Maydell <peter.maydell@linaro.org>
wrote:

> On 17 February 2017 at 16:54, Chad Joan <chadjoan@gmail.com> wrote:
> > so if we can test and
> > approve even /some/ of the more popular mail clients (ex: gmail,
> > thunderbird, outlook, etc) for use, it would help newbies A LOT.
>
> Pretty sure we've seen mangled emails from all of those.
> The problem is that most email clients will automatically
> wrap long lines, which is fine for text but breaks an
> inline patch. Changing trailing whitespace in patches is
> also a common issue.
>
> The documentation in "git help format-patch" has a section
> "MUA-SPECIFIC HINTS" which says that gmail's web interface
> is definitely no good, and Thunderbird defaults to the wrong
> thing and requires a bunch of config changes which you'd then
> have to switch back to your preferences for normal mail when
> you're done sending patches. Outlook is so hopeless for
> patch mail it isn't even listed :-)
>
> The kernel docs have a longer list of mail clients with
> notes about suitability:
> https://kernel.org/doc/html/latest/process/email-clients.html
> but the set of "just works" clients is very small.
>
> thanks
> -- PMM
>

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 19 February 2017 at 07:22, Chad Joan <chadjoan@gmail.com> wrote:
> I suspect I'm going to encounter this problem again as I try to make small
> fixes for more projects, so it might be worth it for me to spend a small
> amount of time at some point setting up a mail client that I can send git
> patches with.  Or perhaps I can just move the patch(es) onto another machine
> (ex: my personal laptop) and send it with 'git send-email' from there,
> instead of needing to install a mail client that is outside of my normal
> workflow.  I am not comfortable with putting smtp login information onto the
> server that I am using for this work.

Yes, I'd do that. If you do git format-patch on one machine you can
then copy the patch files onto another where you do git send-email
(my workflow actually involves something similar to this).

Or if the machine you're running on is a server with limited
development facilities you might prefer to do development
entirely on a different machine and then on the server
just do a git fetch or git pull from the git tree on your
development machine to build it. (I do this when I'm doing
build tests, since there usually isn't a nice dev environment
with my preferred editor and tools and so on on the test
machine.) Then your patches are all on the dev machine to
start with.

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]

On 02/17/2017 11:15 AM, Peter Maydell wrote:
> The kernel docs have a longer list of mail clients with
> notes about suitability:
> https://kernel.org/doc/html/latest/process/email-clients.html
> but the set of "just works" clients is very small.

Currently fails with 403 Forbidden

You don't have permission to access
/doc/html/latest/process/email-clients.html on this server.

Hopefully its transient?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 5226 bytes --]

On 02/17/2017 10:54 AM, Chad Joan wrote:
> Wow, that is some quick turn-around.  Well done!
> 
> My thoughts:
> 
>    - I find this summary info very helpful!  On behalf of the N people
>    trying to heal paper cuts: thank you!
>    - I still recommend an example snippet for shell/bash interaction that
>    demonstrates the workflow you expect from a first-time contributor.  It
>    should be populated with commonly used values (ex: mailing list
>    addresses).  I don't expect this to happen fast like the summary info; I
>    suspect someone is going to have to pretend they are submitting a patch,
>    write down what they do, and then think about how to present this.

For a quickie patch, I make my edits, then 'git commit -a -s' and 'git
send-email -1' - but that works because I've already set up git hooks to
auto-cc the list, and I already debugged my 'get send-email' setup years
ago.  So yeah, doing it from a completely virgin environment could
benefit from a complete command line that reproduces what I take for
granted in my normal environment.  And some email setups are a lot
friendlier than others (I personally do not use gmail, but will readily
admit that it is probably a lot easier to set up my work emails to use
Red Hat's SMTP servers than it is for a gmail contributor to get their
email setup working in a way that does not mangle patches).

>    - Regarding the signature: IIRC, setting up certificates on a machine
>    using gpg can be quite time consuming and learning-intensive if you've
>    never needed to do it before.

But nothing requires you to set up a certificate to submit a patch.  I'm
not sure which piece of the documentation got you steered in that
direction, but gpg signing of patches is only required of maintainers,
not contributors (or maybe you're hinting at the extra effort required
to set up gmail as a valid 'git send-email' target, to which I have no
experience, but which starts to leave the realm of qemu-specific
instructions into something where it would be better to link to a good
git setup tutorial, if one exists).

>  Having that example will go a long way to
>    help with this.

<snip lots of useful ideas for improvement>

>    - I should also mention that I found the rest of the document to be very
>    well-written.  It's comprehensiveness became its weakness, but that's still
>    important long-term, hence why I think an alternative path with a short
>    example for trivial patches is plenty sufficient: from my perspective,
>    there's no need to change the rest of the text; it is already good :).

Thanks for that feedback.  It's often hard for a core contributor to
remember what it was like to submit their first patch years ago, and
having a fresh take on the matter from a new contributor is well worth
the reminders.  It's also nice to hear this as a compliment, and not
just a complaint.

> 
> Note that I'm bothering to stick around and provide feedback, despite other
> pressing life obligations.  Providing advice on submit-a-patch usability
> for QEMU isn't on my schedule, but I don't have the heart to bail on this,
> especially when you all are kindly listening, having high quality
> discussion, and sincerely trying to improve things.  If you read between
> the lines, you see the truth: I am a yak shaver!

At the risk of pushing too hard, you could always turn your (good!)
suggestions into concrete wording, or even request a wiki account to
make the changes yourself. But even if that is beyond your planned level
of involvement, I do hope that various readers will be able to act on
the suggestions in this mail to improve things.

> 
> Oh man, when I hit a topic like "use git send-email", the hair started
> flying: learning new git commands, two-factor auth on gmail, U2F keys,
> governance for the no-mans-land of a server, and so on, a real yak-shaving
> party.  After an hour or two, my safety triggered, and I thought, "man, I
> am spending way too much time perfecting this workflow that I might never
> do again" and I spent a few minutes writing a message in gmail.  I
> certainly don't expect QEMU devs to fix garbled patches either: that also
> seems like a huge waste of valuable time (and for talented and passionate
> individuals, too).  There has to be a better way!  So I hope the whitelist
> idea helps, or maybe it's enough to just call awareness to this potential
> improvement.

Again, part of the problem may be that gmail is not really suited to the
ideal patch flow, and so that's going to be a pain point for any such
contributor.  Submitting patches as an attachment is harder than the
inline version you get with 'git send-email', but it is one of those
one-off manual fixups that a maintainer can overlook, as long as it
really is a one-off thing and not a repeated pattern.  And yes, we
should document that use of an attachment is the most likely to avoid
mangling the patch if you don't have 'git send-email' working, even if
it is harder for the maintainer to apply such a patch.


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Chad Joan]

On Fri, Feb 17, 2017 at 12:17 PM, Eric Blake <eblake@redhat.com> wrote:

> On 02/17/2017 10:54 AM, Chad Joan wrote:
> > Wow, that is some quick turn-around.  Well done!
> >
> > My thoughts:
> >
> >    - I find this summary info very helpful!  On behalf of the N people
> >    trying to heal paper cuts: thank you!
> >    - I still recommend an example snippet for shell/bash interaction that
> >    demonstrates the workflow you expect from a first-time contributor.
> It
> >    should be populated with commonly used values (ex: mailing list
> >    addresses).  I don't expect this to happen fast like the summary
> info; I
> >    suspect someone is going to have to pretend they are submitting a
> patch,
> >    write down what they do, and then think about how to present this.
>
> For a quickie patch, I make my edits, then 'git commit -a -s' and 'git
> send-email -1' - but that works because I've already set up git hooks to
> auto-cc the list, and I already debugged my 'get send-email' setup years
> ago.  So yeah, doing it from a completely virgin environment could
> benefit from a complete command line that reproduces what I take for
> granted in my normal environment.  And some email setups are a lot
> friendlier than others (I personally do not use gmail, but will readily
> admit that it is probably a lot easier to set up my work emails to use
> Red Hat's SMTP servers than it is for a gmail contributor to get their
> email setup working in a way that does not mangle patches).
>
>
I appreciate this retrospective :)

In my case I am working on a machine that wasn't even supposed to see any
development work.  There are no user accounts, just root.  I have tried to
avoid putting any personal information on it.  If I am on it, then I'm
editing files in /etc or installing system-wide software.  I'm realizing
that I might have to change this a bit due to the WIP nature of the
hardened-musl profile: ultimately I *am* doing development work on it, and
that kind of snuck up on me.  If I give myself a user account, then
authoring patches with git (and using send-email) becomes somewhat more
practical (putting smtp login information onto the machine still bugs me).
Still, I can't imagine I'm the only person who runs into this kind of thing
and wants to write quick patches on an impersonal machine.


> [...]
>
> But nothing requires you to set up a certificate to submit a patch.  I'm
> not sure which piece of the documentation got you steered in that
> direction, but gpg signing of patches is only required of maintainers,
> not contributors (or maybe you're hinting at the extra effort required
> to set up gmail as a valid 'git send-email' target, to which I have no
> experience, but which starts to leave the realm of qemu-specific
> instructions into something where it would be better to link to a good
> git setup tutorial, if one exists).
>
>
I think this is just language ambiguity and confirmation bias doing their
thing.  Usually when I read "you have to sign this" in an OSS context, I
think of cryptographic signing.  I haven't encountered the requirement for
non-cryptographic signing before.  Language is arbitrary and we all have
different experiences and backgrounds.

This is one of the reasons why I suggest a simple example: it would be both
very concise and unambiguous.  If there are no signing steps in the example
then you don't even need to spend words telling the reader that
cryptographic signing is unnecessary.  It'll be implied.

Thankfully, this is a separate concern from the 'git send-email' thing.



> >  Having that example will go a long way to
> >    help with this.
>
> <snip lots of useful ideas for improvement>
>
> >    - I should also mention that I found the rest of the document to be
> very
> >    well-written.  It's comprehensiveness became its weakness, but that's
> still
> >    important long-term, hence why I think an alternative path with a
> short
> >    example for trivial patches is plenty sufficient: from my perspective,
> >    there's no need to change the rest of the text; it is already good :).
>
> Thanks for that feedback.  It's often hard for a core contributor to
> remember what it was like to submit their first patch years ago, and
> having a fresh take on the matter from a new contributor is well worth
> the reminders.  It's also nice to hear this as a compliment, and not
> just a complaint.
>
>
Thanks for listening :)

I know what you mean about the complaint thing.  It's unfortunate, but it
can be very hard to know the significance of your words for others.  For
many people (possibly myself included; hard to remember) it may take a long
time to learn and internalize the implications of a truth such as "no one
wants to be hurt".  I try to be the world that I want to live in.  I'm glad
it's working out for everyone this time around.

FWIW, this thread has also left me impressed with the culture in the QEMU
community, and it makes me feel good about choosing this software too.
Good job everyone!


> >
> > Note that I'm bothering to stick around and provide feedback, despite
> other
> > pressing life obligations.  Providing advice on submit-a-patch usability
> > for QEMU isn't on my schedule, but I don't have the heart to bail on
> this,
> > especially when you all are kindly listening, having high quality
> > discussion, and sincerely trying to improve things.  If you read between
> > the lines, you see the truth: I am a yak shaver!
>
> At the risk of pushing too hard, you could always turn your (good!)
> suggestions into concrete wording, or even request a wiki account to
> make the changes yourself. But even if that is beyond your planned level
> of involvement, I do hope that various readers will be able to act on
> the suggestions in this mail to improve things.
>
>
It's always worth asking!

I'm in deep enough now that I wouldn't cheap out on it ;)

I'm more worried that I'm completely unqualified to do this.  I could try
to read the whole thing and make a small 2-4 liner that would do what QEMU
devs want.  But I'm almost certain I'd botch it up somehow.  I've already
misinterpreted the gpg signature thing; there's more where that came from!
I think a QEMU dev would get results that a QEMU dev would expect.

Well, let me know if you want me to try *anyways*.


> >
> > Oh man, when I hit a topic like "use git send-email", the hair started
> > flying: learning new git commands, two-factor auth on gmail, U2F keys,
> > governance for the no-mans-land of a server, and so on, a real
> yak-shaving
> > party.  After an hour or two, my safety triggered, and I thought, "man, I
> > am spending way too much time perfecting this workflow that I might never
> > do again" and I spent a few minutes writing a message in gmail.  I
> > certainly don't expect QEMU devs to fix garbled patches either: that also
> > seems like a huge waste of valuable time (and for talented and passionate
> > individuals, too).  There has to be a better way!  So I hope the
> whitelist
> > idea helps, or maybe it's enough to just call awareness to this potential
> > improvement.
>
> Again, part of the problem may be that gmail is not really suited to the
> ideal patch flow, and so that's going to be a pain point for any such
> contributor.  Submitting patches as an attachment is harder than the
> inline version you get with 'git send-email', but it is one of those
> one-off manual fixups that a maintainer can overlook, as long as it
> really is a one-off thing and not a repeated pattern.  And yes, we
> should document that use of an attachment is the most likely to avoid
> mangling the patch if you don't have 'git send-email' working, even if
> it is harder for the maintainer to apply such a patch.
>
>
> --
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
>
Interesting.  I certainly think that would help.  If the QEMU maintainers
could meet me (the hypothetical mass of first-timers) in the middle and
handle 1-3 attachments or so, then that would make things very painless on
my side.  And at the point where I am submitting more than a few patches, I
know that there is a pattern developing and I wouldn't mind spending the
effort to streamline it for everyone.  tl;dr: This idea makes sense on my
side too.

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 3335 bytes --]

On 02/19/2017 01:02 AM, Chad Joan wrote:
> development work.  There are no user accounts, just root.  I have tried to
> avoid putting any personal information on it.  If I am on it, then I'm
> editing files in /etc or installing system-wide software.  I'm realizing
> that I might have to change this a bit due to the WIP nature of the
> hardened-musl profile: ultimately I *am* doing development work on it, and
> that kind of snuck up on me.  If I give myself a user account, then
> authoring patches with git (and using send-email) becomes somewhat more
> practical (putting smtp login information onto the machine still bugs me).

You don't have to store your SMTP passwords; git is smart enough to ask
you interactively if you (intentionally) omit the passwords from
.gitconfig.  But I agree that even storing your SMTP address and
username in configs can be a bit hairier than you want on some boxes.

> Still, I can't imagine I'm the only person who runs into this kind of thing
> and wants to write quick patches on an impersonal machine.

There's always the option to float the patches back to a personal
machine before posting to the list (yes, it requires more work on your
end, but if it serves as a nice manual wall between your internal and
external machines, it may well be worth the discipline).

> 
> 
>> [...]
>>
>> But nothing requires you to set up a certificate to submit a patch.  I'm
>> not sure which piece of the documentation got you steered in that
>> direction, but gpg signing of patches is only required of maintainers,
>> not contributors (or maybe you're hinting at the extra effort required
>> to set up gmail as a valid 'git send-email' target, to which I have no
>> experience, but which starts to leave the realm of qemu-specific
>> instructions into something where it would be better to link to a good
>> git setup tutorial, if one exists).
>>
>>
> I think this is just language ambiguity and confirmation bias doing their
> thing.  Usually when I read "you have to sign this" in an OSS context, I
> think of cryptographic signing.  I haven't encountered the requirement for
> non-cryptographic signing before.  Language is arbitrary and we all have
> different experiences and backgrounds.

Is it sufficient to just give the example of 'git commit -s' being the
trick to automatically adding the necessary Signed-off-by: line?  (Of
course, automating the process like that, without actually reading
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/SubmittingPatches?id=f6f94e2ab1b33f0082ac22d71f66385a60d8157f#n297
to understand what it means and that you actually comply, is risky)

> 
> This is one of the reasons why I suggest a simple example: it would be both
> very concise and unambiguous.  If there are no signing steps in the example
> then you don't even need to spend words telling the reader that
> cryptographic signing is unnecessary.  It'll be implied.
> 
> Thankfully, this is a separate concern from the 'git send-email' thing.

'git send-email -s' can also add Signed-off-by: lines, if you didn't add
them earlier (but only if you use send-email, rather than attachments) :)

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

[...]
> 'git send-email -s' can also add Signed-off-by: lines, if you didn't add
> them earlier (but only if you use send-email, rather than attachments) :)

That's fine, as you *should* use git send-email.

If it looks unattractive to you because it needs some setup, then that's
because you haven't wasted your time failing at all the alternatives.

Hiding patches in attachments counts as failure.

We should be nice about new contributors failures, of course.  Still,
you might want to spend your new contributor goodwill budget on more
interesting mistakes ;)

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 21 February 2017 at 09:41, Markus Armbruster <armbru@redhat.com> wrote:
> Eric Blake <eblake@redhat.com> writes:
>
> [...]
>> 'git send-email -s' can also add Signed-off-by: lines, if you didn't add
>> them earlier (but only if you use send-email, rather than attachments) :)
>
> That's fine, as you *should* use git send-email.

It has the disadvantage that checkpatch will complain at you
when you test your patches before sending them, because the
sign-off is missing... Personally I recommend adding the signoff
lines earlier in the process.

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[John Snow]

On 02/17/2017 10:34 AM, Eric Blake wrote:
> On 02/17/2017 03:28 AM, Peter Maydell wrote:
>> On 17 February 2017 at 06:43, Fam Zheng <famz@redhat.com> wrote:
>>> But your point is taken, we should make the first (or a one-shot)
>>> contribution as easy as possible.
>>
>> Yes; we could do with providing a "This page seems very long..."
>> introduction section. The absolute bare minimum requirements
>> for a submitter I think are:
>>  * Provide a Signed-off-by: line (this is a hard requirement
>>    because it's how you say "I'm legally OK to contribute this
>>    and am happy for it to go into QEMU")
>>  * send patch by email
>>  * read replies and act on them if you want your patch to go in
>>
>> The larger your contribution is, the more important the other
>> requirements detailed on the page are; but personally I'm
>> happy to manually fix up patches from a first-time submitter,
>> and I think most other maintainers are too.
> 
> I've updated the wiki to put in that nice bullet list, prior to the
> table of contents.
> 

Is there a way we can make the quickstart exist in a pop-box?

Also, any way to wrap the text on the rest of the introductory article?
If you fullscreen this on a widescreen monitor you have to chase the
letters to the far corners of the earth.

Formatting the article more nicely might help its readability. It's also
worth noting that just /reading the outline/ gives you enough advice,
reading the details are actually not strictly necessary.

--js

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Paolo Bonzini]

On 16/02/2017 18:23, Laszlo Ersek wrote:
>> Hi,
>>
>> can you explain exactly which function is missing without
>> -D_XOPEN_SOURCE=500?  If it is curses' wide-char functions, why does it
>> fail with musl but not with glibc?  Is _XOPEN_SOURCE always defined by
>> glibc if you have _D_GNU_SOURCE?
> It is not necessarily auto-defined, but the effect is "as if".
> 
> It is documented in the libc info:
> 
> https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fXOPEN_005fSOURCE
> 

So is this a musl bug?

Paolo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Paolo Bonzini]

On 16/02/2017 18:23, Laszlo Ersek wrote:
> On 02/16/17 17:58, Paolo Bonzini wrote:
>>
>>
>> On 16/02/2017 17:30, Chad Joan wrote:
>>> Hello,
>>>
>>> This is a one-line patch to the configure script that will allow QEMU to be
>>> built on musl-libc based Linux systems.  This problem is only noticeable
>>> when QEMU is built with --enable-curses.
>>>
>>> Detailed reading material if you want to know where this came from:
>>> https://bugs.gentoo.org/show_bug.cgi?id=609364
>>
>> Hi,
>>
>> can you explain exactly which function is missing without
>> -D_XOPEN_SOURCE=500?  If it is curses' wide-char functions, why does it
>> fail with musl but not with glibc?  Is _XOPEN_SOURCE always defined by
>> glibc if you have _D_GNU_SOURCE?
> 
> It is not necessarily auto-defined, but the effect is "as if".

Ok, so the bug is that glibc _does_ auto-define _XOPEN_SOURCE:

#ifdef _GNU_SOURCE
# undef  _ISOC95_SOURCE
# define _ISOC95_SOURCE 1
# undef  _ISOC99_SOURCE
# define _ISOC99_SOURCE 1
# undef  _ISOC11_SOURCE
# define _ISOC11_SOURCE 1
# undef  _POSIX_SOURCE
# define _POSIX_SOURCE  1
# undef  _POSIX_C_SOURCE
# define _POSIX_C_SOURCE        200809L
# undef  _XOPEN_SOURCE
# define _XOPEN_SOURCE  700
# undef  _XOPEN_SOURCE_EXTENDED
# define _XOPEN_SOURCE_EXTENDED 1
# undef  _LARGEFILE64_SOURCE
# define _LARGEFILE64_SOURCE    1
# undef  _DEFAULT_SOURCE
# define _DEFAULT_SOURCE        1
# undef  _ATFILE_SOURCE
# define _ATFILE_SOURCE 1
#endif

and ncursesw looks for _XOPEN_SOURCE exclusively:

#ifndef NCURSES_WIDECHAR
#if defined(_XOPEN_SOURCE_EXTENDED) || (defined(_XOPEN_SOURCE) && (_XOPEN_SOURCE - 0 >= 500))
#define NCURSES_WIDECHAR 1
#else
#define NCURSES_WIDECHAR 0
#endif
#endif /* NCURSES_WIDECHAR */


So I do think that if musl wants to support _GNU_SOURCE, it should also auto-define;
however it doesn't, see https://git.musl-libc.org/cgit/musl/tree/include/features.h.

Alternatively, ncurses should be patched to look at _GNU_SOURCE in addition to
_XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED.

In any case, it's not a QEMU bug.  Other packages likely are going to have the same
issue, and fixing all of them is the wrong course of action.

Paolo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Laszlo Ersek]

On 02/17/17 09:56, Paolo Bonzini wrote:
> 
> 
> On 16/02/2017 18:23, Laszlo Ersek wrote:
>> On 02/16/17 17:58, Paolo Bonzini wrote:
>>>
>>>
>>> On 16/02/2017 17:30, Chad Joan wrote:
>>>> Hello,
>>>>
>>>> This is a one-line patch to the configure script that will allow QEMU to be
>>>> built on musl-libc based Linux systems.  This problem is only noticeable
>>>> when QEMU is built with --enable-curses.
>>>>
>>>> Detailed reading material if you want to know where this came from:
>>>> https://bugs.gentoo.org/show_bug.cgi?id=609364
>>>
>>> Hi,
>>>
>>> can you explain exactly which function is missing without
>>> -D_XOPEN_SOURCE=500?  If it is curses' wide-char functions, why does it
>>> fail with musl but not with glibc?  Is _XOPEN_SOURCE always defined by
>>> glibc if you have _D_GNU_SOURCE?
>>
>> It is not necessarily auto-defined, but the effect is "as if".
> 
> Ok, so the bug is that glibc _does_ auto-define _XOPEN_SOURCE:
> 
> #ifdef _GNU_SOURCE
> # undef  _ISOC95_SOURCE
> # define _ISOC95_SOURCE 1
> # undef  _ISOC99_SOURCE
> # define _ISOC99_SOURCE 1
> # undef  _ISOC11_SOURCE
> # define _ISOC11_SOURCE 1
> # undef  _POSIX_SOURCE
> # define _POSIX_SOURCE  1
> # undef  _POSIX_C_SOURCE
> # define _POSIX_C_SOURCE        200809L
> # undef  _XOPEN_SOURCE
> # define _XOPEN_SOURCE  700
> # undef  _XOPEN_SOURCE_EXTENDED
> # define _XOPEN_SOURCE_EXTENDED 1
> # undef  _LARGEFILE64_SOURCE
> # define _LARGEFILE64_SOURCE    1
> # undef  _DEFAULT_SOURCE
> # define _DEFAULT_SOURCE        1
> # undef  _ATFILE_SOURCE
> # define _ATFILE_SOURCE 1
> #endif
> 
> and ncursesw looks for _XOPEN_SOURCE exclusively:
> 
> #ifndef NCURSES_WIDECHAR
> #if defined(_XOPEN_SOURCE_EXTENDED) || (defined(_XOPEN_SOURCE) && (_XOPEN_SOURCE - 0 >= 500))
> #define NCURSES_WIDECHAR 1
> #else
> #define NCURSES_WIDECHAR 0
> #endif
> #endif /* NCURSES_WIDECHAR */
> 
> 
> So I do think that if musl wants to support _GNU_SOURCE, it should also auto-define;
> however it doesn't, see https://git.musl-libc.org/cgit/musl/tree/include/features.h.
> 
> Alternatively, ncurses should be patched to look at _GNU_SOURCE in addition to
> _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED.
> 
> In any case, it's not a QEMU bug.  Other packages likely are going to have the same
> issue, and fixing all of them is the wrong course of action.

I'm somewhat unsure -- I think it's debatable whether other libc (or
Curses) implementations should care about _GNU_SOURCE at all. After all,
_GNU_SOURCE says "GNU" in the name.

However, defining _XOPEN_SOURCE as 500 is specified in an industry
standard (SUSv2):

http://pubs.opengroup.org/onlinepubs/7908799/xsh/compilation.html

and the same standard defines Curses too (and it describes the
relationship of Curses interfaces with _XOPEN_SOURCE):

http://pubs.opengroup.org/onlinepubs/7908799/xcurses/implement.html

So I think if an application wants to support Curses *without*
necessarily depending on glibc, then the application should explicitly
define _XOPEN_SOURCE. If that other libc and that Curses implementation
don't care about GNU, only SUSv2 (which is entirely in their right),
then they'll look only for _XOPEN_SOURCE.

And, if other packages have the same issue, then they all should be
patched, for better standards conformance.

Of course, *if* that other C library, and the Curses implementation,
claim compatibility with glibc as far as _GNU_SOURCE goes, *then* I
fully agree with you. (IOW, it depends on the condition that you
formulated above, "if musl wants to support _GNU_SOURCE".)

Same for ncurses -- do they aim at SUSv2 conformance, or do they intend
to consider _GNU_SOURCE specifically?

Thanks
Laszlo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Paolo Bonzini]

On 17/02/2017 10:17, Laszlo Ersek wrote:
> Of course, *if* that other C library, and the Curses implementation,
> claim compatibility with glibc as far as _GNU_SOURCE goes, *then* I
> fully agree with you. (IOW, it depends on the condition that you
> formulated above, "if musl wants to support _GNU_SOURCE".)

They at least try, since their features.h supports it.

> Same for ncurses -- do they aim at SUSv2 conformance, or do they intend
> to consider _GNU_SOURCE specifically?

Well, ncurses is a GNU project so I suppose they mostly care about
whatever Autoconf does.  And indeed Autoconf does have some special
casing for -D_XOPEN_SOURCE=500:

  AC_CACHE_CHECK([whether _XOPEN_SOURCE should be defined],
    [ac_cv_should_define__xopen_source],
    [ac_cv_should_define__xopen_source=no
     AC_COMPILE_IFELSE(
       [AC_LANG_PROGRAM([[
          #include <wchar.h>
          mbstate_t x;]])],
       [],
       [AC_COMPILE_IFELSE(
          [AC_LANG_PROGRAM([[
             #define _XOPEN_SOURCE 500
             #include <wchar.h>
             mbstate_t x;]])],
          [ac_cv_should_define__xopen_source=yes])])])
  test $ac_cv_should_define__xopen_source = yes &&
    AC_DEFINE([_XOPEN_SOURCE], [500])

If musl needs _XOPEN_SOURCE for wchar.h, then it would be worth working
around in QEMU.  If it's just happening with ncurses, however, it'd be a
musl bug, which should be fixed in musl or worked around in ncurses.

Chad, can you check whether the programs below fails with musl:

/* First program */
#define _GNU_SOURCE
#include <wchar.h>
mbstate_t x;

/* Second program */
#include <wchar.h>
mbstate_t x;

/* Third program */
#define _XOPEN_SOURCE 500
#include <wchar.h>
mbstate_t x;

Just compile them with "gcc foo.c -O2".

Thanks,

Paolo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Chad Joan]

Sure.

I'll see your -O2 and raise you a -c ;)

It went like this:

$ gcc -c first.c -O2
$ gcc -c second.c -O2
$ gcc -c third.c -O2

No complaints from gcc whatsoever (unless I drop the -c, then ld complains
about the undefined reference to main, which I think we all expect).

Just incase you expected one of them to fail (ex: as a control test), I
verified that the original snippet from the configure script still causes
an error on the system:
#########
$ cat << EOF > reference.c
> #include <locale.h>
> #include <curses.h>
> #include <wchar.h>
> int main(void) {
>   const char *s = curses_version();
>   wchar_t wch = L'w';
>   setlocale(LC_ALL, "");
>   resize_term(0, 0);
>   addwstr(L"wide chars\n");
>   addnwstr(&wch, 1);
>   add_wch(WACS_DEGREE);
>   return s != 0;
> }
> EOF
$ gcc reference.c -O2
reference.c: In function 'main':
reference.c:11:11: error: 'WACS_DEGREE' undeclared (first use in this
function)
   add_wch(WACS_DEGREE);
           ^
reference.c:11:11: note: each undeclared identifier is reported only once
for each function it appears in
#########

Hope that helps.

On Fri, Feb 17, 2017 at 6:11 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:

>
>
> On 17/02/2017 10:17, Laszlo Ersek wrote:
> > Of course, *if* that other C library, and the Curses implementation,
> > claim compatibility with glibc as far as _GNU_SOURCE goes, *then* I
> > fully agree with you. (IOW, it depends on the condition that you
> > formulated above, "if musl wants to support _GNU_SOURCE".)
>
> They at least try, since their features.h supports it.
>
> > Same for ncurses -- do they aim at SUSv2 conformance, or do they intend
> > to consider _GNU_SOURCE specifically?
>
> Well, ncurses is a GNU project so I suppose they mostly care about
> whatever Autoconf does.  And indeed Autoconf does have some special
> casing for -D_XOPEN_SOURCE=500:
>
>   AC_CACHE_CHECK([whether _XOPEN_SOURCE should be defined],
>     [ac_cv_should_define__xopen_source],
>     [ac_cv_should_define__xopen_source=no
>      AC_COMPILE_IFELSE(
>        [AC_LANG_PROGRAM([[
>           #include <wchar.h>
>           mbstate_t x;]])],
>        [],
>        [AC_COMPILE_IFELSE(
>           [AC_LANG_PROGRAM([[
>              #define _XOPEN_SOURCE 500
>              #include <wchar.h>
>              mbstate_t x;]])],
>           [ac_cv_should_define__xopen_source=yes])])])
>   test $ac_cv_should_define__xopen_source = yes &&
>     AC_DEFINE([_XOPEN_SOURCE], [500])
>
> If musl needs _XOPEN_SOURCE for wchar.h, then it would be worth working
> around in QEMU.  If it's just happening with ncurses, however, it'd be a
> musl bug, which should be fixed in musl or worked around in ncurses.
>
> Chad, can you check whether the programs below fails with musl:
>
> /* First program */
> #define _GNU_SOURCE
> #include <wchar.h>
> mbstate_t x;
>
> /* Second program */
> #include <wchar.h>
> mbstate_t x;
>
> /* Third program */
> #define _XOPEN_SOURCE 500
> #include <wchar.h>
> mbstate_t x;
>
> Just compile them with "gcc foo.c -O2".
>
> Thanks,
>
> Paolo
>

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 17 February 2017 at 08:56, Paolo Bonzini <pbonzini@redhat.com> wrote:
> So I do think that if musl wants to support _GNU_SOURCE, it should also auto-define;
> however it doesn't, see https://git.musl-libc.org/cgit/musl/tree/include/features.h.

Yes, I agree that this looks like a musl bug. I spoke to a musl
developer I happen to know, and he agreed to submit a patch to
see what the other musl devs think.

In the meantime, we should probably work around this system
bug for the benefit of the existing musl installs in the field.

Defining _XOPEN_SOURCE is easy enough, and I think we should
do it unconditionally. We should check what effect this has
on the BSD hosts though I guess. (You could argue that we
should be defining _XOPEN_SOURCE anyway for the benefit of
the non-glibc BSD/Solaris/etc platforms.)

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Paolo Bonzini]

On 17/02/2017 11:18, Peter Maydell wrote:
> Defining _XOPEN_SOURCE is easy enough, and I think we should
> do it unconditionally. We should check what effect this has
> on the BSD hosts though I guess. (You could argue that we
> should be defining _XOPEN_SOURCE anyway for the benefit of
> the non-glibc BSD/Solaris/etc platforms.)

Sounds good, then I think we should define it to 700 just like glibc does.

Paolo

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 17 February 2017 at 11:20, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 17/02/2017 11:18, Peter Maydell wrote:
>> Defining _XOPEN_SOURCE is easy enough, and I think we should
>> do it unconditionally. We should check what effect this has
>> on the BSD hosts though I guess. (You could argue that we
>> should be defining _XOPEN_SOURCE anyway for the benefit of
>> the non-glibc BSD/Solaris/etc platforms.)
>
> Sounds good, then I think we should define it to 700 just like glibc does.

Unfortunately this idea turns out to break OSX compiles,
because on OSX saying _XOPEN_SOURCE=anything disables
all the non-X/Open APIs (which you get by default, and
some of which like mkdtemp we use).

Looking at the manpage for Solaris
https://www.freebsd.org/cgi/man.cgi?query=standards&apropos=0&sektion=0&manpath=SunOS+5.8&format=html
it seems to also go with "_XOPEN_SOURCE means *only* those
functions", though its mechanism for saying "and the other
stuff too" is different (define __EXTENSION__).

So perhaps we'd better stick with "only define this for
Linux hosts".

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Rainer Müller]

On 2017-02-17 17:57, Peter Maydell wrote:
> On 17 February 2017 at 11:20, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>>
>> On 17/02/2017 11:18, Peter Maydell wrote:
>>> Defining _XOPEN_SOURCE is easy enough, and I think we should
>>> do it unconditionally. We should check what effect this has
>>> on the BSD hosts though I guess. (You could argue that we
>>> should be defining _XOPEN_SOURCE anyway for the benefit of
>>> the non-glibc BSD/Solaris/etc platforms.)
>>
>> Sounds good, then I think we should define it to 700 just like glibc does.
> 
> Unfortunately this idea turns out to break OSX compiles,
> because on OSX saying _XOPEN_SOURCE=anything disables
> all the non-X/Open APIs (which you get by default, and
> some of which like mkdtemp we use).

A bit late to this thread, but the original problem was also reported
for Mac OS X with --enable-curses in MacPorts. The build fails with the
same symptoms as in the original report.

https://trac.macports.org/ticket/53929

As you identified, the problem is that ncurses expects the define
_XOPEN_SOURCE >= 500 to enable the wide-char function declarations.

The solution to retain access to non-standard API on Mac OS X would be
to also define _DARWIN_C_SOURCE which enables extensions.

$ cat foo.c
#include <unistd.h>
int main() {
    mkdtemp("/tmp/test-XXXXXX");
}
$ cc -D_XOPEN_SOURCE=500 -c foo.c
foo.c:4:5: warning: implicit declaration of function 'mkdtemp' is
invalid in C99 [-Wimplicit-function-declaration]
    mkdtemp("/tmp/test-XXXXXX");
    ^
1 warning generated.
$ cc -D_XOPEN_SOURCE=500 -D_DARWIN_C_SOURCE -c foo.c
$

A quick test on current master with configure patched to define
  QEMU_CFLAGS="-D_XOPEN_SOURCE=500 -D_DARWIN_C_SOURCE $QEMU_CFLAGS"
compiled fine for both a default configure and with --enable-curses.

Rainer

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 6 April 2017 at 19:15, Rainer Müller <raimue@codingfarm.de> wrote:
> A bit late to this thread, but the original problem was also reported
> for Mac OS X with --enable-curses in MacPorts. The build fails with the
> same symptoms as in the original report.
>
> https://trac.macports.org/ticket/53929
>
> As you identified, the problem is that ncurses expects the define
> _XOPEN_SOURCE >= 500 to enable the wide-char function declarations.
>
> The solution to retain access to non-standard API on Mac OS X would be
> to also define _DARWIN_C_SOURCE which enables extensions.

Thanks for the report. I have a feeling that fixing this bug
got lost somewhere -- there was a long thread about it but I
don't think an actual patch ever came out of it :-(

At this point (very near to release of 2.9) I think I'd rather
postpone fixing this until after release. It should be possible
to work around it with an --extra-cflags configure setting,
and our current behaviour is something we've had for many
releases now.

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 6 April 2017 at 19:36, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 6 April 2017 at 19:15, Rainer Müller <raimue@codingfarm.de> wrote:
>> A bit late to this thread, but the original problem was also reported
>> for Mac OS X with --enable-curses in MacPorts. The build fails with the
>> same symptoms as in the original report.
>>
>> https://trac.macports.org/ticket/53929
>>
>> As you identified, the problem is that ncurses expects the define
>> _XOPEN_SOURCE >= 500 to enable the wide-char function declarations.
>>
>> The solution to retain access to non-standard API on Mac OS X would be
>> to also define _DARWIN_C_SOURCE which enables extensions.
>
> Thanks for the report. I have a feeling that fixing this bug
> got lost somewhere -- there was a long thread about it but I
> don't think an actual patch ever came out of it :-(

I've got back to looking at this, and it turns out that the
situation is a complete mess. In particular, on OSX if you
define _XOPEN_SOURCE=600 then it causes _POSIX_C_SOURCE to be
set such that the getpagesize() prototype is not provided, and
even if you set _DARWIN_C_SOURCE this does not override that.

Similarly, on FreeBSD if you define _XOPEN_SOURCE then it
doesn't provide any APIs that aren't in that standard and there
is no provided mechanism at all to say "and also the FreeBSD
APIs" please.

My conclusion from this is that we must not set _XOPEN_SOURCE,
but instead have to tell ncurses directly that we definitely
want the widechar prototypes, by defining NCURSES_WIDECHAR.
Since ncurses 20111030 this is sufficient I think; if you're
still using an older ncurses then you'll need to upgrade it
(older ncurses ignore NCURSES_WIDECHAR and only pay attention
to _XOPEN_SOURCE_EXTENDED, etc).

thanks
-- PMM

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]

On 02/16/2017 10:30 AM, Chad Joan wrote:
> Hello,
> 
> This is a one-line patch to the configure script that will allow QEMU to be
> built on musl-libc based Linux systems.  This problem is only noticeable
> when QEMU is built with --enable-curses.
> 
> Detailed reading material if you want to know where this came from:
> https://bugs.gentoo.org/show_bug.cgi?id=609364
> 
> Hope that helps!
> - Chad
> 
> 
> qemu-2.8.0-musl-ncurses.patch
> 


Thanks for the patch.  However, it is not in the usual format (we prefer
'git send-email's inline format, rather than attachements), and is
missing a Signed-off-by: designation, so it cannot be applied without
that being fixed first.  More information at:

http://wiki.qemu-project.org/Contribute/SubmitAPatch

> 
> diff --git configure configure
> index 1c9655e..9f83176 100755
> --- configure
> +++ configure
> @@ -682,6 +682,7 @@ Haiku)
>    vhost_net="yes"
>    vhost_scsi="yes"
>    vhost_vsock="yes"
> +  QEMU_CFLAGS="-D_XOPEN_SOURCE=500 $QEMU_CFLAGS"
>    QEMU_INCLUDES="-I\$(SRC_PATH)/linux-headers -I$(pwd)/linux-headers $QEMU_INCLUDES"
>  ;;
>  esac
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] Fix build break during configuration on musl-libc based Linux systems.

[Peter Maydell]

On 16 February 2017 at 16:30, Chad Joan <chadjoan@gmail.com> wrote:
> Hello,
>
> This is a one-line patch to the configure script that will allow QEMU to be
> built on musl-libc based Linux systems.  This problem is only noticeable
> when QEMU is built with --enable-curses.
>
> Detailed reading material if you want to know where this came from:
> https://bugs.gentoo.org/show_bug.cgi?id=609364

Thanks for the bug report and the patch. I'm a bit confused
about why it's necessary, though. We already define _GNU_SOURCE,
which the musl docs say implies _XOPEN_SOURCE:
 https://www.musl-libc.org/doc/1.0.0/manual.html
Is that not true for your musl, or is our configure script
failing to pass -D_GNU_SOURCE when building the curses test program?

If we do need an explicit define of _XOPEN_SOURCE I think we should
do it at the same place where we add -D_GNU_SOURCE to the cflags.

thanks
-- PMM