musl thoughts

musl thoughts

[Andreas Müller]

Just prepared meta-networking/networkmanager update and spent hours to
get musl patches applied. Have no idea if musl will build and
currently all my machines are building.

For me - not using musl and looking forward to have a beer with Khem
explaining me why I should want musl [1] - it is just useless effort
and patches are rejected often due to failures on musl build.

So on my way home I though about the options I see to handle musl specifics:

1. Continue as we do - patch recipe-wise: This is lots of efforts and
requires resources we don't have. Most patches are created by Khem and
it would be better for all of us to save his time for other tasks.
2. Sending musl related patches upstream: Surely the cleanest approach
but even more effort because upstream maintainers all have different
thoughts. Some of them might hear of musl for the first time and are
possibly not keen on patches they do not see a need for.
3. Change our musl slightly: Many patches we currently have are
trivial. Missing headers or #defines for missing functions... So if we
add few headers
  * Empty chunks for e.g <net/ethernet.h>
  * Add defines as #define strndupa(x,s) strncpy(alloca(strlen(x)+1),x,s)
  * ...

As you might guess I'd prefer 3 because:
+ Many patches can go and don't need maintenance on upstream refactoring anymore
+ Burden for people sending patches would be reduced
+ Recipes not building with musl currently might work without further
modification
+ Just in case musl stops (we have seen this before with others e.g
ulibc) the cleanup would be reduced

Of course there are disadvantages:
- We turn musl into glibc direction. Posix-purists do not like that.
- Configuration scripts already aware of musl and deciding upon the
existence of headers missing in musl might do the wrong thing
- ?

Some other ideas?

Andreas

[1] http://lists.openembedded.org/pipermail/openembedded-devel/2018-March/193521.html

Re: musl thoughts

[akuster808]

On 3/21/19 6:11 PM, Andreas Müller wrote:
> Just prepared meta-networking/networkmanager update and spent hours to
> get musl patches applied. Have no idea if musl will build and
> currently all my machines are building.
>
> For me - not using musl and looking forward to have a beer with Khem
> explaining me why I should want musl [1] - it is just useless effort
> and patches are rejected often due to failures on musl build.
>
> So on my way home I though about the options I see to handle musl specifics:
>
> 1. Continue as we do - patch recipe-wise: This is lots of efforts and
> requires resources we don't have. Most patches are created by Khem and
> it would be better for all of us to save his time for other tasks.
> 2. Sending musl related patches upstream: Surely the cleanest approach
> but even more effort because upstream maintainers all have different
> thoughts. Some of them might hear of musl for the first time and are
> possibly not keen on patches they do not see a need for.
> 3. Change our musl slightly: Many patches we currently have are
> trivial. Missing headers or #defines for missing functions... So if we
> add few headers
>   * Empty chunks for e.g <net/ethernet.h>
>   * Add defines as #define strndupa(x,s) strncpy(alloca(strlen(x)+1),x,s)
>   * ...
>
> As you might guess I'd prefer 3 because:
> + Many patches can go and don't need maintenance on upstream refactoring anymore
> + Burden for people sending patches would be reduced
> + Recipes not building with musl currently might work without further
> modification
> + Just in case musl stops (we have seen this before with others e.g
> ulibc) the cleanup would be reduced

Didn't we do something like this with glibc? We carried several patches
that where rejected and just recently got removed so there is a
precedent for carrying OE specific changes  so it seems like a
reasonable approach.

- armin
>
> Of course there are disadvantages:
> - We turn musl into glibc direction. Posix-purists do not like that.
> - Configuration scripts already aware of musl and deciding upon the
> existence of headers missing in musl might do the wrong thing
> - ?
>
> Some other ideas?
>
> Andreas
>
> [1] http://lists.openembedded.org/pipermail/openembedded-devel/2018-March/193521.html

Re: musl thoughts

[Adrian Bunk]

On Fri, Mar 22, 2019 at 07:53:03AM -0700, akuster808 wrote:
> On 3/21/19 6:11 PM, Andreas Müller wrote:
>...
> > As you might guess I'd prefer 3 because:
> > + Many patches can go and don't need maintenance on upstream refactoring anymore
> > + Burden for people sending patches would be reduced
> > + Recipes not building with musl currently might work without further
> > modification
> > + Just in case musl stops (we have seen this before with others e.g
> > ulibc) the cleanup would be reduced
> 
> Didn't we do something like this with glibc? We carried several patches
> that where rejected and just recently got removed so there is a
> precedent for carrying OE specific changes  so it seems like a
> reasonable approach.

The glibc patches that got removed were patches where it can be 
questioned whether they should have ever been applied.

glibc still carries far too many patches that simply shouldn't be there.

All this is different from the musl situation where some kind of 
patching is required somewhere at least short-term.

> - armin

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Adrian Bunk]

On Fri, Mar 22, 2019 at 02:11:35AM +0100, Andreas Müller wrote:
>...
> 3. Change our musl slightly: Many patches we currently have are
> trivial. Missing headers or #defines for missing functions... So if we
> add few headers
>   * Empty chunks for e.g <net/ethernet.h>
>...
> Some other ideas?

Not upstreamable hacks aren't a long-term sustainable way,
no matter where you do the patching.

net/ethernet.h header problems are requiring 2 patches in NM.

It would be good to get whatever is the actual root cause fixed properly,
and that fix upstreamed.

E.g. I wonder whether this was supposed to be fixed by
0001-if_ether-move-muslc-ethhdr-protection-to-uapi-file.patch

Or is this patch in linux-libc-headers even the cause of the problem?

linux-libc-headers applying 6 patches just for musl is not a good sign 
in any case.
Is that much patching also done by other distributions that use musl?
Why are these patches Upstream-Status: Pending/Submitted for a  long 
time without having been applied upstream?

Long-term it would be less work if everything would get fixed properly
with fixes reviewed and applied by upstream.

> Andreas
>...

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Khem Raj]

On Fri, Mar 22, 2019 at 1:03 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Fri, Mar 22, 2019 at 02:11:35AM +0100, Andreas Müller wrote:
> >...
> > 3. Change our musl slightly: Many patches we currently have are
> > trivial. Missing headers or #defines for missing functions... So if we
> > add few headers
> >   * Empty chunks for e.g <net/ethernet.h>
> >...
> > Some other ideas?
>
> Not upstreamable hacks aren't a long-term sustainable way,
> no matter where you do the patching.
>
> net/ethernet.h header problems are requiring 2 patches in NM.
>
> It would be good to get whatever is the actual root cause fixed properly,
> and that fix upstreamed.
>
> E.g. I wonder whether this was supposed to be fixed by
> 0001-if_ether-move-muslc-ethhdr-protection-to-uapi-file.patch
>
> Or is this patch in linux-libc-headers even the cause of the problem?
>
> linux-libc-headers applying 6 patches just for musl is not a good sign
> in any case.
> Is that much patching also done by other distributions that use musl?
> Why are these patches Upstream-Status: Pending/Submitted for a  long
> time without having been applied upstream?
>
> Long-term it would be less work if everything would get fixed properly
> with fixes reviewed and applied by upstream.
>

agreed, meanwhile we have to also ensure that musl users keep going and make it
possible, if OE devs are fixing musl related issues in upstream
packages then it is also
good reputation for the project. The real thing is to find time and
propose the patches upstream
more than often they get accepted.

> > Andreas
> >...
>
> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>

Re: musl thoughts

[Adrian Bunk]

On Fri, Mar 22, 2019 at 03:20:19PM -0700, Khem Raj wrote:
> On Fri, Mar 22, 2019 at 1:03 PM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Fri, Mar 22, 2019 at 02:11:35AM +0100, Andreas Müller wrote:
> > >...
> > > 3. Change our musl slightly: Many patches we currently have are
> > > trivial. Missing headers or #defines for missing functions... So if we
> > > add few headers
> > >   * Empty chunks for e.g <net/ethernet.h>
> > >...
> > > Some other ideas?
> >
> > Not upstreamable hacks aren't a long-term sustainable way,
> > no matter where you do the patching.
> >
> > net/ethernet.h header problems are requiring 2 patches in NM.
> >
> > It would be good to get whatever is the actual root cause fixed properly,
> > and that fix upstreamed.
> >
> > E.g. I wonder whether this was supposed to be fixed by
> > 0001-if_ether-move-muslc-ethhdr-protection-to-uapi-file.patch
> >
> > Or is this patch in linux-libc-headers even the cause of the problem?
> >
> > linux-libc-headers applying 6 patches just for musl is not a good sign
> > in any case.
> > Is that much patching also done by other distributions that use musl?
> > Why are these patches Upstream-Status: Pending/Submitted for a  long
> > time without having been applied upstream?
> >
> > Long-term it would be less work if everything would get fixed properly
> > with fixes reviewed and applied by upstream.
> 
> agreed, meanwhile we have to also ensure that musl users keep going and make it
> possible, if OE devs are fixing musl related issues in upstream
> packages then it is also good reputation for the project.
>...

Some of the NM patches are obviously just quick hacks to workaround bugs 
that are elsewhere. Trying to push such stuff upstream would not exactly
improve the reputation.

A problem with a "keep going" approach is that you might end up spending 
much work on working around symptoms, or even working around OE-specific 
bugs without noticing that they are OE-specific.

Building an image (with things like NM included) for musl builds for me, 
and it also builds for me with all 6 musl patches not applied to the 
kernel headers.

Are these patches still required at all?
Could they be the cause of other musl related issues in OE?

Patching leaf packages can create a huge mess if the core packages are 
not in a good state.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Khem Raj]

On Thu, Mar 21, 2019 at 6:11 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
>
> Just prepared meta-networking/networkmanager update and spent hours to
> get musl patches applied. Have no idea if musl will build and
> currently all my machines are building.
>
> For me - not using musl and looking forward to have a beer with Khem

you will have to be sober, because I will show you data that you need
to understand :)

> explaining me why I should want musl [1] - it is just useless effort
> and patches are rejected often due to failures on musl build.
>

If you make a valid case, and seek others to chime in then it could be
considered.
but its desired to keep our musl port to be healthy.

> So on my way home I though about the options I see to handle musl specifics:
>
> 1. Continue as we do - patch recipe-wise: This is lots of efforts and
> requires resources we don't have. Most patches are created by Khem and
> it would be better for all of us to save his time for other tasks.
> 2. Sending musl related patches upstream: Surely the cleanest approach
> but even more effort because upstream maintainers all have different
> thoughts. Some of them might hear of musl for the first time and are
> possibly not keen on patches they do not see a need for.
> 3. Change our musl slightly: Many patches we currently have are
> trivial. Missing headers or #defines for missing functions... So if we
> add few headers
>   * Empty chunks for e.g <net/ethernet.h>
>   * Add defines as #define strndupa(x,s) strncpy(alloca(strlen(x)+1),x,s)
>   * ...
>

There are certain design aspects of musl which are actually turning
out to be good
e.g. there is no __MUSL__ define, so non-portable code can not be
hidden which is a good
thing, similarly, adding these defines to musl itself would probably
be not serving the purpose
of cleaning up the apps and making them portable. The fact they do
exist as patches is a reminder
to clean the application code or rethink to fix it differently. This
has actually resulted in cleaning up
many packages, several of musl initialed patches regarding portability
has been accepted in upstream
and some are not, and some are not proposed. While the eyes may hurt
to see them, it does serve a
good reminder of whats needed for a given package.

> As you might guess I'd prefer 3 because:
> + Many patches can go and don't need maintenance on upstream refactoring anymore
> + Burden for people sending patches would be reduced
> + Recipes not building with musl currently might work without further
> modification
> + Just in case musl stops (we have seen this before with others e.g
> ulibc) the cleanup would be reduced
>

this points to the above as I said. Musl stays or goes cleaning them
up would be good regardless

> Of course there are disadvantages:
> - We turn musl into glibc direction. Posix-purists do not like that.

right. and IMO its valuable for apps to be a bit portable.

> - Configuration scripts already aware of musl and deciding upon the
> existence of headers missing in musl might do the wrong thing
> - ?
>
> Some other ideas?
>

I think we should propose the fixes upstream and see what is acceptable.

> Andreas
>
> [1] http://lists.openembedded.org/pipermail/openembedded-devel/2018-March/193521.html

Re: musl thoughts

[Adrian Bunk]

On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
>...
> There are certain design aspects of musl which are actually turning
> out to be good
> e.g. there is no __MUSL__ define, so non-portable code can not be
> hidden which is a good thing,
>...

Please take a closer look at some of the musl changes to NM that made
upgrading NM so hard for Andreas.

+#if defined(__GLIBC__)
 #include <net/ethernet.h>
+#else /* musl libc */
+#define ETH_ALEN       6               /* Octets in one ethernet addr   */
+#endif

Using __GLIBC__ in workarounds for bugs in musl is wrong,
and cannot be upstreamed since it would do the wrong thing
on other non-broken C libraries.

> While the eyes may hurt
> to see them, it does serve a
> good reminder of whats needed for a given package.
>...

Who is responsible for fixing the root causes of such bugs in musl,
so that the workaround patches can be dropped from packages like NM?

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Andreas Müller]

On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> >...
> > There are certain design aspects of musl which are actually turning
> > out to be good
> > e.g. there is no __MUSL__ define, so non-portable code can not be
> > hidden which is a good thing,
> >...
>
> Please take a closer look at some of the musl changes to NM that made
> upgrading NM so hard for Andreas.
>
> +#if defined(__GLIBC__)
>  #include <net/ethernet.h>
> +#else /* musl libc */
> +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> +#endif
>
> Using __GLIBC__ in workarounds for bugs in musl is wrong,
> and cannot be upstreamed since it would do the wrong thing
> on other non-broken C libraries.
>
> > While the eyes may hurt
> > to see them, it does serve a
> > good reminder of whats needed for a given package.
> >...
>
> Who is responsible for fixing the root causes of such bugs in musl,
> so that the workaround patches can be dropped from packages like NM?
>
> cu
> Adrian
If I am not mistaken nobody is responsible. It is recipe wise: Sending
out a patch that fails for musl is rejected usually.

The last example could be fixed easily at musl shipping a ethernet.h containing
#define ETH_ALEN       6

or at NM
#ifndef ETH_ALEN
#define ETH_ALEN       6
#endif

Andreas

Re: musl thoughts

[Adrian Bunk]

On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > >...
> > > There are certain design aspects of musl which are actually turning
> > > out to be good
> > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > hidden which is a good thing,
> > >...
> >
> > Please take a closer look at some of the musl changes to NM that made
> > upgrading NM so hard for Andreas.
> >
> > +#if defined(__GLIBC__)
> >  #include <net/ethernet.h>
> > +#else /* musl libc */
> > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > +#endif
> >
> > Using __GLIBC__ in workarounds for bugs in musl is wrong,
> > and cannot be upstreamed since it would do the wrong thing
> > on other non-broken C libraries.
> >
> > > While the eyes may hurt
> > > to see them, it does serve a
> > > good reminder of whats needed for a given package.
> > >...
> >
> > Who is responsible for fixing the root causes of such bugs in musl,
> > so that the workaround patches can be dropped from packages like NM?
> >
> > cu
> > Adrian
> If I am not mistaken nobody is responsible. It is recipe wise: Sending
> out a patch that fails for musl is rejected usually.

As you have experienced, it does create a huge technical debt to ship 
workaround patches in several recipes instead of fixing the bug in musl.

> The last example could be fixed easily at musl shipping a ethernet.h containing
> #define ETH_ALEN       6
>...

That's already shipped by musl.

But there seems to be some incompatibility between musl and the 
kernel headers used by musl.

This has to be sorted out in musl and/or the kernel headers.

> Andreas

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Andreas Müller]

On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > >
> > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > >...
> > > > There are certain design aspects of musl which are actually turning
> > > > out to be good
> > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > hidden which is a good thing,
> > > >...
> > >
> > > Please take a closer look at some of the musl changes to NM that made
> > > upgrading NM so hard for Andreas.
> > >
> > > +#if defined(__GLIBC__)
> > >  #include <net/ethernet.h>
> > > +#else /* musl libc */
> > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > +#endif
> > >
> > > Using __GLIBC__ in workarounds for bugs in musl is wrong,
> > > and cannot be upstreamed since it would do the wrong thing
> > > on other non-broken C libraries.
> > >
> > > > While the eyes may hurt
> > > > to see them, it does serve a
> > > > good reminder of whats needed for a given package.
> > > >...
> > >
> > > Who is responsible for fixing the root causes of such bugs in musl,
> > > so that the workaround patches can be dropped from packages like NM?
> > >
> > > cu
> > > Adrian
> > If I am not mistaken nobody is responsible. It is recipe wise: Sending
> > out a patch that fails for musl is rejected usually.
>
> As you have experienced, it does create a huge technical debt to ship
> workaround patches in several recipes instead of fixing the bug in musl.
>
> > The last example could be fixed easily at musl shipping a ethernet.h containing
> > #define ETH_ALEN       6
> >...
>
> That's already shipped by musl.
>
> But there seems to be some incompatibility between musl and the
> kernel headers used by musl.
>
> This has to be sorted out in musl and/or the kernel headers.
>
Although I did not want to I'll start a musl build to create a cleanup
for NM (and maybe) other musl patches.

Andreas

Re: musl thoughts

[Andrea Adami]

On Sat, Mar 23, 2019 at 11:00 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
>
> On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > >
> > > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > > >...
> > > > > There are certain design aspects of musl which are actually turning
> > > > > out to be good
> > > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > > hidden which is a good thing,
> > > > >...
> > > >
> > > > Please take a closer look at some of the musl changes to NM that made
> > > > upgrading NM so hard for Andreas.
> > > >
> > > > +#if defined(__GLIBC__)
> > > >  #include <net/ethernet.h>
> > > > +#else /* musl libc */
> > > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > > +#endif
> > > >
> > > > Using __GLIBC__ in workarounds for bugs in musl is wrong,
> > > > and cannot be upstreamed since it would do the wrong thing
> > > > on other non-broken C libraries.
> > > >
> > > > > While the eyes may hurt
> > > > > to see them, it does serve a
> > > > > good reminder of whats needed for a given package.
> > > > >...
> > > >
> > > > Who is responsible for fixing the root causes of such bugs in musl,
> > > > so that the workaround patches can be dropped from packages like NM?
> > > >
> > > > cu
> > > > Adrian
> > > If I am not mistaken nobody is responsible. It is recipe wise: Sending
> > > out a patch that fails for musl is rejected usually.
> >
> > As you have experienced, it does create a huge technical debt to ship
> > workaround patches in several recipes instead of fixing the bug in musl.
> >
> > > The last example could be fixed easily at musl shipping a ethernet.h containing
> > > #define ETH_ALEN       6
> > >...
> >
> > That's already shipped by musl.
> >
> > But there seems to be some incompatibility between musl and the
> > kernel headers used by musl.
> >
> > This has to be sorted out in musl and/or the kernel headers.
> >
> Although I did not want to I'll start a musl build to create a cleanup
> for NM (and maybe) other musl patches.
>
> Andreas
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Hi,

I am jumping in a little late to take side with Khem :)

What happens now is that more 'bad' sources (written to suit glibc and
thus not portable) are discovered by the wider base of developers and
autobuilders.

I myself did struggle in the years to maintain a few packages deeply
bound to the kernel (mtd-utils, kexec tools) 'compatible' with klibc
so I know the pain.
As Khem said, this just needs time and efforts to clean up and
upstream the patches.

Cheers
Andrea

Re: musl thoughts

[Adrian Bunk]

On Mon, Mar 25, 2019 at 04:36:44PM +0100, Andrea Adami wrote:
> On Sat, Mar 23, 2019 at 11:00 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
> >
> > On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
> > >
> > > On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > > > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > >
> > > > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > > > >...
> > > > > > There are certain design aspects of musl which are actually turning
> > > > > > out to be good
> > > > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > > > hidden which is a good thing,
> > > > > >...
> > > > >
> > > > > Please take a closer look at some of the musl changes to NM that made
> > > > > upgrading NM so hard for Andreas.
> > > > >
> > > > > +#if defined(__GLIBC__)
> > > > >  #include <net/ethernet.h>
> > > > > +#else /* musl libc */
> > > > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > > > +#endif
>...
> 
> Hi,

Hi Andrea,

> I am jumping in a little late to take side with Khem :)
> 
> What happens now is that more 'bad' sources (written to suit glibc and
> thus not portable) are discovered by the wider base of developers and
> autobuilders.
>...

but this does not apply to this example, which is a problem between
musl itself and the kernel headers.

Code can expect #include <foo.h> to work for any headers, and with any 
order of these headers. If it does not, the 'bad' sources are whatever
sources provide the headers in question.

musl does provide net/ethernet.h, but including it causes a compile
error here.

> As Khem said, this just needs time and efforts to clean up and
> upstream the patches.

Any sane upstream will refuse such patches that add musl-specific hacks
with the wrong define to work around problems in musl itself.

> Cheers
> Andrea

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Andreas Müller]

On Mon, Mar 25, 2019 at 5:10 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Mon, Mar 25, 2019 at 04:36:44PM +0100, Andrea Adami wrote:
> > On Sat, Mar 23, 2019 at 11:00 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
> > >
> > > On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > >
> > > > On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > > > > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > >
> > > > > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > > > > >...
> > > > > > > There are certain design aspects of musl which are actually turning
> > > > > > > out to be good
> > > > > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > > > > hidden which is a good thing,
> > > > > > >...
> > > > > >
> > > > > > Please take a closer look at some of the musl changes to NM that made
> > > > > > upgrading NM so hard for Andreas.
> > > > > >
> > > > > > +#if defined(__GLIBC__)
> > > > > >  #include <net/ethernet.h>
> > > > > > +#else /* musl libc */
> > > > > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > > > > +#endif
> >...
> >
> > Hi,
>
> Hi Andrea,
>
> > I am jumping in a little late to take side with Khem :)
> >
> > What happens now is that more 'bad' sources (written to suit glibc and
> > thus not portable) are discovered by the wider base of developers and
> > autobuilders.
> >...
>
> but this does not apply to this example, which is a problem between
> musl itself and the kernel headers.
>
> Code can expect #include <foo.h> to work for any headers, and with any
> order of these headers. If it does not, the 'bad' sources are whatever
> sources provide the headers in question.
>
> musl does provide net/ethernet.h, but including it causes a compile
> error here.
Out of curiosity: you have some log?

Andreas

Re: musl thoughts

[Andreas Müller]

On Mon, Mar 25, 2019 at 5:26 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
>
> On Mon, Mar 25, 2019 at 5:10 PM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Mon, Mar 25, 2019 at 04:36:44PM +0100, Andrea Adami wrote:
> > > On Sat, Mar 23, 2019 at 11:00 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
> > > >
> > > > On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > >
> > > > > On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > > > > > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > > >
> > > > > > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > > > > > >...
> > > > > > > > There are certain design aspects of musl which are actually turning
> > > > > > > > out to be good
> > > > > > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > > > > > hidden which is a good thing,
> > > > > > > >...
> > > > > > >
> > > > > > > Please take a closer look at some of the musl changes to NM that made
> > > > > > > upgrading NM so hard for Andreas.
> > > > > > >
> > > > > > > +#if defined(__GLIBC__)
> > > > > > >  #include <net/ethernet.h>
> > > > > > > +#else /* musl libc */
> > > > > > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > > > > > +#endif
> > >...
> > >
> > > Hi,
> >
> > Hi Andrea,
> >
> > > I am jumping in a little late to take side with Khem :)
> > >
> > > What happens now is that more 'bad' sources (written to suit glibc and
> > > thus not portable) are discovered by the wider base of developers and
> > > autobuilders.
> > >...
> >
> > but this does not apply to this example, which is a problem between
> > musl itself and the kernel headers.
> >
> > Code can expect #include <foo.h> to work for any headers, and with any
> > order of these headers. If it does not, the 'bad' sources are whatever
> > sources provide the headers in question.
> >
> > musl does provide net/ethernet.h, but including it causes a compile
> > error here.
> Out of curiosity: you have some log?
>
Looked into this. Found an old musl build failure of networkmanager
[1] but I think the issue has not changed:

| In file included from
TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/net/ethernet.h:10,
|                  from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:28:
| TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/netinet/if_ether.h:111:8:
error: redefinition of 'struct ethhdr'
|  struct ethhdr {
|         ^~~~~~
| In file included from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:26:
| TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/linux/if_ether.h:167:8:
note: originally defined here
| struct ethhdr {
|         ^~~~~~

glibc does not fail because it does include linux header
| /* Get definitions from kernel header file.  */
| #include <linux/if_ether.h>
and does not define struct ethhdr

linux/if_ether.h says:
/* allow libcs like musl to deactivate this, glibc does not implement this. */
#ifndef __UAPI_DEF_ETHHDR
#define __UAPI_DEF_ETHHDR        1
#endif

#if __UAPI_DEF_ETHHDR
struct ethhdr {
    unsigned char    h_dest[ETH_ALEN];    /* destination eth addr    */
    unsigned char    h_source[ETH_ALEN];    /* source ether addr    */
    __be16        h_proto;        /* packet type ID field    */
} __attribute__((packed));
#endif

musl does not include linux header but defines which is differen from
what linux does:
struct ethhdr {
    uint8_t h_dest[ETH_ALEN];
    uint8_t h_source[ETH_ALEN];
    uint16_t h_proto;
};
and later
#define __UAPI_DEF_ETHHDR       0

So for networkmanager there is either some wrong sequence or it
includes linux headers.

And I am still not confident that it is our job to teach umpteen
projects written for linux how to write portable code (oe-core has 147
musl related patches and meta-openembedded has 140)...

[1] http://errors.yoctoproject.org/Errors/Details/198239/

Andreas

Re: musl thoughts

[Andreas Müller]

On Mon, Mar 25, 2019 at 6:15 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
>
> On Mon, Mar 25, 2019 at 5:26 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
> >
> > On Mon, Mar 25, 2019 at 5:10 PM Adrian Bunk <bunk@stusta.de> wrote:
> > >
> > > On Mon, Mar 25, 2019 at 04:36:44PM +0100, Andrea Adami wrote:
> > > > On Sat, Mar 23, 2019 at 11:00 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
> > > > >
> > > > > On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > >
> > > > > > On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > > > > > > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > > > >
> > > > > > > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > > > > > > >...
> > > > > > > > > There are certain design aspects of musl which are actually turning
> > > > > > > > > out to be good
> > > > > > > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > > > > > > hidden which is a good thing,
> > > > > > > > >...
> > > > > > > >
> > > > > > > > Please take a closer look at some of the musl changes to NM that made
> > > > > > > > upgrading NM so hard for Andreas.
> > > > > > > >
> > > > > > > > +#if defined(__GLIBC__)
> > > > > > > >  #include <net/ethernet.h>
> > > > > > > > +#else /* musl libc */
> > > > > > > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > > > > > > +#endif
> > > >...
> > > >
> > > > Hi,
> > >
> > > Hi Andrea,
> > >
> > > > I am jumping in a little late to take side with Khem :)
> > > >
> > > > What happens now is that more 'bad' sources (written to suit glibc and
> > > > thus not portable) are discovered by the wider base of developers and
> > > > autobuilders.
> > > >...
> > >
> > > but this does not apply to this example, which is a problem between
> > > musl itself and the kernel headers.
> > >
> > > Code can expect #include <foo.h> to work for any headers, and with any
> > > order of these headers. If it does not, the 'bad' sources are whatever
> > > sources provide the headers in question.
> > >
> > > musl does provide net/ethernet.h, but including it causes a compile
> > > error here.
> > Out of curiosity: you have some log?
> >
> Looked into this. Found an old musl build failure of networkmanager
> [1] but I think the issue has not changed:
>
> | In file included from
> TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/net/ethernet.h:10,
> |                  from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:28:
> | TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/netinet/if_ether.h:111:8:
> error: redefinition of 'struct ethhdr'
> |  struct ethhdr {
> |         ^~~~~~
> | In file included from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:26:
> | TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/linux/if_ether.h:167:8:
> note: originally defined here
> | struct ethhdr {
> |         ^~~~~~
>
> glibc does not fail because it does include linux header
> | /* Get definitions from kernel header file.  */
> | #include <linux/if_ether.h>
> and does not define struct ethhdr
>
> linux/if_ether.h says:
> /* allow libcs like musl to deactivate this, glibc does not implement this. */
> #ifndef __UAPI_DEF_ETHHDR
> #define __UAPI_DEF_ETHHDR        1
> #endif
>
> #if __UAPI_DEF_ETHHDR
> struct ethhdr {
>     unsigned char    h_dest[ETH_ALEN];    /* destination eth addr    */
>     unsigned char    h_source[ETH_ALEN];    /* source ether addr    */
>     __be16        h_proto;        /* packet type ID field    */
> } __attribute__((packed));
> #endif
>
> musl does not include linux header but defines which is differen from
> what linux does:
> struct ethhdr {
>     uint8_t h_dest[ETH_ALEN];
>     uint8_t h_source[ETH_ALEN];
>     uint16_t h_proto;
> };
> and later
> #define __UAPI_DEF_ETHHDR       0
>
> So for networkmanager there is either some wrong sequence or it
> includes linux headers.
>
> And I am still not confident that it is our job to teach umpteen
> projects written for linux how to write portable code (oe-core has 147
> musl related patches and meta-openembedded has 140)...
>
> [1] http://errors.yoctoproject.org/Errors/Details/198239/
>
Something to add - haven't yet looked into networkmanger:

Do we have the chance to change the sequence of paths headers are searched for

1. musl
2. linux

That might fix some issues

Andreas

> Andreas

Re: musl thoughts

[Adrian Bunk]

On Mon, Mar 25, 2019 at 06:36:19PM +0100, Andreas Müller wrote:
>...
> Something to add - haven't yet looked into networkmanger:
> 
> Do we have the chance to change the sequence of paths headers are searched for
> 
> 1. musl
> 2. linux
> 
> That might fix some issues

It would not.

The headers are in different namespaces, order is defined solely
by the order of #include statements in source and header files.

> Andreas

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Andreas Müller]

On Mon, Mar 25, 2019 at 7:03 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Mon, Mar 25, 2019 at 06:36:19PM +0100, Andreas Müller wrote:
> >...
> > Something to add - haven't yet looked into networkmanger:
> >
> > Do we have the chance to change the sequence of paths headers are searched for
> >
> > 1. musl
> > 2. linux
> >
> > That might fix some issues
>
> It would not.
>
> The headers are in different namespaces, order is defined solely
> by the order of #include statements in source and header files.

Yes I should have skipped that:

* I missed my bus for writing it
* it does not make sense

Andreas
>
> > Andreas
>
> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>

Re: musl thoughts

[Adrian Bunk]

On Mon, Mar 25, 2019 at 06:15:40PM +0100, Andreas Müller wrote:
>...
> Looked into this. Found an old musl build failure of networkmanager
> [1] but I think the issue has not changed:
> 
> | In file included from
> TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/net/ethernet.h:10,
> |                  from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:28:
> | TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/netinet/if_ether.h:111:8:
> error: redefinition of 'struct ethhdr'
> |  struct ethhdr {
> |         ^~~~~~
> | In file included from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:26:
> | TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/linux/if_ether.h:167:8:
> note: originally defined here
> | struct ethhdr {
> |         ^~~~~~
> 
> glibc does not fail because it does include linux header
> | /* Get definitions from kernel header file.  */
> | #include <linux/if_ether.h>
> and does not define struct ethhdr
> 
> linux/if_ether.h says:
> /* allow libcs like musl to deactivate this, glibc does not implement this. */
> #ifndef __UAPI_DEF_ETHHDR
> #define __UAPI_DEF_ETHHDR        1
> #endif
> 
> #if __UAPI_DEF_ETHHDR
> struct ethhdr {
>     unsigned char    h_dest[ETH_ALEN];    /* destination eth addr    */
>     unsigned char    h_source[ETH_ALEN];    /* source ether addr    */
>     __be16        h_proto;        /* packet type ID field    */
> } __attribute__((packed));
> #endif
> 
> musl does not include linux header but defines which is differen from
> what linux does:
> struct ethhdr {
>     uint8_t h_dest[ETH_ALEN];
>     uint8_t h_source[ETH_ALEN];
>     uint16_t h_proto;
> };
> and later
> #define __UAPI_DEF_ETHHDR       0
> 
> So for networkmanager there is either some wrong sequence or it
> includes linux headers.

musl headers providing own different definitions of kernel interfaces
is a problem in musl.

After reading [1] I think that this is musl upstream having made the 
decision of not even trying to work properly with the kernel headers.

OE adding a not upstreamable patch that removes one of the two 
definitions in musl builds might be the best available solution.

> And I am still not confident that it is our job to teach umpteen
> projects written for linux how to write portable code (oe-core has 147
> musl related patches and meta-openembedded has 140)...
>...

This is not about writing portable code, this is about problems with musl.

Using the Linux userspace headers is obviously not portable to non-Linux,
but many packages like NetworkManager are anyways Linux-only no matter 
what you do.

> Andreas

cu
Adrian

[1] https://wiki.musl-libc.org/faq.html#Q:-Why-am-I-getting-

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Andreas Müller]

On Mon, Mar 25, 2019 at 6:46 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Mon, Mar 25, 2019 at 06:15:40PM +0100, Andreas Müller wrote:
> >...
> > Looked into this. Found an old musl build failure of networkmanager
> > [1] but I think the issue has not changed:
> >
> > | In file included from
> > TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/net/ethernet.h:10,
> > |                  from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:28:
> > | TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/netinet/if_ether.h:111:8:
> > error: redefinition of 'struct ethhdr'
> > |  struct ethhdr {
> > |         ^~~~~~
> > | In file included from ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:26:
> > | TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/linux/if_ether.h:167:8:
> > note: originally defined here
> > | struct ethhdr {
> > |         ^~~~~~
> >
> > glibc does not fail because it does include linux header
> > | /* Get definitions from kernel header file.  */
> > | #include <linux/if_ether.h>
> > and does not define struct ethhdr
> >
> > linux/if_ether.h says:
> > /* allow libcs like musl to deactivate this, glibc does not implement this. */
> > #ifndef __UAPI_DEF_ETHHDR
> > #define __UAPI_DEF_ETHHDR        1
> > #endif
> >
> > #if __UAPI_DEF_ETHHDR
> > struct ethhdr {
> >     unsigned char    h_dest[ETH_ALEN];    /* destination eth addr    */
> >     unsigned char    h_source[ETH_ALEN];    /* source ether addr    */
> >     __be16        h_proto;        /* packet type ID field    */
> > } __attribute__((packed));
> > #endif
> >
> > musl does not include linux header but defines which is differen from
> > what linux does:
> > struct ethhdr {
> >     uint8_t h_dest[ETH_ALEN];
> >     uint8_t h_source[ETH_ALEN];
> >     uint16_t h_proto;
> > };
> > and later
> > #define __UAPI_DEF_ETHHDR       0
> >
> > So for networkmanager there is either some wrong sequence or it
> > includes linux headers.
>
> musl headers providing own different definitions of kernel interfaces
> is a problem in musl.
>
> After reading [1] I think that this is musl upstream having made the
> decision of not even trying to work properly with the kernel headers.
>
> OE adding a not upstreamable patch that removes one of the two
> definitions in musl builds might be the best available solution.
>
> > And I am still not confident that it is our job to teach umpteen
> > projects written for linux how to write portable code (oe-core has 147
> > musl related patches and meta-openembedded has 140)...
> >...
>
> This is not about writing portable code, this is about problems with musl.
>
> Using the Linux userspace headers is obviously not portable to non-Linux,
> but many packages like NetworkManager are anyways Linux-only no matter
> what you do.
>
> > Andreas
>
> cu
> Adrian
>
> [1] https://wiki.musl-libc.org/faq.html#Q:-Why-am-I-getting-
>
Have looked into some papers at musl page and the link above: The more
I read from them the less I am interested in wasting my time on it.
There is too often this 'the world is full of idiots - and it is just
us doing the right thing' between the lines. Starts that they call
musl 'correct' - while their own performance charts are - for my use
case - unacceptable (ok outdated - maybe things changed). One could
say: Go and give it a try but there is just a little problem: I think
it'll take weeks to get images with contents I interested in.

Looked into networkmanager: There are dozen places where linux headers
are used - I don't see a clean solution we could offer upstream.

What bugs me most: Silently a rule was introduced that patches failing
for musl are not accepted - or did I miss something. A project
considering itself as center of the world blocks resources here in
times these are limited.

Just to add that I spent past three weekends to wipe after patches
being removed / 'bit-rottened' stuff was removed from a layer I
maintain right before next release. Ahh maintainer means take care /
answer user questions take / complaints when something does not work -
but when it comes to decisions: Please keep you mouth shut.

it is enough

Andreas

Re: musl thoughts

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 5917 bytes --]

On Mon, Mar 25, 2019 at 2:12 PM Andreas Müller <schnitzeltony@gmail.com>
wrote:

> On Mon, Mar 25, 2019 at 6:46 PM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Mon, Mar 25, 2019 at 06:15:40PM +0100, Andreas Müller wrote:
> > >...
> > > Looked into this. Found an old musl build failure of networkmanager
> > > [1] but I think the issue has not changed:
> > >
> > > | In file included from
> > >
> TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/net/ethernet.h:10,
> > > |                  from
> ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:28:
> > > |
> TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/netinet/if_ether.h:111:8:
> > > error: redefinition of 'struct ethhdr'
> > > |  struct ethhdr {
> > > |         ^~~~~~
> > > | In file included from
> ../NetworkManager-1.14.4/shared/n-acd/src/n-acd.c:26:
> > > |
> TOPDIR/build/tmp/work/mips32r2-yoe-linux-musl/networkmanager/1.14.4-r0/recipe-sysroot/usr/include/linux/if_ether.h:167:8:
> > > note: originally defined here
> > > | struct ethhdr {
> > > |         ^~~~~~
> > >
> > > glibc does not fail because it does include linux header
> > > | /* Get definitions from kernel header file.  */
> > > | #include <linux/if_ether.h>
> > > and does not define struct ethhdr
> > >
> > > linux/if_ether.h says:
> > > /* allow libcs like musl to deactivate this, glibc does not implement
> this. */
> > > #ifndef __UAPI_DEF_ETHHDR
> > > #define __UAPI_DEF_ETHHDR        1
> > > #endif
> > >
> > > #if __UAPI_DEF_ETHHDR
> > > struct ethhdr {
> > >     unsigned char    h_dest[ETH_ALEN];    /* destination eth addr    */
> > >     unsigned char    h_source[ETH_ALEN];    /* source ether addr    */
> > >     __be16        h_proto;        /* packet type ID field    */
> > > } __attribute__((packed));
> > > #endif
> > >
> > > musl does not include linux header but defines which is differen from
> > > what linux does:
> > > struct ethhdr {
> > >     uint8_t h_dest[ETH_ALEN];
> > >     uint8_t h_source[ETH_ALEN];
> > >     uint16_t h_proto;
> > > };
> > > and later
> > > #define __UAPI_DEF_ETHHDR       0
> > >
> > > So for networkmanager there is either some wrong sequence or it
> > > includes linux headers.
> >
> > musl headers providing own different definitions of kernel interfaces
> > is a problem in musl.
> >
> > After reading [1] I think that this is musl upstream having made the
> > decision of not even trying to work properly with the kernel headers.
> >
> > OE adding a not upstreamable patch that removes one of the two
> > definitions in musl builds might be the best available solution.
> >
> > > And I am still not confident that it is our job to teach umpteen
> > > projects written for linux how to write portable code (oe-core has 147
> > > musl related patches and meta-openembedded has 140)...
> > >...
> >
> > This is not about writing portable code, this is about problems with
> musl.
> >
> > Using the Linux userspace headers is obviously not portable to non-Linux,
> > but many packages like NetworkManager are anyways Linux-only no matter
> > what you do.
> >
> > > Andreas
> >
> > cu
> > Adrian
> >
> > [1] https://wiki.musl-libc.org/faq.html#Q:-Why-am-I-getting-
> >
> Have looked into some papers at musl page and the link above: The more
> I read from them the less I am interested in wasting my time on it.
> There is too often this 'the world is full of idiots - and it is just
> us doing the right thing' between the lines. Starts that they call
> musl 'correct' - while their own performance charts are - for my use
> case - unacceptable (ok outdated - maybe things changed). One could
> say: Go and give it a try but there is just a little problem: I think
> it'll take weeks to get images with contents I interested in.
>
> Looked into networkmanager: There are dozen places where linux headers
> are used - I don't see a clean solution we could offer upstream.
>
> What bugs me most: Silently a rule was introduced that patches failing
> for musl are not accepted - or did I miss something. A project
> considering itself as center of the world blocks resources here in
> times these are limited.


No there is no rule like that it’s who shows up with interest and reports
problems I understand your usecase might not involve musl and that’s how
everyone is they have interest in some parts of project but they cooperate
with the things which don’t interest them so when I ask it’s breaking musl
I am trying to nudge them to help fixing it since I do the same when
someone approaches my patches and reports issues that could be fixed in
that and sometimes I express my helplessness and sometimes I do help if I
have knleoledge and time so I expect others to do same

>
>
> Just to add that I spent past three weekends to wipe after patches
> being removed / 'bit-rottened' stuff was removed from a layer I
> maintain right before next release. Ahh maintainer means take care /
> answer user questions take / complaints when something does not work -
> but when it comes to decisions: Please keep you mouth shut.


You can be direct and cite the case and may be it can be discussed further
if you are talking about gstreamer 0.10 then I thought you agreed for
removal since that’s what your email said but if it’s a different case then
I encourage you to bring it forward and discuss

I usually report my problems and of possible I send a fix and if something
of my interest is affected I try to help this works well in most cases

>
>
> it is enough
>
> Andreas
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 7609 bytes --]

Re: musl thoughts

[Andrea Adami]

On Mon, Mar 25, 2019 at 5:10 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Mon, Mar 25, 2019 at 04:36:44PM +0100, Andrea Adami wrote:
> > On Sat, Mar 23, 2019 at 11:00 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
> > >
> > > On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > >
> > > > On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > > > > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > >
> > > > > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > > > > >...
> > > > > > > There are certain design aspects of musl which are actually turning
> > > > > > > out to be good
> > > > > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > > > > hidden which is a good thing,
> > > > > > >...
> > > > > >
> > > > > > Please take a closer look at some of the musl changes to NM that made
> > > > > > upgrading NM so hard for Andreas.
> > > > > >
> > > > > > +#if defined(__GLIBC__)
> > > > > >  #include <net/ethernet.h>
> > > > > > +#else /* musl libc */
> > > > > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > > > > +#endif
> >...
> >
> > Hi,
>
> Hi Andrea,
>
> > I am jumping in a little late to take side with Khem :)
> >
> > What happens now is that more 'bad' sources (written to suit glibc and
> > thus not portable) are discovered by the wider base of developers and
> > autobuilders.
> >...
>
> but this does not apply to this example, which is a problem between
> musl itself and the kernel headers.
>
> Code can expect #include <foo.h> to work for any headers, and with any
> order of these headers. If it does not, the 'bad' sources are whatever
> sources provide the headers in question.
>
> musl does provide net/ethernet.h, but including it causes a compile
> error here.

Adrian,

I don't know in this specific case, you do surely know better about
kernel/headers than me :)
Strangely I remember one issue with net/if.h and netinet/in.h with
kexec-tools and musl: maybe there is really something too special with
those net headers.

Switching the order of the headers did solve back then
https://git.openembedded.org/meta-openembedded/commit/meta-initramfs/recipes-kernel/kexec?id=b97358d5a3568deb2a5e939019bb2acef053e53f

Sorry for the OT


Cheers
Andrea



>
> > As Khem said, this just needs time and efforts to clean up and
> > upstream the patches.
>
> Any sane upstream will refuse such patches that add musl-specific hacks
> with the wrong define to work around problems in musl itself.
>
> > Cheers
> > Andrea
>
> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>

Re: musl thoughts

[Adrian Bunk]

On Mon, Mar 25, 2019 at 05:31:47PM +0100, Andrea Adami wrote:
> On Mon, Mar 25, 2019 at 5:10 PM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Mon, Mar 25, 2019 at 04:36:44PM +0100, Andrea Adami wrote:
> > > On Sat, Mar 23, 2019 at 11:00 PM Andreas Müller <schnitzeltony@gmail.com> wrote:
> > > >
> > > > On Sat, Mar 23, 2019 at 10:53 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > >
> > > > > On Sat, Mar 23, 2019 at 10:22:15PM +0100, Andreas Müller wrote:
> > > > > > On Sat, Mar 23, 2019 at 10:16 PM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > > >
> > > > > > > On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> > > > > > > >...
> > > > > > > > There are certain design aspects of musl which are actually turning
> > > > > > > > out to be good
> > > > > > > > e.g. there is no __MUSL__ define, so non-portable code can not be
> > > > > > > > hidden which is a good thing,
> > > > > > > >...
> > > > > > >
> > > > > > > Please take a closer look at some of the musl changes to NM that made
> > > > > > > upgrading NM so hard for Andreas.
> > > > > > >
> > > > > > > +#if defined(__GLIBC__)
> > > > > > >  #include <net/ethernet.h>
> > > > > > > +#else /* musl libc */
> > > > > > > +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> > > > > > > +#endif
> > >...
> > >
> > > Hi,
> >
> > Hi Andrea,
> >
> > > I am jumping in a little late to take side with Khem :)
> > >
> > > What happens now is that more 'bad' sources (written to suit glibc and
> > > thus not portable) are discovered by the wider base of developers and
> > > autobuilders.
> > >...
> >
> > but this does not apply to this example, which is a problem between
> > musl itself and the kernel headers.
> >
> > Code can expect #include <foo.h> to work for any headers, and with any
> > order of these headers. If it does not, the 'bad' sources are whatever
> > sources provide the headers in question.
> >
> > musl does provide net/ethernet.h, but including it causes a compile
> > error here.
> 
> Adrian,
> 
> I don't know in this specific case, you do surely know better about
> kernel/headers than me :)
> Strangely I remember one issue with net/if.h and netinet/in.h with
> kexec-tools and musl: maybe there is really something too special with
> those net headers.
> 
> Switching the order of the headers did solve back then
> https://git.openembedded.org/meta-openembedded/commit/meta-initramfs/recipes-kernel/kexec?id=b97358d5a3568deb2a5e939019bb2acef053e53f

This changed the order between two headers that are both provided by musl.

> Sorry for the OT

This is not OT, this is a good example for a patch that is a workaround
for some problem in musl (and not a generic portability fix).

> Cheers
> Andrea

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Khem Raj]

On Sat, Mar 23, 2019 at 2:16 PM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Fri, Mar 22, 2019 at 03:18:01PM -0700, Khem Raj wrote:
> >...
> > There are certain design aspects of musl which are actually turning
> > out to be good
> > e.g. there is no __MUSL__ define, so non-portable code can not be
> > hidden which is a good thing,
> >...
>
> Please take a closer look at some of the musl changes to NM that made
> upgrading NM so hard for Andreas.
>
> +#if defined(__GLIBC__)
>  #include <net/ethernet.h>
> +#else /* musl libc */
> +#define ETH_ALEN       6               /* Octets in one ethernet addr   */
> +#endif
>
> Using __GLIBC__ in workarounds for bugs in musl is wrong,
> and cannot be upstreamed since it would do the wrong thing
> on other non-broken C libraries.

yes, not everything that fixed builds for musl are golden, and some
are workarounds
as above which obviously can't be upstreamed, my comments were from
overall perspective
not specific to network manager patches

>
> > While the eyes may hurt
> > to see them, it does serve a
> > good reminder of whats needed for a given package.
> >...
>
> Who is responsible for fixing the root causes of such bugs in musl,

your concluded its a bug in musl, probably based on the patch but its
one sided view
musl is calling out namespace collisions etc. see

https://www.openwall.com/lists/musl/2012/10/11/4

so obviously, we need to make sure that kernel headers are not written
with an assumption
that glibc is only options when it comes to C libraries, but there are
such assumptions as well unfortunately

> so that the workaround patches can be dropped from packages like NM?

I think the right approach is to fix the problem where it belongs, and
generally making changes to headers
is a long term approach but is right one until then live with
temporary solutions. there has been changes in
kernel headers which fixes such assumptions broken by musl e.g. see

https://patchwork.kernel.org/cover/10724549/

We have to take more portable approach and I think its better for everyone.
>
> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>

Re: musl thoughts

[Adrian Bunk]

On Mon, Mar 25, 2019 at 10:44:28AM -0700, Khem Raj wrote:
> On Sat, Mar 23, 2019 at 2:16 PM Adrian Bunk <bunk@stusta.de> wrote:
>...
> > so that the workaround patches can be dropped from packages like NM?
> 
> I think the right approach is to fix the problem where it belongs, and
> generally making changes to headers
> is a long term approach but is right one until then live with
> temporary solutions.
>...

Many of the musl-specific header ordering restrictions could 
alternatively be resolved if there would be an #if defined(__MUSL__)
in the linux/libc-compat.h kernel header to reliably disable conflicting 
declarations in the kernel headers.

I wonder whether there is a better temporary solution by doing something
like that in OE instead of having workaround hacks in various packages
like NetworkManager.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: musl thoughts

[Mark Hatle]

On 3/21/19 8:11 PM, Andreas Müller wrote:
> Just prepared meta-networking/networkmanager update and spent hours to
> get musl patches applied. Have no idea if musl will build and
> currently all my machines are building.
> 
> For me - not using musl and looking forward to have a beer with Khem
> explaining me why I should want musl [1] - it is just useless effort
> and patches are rejected often due to failures on musl build.
> 
> So on my way home I though about the options I see to handle musl specifics:
> 
> 1. Continue as we do - patch recipe-wise: This is lots of efforts and
> requires resources we don't have. Most patches are created by Khem and
> it would be better for all of us to save his time for other tasks.
> 2. Sending musl related patches upstream: Surely the cleanest approach
> but even more effort because upstream maintainers all have different
> thoughts. Some of them might hear of musl for the first time and are
> possibly not keen on patches they do not see a need for.
> 3. Change our musl slightly: Many patches we currently have are
> trivial. Missing headers or #defines for missing functions... So if we
> add few headers
>   * Empty chunks for e.g <net/ethernet.h>
>   * Add defines as #define strndupa(x,s) strncpy(alloca(strlen(x)+1),x,s)
>   * ...
> 
> As you might guess I'd prefer 3 because:
> + Many patches can go and don't need maintenance on upstream refactoring anymore
> + Burden for people sending patches would be reduced
> + Recipes not building with musl currently might work without further
> modification
> + Just in case musl stops (we have seen this before with others e.g
> ulibc) the cleanup would be reduced
> 
> Of course there are disadvantages:
> - We turn musl into glibc direction. Posix-purists do not like that.
> - Configuration scripts already aware of musl and deciding upon the
> existence of headers missing in musl might do the wrong thing
> - ?
> 
> Some other ideas?

I'm late jumping into this.. but for things that may not build due to inherent
expectations from glibc...  you could just block the package if "not-glibc" is
enabled.

You could do this something like:

BLACKLISTMSG = "This package only works with glibc."
BLACKLISTMSG_libc-glibc = ""

PNBLACKLIST[${PN}] = "${BLACKLISTMSG}"

(I'm sure there is a cleaner way -- but the point is, if the package says sorry
-- I don't work with musl -- that is much more preferable to bitbake parses,
unpacks, patches, starts to build and explodes with no explanation...)

--Mark

> Andreas
> 
> [1] http://lists.openembedded.org/pipermail/openembedded-devel/2018-March/193521.html
>