Re: ipv6 redefinition build issue with 4.15-rc8

Re: ipv6 redefinition build issue with 4.15-rc8

[Daniel Wagner]

Hi Neil,

On 01/16/2018 07:51 PM, Neil MacLeod wrote:
> Since this commit in 4.15-rc8:
> 
> https://github.com/torvalds/linux/commit/6926e041a8920c8ec27e4e155efa760aa01551fd
> 
> building connman 1.35 with glibc 2.26 now fails as follows:
> 
> http://ix.io/EbP
> 
> I'm not sure if this is a kernel issue, a glibc issue, or a connman issue.
> 
> Reverting the kernel commit resolves the issue, but isn't ideal (unless 
> it's the correct solution, of course).
> 
> Does anyone have any better ideas?

Since ConnMan does not redefine 'struct in6_addr' and friends I would 
say it is kernel/glibc header include problem. But I might be wrong here.

@Hauke: Do you happen to know what is going on?

Thanks,
Daniel

Re: ipv6 redefinition build issue with 4.15-rc8

[Hauke Mehrtens]

On 01/17/2018 08:59 AM, Daniel Wagner wrote:
> Hi Neil,
> 
> On 01/16/2018 07:51 PM, Neil MacLeod wrote:
>> Since this commit in 4.15-rc8:
>>
>> https://github.com/torvalds/linux/commit/6926e041a8920c8ec27e4e155efa760aa01551fd
>>
>>
>> building connman 1.35 with glibc 2.26 now fails as follows:
>>
>> http://ix.io/EbP
>>
>> I'm not sure if this is a kernel issue, a glibc issue, or a connman
>> issue.
>>
>> Reverting the kernel commit resolves the issue, but isn't ideal
>> (unless it's the correct solution, of course).
>>
>> Does anyone have any better ideas?
> 
> Since ConnMan does not redefine 'struct in6_addr' and friends I would
> say it is kernel/glibc header include problem. But I might be wrong here.
> 
> @Hauke: Do you happen to know what is going on?
> 
> Thanks,
> Daniel

Hi,

my assumption is that connman uses a strange include order. Can you try
to remove the #include <linux/libc-compat.h> from from
include/uapi/linux/if_ether.h please. This would not be the final
solution but would help to understand the problem better.

I will look into this more closely in the evening.

Hauke

Re: ipv6 redefinition build issue with 4.15-rc8

[Jonas Bonn]

On 01/17/2018 08:59 AM, Daniel Wagner wrote:
> Hi Neil,
> 
> On 01/16/2018 07:51 PM, Neil MacLeod wrote:
>> Since this commit in 4.15-rc8:
>>
>> https://github.com/torvalds/linux/commit/6926e041a8920c8ec27e4e155efa760aa01551fd 
>>
>>
>> building connman 1.35 with glibc 2.26 now fails as follows:
>>
>> http://ix.io/EbP
>>
>> I'm not sure if this is a kernel issue, a glibc issue, or a connman 
>> issue.
>>
>> Reverting the kernel commit resolves the issue, but isn't ideal 
>> (unless it's the correct solution, of course).
>>
>> Does anyone have any better ideas?

Try switching the order of these headers around (src/tethering.c)... 
netinet/in.h seems to depend on linux/in.h being included _first_ and 
it's presumably being pulled in via linux/if_bridge.h now as a result of 
the kernel patch (couldn't immediately see why, though... I suspect the 
inclusion of libc-compat.h is the culprit.)

#include <netinet/in.h>
#include <linux/if_bridge.h>

Yes, this is a hack and only masks the issue... nonetheless.

/Jonas


> 
> Since ConnMan does not redefine 'struct in6_addr' and friends I would 
> say it is kernel/glibc header include problem. But I might be wrong here.
> 
> @Hauke: Do you happen to know what is going on?
> 
> Thanks,
> Daniel
> _______________________________________________
> connman mailing list
> connman@lists.01.org
> https://lists.01.org/mailman/listinfo/connman

Re: ipv6 redefinition build issue with 4.15-rc8

[Neil MacLeod]

All

Further to my previous reply (reproduced below having been bounced by
linux-kernel) I have successfully built LibreELEC when using the
ConnMan patch from Jonas - there were no other failures.

I have also built a number of network related packages (iftop, iperf,
ngrp, nmap, sshfs, tcpdump, udpxy, wireless-tools), again without
issue, so this particular 4.15-rc8 kernel change is only affecting
ConnMan as far as I can tell.

Regards
Neil

> All
>
> Many thanks for the replies.
>
> To ensure my build environment is sane I tested again without reverting the kernel commit, and reproduced the connman build failure.
>
> Next I tested the change suggested by Hauke (kernel patch: http://ix.io/Eh5) and connman fails to build, however it fails with a different error this time: http://ix.io/Eh2
>
> I then tested the change suggested by Jonas (connman patch: http://ix.io/Eh6) and connman builds successfully, no failure, so this might be a potential fix.
>
> I'll now try a clean build with Jonas' patch and see if any other packages fail to build for the same reason as connman (I'm building a complete embedded distro with about 700 packages).
>
> I'll post again later with an update.
>
> Thanks
> Neil

On 17 January 2018 at 15:25, Neil MacLeod <neil@nmacleod.com> wrote:
> All
>
> Many thanks for the replies.
>
> To ensure my build environment is sane I tested again without reverting the
> kernel commit, and reproduced the connman build failure.
>
> Next I tested the change suggested by Hauke (kernel patch: http://ix.io/Eh5)
> and connman fails to build, however it fails with a different error this
> time: http://ix.io/Eh2
>
> I then tested the change suggested by Jonas (connman patch:
> http://ix.io/Eh6) and connman builds successfully, no failure, so this might
> be a potential fix.
>
> I'll now try a clean build with Jonas' patch and see if any other packages
> fail to build for the same reason as connman (I'm building a complete
> embedded distro with about 700 packages).
>
> I'll post again later with an update.
>
> Thanks
> Neil
>
> On 17 January 2018 at 09:03, Jonas Bonn <jonas@southpole.se> wrote:
>>
>> On 01/17/2018 08:59 AM, Daniel Wagner wrote:
>>>
>>> Hi Neil,
>>>
>>> On 01/16/2018 07:51 PM, Neil MacLeod wrote:
>>>>
>>>> Since this commit in 4.15-rc8:
>>>>
>>>>
>>>> https://github.com/torvalds/linux/commit/6926e041a8920c8ec27e4e155efa760aa01551fd
>>>>
>>>> building connman 1.35 with glibc 2.26 now fails as follows:
>>>>
>>>> http://ix.io/EbP
>>>>
>>>> I'm not sure if this is a kernel issue, a glibc issue, or a connman
>>>> issue.
>>>>
>>>> Reverting the kernel commit resolves the issue, but isn't ideal (unless
>>>> it's the correct solution, of course).
>>>>
>>>> Does anyone have any better ideas?
>>
>>
>> Try switching the order of these headers around (src/tethering.c)...
>> netinet/in.h seems to depend on linux/in.h being included _first_ and it's
>> presumably being pulled in via linux/if_bridge.h now as a result of the
>> kernel patch (couldn't immediately see why, though... I suspect the
>> inclusion of libc-compat.h is the culprit.)
>>
>> #include <netinet/in.h>
>> #include <linux/if_bridge.h>
>>
>> Yes, this is a hack and only masks the issue... nonetheless.
>>
>> /Jonas
>>
>>
>>>
>>> Since ConnMan does not redefine 'struct in6_addr' and friends I would say
>>> it is kernel/glibc header include problem. But I might be wrong here.
>>>
>>> @Hauke: Do you happen to know what is going on?
>>>
>>> Thanks,
>>> Daniel
>>> _______________________________________________
>>> connman mailing list
>>> connman@lists.01.org
>>> https://lists.01.org/mailman/listinfo/connman
>>
>>
>

Re: ipv6 redefinition build issue with 4.15-rc8

[Hauke Mehrtens]

On 01/17/2018 08:31 PM, Neil MacLeod wrote:
> All
> 
> Further to my previous reply (reproduced below having been bounced by
> linux-kernel) I have successfully built LibreELEC when using the
> ConnMan patch from Jonas - there were no other failures.
> 
> I have also built a number of network related packages (iftop, iperf,
> ngrp, nmap, sshfs, tcpdump, udpxy, wireless-tools), again without
> issue, so this particular 4.15-rc8 kernel change is only affecting
> ConnMan as far as I can tell.

Thanks for testing.

> Regards
> Neil
> 
>> All
>>
>> Many thanks for the replies.
>>
>> To ensure my build environment is sane I tested again without reverting the kernel commit, and reproduced the connman build failure.
>>
>> Next I tested the change suggested by Hauke (kernel patch: http://ix.io/Eh5) and connman fails to build, however it fails with a different error this time: http://ix.io/Eh2
>>
>> I then tested the change suggested by Jonas (connman patch: http://ix.io/Eh6) and connman builds successfully, no failure, so this might be a potential fix.

You should import the libc header files first and then the Linux header
files in user space applications, this is the supported order.

Can you try this patch please:

--- a/src/tethering.c
+++ b/src/tethering.c
@@ -31,11 +31,11 @@
 #include <stdio.h>
 #include <sys/ioctl.h>
 #include <net/if.h>
-#include <linux/sockios.h>
 #include <string.h>
 #include <fcntl.h>
-#include <linux/if_tun.h>
 #include <netinet/in.h>
+#include <linux/sockios.h>
+#include <linux/if_tun.h>
 #include <linux/if_bridge.h>

 #include "connman.h"


Do we want to do any changes to the kernel header files? I do not know
of any clean workaround to make this work, we can probably hack
something for connman, but I think it is not worth the trouble.

Hauke

>> I'll now try a clean build with Jonas' patch and see if any other packages fail to build for the same reason as connman (I'm building a complete embedded distro with about 700 packages).
>>
>> I'll post again later with an update.
>>
>> Thanks
>> Neil
> 
> On 17 January 2018 at 15:25, Neil MacLeod <neil@nmacleod.com> wrote:
>> All
>>
>> Many thanks for the replies.
>>
>> To ensure my build environment is sane I tested again without reverting the
>> kernel commit, and reproduced the connman build failure.
>>
>> Next I tested the change suggested by Hauke (kernel patch: http://ix.io/Eh5)
>> and connman fails to build, however it fails with a different error this
>> time: http://ix.io/Eh2
>>
>> I then tested the change suggested by Jonas (connman patch:
>> http://ix.io/Eh6) and connman builds successfully, no failure, so this might
>> be a potential fix.
>>
>> I'll now try a clean build with Jonas' patch and see if any other packages
>> fail to build for the same reason as connman (I'm building a complete
>> embedded distro with about 700 packages).
>>
>> I'll post again later with an update.
>>
>> Thanks
>> Neil
>>
>> On 17 January 2018 at 09:03, Jonas Bonn <jonas@southpole.se> wrote:
>>>
>>> On 01/17/2018 08:59 AM, Daniel Wagner wrote:
>>>>
>>>> Hi Neil,
>>>>
>>>> On 01/16/2018 07:51 PM, Neil MacLeod wrote:
>>>>>
>>>>> Since this commit in 4.15-rc8:
>>>>>
>>>>>
>>>>> https://github.com/torvalds/linux/commit/6926e041a8920c8ec27e4e155efa760aa01551fd
>>>>>
>>>>> building connman 1.35 with glibc 2.26 now fails as follows:
>>>>>
>>>>> http://ix.io/EbP
>>>>>
>>>>> I'm not sure if this is a kernel issue, a glibc issue, or a connman
>>>>> issue.
>>>>>
>>>>> Reverting the kernel commit resolves the issue, but isn't ideal (unless
>>>>> it's the correct solution, of course).
>>>>>
>>>>> Does anyone have any better ideas?
>>>
>>>
>>> Try switching the order of these headers around (src/tethering.c)...
>>> netinet/in.h seems to depend on linux/in.h being included _first_ and it's
>>> presumably being pulled in via linux/if_bridge.h now as a result of the
>>> kernel patch (couldn't immediately see why, though... I suspect the
>>> inclusion of libc-compat.h is the culprit.)
>>>
>>> #include <netinet/in.h>
>>> #include <linux/if_bridge.h>
>>>
>>> Yes, this is a hack and only masks the issue... nonetheless.
>>>
>>> /Jonas
>>>
>>>
>>>>
>>>> Since ConnMan does not redefine 'struct in6_addr' and friends I would say
>>>> it is kernel/glibc header include problem. But I might be wrong here.
>>>>
>>>> @Hauke: Do you happen to know what is going on?
>>>>
>>>> Thanks,
>>>> Daniel
>>>> _______________________________________________
>>>> connman mailing list
>>>> connman@lists.01.org
>>>> https://lists.01.org/mailman/listinfo/connman
>>>
>>>
>>

Re: ipv6 redefinition build issue with 4.15-rc8

[Daniel Wagner]

On 01/17/2018 11:20 PM, Hauke Mehrtens wrote:
> On 01/17/2018 08:31 PM, Neil MacLeod wrote:
>> All
>>
>> Further to my previous reply (reproduced below having been bounced by
>> linux-kernel) I have successfully built LibreELEC when using the
>> ConnMan patch from Jonas - there were no other failures.
>>
>> I have also built a number of network related packages (iftop, iperf,
>> ngrp, nmap, sshfs, tcpdump, udpxy, wireless-tools), again without
>> issue, so this particular 4.15-rc8 kernel change is only affecting
>> ConnMan as far as I can tell.
> 
> Thanks for testing.
> 
>> Regards
>> Neil
>>
>>> All
>>>
>>> Many thanks for the replies.
>>>
>>> To ensure my build environment is sane I tested again without reverting the kernel commit, and reproduced the connman build failure.
>>>
>>> Next I tested the change suggested by Hauke (kernel patch: http://ix.io/Eh5) and connman fails to build, however it fails with a different error this time: http://ix.io/Eh2
>>>
>>> I then tested the change suggested by Jonas (connman patch: http://ix.io/Eh6) and connman builds successfully, no failure, so this might be a potential fix.
> 
> You should import the libc header files first and then the Linux header
> files in user space applications, this is the supported order.
> 
> Can you try this patch please:
> 
> --- a/src/tethering.c
> +++ b/src/tethering.c
> @@ -31,11 +31,11 @@
>   #include <stdio.h>
>   #include <sys/ioctl.h>
>   #include <net/if.h>
> -#include <linux/sockios.h>
>   #include <string.h>
>   #include <fcntl.h>
> -#include <linux/if_tun.h>
>   #include <netinet/in.h>
> +#include <linux/sockios.h>
> +#include <linux/if_tun.h>
>   #include <linux/if_bridge.h>
> 
>   #include "connman.h"
> 
> 
> Do we want to do any changes to the kernel header files? I do not know
> of any clean workaround to make this work, we can probably hack
> something for connman, but I think it is not worth the trouble.

Nah, don't worry, we just fix this in ConnMan. Thanks a lot for the 
excellent input!

I'll cook a patch for this.

Thanks,
Daniel

Re: ipv6 redefinition build issue with 4.15-rc8

[Jonas Bonn]

On 01/17/2018 11:34 PM, Daniel Wagner wrote:
> 
> On 01/17/2018 11:20 PM, Hauke Mehrtens wrote:
>>
>> Do we want to do any changes to the kernel header files? I do not know
>> of any clean workaround to make this work, we can probably hack
>> something for connman, but I think it is not worth the trouble.
> 

Well, it's not _just_ a connman issue, even though it apparently only 
shows up there, currently.

The problem with the kernel patch is that it now pulls in lib-compat.h 
which causes problems if it appears before netinet/in.h.  The following 
code is sufficient to show the issue:

#include <linux/libc-compat.h>
#include <netinet/in.h>
#include <linux/in6.h>

int main(int argc, char** argv)
{
}

lib-compat checks if _NETINET_IN_H is defined... it's not.  So it 
defines __UAPI_DEF_IN6_ADDR.

Then netinet/in.h checks (via bits/in.h) if _LINUX_IN6_H is defined... 
it's not, so it defines the struct in6_addr (and others).

Then linux/in6.h gets pulled in and redefines the function because the 
earlier libc-compat check told it to do so.

If you comment out the first #include statement then it compiles fine.

libc-compat has, as you say, a requirement to be ordered after system 
headers in order for this to work... that doesn't feel terribly robust.

Anyway, the bug is probably in the glibc headers that are not checking 
the __UAPI_DEF*'s but rather using another broken heuristic... right 
place to fix this is probably there.

/Jonas

Re: ipv6 redefinition build issue with 4.15-rc8

[Hauke Mehrtens]

On 01/18/2018 09:49 AM, Jonas Bonn wrote:
> On 01/17/2018 11:34 PM, Daniel Wagner wrote:
>>
>> On 01/17/2018 11:20 PM, Hauke Mehrtens wrote:
>>>
>>> Do we want to do any changes to the kernel header files? I do not know
>>> of any clean workaround to make this work, we can probably hack
>>> something for connman, but I think it is not worth the trouble.
>>
> 
> Well, it's not _just_ a connman issue, even though it apparently only
> shows up there, currently.
> 
> The problem with the kernel patch is that it now pulls in lib-compat.h
> which causes problems if it appears before netinet/in.h.  The following
> code is sufficient to show the issue:
> 
> #include <linux/libc-compat.h>
> #include <netinet/in.h>
> #include <linux/in6.h>
> 
> int main(int argc, char** argv)
> {
> }
> 
> lib-compat checks if _NETINET_IN_H is defined... it's not.  So it
> defines __UAPI_DEF_IN6_ADDR.
> 
> Then netinet/in.h checks (via bits/in.h) if _LINUX_IN6_H is defined...
> it's not, so it defines the struct in6_addr (and others).
> 
> Then linux/in6.h gets pulled in and redefines the function because the
> earlier libc-compat check told it to do so.
> 
> If you comment out the first #include statement then it compiles fine.
> 
> libc-compat has, as you say, a requirement to be ordered after system
> headers in order for this to work... that doesn't feel terribly robust.
> 
> Anyway, the bug is probably in the glibc headers that are not checking
> the __UAPI_DEF*'s but rather using another broken heuristic... right
> place to fix this is probably there.
> 
> /Jonas

Florian Weimer said here "A lot of combinations are broken, especially
when kernel headers are included first.":
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1411192.html

That was on a older version of these two patches:
https://www.mail-archive.com/search?l=linux-kernel@vger.kernel.org&q=subject:%22Re%5C%3A+%5C%5Bmusl%5C%5D+Re%5C%3A+%5C%5BPATCH+resent%5C%5D+uapi+libc+compat%5C%3A+allow+non%5C-glibc+to+opt+out+of+uapi+definitions%22&o=newest&f=1

My understanding is that you should include libc headers before Linux
headers, otherwise you *could* run into problems. There are some
workarounds done to also support including Linux headers first, but they
are not working in all cases.

Hauke