From: Hannes Frederic Sowa <hannes@redhat.com>
To: Nicholas Piggin <npiggin@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Stanislav Kozina <skozina@redhat.com>,
Don Zickus <dzickus@redhat.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Ben Hutchings <ben@decadent.org.uk>,
Michal Marek <mmarek@suse.com>,
Adam Borowski <kilobyte@angband.pl>,
Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
Debian kernel maintainers <debian-kernel@lists.debian.org>,
"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
Arnd Bergmann <arnd@arndb.de>, Ingo Molnar <mingo@kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm
Date: Thu, 15 Dec 2016 14:15:31 +0100 [thread overview]
Message-ID: <027b6dd6-2117-2ff9-7308-e0b235bbd1d7@redhat.com> (raw)
In-Reply-To: <20161215220322.63e72ff5@roar.ozlabs.ibm.com>
On 15.12.2016 13:03, Nicholas Piggin wrote:
> On Thu, 15 Dec 2016 12:19:02 +0100
> Hannes Frederic Sowa <hannes@redhat.com> wrote:
>
>> On 15.12.2016 03:06, Nicholas Piggin wrote:
>>> On Wed, 14 Dec 2016 15:04:36 +0100
>>> Hannes Frederic Sowa <hannes@redhat.com> wrote:
>>>
>>>> On 09.12.2016 17:03, Greg Kroah-Hartman wrote:
>>>>> On Sat, Dec 10, 2016 at 01:56:53AM +1000, Nicholas Piggin wrote:
>>>>>> On Fri, 9 Dec 2016 15:36:04 +0100
>>>>>> Stanislav Kozina <skozina@redhat.com> wrote:
>>>>>>
>>>>>>>>>>> The question is how to provide a similar guarantee if a different way?
>>>>>>>>>> As a tool to aid distro reviewers, modversions has some value, but the
>>>>>>>>>> debug info parsing tools that have been mentioned in this thread seem
>>>>>>>>>> superior (not that I've tested them).
>>>>>>>>> On the other hand the big advantage of modversions is that it also
>>>>>>>>> verifies the checksum during runtime (module loading). In other words, I
>>>>>>>>> believe that any other solution should still generate some form of
>>>>>>>>> checksum/watermark which can be easily checked for compatibility on
>>>>>>>>> module load.
>>>>>>>>> It should not be hard to add to the DWARF based tools though. We'd just
>>>>>>>>> parse DWARF data instead of the C code.
>>>>>>>> A runtime check is still done, with per-module vermagic which distros
>>>>>>>> can change when they bump the ABI version. Is it really necessary to
>>>>>>>> have more than that (i.e., per-symbol versioning)?
>>>>>>>
>>>>>>> From my point of view, it is. We need to allow changing ABI for some
>>>>>>> modules while maintaining it for others.
>>>>>>> In fact I think that there should be version not only for every exported
>>>>>>> symbol (in the EXPORT_SYMBOL() sense), but also for every public type
>>>>>>> (in the sense of eg. structure defined in the public header file).
>>>>>>
>>>>>> Well the distro can just append _v2, _v3 to the name of the function
>>>>>> or type if it has to break compat for some reason. Would that be enough?
>>>>>
>>>>> There are other ways that distros can work around when upstream "breaks"
>>>>> the ABI, sometimes they can rename functions, and others they can
>>>>> "preload" structures with padding in anticipation for when/if fields get
>>>>> added to them. But that's all up to the distros, no need for us to
>>>>> worry about that at all :)
>>>>
>>>> The _v2 and _v3 functions are probably the ones that also get used by
>>>> future backports in the distro kernel itself and are probably the reason
>>>> for the ABI change in the first place. Thus going down this route will
>>>> basically require distros to touch every future backport patch and will
>>>> in general generate a big mess internally.
>>>
>>> What kind of big mess? You have to check the logic of each backport even
>>> if it does apply cleanly, so the added overhead of the name change should
>>> be relatively tiny, no?
>>
>> Basically single patches are backported in huge series. Reviewing each
>> single patch also definitely makes sense, a review of the series as a
>> whole is much more worthwhile because it focuses more on logic.
>>
>> The patches themselves are checked by individual robots or humans
>> against merge conflict introduced mistakes which ring alarm bells for
>> people to look more closely during review.
>>
>> Merge conflicts introduced mistakes definitely can happen because
>> developers/backporters lose the focus from the actual logic but deal
>> with shifting lines around or just fixing up postfixes to function names.
>>
>> We still try to align the kernel as much as possible with upstream,
>> because most developers can't really hold the differences between
>> upstream and the internal functions in their heads (is this function RMW
>> safe in this version but not that kernel version...).
>
> I agree with all this, but in the case of a function rename, you can
> automate it all with scripts if that's what you want.
>
> When you have your list of exported symbols with non-zero version number,
> then you can script that __abivXXX into the changeset applying process,
> or alternatively apply the rename after your patches are applied, or
> use the c preprocessor to define names to something else.
Yes, probably one could come up with coccinelle patches to do this,
preprocessing/string matching could have false positives. But as I wrote
above, we need one stable ABI and not multiple for our particular
kernels, so it seems like a lot of overhead to rename particular
functions internally all the time to make them inaccessible for external
modules.
>> Anyway, I don't think we will at any time have multiple versions of a
>> function exported to 3rd party kernel modules. The headaches are just
>> too big. Basically we would have to version structs and not functions
>> (this is our bigger problem), thus exporting new versions of functions
>> don't really help at all. Having multiple versions of structs really
>> scares me. ;)
>>
>> We already pad structs to allow for additional struct members to be
>> added, which helps a lot.
>>
>> If versioning of function symbols would be an issue we probably would
>> have switched to ELF function versioning (like glibc does it) long time ago.
>>
>>>> I think it is important to keep versioning information outside of the
>>>> source code. Some kind of modversions will still be required, but
>>>> distros should be able to decide if they put in some kind of checksum or
>>>> a string, what suites them most.
>>>
>>> The module crc symbols are just an integer that requires a match, so it
>>> could easily be populated by a list that the distro keeps, rather than
>>> by genksyms. Most of the complexity is on the build side, so that would
>>> still be an improvement for the kernel. So we *could* do this if the
>>> distros need it.
>>
>> Like Don also already said, genksyms already did a pretty good job so
>> far. We are right now working with Dodji to come up with a way to
>> replace genksyms, in case people really want to have very specific
>> control about what causes the symbol version to be changed.
>
> Yeah it's great work, so is Stanislav's checker. I wouldn't mind having
> a kernel-centric checker tool merged in the kernel if it is small,
> maintained, and does a sufficient job for distros.
Yes, I think this needs more experimentation and thought right now
before we can make a decision.
>> Also I wonder what Ben's opinion on this is.. As I understood that he
>> wants to maintain a super-long-term stable kernel with kabi guarantees.
>>
>> Note, what we want is to weaken the check for kabi, by excluding parts
>> of the struct from genksyms with libabigail. For Red Hat genksyms is too
>> strict in the checks.
>
> Sure, that makes sense.
>
> So if I understand where we are, moving the ABI compatibility checking
> to one of these tools looks possible. What to do when we have an ABI change
> is not settled, but feeding version numbers explicitly into modversions
> is an option that would be close to what distros do today.
Agreed!
Thanks also,
Hannes
next prev parent reply other threads:[~2016-12-15 13:15 UTC|newest]
Thread overview: 113+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <a73aec83-ddad-2bdf-e612-178c9936a16f@manjaro.org>
[not found] ` <20161102004639.6870806d@roar.ozlabs.ibm.com>
2016-11-23 20:08 ` BUG: 4.9-rc6 Still "no symbol version" on boot Philip Müller
2016-11-23 20:14 ` Robert LeBlanc
2016-11-23 20:27 ` Philip Müller
2016-11-23 20:53 ` Adam Borowski
2016-11-23 21:01 ` Robert LeBlanc
2016-11-23 21:02 ` [PATCH] x86/kbuild: enable modversions for symbols exported from asm Adam Borowski
2016-11-23 23:10 ` Philip Müller
2016-11-24 4:40 ` Ingo Molnar
2016-11-24 5:20 ` Nicholas Piggin
2016-11-24 6:00 ` Ingo Molnar
2016-11-24 7:20 ` Nicholas Piggin
2016-11-24 7:36 ` Greg Kroah-Hartman
2016-11-24 7:53 ` Nicholas Piggin
2016-11-24 9:32 ` Michal Marek
2016-11-24 10:03 ` Nicholas Piggin
2016-11-24 10:51 ` Michal Marek
2016-11-24 9:38 ` Arnd Bergmann
2016-11-24 10:01 ` Nicholas Piggin
2016-11-24 9:56 ` Greg Kroah-Hartman
2016-11-24 10:31 ` Nicholas Piggin
2016-11-24 15:24 ` Greg Kroah-Hartman
2016-11-25 0:40 ` Nicholas Piggin
2016-11-25 18:00 ` Linus Torvalds
2016-11-26 0:37 ` Nicholas Piggin
2016-11-29 1:15 ` Ben Hutchings
2016-11-29 2:31 ` Nicholas Piggin
2016-11-29 9:14 ` Michal Marek
2016-11-29 4:08 ` Linus Torvalds
2016-11-29 13:19 ` Adam Borowski
2016-11-29 13:29 ` Ingo Molnar
2016-11-29 14:24 ` Adam Borowski
2016-11-29 13:51 ` Adam Borowski
[not found] ` <CA+55aFyZiB4YkwvqzrXO=HD8bcnc2xHkAYrek2QHVnhVvAi3Fw@mail.gmail.com>
2016-11-29 16:03 ` Michal Marek
2016-11-29 16:17 ` Linus Torvalds
2016-11-29 19:57 ` Ben Hutchings
2016-11-29 20:35 ` Linus Torvalds
2016-11-30 18:18 ` Nicholas Piggin
2016-11-30 18:40 ` Linus Torvalds
2016-11-30 21:33 ` Ben Hutchings
2016-12-01 1:55 ` Nicholas Piggin
2016-12-01 2:35 ` Ben Hutchings
2016-12-01 3:39 ` Nicholas Piggin
2016-12-01 16:12 ` Michal Marek
2016-12-02 14:36 ` Hannes Frederic Sowa
2016-12-09 3:33 ` Nicholas Piggin
2016-12-09 15:21 ` Ian Campbell
2016-12-09 16:15 ` Nicholas Piggin
2016-12-09 22:46 ` Dodji Seketeli
2016-12-10 12:41 ` Greg Kroah-Hartman
2016-12-12 3:50 ` Nicholas Piggin
2016-12-12 9:08 ` Ian Campbell
2016-12-14 17:59 ` Don Zickus
2016-12-13 1:07 ` Stanislav Kozina
2016-12-13 22:51 ` Michal Marek
2016-12-14 8:58 ` Dodji Seketeli
2016-12-14 9:15 ` Michal Marek
2016-12-14 9:36 ` Dodji Seketeli
2016-12-14 9:44 ` Michal Marek
2016-12-14 10:02 ` Dodji Seketeli
2016-12-14 10:15 ` Michal Marek
2016-12-14 9:56 ` Dodji Seketeli
2016-12-14 9:37 ` Michal Marek
2016-12-01 4:13 ` Don Zickus
2016-12-01 4:32 ` Nicholas Piggin
2016-12-01 15:20 ` Don Zickus
2016-12-01 15:26 ` Christoph Hellwig
2016-12-01 15:40 ` Don Zickus
2016-12-01 16:06 ` Greg Kroah-Hartman
2016-12-01 18:42 ` Don Zickus
2016-12-09 3:50 ` Nicholas Piggin
2016-12-09 7:55 ` Stanislav Kozina
2016-12-09 8:14 ` Nicholas Piggin
2016-12-09 14:36 ` Stanislav Kozina
2016-12-09 15:56 ` Nicholas Piggin
2016-12-09 16:03 ` Greg Kroah-Hartman
2016-12-12 9:48 ` Stanislav Kozina
2016-12-13 7:25 ` Nicholas Piggin
2016-12-14 14:04 ` Hannes Frederic Sowa
2016-12-15 2:06 ` Nicholas Piggin
2016-12-15 11:19 ` Hannes Frederic Sowa
2016-12-15 12:03 ` Nicholas Piggin
2016-12-15 13:15 ` Hannes Frederic Sowa [this message]
2016-12-15 14:15 ` Nicholas Piggin
2016-12-15 15:17 ` Hannes Frederic Sowa
2016-12-15 13:35 ` Stanislav Kozina
2016-12-09 16:16 ` Don Zickus
2016-12-01 10:48 ` Stanislav Kozina
2016-12-01 11:09 ` Nicholas Piggin
2016-12-01 11:33 ` Stanislav Kozina
2016-12-01 12:39 ` Nicholas Piggin
2016-12-01 15:19 ` Dodji Seketeli
2016-12-01 16:14 ` Michal Marek
2016-11-29 17:05 ` Adam Borowski
2016-11-29 17:10 ` Linus Torvalds
2016-11-29 17:14 ` Linus Torvalds
2016-12-01 13:58 ` Arnd Bergmann
2016-12-01 16:21 ` Michal Marek
2016-12-01 18:26 ` Linus Torvalds
2016-12-02 10:55 ` Arnd Bergmann
2016-12-02 12:40 ` [RFC, PATCH, v3.9] default exported asm symbols to zero Arnd Bergmann
2016-12-02 12:59 ` Geert Uytterhoeven
2016-12-02 14:51 ` Arnd Bergmann
2016-12-02 15:35 ` Adam Borowski
2016-12-03 4:36 ` Ben Hutchings
2016-12-03 10:43 ` Arnd Bergmann
2016-12-02 17:04 ` [PATCH] x86/kbuild: enable modversions for symbols exported from asm Linus Torvalds
2016-12-04 7:44 ` Alan Modra
2016-12-04 20:44 ` Linus Torvalds
2016-11-29 21:23 ` Michal Marek
2016-11-24 9:25 ` Michal Marek
2016-11-24 11:42 ` Regression: " Kalle Valo
2016-11-23 23:07 ` BUG: 4.9-rc6 Still "no symbol version" on boot Philip Müller
2016-11-28 17:10 ` Robert LeBlanc
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=027b6dd6-2117-2ff9-7308-e0b235bbd1d7@redhat.com \
--to=hannes@redhat.com \
--cc=arnd@arndb.de \
--cc=ben@decadent.org.uk \
--cc=debian-kernel@lists.debian.org \
--cc=dzickus@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=kilobyte@angband.pl \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=mmarek@suse.com \
--cc=npiggin@gmail.com \
--cc=skozina@redhat.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).