* Documenting the (dynamic) linking rules for symbol versioning @ 2017-04-19 15:07 Michael Kerrisk (man-pages) [not found] ` <b3a962de-6703-d8b9-18f7-138185171475-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-04-19 15:07 UTC (permalink / raw) To: libc-alpha Cc: mtk.manpages, linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu Hello libc folk, The documentation around symbol versioning as used by the glibc dynamic linker (DL) is currently rather weak, and I'd like to add some pieces to various man pages (ld.so(8), dlsym(3), and possibly others) to improve this situation. Before that though, I'd rather like to check my understanding of the rules. The following are the rules as I understand them. Please let me know of corrections and additions: 1. If looking for a versioned symbol (NAME@VERSION), the DL will search starting from the start of the link map ("namespace") until it finds the first instance of either a matching unversioned NAME or an exact version match on NAME@VERSION. Preloading takes advantage of the former case to allow easy overriding of versioned symbols in a library that is loaded later in the link map. 2. The version notation NAME@@VERSION denotes the default version for NAME. This default version is used in the following places: a) At static link time, this is the version that the static linker will bind to when creating the relocation record that will be used by the DL. b) When doing a dlsym() look-up on the unversioned symbol NAME. (See check_match() in elf/dl-lookup.c) Is the default version used in any other circumstance? 3. There can of course be only one NAME@@VERSION definition. 4. The version notation NAME@VERSION denotes a "hidden" version of the symbol. Such versions are not directly accessible, but can be accessed via asm(".symver") magic. There can be multiple "hidden" versions of a symbol. 5. When resolving a reference to an unversioned symbol, NAME, in an executable that was linked against a nonsymbol-versioned library, the DL will, if it finds a symbol-versioned library in the link map use the earliest version of the symbol provided by that library. I presume that this behavior exists to allow easy migration of a non-symbol-versioned application onto a system with a symbol-versioned versioned library that uses the same major version real name for the library as was formerly used by a non-symbol-versioned library. (My knowledge of this area was pretty much nonexistent at that time, but presumably this is what was done in the transition from glibc 2.0 to glibc 2.1.) To clarify the scenario I am talking about: a) We have prog.c which calls xyz() and is linked against a non-symbol-versioned libxyz.so.2. b) Later, a symbol-versioned libxyz.so.2 is created that defines (for example): xyz@@VER_3 xyz@VER_2 xyz@VER_1 (Alternatively, we preload a shared library that defines these three versions of 'xyz'.) c) If we run the ancient binary 'prog' which requests refers to an unversioned 'xyz', that will resolve to xyz@VER_1. 6. [An additional detail to 5, which surprised me at first, but I can sort of convince myself it makes sense...] In the scenario described in point 5, an unversioned reference to NAME will be resolved to the earliest versioned symbol NAME inside a symbol-versioned library if there is is a version of NAME in the *lowest* version provided by the library. Otherwise, it will resolve to the *latest* version of NAME (and *not* to the default NAME@@VERSION version of the symbol). To clarify with an example: We have prog.c that calls abc() and xyz(), and is linked against a non-symbol-versioned library, lib_nonver.so, that provides definitions of abc() and xyz(). Then, we have a symbol-versioned library, lib_ver.so, that has three versions, VER_1, VER_2, and VER_3, and defines the following symbols: xyz@@VER_3 xyz@VER_2 xyz@VER_1 abc@@VER_3 abc@VER_2 Then we run 'prog' using: LD_PRELOAD=./lib_ver.so ./prog In this case, 'prog' will call xyz@VER_1 and abc@@VER_3 (*not* abc@VER_2) from lib_ver.so. I can convince myself (sort of) that this makes some sense by thinking about things from the perspective of the scenario of migrating from the non-symbol-versioned shared library to the symbol-versioned shared library: the old non-symbol-versioned library never provided a symbol 'abc()' so in this scenario, use the latest version of 'abc'. This applies even if the the latest version is not the 'default'. In other words, even if the versions of 'abc' provided by lib_ver.so were the following, it would still be the VER_3 of abc() that is called: abc@VER_3 abc@@VER_2 Am I right about my rough guess for the rationale for point 6, or is there something else I should know/write about? 7. The way to remove a versioned symbol from a new release of a shared library is to not define a default version (NAME@@VERSION) for that symbol. (Right?) In other words, if we wanted to create a VER_4 of lib_ver.so that removed the symbol 'abc', we simply don't create use the usual asm(".symver") magic to create abc@VER_4. And of course if there are other symbol versioning details that should be documented, please let me know. Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <b3a962de-6703-d8b9-18f7-138185171475-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <b3a962de-6703-d8b9-18f7-138185171475-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> @ 2017-04-19 15:48 ` Florian Weimer [not found] ` <ee5e8057-7afa-c919-8ccb-9c8e6d0833c4-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2017-04-20 6:05 ` Siddhesh Poyarekar ` (2 subsequent siblings) 3 siblings, 1 reply; 21+ messages in thread From: Florian Weimer @ 2017-04-19 15:48 UTC (permalink / raw) To: Michael Kerrisk (man-pages), libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu On 04/19/2017 05:07 PM, Michael Kerrisk (man-pages) wrote: > Am I right about my rough guess for the rationale for point 6, > or is there something else I should know/write about? We currently have a bug where the symbol resolution depends on the order of alternatives along a hash bucket list: <https://sourceware.org/bugzilla/show_bug.cgi?id=12977#c2> Another open problem is what happens when a versioned symbol moves from one DSO to another. This is not a problem for unversioned symbols, but we currently have a soname check for versioned symbols. This is rather odd because this check isn't used to accelerate lookups. It does not prevent symbol interposition from other DSOs, it merely introduces spurious failures. > 7. The way to remove a versioned symbol from a new release > of a shared library is to not define a default version > (NAME@@VERSION) for that symbol. (Right?) > > In other words, if we wanted to create a VER_4 of lib_ver.so > that removed the symbol 'abc', we simply don't create use > the usual asm(".symver") magic to create abc@VER_4. You still need to use .symver, but with a @ version instead of a @@ version. Thanks, Florian -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <ee5e8057-7afa-c919-8ccb-9c8e6d0833c4-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <ee5e8057-7afa-c919-8ccb-9c8e6d0833c4-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2017-04-19 19:49 ` Michael Kerrisk (man-pages) [not found] ` <517c3e75-93b5-0762-d6a4-7a17d196654e-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-04-19 19:49 UTC (permalink / raw) To: Florian Weimer, libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu Hi Florian, Thanks for your answers. On 04/19/2017 05:48 PM, Florian Weimer wrote: > On 04/19/2017 05:07 PM, Michael Kerrisk (man-pages) wrote: >> Am I right about my rough guess for the rationale for point 6, >> or is there something else I should know/write about? > > We currently have a bug where the symbol resolution depends on the order > of alternatives along a hash bucket list: > > <https://sourceware.org/bugzilla/show_bug.cgi?id=12977#c2> Okay. > Another open problem is what happens when a versioned symbol moves from > one DSO to another. This is not a problem for unversioned symbols, but > we currently have a soname check for versioned symbols. This is rather > odd because this check isn't used to accelerate lookups. It does not > prevent symbol interposition from other DSOs, it merely introduces > spurious failures. I have a vague recollection that this problem has been around for a very long time, right? >> 7. The way to remove a versioned symbol from a new release >> of a shared library is to not define a default version >> (NAME@@VERSION) for that symbol. (Right?) >> >> In other words, if we wanted to create a VER_4 of lib_ver.so >> that removed the symbol 'abc', we simply don't create use >> the usual asm(".symver") magic to create abc@VER_4. > > You still need to use .symver, but with a @ version instead of a @@ version. Why is that? What functionally is the difference between having no .symver and a .symver with an @ version? (I tried both, and they *both* result in undefined symbol errors from ld(1), as I expected. Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <517c3e75-93b5-0762-d6a4-7a17d196654e-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <517c3e75-93b5-0762-d6a4-7a17d196654e-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> @ 2017-04-20 8:49 ` Florian Weimer [not found] ` <3edb27c6-c9b6-df95-3810-a8b5abc740fb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Florian Weimer @ 2017-04-20 8:49 UTC (permalink / raw) To: Michael Kerrisk (man-pages), libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu On 04/19/2017 09:49 PM, Michael Kerrisk (man-pages) wrote: > Hi Florian, > > Thanks for your answers. > > On 04/19/2017 05:48 PM, Florian Weimer wrote: >> On 04/19/2017 05:07 PM, Michael Kerrisk (man-pages) wrote: >>> Am I right about my rough guess for the rationale for point 6, >>> or is there something else I should know/write about? >> >> We currently have a bug where the symbol resolution depends on the order >> of alternatives along a hash bucket list: >> >> <https://sourceware.org/bugzilla/show_bug.cgi?id=12977#c2> > > Okay. What I was trying to say is that point 6 might be a bug. But I think the problem there is that the file format only has one global base version, not different per-symbol base versions. Your second symbol with the unexpected binding simply does not have a base version at all. >> Another open problem is what happens when a versioned symbol moves from >> one DSO to another. This is not a problem for unversioned symbols, but >> we currently have a soname check for versioned symbols. This is rather >> odd because this check isn't used to accelerate lookups. It does not >> prevent symbol interposition from other DSOs, it merely introduces >> spurious failures. > > I have a vague recollection that this problem has been around for a > very long time, right? Yes, it has. >>> 7. The way to remove a versioned symbol from a new release >>> of a shared library is to not define a default version >>> (NAME@@VERSION) for that symbol. (Right?) >>> >>> In other words, if we wanted to create a VER_4 of lib_ver.so >>> that removed the symbol 'abc', we simply don't create use >>> the usual asm(".symver") magic to create abc@VER_4. >> >> You still need to use .symver, but with a @ version instead of a @@ version. > > Why is that? What functionally is the difference between > having no .symver and a .symver with an @ version? (I tried > both, and they *both* result in undefined symbol errors from > ld(1), as I expected. I was assuming that you want to preserve ABI. People who are interested in symbol versioning usually want that. :) Thanks, Florian -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <3edb27c6-c9b6-df95-3810-a8b5abc740fb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <3edb27c6-c9b6-df95-3810-a8b5abc740fb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2017-04-20 11:45 ` Michael Kerrisk (man-pages) 2017-04-20 13:17 ` Florian Weimer 0 siblings, 1 reply; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-04-20 11:45 UTC (permalink / raw) To: Florian Weimer, libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 10:49 AM, Florian Weimer wrote: > On 04/19/2017 09:49 PM, Michael Kerrisk (man-pages) wrote: >> Hi Florian, >> >> Thanks for your answers. >> >> On 04/19/2017 05:48 PM, Florian Weimer wrote: >>> On 04/19/2017 05:07 PM, Michael Kerrisk (man-pages) wrote: >>>> Am I right about my rough guess for the rationale for point 6, >>>> or is there something else I should know/write about? >>> >>> We currently have a bug where the symbol resolution depends on the order >>> of alternatives along a hash bucket list: >>> >>> <https://sourceware.org/bugzilla/show_bug.cgi?id=12977#c2> >> >> Okay. > > What I was trying to say is that point 6 might be a bug. Ahhh -- okay. Thanks for the clarification. > But I think the problem there is that the file format only has one > global base version, not different per-symbol base versions. Your > second symbol with the unexpected binding simply does not have a base > version at all. Yes, that all makes sense. >>> Another open problem is what happens when a versioned symbol moves from >>> one DSO to another. This is not a problem for unversioned symbols, but >>> we currently have a soname check for versioned symbols. This is rather >>> odd because this check isn't used to accelerate lookups. It does not >>> prevent symbol interposition from other DSOs, it merely introduces >>> spurious failures. >> >> I have a vague recollection that this problem has been around for a >> very long time, right? > > Yes, it has. > >>>> 7. The way to remove a versioned symbol from a new release >>>> of a shared library is to not define a default version >>>> (NAME@@VERSION) for that symbol. (Right?) >>>> >>>> In other words, if we wanted to create a VER_4 of lib_ver.so >>>> that removed the symbol 'abc', we simply don't create use >>>> the usual asm(".symver") magic to create abc@VER_4. >>> >>> You still need to use .symver, but with a @ version instead of a @@ version. >> >> Why is that? What functionally is the difference between >> having no .symver and a .symver with an @ version? (I tried >> both, and they *both* result in undefined symbol errors from >> ld(1), as I expected. > > I was assuming that you want to preserve ABI. People who are interested > in symbol versioning usually want that. :) I am thinking of the situation where one explicitly wants to deprecate a function in say VER_4, while still allowing that symbol to be available in VER_1 to VER_3. For example, one might possibly want to do this one day for the gets() function that has been removed in C11 and is deprecated in the last POSIX.1 (2008). The point is that removing a symbol is possible in the traditional (filename-based) major versioning scheme where we create a new incompatible major version that drops the symbol. I've occasionally had people ask me how you would do something similar with symbol versioning. To return to my question: given the two options, having no abc@VER_4 defined and having abc@VER_4 (not abc@@VER_4), then the effects are as follows: * In both cases, the static linker would emit an error when trying to resolve a reference to 'abc'. * In the case where we had a symbol abc@VER_4 in the SO, then that symbol would be accessible to the static linker using asm(".symver") (and obviously this wouldn't be possible if there was no abc@VER_4 defined). Any other differences to be aware of? Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning 2017-04-20 11:45 ` Michael Kerrisk (man-pages) @ 2017-04-20 13:17 ` Florian Weimer 2017-04-20 14:07 ` Michael Kerrisk (man-pages) 0 siblings, 1 reply; 21+ messages in thread From: Florian Weimer @ 2017-04-20 13:17 UTC (permalink / raw) To: Michael Kerrisk (man-pages), libc-alpha Cc: linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 01:45 PM, Michael Kerrisk (man-pages) wrote: >>>>> 7. The way to remove a versioned symbol from a new release >>>>> of a shared library is to not define a default version >>>>> (NAME@@VERSION) for that symbol. (Right?) >>>>> >>>>> In other words, if we wanted to create a VER_4 of lib_ver.so >>>>> that removed the symbol 'abc', we simply don't create use >>>>> the usual asm(".symver") magic to create abc@VER_4. >>>> >>>> You still need to use .symver, but with a @ version instead of a @@ version. >>> >>> Why is that? What functionally is the difference between >>> having no .symver and a .symver with an @ version? (I tried >>> both, and they *both* result in undefined symbol errors from >>> ld(1), as I expected. >> >> I was assuming that you want to preserve ABI. People who are interested >> in symbol versioning usually want that. :) > > I am thinking of the situation where one explicitly wants to > deprecate a function in say VER_4, while still allowing that symbol > to be available in VER_1 to VER_3. For example, one might possibly > want to do this one day for the gets() function that has been > removed in C11 and is deprecated in the last POSIX.1 (2008). They way this usually works is that deprecation involves removing the default, but keeping the version itself around. This means that existing binaries will continue to work, but new programs won't be able to use the symbol anymore. > The point is that removing a symbol is possible in the traditional > (filename-based) major versioning scheme where we create a new > incompatible major version that drops the symbol. I've occasionally > had people ask me how you would do something similar with symbol > versioning. Yes. Removing the entire version still requires a soname bump (at which point you can remove all non-default versions because there aren't any legacy binaries anymore), which is why I believe you typically do not want to do that. > To return to my question: given the two options, having no abc@VER_4 I think you mean “no abc” (with any version whatsoever). > defined and having abc@VER_4 (not abc@@VER_4), then the effects are > as follows: > > * In both cases, the static linker would emit an error when trying > to resolve a reference to 'abc'. Right. > * In the case where we had a symbol abc@VER_4 in the SO, then > that symbol would be accessible to the static linker using > asm(".symver") (and obviously this wouldn't be possible > if there was no abc@VER_4 defined). > > Any other differences to be aware of? The key difference is that if there used to be an abc@@VER_4 default version, then older binaries did not have to use the .symver kludge to get the definition. The static linker would have magically applied that version. In fact, the general expectation is that the .symver backdoor to get symbols at non-default versions doesn't really exist. :) Thanks, Florian ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning 2017-04-20 13:17 ` Florian Weimer @ 2017-04-20 14:07 ` Michael Kerrisk (man-pages) [not found] ` <0409f767-3ae3-48f0-4836-8694361c755c-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-04-20 14:07 UTC (permalink / raw) To: Florian Weimer, libc-alpha Cc: mtk.manpages, linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu Hi Florian, On 04/20/2017 03:17 PM, Florian Weimer wrote: > On 04/20/2017 01:45 PM, Michael Kerrisk (man-pages) wrote: > >>>>>> 7. The way to remove a versioned symbol from a new release >>>>>> of a shared library is to not define a default version >>>>>> (NAME@@VERSION) for that symbol. (Right?) >>>>>> >>>>>> In other words, if we wanted to create a VER_4 of lib_ver.so >>>>>> that removed the symbol 'abc', we simply don't create use >>>>>> the usual asm(".symver") magic to create abc@VER_4. >>>>> >>>>> You still need to use .symver, but with a @ version instead of a @@ version. >>>> >>>> Why is that? What functionally is the difference between >>>> having no .symver and a .symver with an @ version? (I tried >>>> both, and they *both* result in undefined symbol errors from >>>> ld(1), as I expected. >>> >>> I was assuming that you want to preserve ABI. People who are interested >>> in symbol versioning usually want that. :) >> >> I am thinking of the situation where one explicitly wants to >> deprecate a function in say VER_4, while still allowing that symbol >> to be available in VER_1 to VER_3. For example, one might possibly >> want to do this one day for the gets() function that has been >> removed in C11 and is deprecated in the last POSIX.1 (2008). > > They way this usually works is that deprecation involves removing the > default, but keeping the version itself around. This means that > existing binaries will continue to work, but new programs won't be able > to use the symbol anymore. > >> The point is that removing a symbol is possible in the traditional >> (filename-based) major versioning scheme where we create a new >> incompatible major version that drops the symbol. I've occasionally >> had people ask me how you would do something similar with symbol >> versioning. > > Yes. Removing the entire version still requires a soname bump (at which > point you can remove all non-default versions because there aren't any > legacy binaries anymore), which is why I believe you typically do not > want to do that. > >> To return to my question: given the two options, having no abc@VER_4 > > I think you mean “no abc” (with any version whatsoever). No, I meant having no abc@VER_4. Let me explain with a more concrete example. (I'm actually beginning to think we agree on the details here, but I didn't convey my scenario clearly enough.) Suppose we have a version script: [[ VER_1 { global: xyz; local: *; }; VER_2 { ... } VER_1; VER_3 { ... } VER_2; ]] And we have a C file that defines xyz@VER_1, xyz@VER_2, and xyz@VER_3. Now suppose that we want to add a VER_4 tag to the map, and a new function, abc(), to our C source file (so that we will end up with a symbol abc@VER_4). Suppose also that in programs that link against the new library (built with the VER_4 map), we want the symbol xyz() to no longer be accessible to ld(1). There appears to be two ways to do this: * Don't define an xyz@VER_4 (or, of course, xyz@@VER_4) in our C source file. * Define an xyz@VER_4. In both cases, the linker emits an error if a program that uses 'xyz' tries to link against the library. But the former approach seems more natural to me. (I mean, why add an xyz@VER_4 definition in the C file?) And in both cases, the VER_4 map file would look like this: [[ VER_1 { global: xyz; local: *; }; VER_2 { global: ...; } VER_1; VER_3 { global: ...; } VER_2; VER_4 { global: abc; } VER_3; ]] Cheers, Michael >> defined and having abc@VER_4 (not abc@@VER_4), then the effects are >> as follows: >> >> * In both cases, the static linker would emit an error when trying >> to resolve a reference to 'abc'. > > Right. > >> * In the case where we had a symbol abc@VER_4 in the SO, then >> that symbol would be accessible to the static linker using >> asm(".symver") (and obviously this wouldn't be possible >> if there was no abc@VER_4 defined). >> >> Any other differences to be aware of? > > The key difference is that if there used to be an abc@@VER_4 default > version, then older binaries did not have to use the .symver kludge to > get the definition. The static linker would have magically applied that > version. > > In fact, the general expectation is that the .symver backdoor to get > symbols at non-default versions doesn't really exist. :) > > Thanks, > Florian > -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <0409f767-3ae3-48f0-4836-8694361c755c-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <0409f767-3ae3-48f0-4836-8694361c755c-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> @ 2017-04-28 14:19 ` Florian Weimer 2017-05-01 18:34 ` Michael Kerrisk (man-pages) 0 siblings, 1 reply; 21+ messages in thread From: Florian Weimer @ 2017-04-28 14:19 UTC (permalink / raw) To: Michael Kerrisk (man-pages), libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 04:07 PM, Michael Kerrisk (man-pages) wrote: > Suppose we have a version script: > > [[ > VER_1 { > global: xyz; > local: *; > }; > > VER_2 { > ... > } VER_1; > > VER_3 { > ... > } VER_2; > ]] > > And we have a C file that defines xyz@VER_1, xyz@VER_2, and xyz@VER_3. > > Now suppose that we want to add a VER_4 tag to the map, and a new > function, abc(), to our C source file (so that we will end up with > a symbol abc@VER_4). > > Suppose also that in programs that link against the new library (built > with the VER_4 map), we want the symbol xyz() to no longer be accessible > to ld(1). There appears to be two ways to do this: > > * Don't define an xyz@VER_4 (or, of course, xyz@@VER_4) in our > C source file. > * Define an xyz@VER_4. Okay, I think I'm getting our misunderstanding now. If xyz was previously part of the public API, then one of the versions xyz@VER_1, xyz@VER2, xyz@VER_3 must have been a default version, so probably xyz@@VER_3. If I understand you correctly, you want to remove xyz from the API. So we need to get rid of the default version. In this case, I think most people just use a .symver directive with a non-default versions, in this example xyz@VER_3. This overrides the version script as far as this particular symbol is concerned. There is no need to introduce VER_4 for this symbol, default or not. Does this clarify matters? Thanks, Florian -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning 2017-04-28 14:19 ` Florian Weimer @ 2017-05-01 18:34 ` Michael Kerrisk (man-pages) 0 siblings, 0 replies; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-05-01 18:34 UTC (permalink / raw) To: Florian Weimer, libc-alpha Cc: mtk.manpages, linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu Hi Florian, On 04/28/2017 04:19 PM, Florian Weimer wrote: > On 04/20/2017 04:07 PM, Michael Kerrisk (man-pages) wrote: > >> Suppose we have a version script: >> >> [[ >> VER_1 { >> global: xyz; >> local: *; >> }; >> >> VER_2 { >> ... >> } VER_1; >> >> VER_3 { >> ... >> } VER_2; >> ]] >> >> And we have a C file that defines xyz@VER_1, xyz@VER_2, and xyz@VER_3. >> >> Now suppose that we want to add a VER_4 tag to the map, and a new >> function, abc(), to our C source file (so that we will end up with >> a symbol abc@VER_4). >> >> Suppose also that in programs that link against the new library (built >> with the VER_4 map), we want the symbol xyz() to no longer be accessible >> to ld(1). There appears to be two ways to do this: >> >> * Don't define an xyz@VER_4 (or, of course, xyz@@VER_4) in our >> C source file. >> * Define an xyz@VER_4. > > Okay, I think I'm getting our misunderstanding now. > > If xyz was previously part of the public API, then one of the versions > xyz@VER_1, xyz@VER2, xyz@VER_3 must have been a default version, so > probably xyz@@VER_3. Yep, that's what I mean... > If I understand you correctly, you want to remove xyz from the API. Yep. > So > we need to get rid of the default version. In this case, I think most > people just use a .symver directive with a non-default versions, in this > example xyz@VER_3. This overrides the version script as far as this > particular symbol is concerned. > > There is no need to introduce VER_4 for this symbol, default or not. > > Does this clarify matters? Yes. I think we understood the same thing just from slightly different perspectives. I wasn't sure though, because I'm not completely secure in my knowledge of the details. Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <b3a962de-6703-d8b9-18f7-138185171475-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-04-19 15:48 ` Florian Weimer @ 2017-04-20 6:05 ` Siddhesh Poyarekar [not found] ` <22f26755-f7f0-898a-ac74-3f6df92a22d7-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> 2017-04-26 19:57 ` Torvald Riegel 2017-05-05 19:51 ` Carlos O'Donell 3 siblings, 1 reply; 21+ messages in thread From: Siddhesh Poyarekar @ 2017-04-20 6:05 UTC (permalink / raw) To: Michael Kerrisk (man-pages), libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On Wednesday 19 April 2017 08:37 PM, Michael Kerrisk (man-pages) wrote: > 1. If looking for a versioned symbol (NAME@VERSION), the DL will search > starting from the start of the link map ("namespace") until it finds the > first instance of either a matching unversioned NAME or an exact version > match on NAME@VERSION. Preloading takes advantage of the former case to > allow easy overriding of versioned symbols in a library that is loaded > later in the link map. I believe it is the other way around, i.e. it looks for the versioned symbol first and if it is not found, link against the unversioned symbol provided that it is public. > 2. The version notation NAME@@VERSION denotes the default version > for NAME. This default version is used in the following places: > > a) At static link time, this is the version that the static > linker will bind to when creating the relocation record > that will be used by the DL. > b) When doing a dlsym() look-up on the unversioned symbol NAME. > (See check_match() in elf/dl-lookup.c) > > Is the default version used in any other circumstance? Only (a) afaict, where do you see (2) happening? Unversioned symbol lookups seem to happen independent of the @@ version. > 3. There can of course be only one NAME@@VERSION definition. Right. > 4. The version notation NAME@VERSION denotes a "hidden" version of the > symbol. Such versions are not directly accessible, but can be > accessed via asm(".symver") magic. There can be multiple "hidden" > versions of a symbol. It is hidden only to the static linker, i.e. it links against either unversioned or @@ versions of a symbol. > 5. When resolving a reference to an unversioned symbol, NAME, > in an executable that was linked against a nonsymbol-versioned > library, the DL will, if it finds a symbol-versioned library > in the link map use the earliest version of the symbol provided > by that library. > > I presume that this behavior exists to allow easy migration > of a non-symbol-versioned application onto a system with > a symbol-versioned versioned library that uses the same major > version real name for the library as was formerly used by > a non-symbol-versioned library. (My knowledge of this area > was pretty much nonexistent at that time, but presumably this > is what was done in the transition from glibc 2.0 to glibc 2.1.) > > To clarify the scenario I am talking about: > > a) We have prog.c which calls xyz() and is linked against a > non-symbol-versioned libxyz.so.2. > > b) Later, a symbol-versioned libxyz.so.2 is created that defines > (for example): > > xyz@@VER_3 > xyz@VER_2 > xyz@VER_1 > > (Alternatively, we preload a shared library that defines > these three versions of 'xyz'.) > > c) If we run the ancient binary 'prog' which requests refers > to an unversioned 'xyz', that will resolve to xyz@VER_1. That seems correct. The VERSYM section orders the versions by index (which seems to be based on ordering of the symbols in the version script) and the odest in that sequence seems to win for unversioned lookup. For a dlsym(), the newest one wins. > 6. [An additional detail to 5, which surprised me at first, but > I can sort of convince myself it makes sense...] > > In the scenario described in point 5, an unversioned > reference to NAME will be resolved to the earliest versioned > symbol NAME inside a symbol-versioned library if there is > is a version of NAME in the *lowest* version provided > by the library. Otherwise, it will resolve to the *latest* > version of NAME (and *not* to the default NAME@@VERSION > version of the symbol). > > To clarify with an example: > > We have prog.c that calls abc() and xyz(), and is linked > against a non-symbol-versioned library, lib_nonver.so, > that provides definitions of abc() and xyz(). > > Then, we have a symbol-versioned library, lib_ver.so, > that has three versions, VER_1, VER_2, and VER_3, and defines > the following symbols: > > xyz@@VER_3 > xyz@VER_2 > xyz@VER_1 > > abc@@VER_3 > abc@VER_2 > > Then we run 'prog' using: > > LD_PRELOAD=./lib_ver.so ./prog > > In this case, 'prog' will call xyz@VER_1 and abc@@VER_3 > (*not* abc@VER_2) from lib_ver.so. > > I can convince myself (sort of) that this makes some sense by > thinking about things from the perspective of the scenario of > migrating from the non-symbol-versioned shared library to the > symbol-versioned shared library: the old non-symbol-versioned library > never provided a symbol 'abc()' so in this scenario, use the latest > version of 'abc'. This applies even if the the latest version is not > the 'default'. In other words, even if the versions of 'abc' > provided by lib_ver.so were the following, it would still be the > VER_3 of abc() that is called: > > abc@VER_3 > abc@@VER_2 > > Am I right about my rough guess for the rationale for point 6, > or is there something else I should know/write about? This seems odd, I hope someone here knows why this really is and (hopefully) point to resources. Documentation about the dynamic linker are generally very hard to find, so I'm glad you're doing this. Siddhesh -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <22f26755-f7f0-898a-ac74-3f6df92a22d7-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <22f26755-f7f0-898a-ac74-3f6df92a22d7-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> @ 2017-04-20 12:40 ` Michael Kerrisk (man-pages) 2017-04-20 12:58 ` Siddhesh Poyarekar 0 siblings, 1 reply; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-04-20 12:40 UTC (permalink / raw) To: siddhesh-9JcytcrH/bA+uJoB2kUjGw, libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-man, Carlos O'Donell, Rich Felker, H.J. Lu [-- Attachment #1: Type: text/plain, Size: 9880 bytes --] Hello Siddhesh, Thanks for your response! On 04/20/2017 08:05 AM, Siddhesh Poyarekar wrote: > On Wednesday 19 April 2017 08:37 PM, Michael Kerrisk (man-pages) wrote: >> 1. If looking for a versioned symbol (NAME@VERSION), the DL will search >> starting from the start of the link map ("namespace") until it finds the >> first instance of either a matching unversioned NAME or an exact version >> match on NAME@VERSION. Preloading takes advantage of the former case to >> allow easy overriding of versioned symbols in a library that is loaded >> later in the link map. > > I believe it is the other way around, i.e. it looks for the versioned > symbol first and if it is not found, link against the unversioned symbol > provided that it is public. I think that I have failed to provide enough detail for you to understand what I meant. Consider the following: 1. We want to interpose some symbol in glibc (say, "malloc@GLIBC_2.0") with a symbol of our own (perhaps via a preloaded library). 2. In our preloaded shared library, the interposing "malloc" need not be a versioned symbol. At least >> 2. The version notation NAME@@VERSION denotes the default version >> for NAME. This default version is used in the following places: >> >> a) At static link time, this is the version that the static >> linker will bind to when creating the relocation record >> that will be used by the DL. >> b) When doing a dlsym() look-up on the unversioned symbol NAME. >> (See check_match() in elf/dl-lookup.c) >> >> Is the default version used in any other circumstance? > > Only (a) afaict, where do you see (2) happening? Unversioned symbol > lookups seem to happen independent of the @@ version. See the following (tarball of code attached): $ cat sv_lib_v3.c /*#* sv_lib_v3.c COPYRIGHT-NOTICE */ #include <stdio.h> #ifndef DEF_XYZ_V2 __asm__(".symver xyz_newest,xyz@@VER_3"); __asm__(".symver xyz_new,xyz@VER_2"); #else __asm__(".symver xyz_newest,xyz@VER_3"); __asm__(".symver xyz_new,xyz@@VER_2"); #endif __asm__(".symver xyz_old,xyz@VER_1"); __asm__(".symver pqr_new,pqr@@VER_3"); __asm__(".symver pqr_old,pqr@VER_2"); __asm__(".symver tuv_newest,tuv@@VER_3"); __asm__(".symver tuv_new,tuv@VER_2"); __asm__(".symver tuv_old,tuv@VER_1"); void xyz_old(void) { printf("v1 xyz\n"); } void xyz_new(void) { printf("v2 xyz\n"); } void xyz_newest(void) { printf("v3 xyz\n"); } void tuv_old(void) { printf("v1 tuv\n"); } void tuv_new(void) { printf("v2 tuv\n"); } void tuv_newest(void) { printf("v3 tuv\n"); } void pqr_new(void) { printf("v3 pqr\n"); } void pqr_old(void) { printf("v2 pqr\n"); } void abc(void) { printf("v3 abc\n"); } void v123(void) { printf("v3 v123\n"); } $ cat sv_v3.map VER_1 { global: xyz; tuv; local: [a-uw-z]*; # Hide all other symbols }; VER_2 { global: pqr; } VER_1; VER_3 { global: abc; } VER_2; $ # Build version 3 of shared library, where the default (@@) version $ # of xyz is VER_3 $ gcc -g -c -fPIC -Wall sv_lib_v3.c $ gcc -g -shared -o libsv.so sv_lib_v3.o -Wl,--version-script,sv_v3.map $ # Build version 3 of shared library, where the default (@@) version $ # of xyz is VER_2 $ gcc -DDEF_XYZ_V2 -g -c -fPIC -Wall sv_lib_v3.c $ gcc -g -shared -o libsv_def_xyz_v2.so sv_lib_v3.o -Wl,--version-script,sv_v3.map $ # Verify symbol versions in the two DSOs: $ $ readelf --dyn-syms libsv.so | grep xyz 20: 0000000000000930 19 FUNC GLOBAL DEFAULT 12 xyz@VER_1 21: 0000000000000956 19 FUNC GLOBAL DEFAULT 12 xyz@@VER_3 22: 0000000000000943 19 FUNC GLOBAL DEFAULT 12 xyz@VER_2 $ readelf --dyn-syms libsv_def_xyz_v2.so | grep xyz 20: 0000000000000943 19 FUNC GLOBAL DEFAULT 12 xyz@@VER_2 21: 0000000000000956 19 FUNC GLOBAL DEFAULT 12 xyz@VER_3 22: 0000000000000930 19 FUNC GLOBAL DEFAULT 12 xyz@VER_1 $ cat dynload.c #include <dlfcn.h> #include <sys/types.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <string.h> #include <errno.h> int main(int argc, char *argv[]) { void *libHandle; /* Handle for shared library */ void (*funcp)(void); /* Pointer to function with no arguments */ const char *err; if (argc != 3 || strcmp(argv[1], "--help") == 0) { fprintf(stderr, "Usage: %s lib-path func-name\n", argv[0]); exit(EXIT_FAILURE); } /* Load the shared library and get a handle for later use */ libHandle = dlopen(argv[1], RTLD_LAZY); if (libHandle == NULL) { fprintf(stderr, "dlopen: %s\n", dlerror()); exit(EXIT_FAILURE); } /* Search library for symbol named in argv[2] */ (void) dlerror(); /* Clear dlerror() */ *(void **) (&funcp) = dlsym(libHandle, argv[2]); err = dlerror(); if (err != NULL) { fprintf(stderr, "dlsym: %s\n", err); exit(EXIT_FAILURE); } /* Try calling the address returned by dlsym() as a function that takes no arguments */ (*funcp)(); dlclose(libHandle); /* Close the library */ exit(EXIT_SUCCESS); } $ gcc -o dynload dynload.c -ldl $ ./dynload ./libsv.so xyz v3 xyz $ ./dynload ./libsv_def_xyz_v2.so xyz v2 xyz Note the last line: dlsym() found xyz@@VER_2 (not xyz@VER_3). >> 3. There can of course be only one NAME@@VERSION definition. > > Right. > >> 4. The version notation NAME@VERSION denotes a "hidden" version of the >> symbol. Such versions are not directly accessible, but can be >> accessed via asm(".symver") magic. There can be multiple "hidden" >> versions of a symbol. > > It is hidden only to the static linker, i.e. it links against either > unversioned or @@ versions of a symbol. Yes. >> 5. When resolving a reference to an unversioned symbol, NAME, >> in an executable that was linked against a nonsymbol-versioned >> library, the DL will, if it finds a symbol-versioned library >> in the link map use the earliest version of the symbol provided >> by that library. >> >> I presume that this behavior exists to allow easy migration >> of a non-symbol-versioned application onto a system with >> a symbol-versioned versioned library that uses the same major >> version real name for the library as was formerly used by >> a non-symbol-versioned library. (My knowledge of this area >> was pretty much nonexistent at that time, but presumably this >> is what was done in the transition from glibc 2.0 to glibc 2.1.) >> >> To clarify the scenario I am talking about: >> >> a) We have prog.c which calls xyz() and is linked against a >> non-symbol-versioned libxyz.so.2. >> >> b) Later, a symbol-versioned libxyz.so.2 is created that defines >> (for example): >> >> xyz@@VER_3 >> xyz@VER_2 >> xyz@VER_1 >> >> (Alternatively, we preload a shared library that defines >> these three versions of 'xyz'.) >> >> c) If we run the ancient binary 'prog' which requests refers >> to an unversioned 'xyz', that will resolve to xyz@VER_1. > > That seems correct. The VERSYM section orders the versions by index > (which seems to be based on ordering of the symbols in the version > script) and the odest in that sequence seems to win for unversioned > lookup. For a dlsym(), the newest one wins. Thanks for the confirmation. >> 6. [An additional detail to 5, which surprised me at first, but >> I can sort of convince myself it makes sense...] >> >> In the scenario described in point 5, an unversioned >> reference to NAME will be resolved to the earliest versioned >> symbol NAME inside a symbol-versioned library if there is >> is a version of NAME in the *lowest* version provided >> by the library. Otherwise, it will resolve to the *latest* >> version of NAME (and *not* to the default NAME@@VERSION >> version of the symbol). >> >> To clarify with an example: >> >> We have prog.c that calls abc() and xyz(), and is linked >> against a non-symbol-versioned library, lib_nonver.so, >> that provides definitions of abc() and xyz(). >> >> Then, we have a symbol-versioned library, lib_ver.so, >> that has three versions, VER_1, VER_2, and VER_3, and defines >> the following symbols: >> >> xyz@@VER_3 >> xyz@VER_2 >> xyz@VER_1 >> >> abc@@VER_3 >> abc@VER_2 >> >> Then we run 'prog' using: >> >> LD_PRELOAD=./lib_ver.so ./prog >> >> In this case, 'prog' will call xyz@VER_1 and abc@@VER_3 >> (*not* abc@VER_2) from lib_ver.so. >> >> I can convince myself (sort of) that this makes some sense by >> thinking about things from the perspective of the scenario of >> migrating from the non-symbol-versioned shared library to the >> symbol-versioned shared library: the old non-symbol-versioned library >> never provided a symbol 'abc()' so in this scenario, use the latest >> version of 'abc'. This applies even if the the latest version is not >> the 'default'. In other words, even if the versions of 'abc' >> provided by lib_ver.so were the following, it would still be the >> VER_3 of abc() that is called: >> >> abc@VER_3 >> abc@@VER_2 >> >> Am I right about my rough guess for the rationale for point 6, >> or is there something else I should know/write about? > > This seems odd, I hope someone here knows why this really is and > (hopefully) point to resources. Florian commented on this point already. See his mail. > Documentation about the dynamic linker > are generally very hard to find, It sure is... > so I'm glad you're doing this. Let's see if I can make something useful... Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ [-- Attachment #2: symver_default.tar.gz --] [-- Type: application/gzip, Size: 1069 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning 2017-04-20 12:40 ` Michael Kerrisk (man-pages) @ 2017-04-20 12:58 ` Siddhesh Poyarekar [not found] ` <eb5bea0c-1f54-1b20-dc78-999160738ed3-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Siddhesh Poyarekar @ 2017-04-20 12:58 UTC (permalink / raw) To: Michael Kerrisk (man-pages), libc-alpha Cc: linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On Thursday 20 April 2017 06:10 PM, Michael Kerrisk (man-pages) wrote: > I think that I have failed to provide enough detail for > you to understand what I meant. Consider the following: > > 1. We want to interpose some symbol in glibc (say, "malloc@GLIBC_2.0") > with a symbol of our own (perhaps via a preloaded library). > 2. In our preloaded shared library, the interposing "malloc" > need not be a versioned symbol. > > At least Ah ok, got it. The dlsym will result in picking up the latest symbol. > Note the last line: dlsym() found xyz@@VER_2 (not xyz@VER_3). Hmm interesting, I thought 'latest' would imply the last version in the sequence of versions in the map, but I guess it kinda makes sense that it is the @@ default, similar to how a static linker would pick it up. Siddhesh ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <eb5bea0c-1f54-1b20-dc78-999160738ed3-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <eb5bea0c-1f54-1b20-dc78-999160738ed3-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> @ 2017-04-20 13:01 ` Florian Weimer 2017-04-20 13:15 ` Siddhesh Poyarekar 0 siblings, 1 reply; 21+ messages in thread From: Florian Weimer @ 2017-04-20 13:01 UTC (permalink / raw) To: siddhesh-9JcytcrH/bA+uJoB2kUjGw, Michael Kerrisk (man-pages), libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 02:58 PM, Siddhesh Poyarekar wrote: > On Thursday 20 April 2017 06:10 PM, Michael Kerrisk (man-pages) wrote: >> I think that I have failed to provide enough detail for >> you to understand what I meant. Consider the following: >> >> 1. We want to interpose some symbol in glibc (say, "malloc@GLIBC_2.0") >> with a symbol of our own (perhaps via a preloaded library). >> 2. In our preloaded shared library, the interposing "malloc" >> need not be a versioned symbol. >> >> At least > > Ah ok, got it. The dlsym will result in picking up the latest symbol. > >> Note the last line: dlsym() found xyz@@VER_2 (not xyz@VER_3). > > Hmm interesting, I thought 'latest' would imply the last version in the > sequence of versions in the map, but I guess it kinda makes sense that > it is the @@ default, similar to how a static linker would pick it up. It might be another instance of bug 12977. At least its fix will involve preferring the default version in this case. I don't know what to do if there is no default version. We currently do not perform a topological sort on the version graph to find the maximum version. Florian -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning 2017-04-20 13:01 ` Florian Weimer @ 2017-04-20 13:15 ` Siddhesh Poyarekar [not found] ` <c31e55fb-25af-bfe9-09db-83e622ec5e3f-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Siddhesh Poyarekar @ 2017-04-20 13:15 UTC (permalink / raw) To: Florian Weimer, Michael Kerrisk (man-pages), libc-alpha Cc: linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On Thursday 20 April 2017 06:31 PM, Florian Weimer wrote: >> Hmm interesting, I thought 'latest' would imply the last version in the >> sequence of versions in the map, but I guess it kinda makes sense that >> it is the @@ default, similar to how a static linker would pick it up. > > It might be another instance of bug 12977. At least its fix will > involve preferring the default version in this case. I don't know what >From Michael's test case it seems like it already is preferring the default version. The fix would have to be to the comment that says prefer the oldest version for regular unversioned lookups and the *latest* for the dlsym lookups. That or I misunderstood what you said. > to do if there is no default version. We currently do not perform a > topological sort on the version graph to find the maximum version. My understanding is that the VERSYM section is set up in a chained manner and the oldest and the newest can be derived from it. Siddhesh ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <c31e55fb-25af-bfe9-09db-83e622ec5e3f-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <c31e55fb-25af-bfe9-09db-83e622ec5e3f-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> @ 2017-04-20 13:45 ` Florian Weimer [not found] ` <89907506-fddb-2429-7e18-b00b8a560070-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Florian Weimer @ 2017-04-20 13:45 UTC (permalink / raw) To: siddhesh-9JcytcrH/bA+uJoB2kUjGw, Michael Kerrisk (man-pages), libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 03:15 PM, Siddhesh Poyarekar wrote: > On Thursday 20 April 2017 06:31 PM, Florian Weimer wrote: >>> Hmm interesting, I thought 'latest' would imply the last version in the >>> sequence of versions in the map, but I guess it kinda makes sense that >>> it is the @@ default, similar to how a static linker would pick it up. >> >> It might be another instance of bug 12977. At least its fix will >> involve preferring the default version in this case. I don't know what > > From Michael's test case it seems like it already is preferring the > default version. The fix would have to be to the comment that says > prefer the oldest version for regular unversioned lookups and the > *latest* for the dlsym lookups. > > That or I misunderstood what you said. I think it picked the default version by accident because of the way the linker ordered the list. But I could be mistaken. >> to do if there is no default version. We currently do not perform a >> topological sort on the version graph to find the maximum version. > > My understanding is that the VERSYM section is set up in a chained > manner and the oldest and the newest can be derived from it. They are chained, but the specification does not tell us whether the version definition records are stored in a topologically sorted order in the file. We may have to re-sort them. Thanks, Florian -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <89907506-fddb-2429-7e18-b00b8a560070-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <89907506-fddb-2429-7e18-b00b8a560070-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2017-04-20 14:09 ` Michael Kerrisk (man-pages) [not found] ` <c1b5fd84-22ec-56de-b169-502d8072d188-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 0 siblings, 1 reply; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-04-20 14:09 UTC (permalink / raw) To: Florian Weimer, siddhesh-9JcytcrH/bA+uJoB2kUjGw, libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 03:45 PM, Florian Weimer wrote: > On 04/20/2017 03:15 PM, Siddhesh Poyarekar wrote: >> On Thursday 20 April 2017 06:31 PM, Florian Weimer wrote: >>>> Hmm interesting, I thought 'latest' would imply the last version in the >>>> sequence of versions in the map, but I guess it kinda makes sense that >>>> it is the @@ default, similar to how a static linker would pick it up. >>> >>> It might be another instance of bug 12977. At least its fix will >>> involve preferring the default version in this case. I don't know what >> >> From Michael's test case it seems like it already is preferring the >> default version. The fix would have to be to the comment that says >> prefer the oldest version for regular unversioned lookups and the >> *latest* for the dlsym lookups. >> >> That or I misunderstood what you said. > > I think it picked the default version by accident because of the way the > linker ordered the list. But I could be mistaken. How could I test your hypothesis? Just a longer chain of versions, maybe? Or oddly named version tags? Cheers, Michael >>> to do if there is no default version. We currently do not perform a >>> topological sort on the version graph to find the maximum version. >> >> My understanding is that the VERSYM section is set up in a chained >> manner and the oldest and the newest can be derived from it. > > They are chained, but the specification does not tell us whether the > version definition records are stored in a topologically sorted order in > the file. We may have to re-sort them. > > Thanks, > Florian > -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <c1b5fd84-22ec-56de-b169-502d8072d188-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <c1b5fd84-22ec-56de-b169-502d8072d188-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> @ 2017-04-20 14:35 ` Michael Kerrisk (man-pages) 2017-05-05 14:10 ` Florian Weimer 1 sibling, 0 replies; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-04-20 14:35 UTC (permalink / raw) To: Florian Weimer, siddhesh-9JcytcrH/bA+uJoB2kUjGw, libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 04:09 PM, Michael Kerrisk (man-pages) wrote: > On 04/20/2017 03:45 PM, Florian Weimer wrote: >> On 04/20/2017 03:15 PM, Siddhesh Poyarekar wrote: >>> On Thursday 20 April 2017 06:31 PM, Florian Weimer wrote: >>>>> Hmm interesting, I thought 'latest' would imply the last version in the >>>>> sequence of versions in the map, but I guess it kinda makes sense that >>>>> it is the @@ default, similar to how a static linker would pick it up. >>>> >>>> It might be another instance of bug 12977. At least its fix will >>>> involve preferring the default version in this case. I don't know what >>> >>> From Michael's test case it seems like it already is preferring the >>> default version. The fix would have to be to the comment that says >>> prefer the oldest version for regular unversioned lookups and the >>> *latest* for the dlsym lookups. >>> >>> That or I misunderstood what you said. >> >> I think it picked the default version by accident because of the way the >> linker ordered the list. But I could be mistaken. > > How could I test your hypothesis? Just a longer chain of versions, > maybe? Or oddly named version tags? So, some quick testing. 1. A map file that have tags VER_1 through VER_5. 2. A C file with five implementations of xyz corresponding to those 5 VER_* tags. 3. My 'dynload' program that loads a shared library and looks up (dlsym) and calls the named function symbol. I tried variously marking all five versions of xyz with '@@', rebuilding the library, and running the command ./dynload libsv.o xyz In each case, it was the xyz@@ symbol that was found by dlsym(). If I marked none of five versions of xyz as default (i.e., @@), then I get the error: dlsym: ./libsv.so: undefined symbol: xyz All of this *suggests* that the observed behavior is not accidental. And then there is this comment in elf/dl-lookup.c::check_match(): /* No specific version is selected. There are two ways we can got here: - a binary which does not include versioning information is loaded - dlsym() instead of dlvsym() is used to get a symbol which might exist in more than one form If the library does not provide symbol version information there is no problem at all: we simply use the symbol if it is defined. These two lookups need to be handled differently if the library defines versions. In the case of the old unversioned application the oldest (default) version should be used. In case of a dlsym() call the latest and public interface should be returned. */ The last sentence doesn't seem quite right though: it appears that it is the default symbol that is returned, not the latest (which would always have been VER_5 in my example). Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <c1b5fd84-22ec-56de-b169-502d8072d188-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-04-20 14:35 ` Michael Kerrisk (man-pages) @ 2017-05-05 14:10 ` Florian Weimer 1 sibling, 0 replies; 21+ messages in thread From: Florian Weimer @ 2017-05-05 14:10 UTC (permalink / raw) To: Michael Kerrisk (man-pages), siddhesh-9JcytcrH/bA+uJoB2kUjGw, libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Carlos O'Donell, Rich Felker, H.J. Lu On 04/20/2017 04:09 PM, Michael Kerrisk (man-pages) wrote: > On 04/20/2017 03:45 PM, Florian Weimer wrote: >> On 04/20/2017 03:15 PM, Siddhesh Poyarekar wrote: >>> On Thursday 20 April 2017 06:31 PM, Florian Weimer wrote: >>>>> Hmm interesting, I thought 'latest' would imply the last version in the >>>>> sequence of versions in the map, but I guess it kinda makes sense that >>>>> it is the @@ default, similar to how a static linker would pick it up. >>>> >>>> It might be another instance of bug 12977. At least its fix will >>>> involve preferring the default version in this case. I don't know what >>> >>> From Michael's test case it seems like it already is preferring the >>> default version. The fix would have to be to the comment that says >>> prefer the oldest version for regular unversioned lookups and the >>> *latest* for the dlsym lookups. >>> >>> That or I misunderstood what you said. >> >> I think it picked the default version by accident because of the way the >> linker ordered the list. But I could be mistaken. > > How could I test your hypothesis? Just a longer chain of versions, > maybe? Or oddly named version tags? I'm no longer sure what I actually saw during testing. May be the results also depend on the binutils version. Based on source code inspection in glibc, I think there actually is a problem here, but I can't reproduce it with stock binaries. Thanks, Florian -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <b3a962de-6703-d8b9-18f7-138185171475-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-04-19 15:48 ` Florian Weimer 2017-04-20 6:05 ` Siddhesh Poyarekar @ 2017-04-26 19:57 ` Torvald Riegel 2017-05-05 19:51 ` Carlos O'Donell 3 siblings, 0 replies; 21+ messages in thread From: Torvald Riegel @ 2017-04-26 19:57 UTC (permalink / raw) To: Michael Kerrisk (man-pages) Cc: libc-alpha-9JcytcrH/bA+uJoB2kUjGw, linux-man, Siddhesh Poyarekar, Carlos O'Donell, Rich Felker, H.J. Lu On Wed, 2017-04-19 at 17:07 +0200, Michael Kerrisk (man-pages) wrote: > The documentation around symbol versioning as used by the glibc dynamic > linker (DL) is currently rather weak, and I'd like to add some pieces to > various man pages (ld.so(8), dlsym(3), and possibly others) to improve > this situation. Before that though, I'd rather like to check my > understanding of the rules. I can't comment on the specific questions you have, but I'd suggest to reach out to Stephen Kell and Peter Sewell, as they have worked on rigorous specifications of linking: http://www.cl.cam.ac.uk/~pes20/rems/papers/oopsla-elf-linking-2016.pdf That's static linking of course, but perhaps they'd be interested to extend that work (or perhaps are already looking at this). Sewell et al. have worked on rigorous specifications of memory models in the past (eg, for C++), and those are *really* useful in practice. -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <b3a962de-6703-d8b9-18f7-138185171475-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> ` (2 preceding siblings ...) 2017-04-26 19:57 ` Torvald Riegel @ 2017-05-05 19:51 ` Carlos O'Donell [not found] ` <66c61101-f44f-2bbb-5ed2-b43c5d764e76-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 3 siblings, 1 reply; 21+ messages in thread From: Carlos O'Donell @ 2017-05-05 19:51 UTC (permalink / raw) To: Michael Kerrisk (man-pages), libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: linux-man, Siddhesh Poyarekar, Rich Felker, H.J. Lu On 04/19/2017 11:07 AM, Michael Kerrisk (man-pages) wrote: > The documentation around symbol versioning as used by the glibc dynamic > linker (DL) is currently rather weak, and I'd like to add some pieces to > various man pages (ld.so(8), dlsym(3), and possibly others) to improve > this situation. Before that though, I'd rather like to check my > understanding of the rules. This is great! Can I make one simple request? Please post all of your test case and we help you get them checked into glibc's regression testsuite so we don't change any of the behaviour you have documented without noticing it. I'd like us to agree to ensure we cover everything you mention in a regression test. -- Cheers, Carlos. -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <66c61101-f44f-2bbb-5ed2-b43c5d764e76-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Documenting the (dynamic) linking rules for symbol versioning [not found] ` <66c61101-f44f-2bbb-5ed2-b43c5d764e76-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2017-05-13 12:10 ` Michael Kerrisk (man-pages) 0 siblings, 0 replies; 21+ messages in thread From: Michael Kerrisk (man-pages) @ 2017-05-13 12:10 UTC (permalink / raw) To: Carlos O'Donell, libc-alpha-9JcytcrH/bA+uJoB2kUjGw Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, linux-man, Siddhesh Poyarekar, Rich Felker, H.J. Lu Hi Carlos, On 05/05/2017 09:51 PM, Carlos O'Donell wrote: > On 04/19/2017 11:07 AM, Michael Kerrisk (man-pages) wrote: >> The documentation around symbol versioning as used by the glibc dynamic >> linker (DL) is currently rather weak, and I'd like to add some pieces to >> various man pages (ld.so(8), dlsym(3), and possibly others) to improve >> this situation. Before that though, I'd rather like to check my >> understanding of the rules. > > This is great! > > Can I make one simple request? > > Please post all of your test case and we help you get them checked into > glibc's regression testsuite so we don't change any of the behaviour you > have documented without noticing it. > > I'd like us to agree to ensure we cover everything you mention in a > regression test. I'm happy to help here, bug do not have a CLA, and do not plan to get one. But, I can place whatever code might be useful under a permissive license (assuming that works for you). Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2017-05-13 12:10 UTC | newest] Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-04-19 15:07 Documenting the (dynamic) linking rules for symbol versioning Michael Kerrisk (man-pages) [not found] ` <b3a962de-6703-d8b9-18f7-138185171475-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-04-19 15:48 ` Florian Weimer [not found] ` <ee5e8057-7afa-c919-8ccb-9c8e6d0833c4-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2017-04-19 19:49 ` Michael Kerrisk (man-pages) [not found] ` <517c3e75-93b5-0762-d6a4-7a17d196654e-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-04-20 8:49 ` Florian Weimer [not found] ` <3edb27c6-c9b6-df95-3810-a8b5abc740fb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2017-04-20 11:45 ` Michael Kerrisk (man-pages) 2017-04-20 13:17 ` Florian Weimer 2017-04-20 14:07 ` Michael Kerrisk (man-pages) [not found] ` <0409f767-3ae3-48f0-4836-8694361c755c-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-04-28 14:19 ` Florian Weimer 2017-05-01 18:34 ` Michael Kerrisk (man-pages) 2017-04-20 6:05 ` Siddhesh Poyarekar [not found] ` <22f26755-f7f0-898a-ac74-3f6df92a22d7-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> 2017-04-20 12:40 ` Michael Kerrisk (man-pages) 2017-04-20 12:58 ` Siddhesh Poyarekar [not found] ` <eb5bea0c-1f54-1b20-dc78-999160738ed3-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> 2017-04-20 13:01 ` Florian Weimer 2017-04-20 13:15 ` Siddhesh Poyarekar [not found] ` <c31e55fb-25af-bfe9-09db-83e622ec5e3f-9JcytcrH/bA+uJoB2kUjGw@public.gmane.org> 2017-04-20 13:45 ` Florian Weimer [not found] ` <89907506-fddb-2429-7e18-b00b8a560070-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2017-04-20 14:09 ` Michael Kerrisk (man-pages) [not found] ` <c1b5fd84-22ec-56de-b169-502d8072d188-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-04-20 14:35 ` Michael Kerrisk (man-pages) 2017-05-05 14:10 ` Florian Weimer 2017-04-26 19:57 ` Torvald Riegel 2017-05-05 19:51 ` Carlos O'Donell [not found] ` <66c61101-f44f-2bbb-5ed2-b43c5d764e76-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2017-05-13 12:10 ` Michael Kerrisk (man-pages)
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.