vdso feature requests from the Go people

vdso feature requests from the Go people

[Andy Lutomirski]

I was talking to some of the Go people, and they have a couple of IMO
reasonable feature requests.

1. Parsing the vDSO is a PITA.  What if we bundled the reference
parser inside the vdso?  Concretely, we could have AT_VDSO_FINDENTRY
point to a function like:

void *vdso_find_entry(const char *name, const char *version)

Then things like Go and maybe even musl (and klibc?) could just call
that function.  And we'd never have to worry about maintaining
compatibility with more and more weird vdso parsers.

Implementing this could be as simple as shoving parse_vdso.c into the
vdso, although vdso2c could help and allow a really simple in-vdso
implementation.

2. Go uses a segmented stack, and the vdso is quite unfriendly for
segmented stack.  If we can get compiler support, is there a
reasonable way that we could advertise the maximum stack usage of each
vdso entry point?


If we were to implement both, maybe we'd actually want to provide
something like:

struct vdso_entry {
  unsigned long vdso_entry_struct_size;  /* so we can add fields later on */
  void *func;
  unsigned int max_stack;  /* zero if not known */
};

vdso_entry *vdso_find_entry(const char *name, const char *version);

AT_VDSO_FINDENTRY would be set to &vdso_find_entry.

This shouldn't even break CRIU :)

--Andy

Re: vdso feature requests from the Go people

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 9:36 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> I was talking to some of the Go people, and they have a couple of IMO
> reasonable feature requests.
>
> 1. Parsing the vDSO is a PITA.  What if we bundled the reference
> parser inside the vdso?  Concretely, we could have AT_VDSO_FINDENTRY
> point to a function like:
>
> void *vdso_find_entry(const char *name, const char *version)
>
> Then things like Go and maybe even musl (and klibc?) could just call
> that function.  And we'd never have to worry about maintaining
> compatibility with more and more weird vdso parsers.
>
> Implementing this could be as simple as shoving parse_vdso.c into the
> vdso, although vdso2c could help and allow a really simple in-vdso
> implementation.
>
> 2. Go uses a segmented stack, and the vdso is quite unfriendly for
> segmented stack.  If we can get compiler support, is there a
> reasonable way that we could advertise the maximum stack usage of each
> vdso entry point?
>
>
> If we were to implement both, maybe we'd actually want to provide
> something like:
>
> struct vdso_entry {
>   unsigned long vdso_entry_struct_size;  /* so we can add fields later on */
>   void *func;
>   unsigned int max_stack;  /* zero if not known */
> };
>
> vdso_entry *vdso_find_entry(const char *name, const char *version);
>
> AT_VDSO_FINDENTRY would be set to &vdso_find_entry.
>
> This shouldn't even break CRIU :)
>

Alternatively we could just assign each entry a stable index and
supply a table: AT_VDSO_ENTRIES would point to:

struct {
  unsigned long nentries;
  vdso_entry entries[];
}

Simpler, faster, but less flexible.  This version wouldn't even need
help from vdso2c.

--Andy

Re: vdso feature requests from the Go people

[H. Peter Anvin]

On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
> 
> If we were to implement both, maybe we'd actually want to provide
> something like:
> 
> struct vdso_entry {
>   unsigned long vdso_entry_struct_size;  /* so we can add fields later on */
>   void *func;
>   unsigned int max_stack;  /* zero if not known */
> };
> 

How different is this really from the .dynsym table -- other than that
you have to find the latter via the dynamic segment?

The hash part is just to speed up lookup...

	-hpa

Re: vdso feature requests from the Go people

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 10:15 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
>>
>> If we were to implement both, maybe we'd actually want to provide
>> something like:
>>
>> struct vdso_entry {
>>   unsigned long vdso_entry_struct_size;  /* so we can add fields later on */
>>   void *func;
>>   unsigned int max_stack;  /* zero if not known */
>> };
>>
>
> How different is this really from the .dynsym table -- other than that
> you have to find the latter via the dynamic segment?
>
> The hash part is just to speed up lookup...

As far as I know, there's no reliable way to just read the dynsym
table -- the thing doesn't have a specified length, which is what
broke Go in the first place.

Parsing the ELF dynamic tables is kind of annoyingly complicated, and
understanding the format is a real PITA -- the documentation I've been
able to find is outright terrible.

--Andy

Re: vdso feature requests from the Go people

[H. Peter Anvin]

On 06/12/2014 10:23 PM, Andy Lutomirski wrote:
> 
> As far as I know, there's no reliable way to just read the dynsym
> table -- the thing doesn't have a specified length, which is what
> broke Go in the first place.
> 

Ah yes, you're right.

>
> Parsing the ELF dynamic tables is kind of annoyingly complicated, and
> understanding the format is a real PITA -- the documentation I've been
> able to find is outright terrible.
> 

Yes... not entirely clear that putting it in the vdso is a net win,
though.  There are some people who would like to put a significant chunk
of libc in the vdso, and it just doesn't seem like a good idea.

An index table is somewhat more reasonable, but (a) not backwards
compatible (so wouldn't work with older kernels) and (b) has shades of
Windows...

	-hpa

Re: vdso feature requests from the Go people

[H. Peter Anvin]

On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
> 
> 1. Parsing the vDSO is a PITA.  What if we bundled the reference
> parser inside the vdso?  Concretely, we could have AT_VDSO_FINDENTRY
> point to a function like:
> 
> void *vdso_find_entry(const char *name, const char *version)
> 
> Then things like Go and maybe even musl (and klibc?) could just call
> that function.  And we'd never have to worry about maintaining
> compatibility with more and more weird vdso parsers.
> 
> Implementing this could be as simple as shoving parse_vdso.c into the
> vdso, although vdso2c could help and allow a really simple in-vdso
> implementation.
> 

I'm not really sure how much of a win that is... you have to parse
*something*, and for the vast majority of all implementations there will
be a dynamic linker just sitting there, and that is what it *does*.

> 2. Go uses a segmented stack, and the vdso is quite unfriendly for
> segmented stack.  If we can get compiler support, is there a
> reasonable way that we could advertise the maximum stack usage of each
> vdso entry point?

I suspect an easier way to do that would just be to define a maximum
stack usage for *any* vdso entry point, and then enable the gcc stack
depth warning (perhaps even with Werror)... we can do this now.

	-hpa

Re: vdso feature requests from the Go people

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 10:39 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
>>
>> 1. Parsing the vDSO is a PITA.  What if we bundled the reference
>> parser inside the vdso?  Concretely, we could have AT_VDSO_FINDENTRY
>> point to a function like:
>>
>> void *vdso_find_entry(const char *name, const char *version)
>>
>> Then things like Go and maybe even musl (and klibc?) could just call
>> that function.  And we'd never have to worry about maintaining
>> compatibility with more and more weird vdso parsers.
>>
>> Implementing this could be as simple as shoving parse_vdso.c into the
>> vdso, although vdso2c could help and allow a really simple in-vdso
>> implementation.
>>
>
> I'm not really sure how much of a win that is... you have to parse
> *something*, and for the vast majority of all implementations there will
> be a dynamic linker just sitting there, and that is what it *does*.

I'm only aware of two implementations that work like that: glibc and
musl.  AFAIK neither one even tries to use the vdso when statically
linked.  IIRC, Bionic doesn't support the vdso at all, and Go has the
present issue.

And ELF parsing is a giant mess.  Currently the vdso doesn't use
DT_GNU_HASH (easy to fix) but no one can safely rely on DT_GNU_HASH
being there, and DT_GNU_HASH isn't actually easier to parse.

My point is that, for things that otherwise need to carry around a
full ELF loader, having a really easy, guaranteed-correct way to use
the vdso would be useful.

I can see how much text size it would add.  If the parser were part of
the vdso, it could probably be trimmed down a lot.  For example, there
is currently exactly one version definition, and it could be
hard-coded.  If someone were to add another version definition, they
could fix the parser at the same time.

Basically, everything except for the vdso_sym function in my parser could go.

>
>> 2. Go uses a segmented stack, and the vdso is quite unfriendly for
>> segmented stack.  If we can get compiler support, is there a
>> reasonable way that we could advertise the maximum stack usage of each
>> vdso entry point?
>
> I suspect an easier way to do that would just be to define a maximum
> stack usage for *any* vdso entry point, and then enable the gcc stack
> depth warning (perhaps even with Werror)... we can do this now.

I can imagine this causing lots of pain when gcc 4.11 comes out with
some issue that blows up the stack usage.  Or when akpm compiles on
Fedora Core 6 using some ancient toolchain that spills every local
variable three or four times and assigns every possible inline
function its own non-overlapping stack range.

My copy of gcc supports -fstack-usage, which seems like an easyish way
to obtain the information.  I'm not entirely sure whether
-fstack-usage refers to the whole call tree or just to the specific
function.

--Andy

Re: vdso feature requests from the Go people

[H. Peter Anvin]

On 06/13/2014 08:34 AM, Andy Lutomirski wrote:
> 
> I'm only aware of two implementations that work like that: glibc and
> musl.  AFAIK neither one even tries to use the vdso when statically
> linked.  IIRC, Bionic doesn't support the vdso at all, and Go has the
> present issue.
> 

I would expect uClibc to behave similarly.  Bionic does, indeed, not
support the vdso, but that is not for the lack of a linker but is really
a shortcoming in Bionic.

> And ELF parsing is a giant mess.  Currently the vdso doesn't use
> DT_GNU_HASH (easy to fix) but no one can safely rely on DT_GNU_HASH
> being there, and DT_GNU_HASH isn't actually easier to parse.

Right... and the vdso is small enough that the performance doesn't
matter.  However, we probably *ought* to publish DT_GNU_HASH data.

>>> 2. Go uses a segmented stack, and the vdso is quite unfriendly for
>>> segmented stack.  If we can get compiler support, is there a
>>> reasonable way that we could advertise the maximum stack usage of each
>>> vdso entry point?
>>
>> I suspect an easier way to do that would just be to define a maximum
>> stack usage for *any* vdso entry point, and then enable the gcc stack
>> depth warning (perhaps even with Werror)... we can do this now.
> 
> I can imagine this causing lots of pain when gcc 4.11 comes out with
> some issue that blows up the stack usage.  Or when akpm compiles on
> Fedora Core 6 using some ancient toolchain that spills every local
> variable three or four times and assigns every possible inline
> function its own non-overlapping stack range.
> 
> My copy of gcc supports -fstack-usage, which seems like an easyish way
> to obtain the information.  I'm not entirely sure whether
> -fstack-usage refers to the whole call tree or just to the specific
> function.

There are issues either way.  However, most vdso code doesn't use much
stack at all, and it seems reasonable to put a (conservative) cap on it
as a matter of policy.

	-hpa

Re: vdso feature requests from the Go people

[Stijn Volckaert]

Andy Lutomirski schreef op 13/06/2014 7:23:
> On Thu, Jun 12, 2014 at 10:15 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>> On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
>>>
>>> If we were to implement both, maybe we'd actually want to provide
>>> something like:
>>>
>>> struct vdso_entry {
>>>    unsigned long vdso_entry_struct_size;  /* so we can add fields later on */
>>>    void *func;
>>>    unsigned int max_stack;  /* zero if not known */
>>> };
>>>
>>
>> How different is this really from the .dynsym table -- other than that
>> you have to find the latter via the dynamic segment?
>>
>> The hash part is just to speed up lookup...
>
> As far as I know, there's no reliable way to just read the dynsym
> table -- the thing doesn't have a specified length, which is what
> broke Go in the first place.
>
> Parsing the ELF dynamic tables is kind of annoyingly complicated, and
> understanding the format is a real PITA -- the documentation I've been
> able to find is outright terrible.
>
> --Andy
>

Why does it need a specified length? It's pretty easy to find, the 
entries have a fixed length and the last entry in the table is NULLed 
out (i.e. the d_tag is DT_NULL).

Just look for the PT_DYNAMIC segment in the program headers and iterate 
over the fixed length Elf_Dyn entries to find what you're looking for.

As for ELF documentation, I've never had any problems understanding the
official specification:
http://refspecs.linuxbase.org/elf/elf.pdf

--Stijn

Re: vdso feature requests from the Go people

[Stijn Volckaert]

Andy Lutomirski schreef op 13/06/2014 17:34:
> On Thu, Jun 12, 2014 at 10:39 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>> On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
>>>
>>> 1. Parsing the vDSO is a PITA.  What if we bundled the reference
>>> parser inside the vdso?  Concretely, we could have AT_VDSO_FINDENTRY
>>> point to a function like:
>>>
>>> void *vdso_find_entry(const char *name, const char *version)
>>>
>>> Then things like Go and maybe even musl (and klibc?) could just call
>>> that function.  And we'd never have to worry about maintaining
>>> compatibility with more and more weird vdso parsers.
>>>
>>> Implementing this could be as simple as shoving parse_vdso.c into the
>>> vdso, although vdso2c could help and allow a really simple in-vdso
>>> implementation.
>>>
>>
>> I'm not really sure how much of a win that is... you have to parse
>> *something*, and for the vast majority of all implementations there will
>> be a dynamic linker just sitting there, and that is what it *does*.
>
> I'm only aware of two implementations that work like that: glibc and
> musl.  AFAIK neither one even tries to use the vdso when statically
> linked.  IIRC, Bionic doesn't support the vdso at all, and Go has the
> present issue.

Glibc DOES use the VDSO even when it's statically linked. It uses the 
AT_SYSINFO_EHDR it gets from the kernel to find the VDSO and it will 
find any syms it needs dynamically.

--Stijn