Broken vDSO on built kernel

Broken vDSO on built kernel

[fREW Schmidt]

Hello!

I was debugging an issue I ran into (
https://github.com/docker/docker/issues/23378) and after chatting with the
folks in #musl and we triaged it down to a broken vDSO (tested by running
`strace date` and seeing a clock_gettime call.)

The theory is that the toolchain was somehow subtly broken and made a
broken kernel, I think.

The built binaries in this case are from
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.7-rc2-yakkety/

I am not a Kernel dev or anything, just following the issues.  Hope this
helps!

(sorry Andy and musl, resending as plaintext)

-- 
fREW Schmidt
http://blog.afoolishmanifesto.com

Re: [musl] Broken vDSO on built kernel

[Alexander Monakov]

On Wed, 8 Jun 2016, fREW Schmidt wrote:
> I was debugging an issue I ran into (
> https://github.com/docker/docker/issues/23378) and after chatting with the
> folks in #musl and we triaged it down to a broken vDSO (tested by running
> `strace date` and seeing a clock_gettime call.)

To provide a bit more detail, we've found that the vdso mappend into the
application has no dynamic symbols; Glibc ignores the vdso, so it continues to
work, but musl segfaults since the vdso is invalid.

I'm pasting below `readelf -aW` output on the vdso dumped on the affected
system (via 'fwrite((void*)getauxval(AT_SYSINFO_EHDR), 8192, 1, stdout)';
as you can see, there's no dynamic symbols and symbol hash tables.  It may
be a toolchain bug since there was no issue with 4.6 kernel (and reportedly
there were no significant vdso changes merged into 4.7 -- the issue is seen
on Ubuntu's 4.7rc2).

Alexander

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x600
  Start of program headers:          64 (bytes into file)
  Start of section headers:          2976 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         4
  Size of section headers:           64 (bytes)
  Number of section headers:         10
  Section header string table index: 9

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .rodata           PROGBITS        0000000000000120 000120 000340 00  WA  0   0  1
  [ 2] .note             NOTE            0000000000000460 000460 00003c 00   A  0   0  4
  [ 3] .eh_frame_hdr     PROGBITS        000000000000049c 00049c 00003c 00   A  0   0  4
  [ 4] .eh_frame         PROGBITS        00000000000004d8 0004d8 000120 00   A  0   0  8
  [ 5] .text             PROGBITS        0000000000000600 000600 0004c9 00  AX  0   0 16
  [ 6] .altinstructions  PROGBITS        0000000000000ac9 000ac9 000034 00   A  0   0  1
  [ 7] .altinstr_replacement PROGBITS        0000000000000afd 000afd 00000c 00  AX  0   0  1
  [ 8] .comment          PROGBITS        0000000000000000 000b09 00002e 01  MS  0   0  1
  [ 9] .shstrtab         STRTAB          0000000000000000 000b37 000067 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

There are no section groups in this file.

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x000b09 0x000b09 R E 0x1000
  DYNAMIC        0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 R   0x8
  NOTE           0x000460 0x0000000000000460 0x0000000000000460 0x00003c 0x00003c R   0x4
  GNU_EH_FRAME   0x00049c 0x000000000000049c 0x000000000000049c 0x00003c 0x00003c R   0x4

 Section to Segment mapping:
  Segment Sections...
   00     .rodata .note .eh_frame_hdr .eh_frame .text .altinstructions .altinstr_replacement 
   01     
   02     .note 
   03     .eh_frame_hdr 

There is no dynamic section in this file.

There are no relocations in this file.

The decoding of unwind sections for machine type Advanced Micro Devices X86-64 is not currently supported.

No version information found in this file.

Displaying notes found at file offset 0x00000460 with length 0x0000003c:
  Owner                 Data size	Description
  Linux                0x00000004	Unknown note type: (0x00000000)
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 496e9db3f494533ffaaf39737eb1023938861349

Re: [musl] Broken vDSO on built kernel

[Andy Lutomirski]

On Wed, Jun 8, 2016 at 2:36 PM, Alexander Monakov <amonakov@ispras.ru> wrote:
> On Wed, 8 Jun 2016, fREW Schmidt wrote:
>> I was debugging an issue I ran into (
>> https://github.com/docker/docker/issues/23378) and after chatting with the
>> folks in #musl and we triaged it down to a broken vDSO (tested by running
>> `strace date` and seeing a clock_gettime call.)
>
> To provide a bit more detail, we've found that the vdso mappend into the
> application has no dynamic symbols; Glibc ignores the vdso, so it continues to
> work, but musl segfaults since the vdso is invalid.
>
> I'm pasting below `readelf -aW` output on the vdso dumped on the affected
> system (via 'fwrite((void*)getauxval(AT_SYSINFO_EHDR), 8192, 1, stdout)';
> as you can see, there's no dynamic symbols and symbol hash tables.  It may
> be a toolchain bug since there was no issue with 4.6 kernel (and reportedly
> there were no significant vdso changes merged into 4.7 -- the issue is seen
> on Ubuntu's 4.7rc2).

Hmm.  The vdso64.so in /lib/firmware in that image has the same
problem, so this isn't vdso2c's fault.  I added this to my -next queue
to help diagnose it:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/vdso&id=6436d4c1a83cbe083e54c53bb2b16efaab9b62f4

--Andy