Re: C vdso

Re: C vdso

[Christophe Leroy]

Hi Michael

Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
> Hi Christophe,
> 
> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>> Hi Michael,
>>
>> What is the status with the generic C vdso merge ?
>> In some mail, you mentionned having difficulties getting it working on
>> ppc64, any progress ? What's the problem ? Can I help ?
> 
> Yeah sorry I was hoping to get time to work on it but haven't been able
> to.
> 
> It's causing crashes on ppc64 ie. big endian.
> 

> 
> As you can see from the instruction dump we have jumped into the weeds somewhere.
> 
> We also had the report from the kbuild robot about rela.opd being
> discarded, which I think is indicative of a bigger problem. ie. we don't
> process relocations for the VDSO, but opds require relocations (they
> contain an absolute pointer).
> 
> I thought we could get away with that, because the VDSO entry points
> aren't proper functions (so they don't have opds), and I didn't think
> we'd be calling via function pointers in the VDSO code (which would
> require opds). But seems something is not working right.
> 
> Sorry I haven't got back to you with those details. Things are a bit of
> a mess inside IBM at the moment (always?), and I've been trying to get
> everything done before I take a holiday next week.
> 


Can you tell what defconfig you are using ? I have been able to setup a 
full glibc PPC64 cross compilation chain and been able to test it under 
QEMU with success, using Nathan's vdsotest tool.

I tested with both ppc64_defconfig and pseries_defconfig.

The only problem I got is with getcpu, which segfaults but both before 
and after applying my series, so I guess this is unrelated.

Not sure we can pay too much attention to the exact measurement as it is 
a ppc64 QEMU running on a x86 Linux which is running in a Virtual Box on 
a x86 windows Laptop, but at least it works:

Without the series:

clock-getres-monotonic:    vdso: 389 nsec/call
clock-gettime-monotonic:    vdso: 781 nsec/call
clock-getres-monotonic-coarse:    vdso: 13715 nsec/call
clock-gettime-monotonic-coarse:    vdso: 312 nsec/call
clock-getres-monotonic-raw:    vdso: 13589 nsec/call
clock-getres-tai:    vdso: 13827 nsec/call
clock-gettime-tai:    vdso: 14846 nsec/call
clock-getres-boottime:    vdso: 13596 nsec/call
clock-gettime-boottime:    vdso: 14758 nsec/call
clock-getres-realtime:    vdso: 327 nsec/call
clock-gettime-realtime:    vdso: 717 nsec/call
clock-getres-realtime-coarse:    vdso: 14102 nsec/call
clock-gettime-realtime-coarse:    vdso: 299 nsec/call
gettimeofday:    vdso: 771 nsec/call

With the series:

clock-getres-monotonic:    vdso: 350 nsec/call
clock-gettime-monotonic:    vdso: 726 nsec/call
clock-getres-monotonic-coarse:    vdso: 356 nsec/call
clock-gettime-monotonic-coarse:    vdso: 423 nsec/call
clock-getres-monotonic-raw:    vdso: 349 nsec/call
clock-getres-tai:    vdso: 419 nsec/call
clock-gettime-tai:    vdso: 724 nsec/call
clock-getres-boottime:    vdso: 352 nsec/call
clock-gettime-boottime:    vdso: 752 nsec/call
clock-getres-realtime:    vdso: 351 nsec/call
clock-gettime-realtime:    vdso: 733 nsec/call
clock-getres-realtime-coarse:    vdso: 356 nsec/call
clock-gettime-realtime-coarse:    vdso: 367 nsec/call
gettimeofday:    vdso: 796 nsec/call


Thanks
Christophe

Re: C vdso

[Christophe Leroy]

Hi Michael,

Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
> Hi Michael
> 
> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>> Hi Christophe,
>>
>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>> Hi Michael,
>>>
>>> What is the status with the generic C vdso merge ?
>>> In some mail, you mentionned having difficulties getting it working on
>>> ppc64, any progress ? What's the problem ? Can I help ?
>>
>> Yeah sorry I was hoping to get time to work on it but haven't been able
>> to.
>>
>> It's causing crashes on ppc64 ie. big endian.
>>
> 
>>
>> As you can see from the instruction dump we have jumped into the weeds somewhere.
>>
>> We also had the report from the kbuild robot about rela.opd being
>> discarded, which I think is indicative of a bigger problem. ie. we don't
>> process relocations for the VDSO, but opds require relocations (they
>> contain an absolute pointer).
>>
>> I thought we could get away with that, because the VDSO entry points
>> aren't proper functions (so they don't have opds), and I didn't think
>> we'd be calling via function pointers in the VDSO code (which would
>> require opds). But seems something is not working right.
>>
>> Sorry I haven't got back to you with those details. Things are a bit of
>> a mess inside IBM at the moment (always?), and I've been trying to get
>> everything done before I take a holiday next week.
>>
> 
> 
> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross 
> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.


What config are you using ?

Christophe

> 
> I tested with both ppc64_defconfig and pseries_defconfig.
> 
> The only problem I got is with getcpu, which segfaults but both before and after applying my series, 
> so I guess this is unrelated.
> 
> Not sure we can pay too much attention to the exact measurement as it is a ppc64 QEMU running on a 
> x86 Linux which is running in a Virtual Box on a x86 windows Laptop, but at least it works:
> 
> Without the series:
> 
> clock-getres-monotonic:    vdso: 389 nsec/call
> clock-gettime-monotonic:    vdso: 781 nsec/call
> clock-getres-monotonic-coarse:    vdso: 13715 nsec/call
> clock-gettime-monotonic-coarse:    vdso: 312 nsec/call
> clock-getres-monotonic-raw:    vdso: 13589 nsec/call
> clock-getres-tai:    vdso: 13827 nsec/call
> clock-gettime-tai:    vdso: 14846 nsec/call
> clock-getres-boottime:    vdso: 13596 nsec/call
> clock-gettime-boottime:    vdso: 14758 nsec/call
> clock-getres-realtime:    vdso: 327 nsec/call
> clock-gettime-realtime:    vdso: 717 nsec/call
> clock-getres-realtime-coarse:    vdso: 14102 nsec/call
> clock-gettime-realtime-coarse:    vdso: 299 nsec/call
> gettimeofday:    vdso: 771 nsec/call
> 
> With the series:
> 
> clock-getres-monotonic:    vdso: 350 nsec/call
> clock-gettime-monotonic:    vdso: 726 nsec/call
> clock-getres-monotonic-coarse:    vdso: 356 nsec/call
> clock-gettime-monotonic-coarse:    vdso: 423 nsec/call
> clock-getres-monotonic-raw:    vdso: 349 nsec/call
> clock-getres-tai:    vdso: 419 nsec/call
> clock-gettime-tai:    vdso: 724 nsec/call
> clock-getres-boottime:    vdso: 352 nsec/call
> clock-gettime-boottime:    vdso: 752 nsec/call
> clock-getres-realtime:    vdso: 351 nsec/call
> clock-gettime-realtime:    vdso: 733 nsec/call
> clock-getres-realtime-coarse:    vdso: 356 nsec/call
> clock-gettime-realtime-coarse:    vdso: 367 nsec/call
> gettimeofday:    vdso: 796 nsec/call
> 
> 
> Thanks
> Christophe

Re: C vdso

[Michael Ellerman]

Christophe Leroy <christophe.leroy@csgroup.eu> writes:
> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>
>>>> What is the status with the generic C vdso merge ?
>>>> In some mail, you mentionned having difficulties getting it working on
>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>
>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>> to.
>>>
>>> It's causing crashes on ppc64 ie. big endian.
...
>> 
>> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross 
>> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.
>
> What config are you using ?

ppc64_defconfig + guest.config

Or pseries_defconfig.

I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other toolchains too.

At a minimum we're seeing relocations in the output, which is a problem:

  $ readelf -r build\~/arch/powerpc/kernel/vdso64/vdso64.so
  
  Relocation section '.rela.dyn' at offset 0x12a8 contains 8 entries:
    Offset          Info           Type           Sym. Value    Sym. Name + Addend
  000000001368  000000000016 R_PPC64_RELATIVE                     7c0
  000000001370  000000000016 R_PPC64_RELATIVE                     9300
  000000001380  000000000016 R_PPC64_RELATIVE                     970
  000000001388  000000000016 R_PPC64_RELATIVE                     9300
  000000001398  000000000016 R_PPC64_RELATIVE                     a90
  0000000013a0  000000000016 R_PPC64_RELATIVE                     9300
  0000000013b0  000000000016 R_PPC64_RELATIVE                     b20
  0000000013b8  000000000016 R_PPC64_RELATIVE                     9300



cheers

Re: C vdso

[Michael Ellerman]

Michael Ellerman <mpe@ellerman.id.au> writes:
> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>
>>>>> What is the status with the generic C vdso merge ?
>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>
>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>> to.
>>>>
>>>> It's causing crashes on ppc64 ie. big endian.
> ...
>>> 
>>> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross 
>>> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.
>>
>> What config are you using ?
>
> ppc64_defconfig + guest.config
>
> Or pseries_defconfig.
>
> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other toolchains too.

I'm also seeing warnings because of the feature fixups:

  ------------[ cut here ]------------
  WARNING: CPU: 0 PID: 1 at arch/powerpc/lib/feature-fixups.c:109 .do_feature_fixups+0x80/0xc0
  Modules linked in:
  CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.9.0-rc2-00261-g107a86292cc4 #11
  NIP:  c0000000000a3660 LR: c0000000000a362c CTR: 0000000000000000
  REGS: c00000007e3a3790 TRAP: 0700   Not tainted  (5.9.0-rc2-00261-g107a86292cc4)
  MSR:  8000000002029032 <SF,VEC,EE,ME,IR,DR,RI>  CR: 48000422  XER: 00000000
  CFAR: c0000000000a3630 IRQMASK: 0
  GPR00: c0000000011e8964 c00000007e3a3a20 c000000001bb2b00 0000000000000001
  GPR04: c000000001bc0bc0 c000000001bc0bf0 0000000066736574 00000000fffffe00
  GPR08: 0000000300000004 0000000000000000 0000000000000000 0000000000000db8
  GPR12: 0000000028000224 c000000001dc0000 c000000000012d40 0000000000000000
  GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
  GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
  GPR24: c000000001aae0f0 c000000001063f08 c000000001063f18 c000000001063f28
  GPR28: c00000000106e188 000000eb8f4d91a7 c000000001bc0bf0 c000000001bc0bc0
  NIP [c0000000000a3660] .do_feature_fixups+0x80/0xc0
  LR [c0000000000a362c] .do_feature_fixups+0x4c/0xc0
  Call Trace:
  [c00000007e3a3a20] [c0000000000a362c] .do_feature_fixups+0x4c/0xc0 (unreliable)
  [c00000007e3a3ab0] [c0000000011e8964] .vdso_init+0x498/0x95c
  [c00000007e3a3bd0] [c000000000012780] .do_one_initcall+0x60/0x2b8
  [c00000007e3a3cb0] [c0000000011e4d8c] .kernel_init_freeable+0x2d8/0x370
  [c00000007e3a3da0] [c000000000012d64] .kernel_init+0x24/0x150
  [c00000007e3a3e20] [c00000000000e24c] .ret_from_kernel_thread+0x58/0x6c
  Instruction dump:
  40820030 3bff0030 7c3ef840 4181ffe4 38210090 e8010010 eb81ffe0 eba1ffe8
  ebc1fff0 7c0803a6 ebe1fff8 4e800020 <0fe00000> e8ff0028 e8df0020 7f83e378
  ---[ end trace ece1c957ca5bd6e9 ]---
  Unable to patch feature section at bffffffd01bc0bbc - c000000001bc0bc0 with bffffd9101bc0958 - bfffffe501bc0ba4


That's happening because the 32-bit VDSO is built with CONFIG_PPC32=y,
due to config-fake32.h, and that causes the feature fixup entries to be
the wrong size.

See the logic in feature-fixup.h:

  #if defined(CONFIG_PPC64) && !defined(__powerpc64__)
  /* 64 bits kernel, 32 bits code (ie. vdso32) */
  #define FTR_ENTRY_LONG		.8byte
  #define FTR_ENTRY_OFFSET	.long 0xffffffff; .long
  #elif defined(CONFIG_PPC64)
  #define FTR_ENTRY_LONG		.8byte
  #define FTR_ENTRY_OFFSET	.8byte
  #else
  #define FTR_ENTRY_LONG		.long
  #define FTR_ENTRY_OFFSET	.long
  #endif


We expect the fixup entries to still use 64-bit values, even for the
32-bit VDSO in a 64-bit kernel.

TBH I'm not sure how config-fake32.h can work long term, it's so fragile
to be defining/redefining a handful of CONFIG symbols like that.

The generic VDSO code is fairly careful to only include uapi and vdso
headers, not linux ones. So I think we need to better split our headers
so that we can build the VDSO code with few or no linux headers, and so
avoid the need to define any (or most) CONFIG symbols.

cheers

Re: C vdso

[Christophe Leroy]

Le 24/10/2020 à 12:07, Michael Ellerman a écrit :
> Michael Ellerman <mpe@ellerman.id.au> writes:
>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>>
>>>>>> What is the status with the generic C vdso merge ?
>>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>>
>>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>>> to.
>>>>>
>>>>> It's causing crashes on ppc64 ie. big endian.
>> ...
>>>>
>>>> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross
>>>> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.
>>>
>>> What config are you using ?
>>
>> ppc64_defconfig + guest.config
>>
>> Or pseries_defconfig.
>>
>> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other toolchains too.
> 
> I'm also seeing warnings because of the feature fixups:
> 
> 
> That's happening because the 32-bit VDSO is built with CONFIG_PPC32=y,
> due to config-fake32.h, and that causes the feature fixup entries to be
> the wrong size.
> 
> See the logic in feature-fixup.h:
> 
> 
> 
> We expect the fixup entries to still use 64-bit values, even for the
> 32-bit VDSO in a 64-bit kernel.
> 
> TBH I'm not sure how config-fake32.h can work long term, it's so fragile
> to be defining/redefining a handful of CONFIG symbols like that.

I took the idea from mips (arch/mips/vdso/config-n32-o32-env.c) after struggling in different 
direction. At that time the generic VDSO code was far less careful and was including several linux 
headers IIRC.
I agree with you that it's rather fragile.

> 
> The generic VDSO code is fairly careful to only include uapi and vdso
> headers, not linux ones. So I think we need to better split our headers
> so that we can build the VDSO code with few or no linux headers, and so
> avoid the need to define any (or most) CONFIG symbols.
> 

I'll revisit it when I'm back from vacation (I'm leaving now for two weeks).

Christophe

Re: C vdso

[Christophe Leroy]

Le 24/10/2020 à 12:07, Michael Ellerman a écrit :
> Michael Ellerman <mpe@ellerman.id.au> writes:
>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>>
>>>>>> What is the status with the generic C vdso merge ?
>>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>>
>>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>>> to.
>>>>>
>>>>> It's causing crashes on ppc64 ie. big endian.
>> ...
>>>>
>>>> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross
>>>> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.
>>>
>>> What config are you using ?
>>
>> ppc64_defconfig + guest.config
>>
>> Or pseries_defconfig.
>>
>> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other toolchains too.
> 
> I'm also seeing warnings because of the feature fixups:
> 

[...]

> 
> That's happening because the 32-bit VDSO is built with CONFIG_PPC32=y,
> due to config-fake32.h, and that causes the feature fixup entries to be
> the wrong size.
> 
> See the logic in feature-fixup.h:
> 
>    #if defined(CONFIG_PPC64) && !defined(__powerpc64__)
>    /* 64 bits kernel, 32 bits code (ie. vdso32) */
>    #define FTR_ENTRY_LONG		.8byte
>    #define FTR_ENTRY_OFFSET	.long 0xffffffff; .long
>    #elif defined(CONFIG_PPC64)
>    #define FTR_ENTRY_LONG		.8byte
>    #define FTR_ENTRY_OFFSET	.8byte
>    #else
>    #define FTR_ENTRY_LONG		.long
>    #define FTR_ENTRY_OFFSET	.long
>    #endif
> 
> 
> We expect the fixup entries to still use 64-bit values, even for the
> 32-bit VDSO in a 64-bit kernel.
> 
> TBH I'm not sure how config-fake32.h can work long term, it's so fragile
> to be defining/redefining a handful of CONFIG symbols like that.
> 
> The generic VDSO code is fairly careful to only include uapi and vdso
> headers, not linux ones. So I think we need to better split our headers
> so that we can build the VDSO code with few or no linux headers, and so
> avoid the need to define any (or most) CONFIG symbols.
> 

Finally, it was easy to do, just had to change a couple of __powerpc64__ into CONFIG_PPC64 in 
asm/cputable.h, and move asm/time.h functions playing with timebase into asm/timebase.h

Christophe

Re: C vdso

[Christophe Leroy]

Le 23/10/2020 à 15:24, Michael Ellerman a écrit :
> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>
>>>>> What is the status with the generic C vdso merge ?
>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>
>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>> to.
>>>>
>>>> It's causing crashes on ppc64 ie. big endian.
> ...
>>>
>>> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross
>>> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.
>>
>> What config are you using ?
> 
> ppc64_defconfig + guest.config
> 
> Or pseries_defconfig.
> 
> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other toolchains too.
> 
> At a minimum we're seeing relocations in the output, which is a problem:
> 
>    $ readelf -r build\~/arch/powerpc/kernel/vdso64/vdso64.so
>    
>    Relocation section '.rela.dyn' at offset 0x12a8 contains 8 entries:
>      Offset          Info           Type           Sym. Value    Sym. Name + Addend
>    000000001368  000000000016 R_PPC64_RELATIVE                     7c0
>    000000001370  000000000016 R_PPC64_RELATIVE                     9300
>    000000001380  000000000016 R_PPC64_RELATIVE                     970
>    000000001388  000000000016 R_PPC64_RELATIVE                     9300
>    000000001398  000000000016 R_PPC64_RELATIVE                     a90
>    0000000013a0  000000000016 R_PPC64_RELATIVE                     9300
>    0000000013b0  000000000016 R_PPC64_RELATIVE                     b20
>    0000000013b8  000000000016 R_PPC64_RELATIVE                     9300

Looks like it's due to the OPD and relation between the function() and .function()

By using DOTSYM() in the 'bl' call, that's directly the dot function which is called and the OPD is 
not used anymore, it can get dropped.

Now I get .rela.dyn full of 0, don't know if we should drop it explicitely.

Christophe

Re: C vdso

[Christophe Leroy]

Hi Michael,

Le 03/11/2020 à 19:13, Christophe Leroy a écrit :
> 
> 
> Le 23/10/2020 à 15:24, Michael Ellerman a écrit :
>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>>
>>>>>> What is the status with the generic C vdso merge ?
>>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>>
>>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>>> to.
>>>>>
>>>>> It's causing crashes on ppc64 ie. big endian.
>> ...
>>>>
>>>> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross
>>>> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.
>>>
>>> What config are you using ?
>>
>> ppc64_defconfig + guest.config
>>
>> Or pseries_defconfig.
>>
>> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other toolchains too.
>>
>> At a minimum we're seeing relocations in the output, which is a problem:
>>
>>    $ readelf -r build\~/arch/powerpc/kernel/vdso64/vdso64.so
>>    Relocation section '.rela.dyn' at offset 0x12a8 contains 8 entries:
>>      Offset          Info           Type           Sym. Value    Sym. Name + Addend
>>    000000001368  000000000016 R_PPC64_RELATIVE                     7c0
>>    000000001370  000000000016 R_PPC64_RELATIVE                     9300
>>    000000001380  000000000016 R_PPC64_RELATIVE                     970
>>    000000001388  000000000016 R_PPC64_RELATIVE                     9300
>>    000000001398  000000000016 R_PPC64_RELATIVE                     a90
>>    0000000013a0  000000000016 R_PPC64_RELATIVE                     9300
>>    0000000013b0  000000000016 R_PPC64_RELATIVE                     b20
>>    0000000013b8  000000000016 R_PPC64_RELATIVE                     9300
> 
> Looks like it's due to the OPD and relation between the function() and .function()
> 
> By using DOTSYM() in the 'bl' call, that's directly the dot function which is called and the OPD is 
> not used anymore, it can get dropped.
> 
> Now I get .rela.dyn full of 0, don't know if we should drop it explicitely.
> 

What is the status now with latest version of CVDSO ? I saw you had it in next-test for some time, 
it is not there anymore today.

Thanks,
Christophe

Re: C vdso

[Michael Ellerman]

Christophe Leroy <christophe.leroy@csgroup.eu> writes:
> Le 03/11/2020 à 19:13, Christophe Leroy a écrit :
>> Le 23/10/2020 à 15:24, Michael Ellerman a écrit :
>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>>>
>>>>>>> What is the status with the generic C vdso merge ?
>>>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>>>
>>>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>>>> to.
>>>>>>
>>>>>> It's causing crashes on ppc64 ie. big endian.
>>> ...
>>>>>
>>>>> Can you tell what defconfig you are using ? I have been able to setup a full glibc PPC64 cross
>>>>> compilation chain and been able to test it under QEMU with success, using Nathan's vdsotest tool.
>>>>
>>>> What config are you using ?
>>>
>>> ppc64_defconfig + guest.config
>>>
>>> Or pseries_defconfig.
>>>
>>> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other toolchains too.
>>>
>>> At a minimum we're seeing relocations in the output, which is a problem:
>>>
>>>    $ readelf -r build\~/arch/powerpc/kernel/vdso64/vdso64.so
>>>    Relocation section '.rela.dyn' at offset 0x12a8 contains 8 entries:
>>>      Offset          Info           Type           Sym. Value    Sym. Name + Addend
>>>    000000001368  000000000016 R_PPC64_RELATIVE                     7c0
>>>    000000001370  000000000016 R_PPC64_RELATIVE                     9300
>>>    000000001380  000000000016 R_PPC64_RELATIVE                     970
>>>    000000001388  000000000016 R_PPC64_RELATIVE                     9300
>>>    000000001398  000000000016 R_PPC64_RELATIVE                     a90
>>>    0000000013a0  000000000016 R_PPC64_RELATIVE                     9300
>>>    0000000013b0  000000000016 R_PPC64_RELATIVE                     b20
>>>    0000000013b8  000000000016 R_PPC64_RELATIVE                     9300
>> 
>> Looks like it's due to the OPD and relation between the function() and .function()
>> 
>> By using DOTSYM() in the 'bl' call, that's directly the dot function which is called and the OPD is 
>> not used anymore, it can get dropped.
>> 
>> Now I get .rela.dyn full of 0, don't know if we should drop it explicitely.
>
> What is the status now with latest version of CVDSO ? I saw you had it in next-test for some time, 
> it is not there anymore today.

Still having some trouble with the compat VDSO.

eg:

$ ./vdsotest clock-gettime-monotonic verify
timestamp obtained from kernel predates timestamp
previously obtained from libc/vDSO:
	[1346, 821441653] (vDSO)
	[570, 769440040] (kernel)


And similar for all clocks except the coarse ones.

Hopefully I can find time to dig into it.

cheers

Re: C vdso

[Christophe Leroy]

Quoting Michael Ellerman <mpe@ellerman.id.au>:

> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>> Le 03/11/2020 à 19:13, Christophe Leroy a écrit :
>>> Le 23/10/2020 à 15:24, Michael Ellerman a écrit :
>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>>>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>>>>
>>>>>>>> What is the status with the generic C vdso merge ?
>>>>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>>>>
>>>>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>>>>> to.
>>>>>>>
>>>>>>> It's causing crashes on ppc64 ie. big endian.
>>>> ...
>>>>>>
>>>>>> Can you tell what defconfig you are using ? I have been able to  
>>>>>> setup a full glibc PPC64 cross
>>>>>> compilation chain and been able to test it under QEMU with  
>>>>>> success, using Nathan's vdsotest tool.
>>>>>
>>>>> What config are you using ?
>>>>
>>>> ppc64_defconfig + guest.config
>>>>
>>>> Or pseries_defconfig.
>>>>
>>>> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other  
>>>> toolchains too.
>>>>
>>>> At a minimum we're seeing relocations in the output, which is a problem:
>>>>
>>>>    $ readelf -r build\~/arch/powerpc/kernel/vdso64/vdso64.so
>>>>    Relocation section '.rela.dyn' at offset 0x12a8 contains 8 entries:
>>>>      Offset          Info           Type           Sym. Value     
>>>> Sym. Name + Addend
>>>>    000000001368  000000000016 R_PPC64_RELATIVE                     7c0
>>>>    000000001370  000000000016 R_PPC64_RELATIVE                     9300
>>>>    000000001380  000000000016 R_PPC64_RELATIVE                     970
>>>>    000000001388  000000000016 R_PPC64_RELATIVE                     9300
>>>>    000000001398  000000000016 R_PPC64_RELATIVE                     a90
>>>>    0000000013a0  000000000016 R_PPC64_RELATIVE                     9300
>>>>    0000000013b0  000000000016 R_PPC64_RELATIVE                     b20
>>>>    0000000013b8  000000000016 R_PPC64_RELATIVE                     9300
>>>
>>> Looks like it's due to the OPD and relation between the function()  
>>> and .function()
>>>
>>> By using DOTSYM() in the 'bl' call, that's directly the dot  
>>> function which is called and the OPD is
>>> not used anymore, it can get dropped.
>>>
>>> Now I get .rela.dyn full of 0, don't know if we should drop it explicitely.
>>
>> What is the status now with latest version of CVDSO ? I saw you had  
>> it in next-test for some time,
>> it is not there anymore today.
>
> Still having some trouble with the compat VDSO.
>
> eg:
>
> $ ./vdsotest clock-gettime-monotonic verify
> timestamp obtained from kernel predates timestamp
> previously obtained from libc/vDSO:
> 	[1346, 821441653] (vDSO)
> 	[570, 769440040] (kernel)
>
>
> And similar for all clocks except the coarse ones.
>

Ok, I managed to get the same with QEMU. Looking at the binary, I only  
see an mftb instead of the mftbu/mftb/mftbu triplet.

Fix below. Can you carry it, or do you prefer a full patch from me ?  
The easiest would be either to squash it into [v13,4/8]  
("powerpc/time: Move timebase functions into new asm/timebase.h"), or  
to add it between patch 4 and 5 ?

diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index f877a576b338..c3473eb031a3 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -1419,7 +1419,7 @@ static inline void msr_check_and_clear(unsigned  
long bits)
  		__msr_check_and_clear(bits);
  }

-#if defined(CONFIG_PPC_CELL) || defined(CONFIG_E500)
+#if defined(__powerpc64__) && (defined(CONFIG_PPC_CELL) ||  
defined(CONFIG_E500))
  #define mftb()		({unsigned long rval;				\
  			asm volatile(					\
  				"90:	mfspr %0, %2;\n"		\
diff --git a/arch/powerpc/include/asm/timebase.h  
b/arch/powerpc/include/asm/timebase.h
index a8eae3adaa91..7b372976f5a5 100644
--- a/arch/powerpc/include/asm/timebase.h
+++ b/arch/powerpc/include/asm/timebase.h
@@ -21,7 +21,7 @@ static inline u64 get_tb(void)
  {
  	unsigned int tbhi, tblo, tbhi2;

-	if (IS_ENABLED(CONFIG_PPC64))
+	if (IS_BUILTIN(__powerpc64__))
  		return mftb();

  	do {

Re: C vdso

[Michael Ellerman]

Christophe Leroy <christophe.leroy@csgroup.eu> writes:
> Quoting Michael Ellerman <mpe@ellerman.id.au>:
>
>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>> Le 03/11/2020 à 19:13, Christophe Leroy a écrit :
>>>> Le 23/10/2020 à 15:24, Michael Ellerman a écrit :
>>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>> Le 24/09/2020 à 15:17, Christophe Leroy a écrit :
>>>>>>> Le 17/09/2020 à 14:33, Michael Ellerman a écrit :
>>>>>>>> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>>>>>>>>>
>>>>>>>>> What is the status with the generic C vdso merge ?
>>>>>>>>> In some mail, you mentionned having difficulties getting it working on
>>>>>>>>> ppc64, any progress ? What's the problem ? Can I help ?
>>>>>>>>
>>>>>>>> Yeah sorry I was hoping to get time to work on it but haven't been able
>>>>>>>> to.
>>>>>>>>
>>>>>>>> It's causing crashes on ppc64 ie. big endian.
>>>>> ...
>>>>>>>
>>>>>>> Can you tell what defconfig you are using ? I have been able to  
>>>>>>> setup a full glibc PPC64 cross
>>>>>>> compilation chain and been able to test it under QEMU with  
>>>>>>> success, using Nathan's vdsotest tool.
>>>>>>
>>>>>> What config are you using ?
>>>>>
>>>>> ppc64_defconfig + guest.config
>>>>>
>>>>> Or pseries_defconfig.
>>>>>
>>>>> I'm using Ubuntu GCC 9.3.0 mostly, but it happens with other  
>>>>> toolchains too.
>>>>>
>>>>> At a minimum we're seeing relocations in the output, which is a problem:
>>>>>
>>>>>    $ readelf -r build\~/arch/powerpc/kernel/vdso64/vdso64.so
>>>>>    Relocation section '.rela.dyn' at offset 0x12a8 contains 8 entries:
>>>>>      Offset          Info           Type           Sym. Value     
>>>>> Sym. Name + Addend
>>>>>    000000001368  000000000016 R_PPC64_RELATIVE                     7c0
>>>>>    000000001370  000000000016 R_PPC64_RELATIVE                     9300
>>>>>    000000001380  000000000016 R_PPC64_RELATIVE                     970
>>>>>    000000001388  000000000016 R_PPC64_RELATIVE                     9300
>>>>>    000000001398  000000000016 R_PPC64_RELATIVE                     a90
>>>>>    0000000013a0  000000000016 R_PPC64_RELATIVE                     9300
>>>>>    0000000013b0  000000000016 R_PPC64_RELATIVE                     b20
>>>>>    0000000013b8  000000000016 R_PPC64_RELATIVE                     9300
>>>>
>>>> Looks like it's due to the OPD and relation between the function()  
>>>> and .function()
>>>>
>>>> By using DOTSYM() in the 'bl' call, that's directly the dot  
>>>> function which is called and the OPD is
>>>> not used anymore, it can get dropped.
>>>>
>>>> Now I get .rela.dyn full of 0, don't know if we should drop it explicitely.
>>>
>>> What is the status now with latest version of CVDSO ? I saw you had  
>>> it in next-test for some time,
>>> it is not there anymore today.
>>
>> Still having some trouble with the compat VDSO.
>>
>> eg:
>>
>> $ ./vdsotest clock-gettime-monotonic verify
>> timestamp obtained from kernel predates timestamp
>> previously obtained from libc/vDSO:
>> 	[1346, 821441653] (vDSO)
>> 	[570, 769440040] (kernel)
>>
>>
>> And similar for all clocks except the coarse ones.
>>
>
> Ok, I managed to get the same with QEMU. Looking at the binary, I only  
> see an mftb instead of the mftbu/mftb/mftbu triplet.
>
> Fix below. Can you carry it, or do you prefer a full patch from me ?  
> The easiest would be either to squash it into [v13,4/8]  
> ("powerpc/time: Move timebase functions into new asm/timebase.h"), or  
> to add it between patch 4 and 5 ?

I can squash it in.

cheers