* [PATCH 1/8] powerpc/64s/exception: Clean up a missed SRR specifier
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
@ 2021-02-25 3:09 ` Daniel Axtens
2021-02-25 3:10 ` [RFC PATCH 2/8] powerpc: check for support for -Wa,-m{power4,any} Daniel Axtens
` (7 subsequent siblings)
8 siblings, 0 replies; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:09 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Nicholas Piggin, Daniel Axtens
Nick's patch cleaning up the SRR specifiers in exception-64s.S
missed a single instance of EXC_HV_OR_STD. Clean that up.
Caught by clang's integrated assembler.
Fixes: 3f7fbd97d07d ("powerpc/64s/exception: Clean up SRR specifiers")
Acked-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/kernel/exceptions-64s.S | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index a4bd3c114a0a..720fb9892745 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -466,7 +466,7 @@ DEFINE_FIXED_SYMBOL(\name\()_common_real)
ld r10,PACAKMSR(r13) /* get MSR value for kernel */
/* MSR[RI] is clear iff using SRR regs */
- .if IHSRR == EXC_HV_OR_STD
+ .if IHSRR_IF_HVMODE
BEGIN_FTR_SECTION
xori r10,r10,MSR_RI
END_FTR_SECTION_IFCLR(CPU_FTR_HVMODE)
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [RFC PATCH 2/8] powerpc: check for support for -Wa,-m{power4,any}
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
2021-02-25 3:09 ` [PATCH 1/8] powerpc/64s/exception: Clean up a missed SRR specifier Daniel Axtens
@ 2021-02-25 3:10 ` Daniel Axtens
2021-03-19 1:32 ` [RFC PATCH 2/8] powerpc: check for support for -Wa, -m{power4, any} Nicholas Piggin
2021-02-25 3:10 ` [RFC PATCH 3/8] powerpc/head-64: do less gas-specific stuff with sections Daniel Axtens
` (6 subsequent siblings)
8 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:10 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Daniel Axtens
LLVM's integrated assembler does not like either -Wa,-mpower4
or -Wa,-many. So just don't pass them if they're not supported.
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/Makefile | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
index 08cf0eade56a..3e2c72d20bb8 100644
--- a/arch/powerpc/Makefile
+++ b/arch/powerpc/Makefile
@@ -252,7 +252,9 @@ cpu-as-$(CONFIG_E500) += -Wa,-me500
# When using '-many -mpower4' gas will first try and find a matching power4
# mnemonic and failing that it will allow any valid mnemonic that GAS knows
# about. GCC will pass -many to GAS when assembling, clang does not.
-cpu-as-$(CONFIG_PPC_BOOK3S_64) += -Wa,-mpower4 -Wa,-many
+# LLVM IAS doesn't understand either flag: https://github.com/ClangBuiltLinux/linux/issues/675
+# but LLVM IAS only supports ISA >= 2.06 for Book3S 64 anyway...
+cpu-as-$(CONFIG_PPC_BOOK3S_64) += $(call as-option,-Wa$(comma)-mpower4) $(call as-option,-Wa$(comma)-many)
cpu-as-$(CONFIG_PPC_E500MC) += $(call as-option,-Wa$(comma)-me500mc)
KBUILD_AFLAGS += $(cpu-as-y)
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 2/8] powerpc: check for support for -Wa, -m{power4, any}
2021-02-25 3:10 ` [RFC PATCH 2/8] powerpc: check for support for -Wa,-m{power4,any} Daniel Axtens
@ 2021-03-19 1:32 ` Nicholas Piggin
2021-03-19 10:53 ` Michael Ellerman
0 siblings, 1 reply; 30+ messages in thread
From: Nicholas Piggin @ 2021-03-19 1:32 UTC (permalink / raw)
To: Daniel Axtens, linuxppc-dev, llvmlinux
Excerpts from Daniel Axtens's message of February 25, 2021 1:10 pm:
> LLVM's integrated assembler does not like either -Wa,-mpower4
> or -Wa,-many. So just don't pass them if they're not supported.
>
> Signed-off-by: Daniel Axtens <dja@axtens.net>
> ---
> arch/powerpc/Makefile | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
> index 08cf0eade56a..3e2c72d20bb8 100644
> --- a/arch/powerpc/Makefile
> +++ b/arch/powerpc/Makefile
> @@ -252,7 +252,9 @@ cpu-as-$(CONFIG_E500) += -Wa,-me500
> # When using '-many -mpower4' gas will first try and find a matching power4
> # mnemonic and failing that it will allow any valid mnemonic that GAS knows
> # about. GCC will pass -many to GAS when assembling, clang does not.
> -cpu-as-$(CONFIG_PPC_BOOK3S_64) += -Wa,-mpower4 -Wa,-many
> +# LLVM IAS doesn't understand either flag: https://github.com/ClangBuiltLinux/linux/issues/675
> +# but LLVM IAS only supports ISA >= 2.06 for Book3S 64 anyway...
> +cpu-as-$(CONFIG_PPC_BOOK3S_64) += $(call as-option,-Wa$(comma)-mpower4) $(call as-option,-Wa$(comma)-many)
> cpu-as-$(CONFIG_PPC_E500MC) += $(call as-option,-Wa$(comma)-me500mc)
>
> KBUILD_AFLAGS += $(cpu-as-y)
I'm wondering why we even have this now. Kbuild's "AS" command goes
through the C compiler now with relevant options like -mcpu. I assume it
used to be useful for cross compiling when as was called directly but
I'm not sure.
Thanks,
Nick
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 2/8] powerpc: check for support for -Wa, -m{power4, any}
2021-03-19 1:32 ` [RFC PATCH 2/8] powerpc: check for support for -Wa, -m{power4, any} Nicholas Piggin
@ 2021-03-19 10:53 ` Michael Ellerman
0 siblings, 0 replies; 30+ messages in thread
From: Michael Ellerman @ 2021-03-19 10:53 UTC (permalink / raw)
To: Nicholas Piggin, Daniel Axtens, linuxppc-dev, llvmlinux
Nicholas Piggin <npiggin@gmail.com> writes:
> Excerpts from Daniel Axtens's message of February 25, 2021 1:10 pm:
>> LLVM's integrated assembler does not like either -Wa,-mpower4
>> or -Wa,-many. So just don't pass them if they're not supported.
>>
>> Signed-off-by: Daniel Axtens <dja@axtens.net>
>> ---
>> arch/powerpc/Makefile | 4 +++-
>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
>> index 08cf0eade56a..3e2c72d20bb8 100644
>> --- a/arch/powerpc/Makefile
>> +++ b/arch/powerpc/Makefile
>> @@ -252,7 +252,9 @@ cpu-as-$(CONFIG_E500) += -Wa,-me500
>> # When using '-many -mpower4' gas will first try and find a matching power4
>> # mnemonic and failing that it will allow any valid mnemonic that GAS knows
>> # about. GCC will pass -many to GAS when assembling, clang does not.
>> -cpu-as-$(CONFIG_PPC_BOOK3S_64) += -Wa,-mpower4 -Wa,-many
>> +# LLVM IAS doesn't understand either flag: https://github.com/ClangBuiltLinux/linux/issues/675
>> +# but LLVM IAS only supports ISA >= 2.06 for Book3S 64 anyway...
>> +cpu-as-$(CONFIG_PPC_BOOK3S_64) += $(call as-option,-Wa$(comma)-mpower4) $(call as-option,-Wa$(comma)-many)
>> cpu-as-$(CONFIG_PPC_E500MC) += $(call as-option,-Wa$(comma)-me500mc)
>>
>> KBUILD_AFLAGS += $(cpu-as-y)
>
> I'm wondering why we even have this now. Kbuild's "AS" command goes
> through the C compiler now with relevant options like -mcpu.
It uses $(CC) but it doesn't pass it $CFLAGS AFAIK. So it would use
whatever the compiler default is for -mcpu etc. I think.
> I assume it used to be useful for cross compiling when as was called
> directly but I'm not sure.
We still use it directly in vdso32/Makefile.
cheers
^ permalink raw reply [flat|nested] 30+ messages in thread
* [RFC PATCH 3/8] powerpc/head-64: do less gas-specific stuff with sections
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
2021-02-25 3:09 ` [PATCH 1/8] powerpc/64s/exception: Clean up a missed SRR specifier Daniel Axtens
2021-02-25 3:10 ` [RFC PATCH 2/8] powerpc: check for support for -Wa,-m{power4,any} Daniel Axtens
@ 2021-02-25 3:10 ` Daniel Axtens
2021-03-19 1:35 ` Nicholas Piggin
2021-02-25 3:10 ` [RFC PATCH 4/8] powerpc/ppc_asm: use plain numbers for registers Daniel Axtens
` (5 subsequent siblings)
8 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:10 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Daniel Axtens
Reopening the section without specifying the same flags breaks
the llvm integrated assembler. Don't do it: just specify all the
flags all the time.
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/include/asm/head-64.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/head-64.h b/arch/powerpc/include/asm/head-64.h
index 4cb9efa2eb21..7d8ccab47e86 100644
--- a/arch/powerpc/include/asm/head-64.h
+++ b/arch/powerpc/include/asm/head-64.h
@@ -15,10 +15,10 @@
.macro define_data_ftsec name
.section ".head.data.\name\()","a",@progbits
.endm
-.macro use_ftsec name
- .section ".head.text.\name\()"
-.endm
-
+//.macro use_ftsec name
+// .section ".head.text.\name\()"
+//.endm
+#define use_ftsec define_ftsec
/*
* Fixed (location) sections are used by opening fixed sections and emitting
* fixed section entries into them before closing them. Multiple fixed sections
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 3/8] powerpc/head-64: do less gas-specific stuff with sections
2021-02-25 3:10 ` [RFC PATCH 3/8] powerpc/head-64: do less gas-specific stuff with sections Daniel Axtens
@ 2021-03-19 1:35 ` Nicholas Piggin
0 siblings, 0 replies; 30+ messages in thread
From: Nicholas Piggin @ 2021-03-19 1:35 UTC (permalink / raw)
To: Daniel Axtens, linuxppc-dev, llvmlinux
Excerpts from Daniel Axtens's message of February 25, 2021 1:10 pm:
> Reopening the section without specifying the same flags breaks
> the llvm integrated assembler. Don't do it: just specify all the
> flags all the time.
I don't have a problem with this but llvm might want to track the issue
if it aims to be compatible with gas if you haven't alread opened an
issue.
When you fix the patch (perhaps add a quick comment as well?), then
Acked-by: Nicholas Piggin <npiggin@gmail.com>
Thanks,
Nick
>
> Signed-off-by: Daniel Axtens <dja@axtens.net>
> ---
> arch/powerpc/include/asm/head-64.h | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/head-64.h b/arch/powerpc/include/asm/head-64.h
> index 4cb9efa2eb21..7d8ccab47e86 100644
> --- a/arch/powerpc/include/asm/head-64.h
> +++ b/arch/powerpc/include/asm/head-64.h
> @@ -15,10 +15,10 @@
> .macro define_data_ftsec name
> .section ".head.data.\name\()","a",@progbits
> .endm
> -.macro use_ftsec name
> - .section ".head.text.\name\()"
> -.endm
> -
> +//.macro use_ftsec name
> +// .section ".head.text.\name\()"
> +//.endm
> +#define use_ftsec define_ftsec
> /*
> * Fixed (location) sections are used by opening fixed sections and emitting
> * fixed section entries into them before closing them. Multiple fixed sections
> --
> 2.27.0
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* [RFC PATCH 4/8] powerpc/ppc_asm: use plain numbers for registers
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
` (2 preceding siblings ...)
2021-02-25 3:10 ` [RFC PATCH 3/8] powerpc/head-64: do less gas-specific stuff with sections Daniel Axtens
@ 2021-02-25 3:10 ` Daniel Axtens
2021-02-25 15:25 ` Segher Boessenkool
2021-02-25 3:10 ` [RFC PATCH 5/8] poweprc/lib/quad: Provide macros for lq/stq Daniel Axtens
` (4 subsequent siblings)
8 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:10 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Daniel Axtens
This is dumb but makes the llvm integrated assembler happy.
https://github.com/ClangBuiltLinux/linux/issues/764
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/include/asm/ppc_asm.h | 64 +++++++++++++++---------------
1 file changed, 32 insertions(+), 32 deletions(-)
diff --git a/arch/powerpc/include/asm/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h
index 3dceb64fc9af..49da2cf4c2d5 100644
--- a/arch/powerpc/include/asm/ppc_asm.h
+++ b/arch/powerpc/include/asm/ppc_asm.h
@@ -509,38 +509,38 @@ END_FTR_SECTION_NESTED(CPU_FTR_CELL_TB_BUG, CPU_FTR_CELL_TB_BUG, 96)
* Use R0-31 only when really nessesary.
*/
-#define r0 %r0
-#define r1 %r1
-#define r2 %r2
-#define r3 %r3
-#define r4 %r4
-#define r5 %r5
-#define r6 %r6
-#define r7 %r7
-#define r8 %r8
-#define r9 %r9
-#define r10 %r10
-#define r11 %r11
-#define r12 %r12
-#define r13 %r13
-#define r14 %r14
-#define r15 %r15
-#define r16 %r16
-#define r17 %r17
-#define r18 %r18
-#define r19 %r19
-#define r20 %r20
-#define r21 %r21
-#define r22 %r22
-#define r23 %r23
-#define r24 %r24
-#define r25 %r25
-#define r26 %r26
-#define r27 %r27
-#define r28 %r28
-#define r29 %r29
-#define r30 %r30
-#define r31 %r31
+#define r0 0
+#define r1 1
+#define r2 2
+#define r3 3
+#define r4 4
+#define r5 5
+#define r6 6
+#define r7 7
+#define r8 8
+#define r9 9
+#define r10 10
+#define r11 11
+#define r12 12
+#define r13 13
+#define r14 14
+#define r15 15
+#define r16 16
+#define r17 17
+#define r18 18
+#define r19 19
+#define r20 20
+#define r21 21
+#define r22 22
+#define r23 23
+#define r24 24
+#define r25 25
+#define r26 26
+#define r27 27
+#define r28 28
+#define r29 29
+#define r30 30
+#define r31 31
/* Floating Point Registers (FPRs) */
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 4/8] powerpc/ppc_asm: use plain numbers for registers
2021-02-25 3:10 ` [RFC PATCH 4/8] powerpc/ppc_asm: use plain numbers for registers Daniel Axtens
@ 2021-02-25 15:25 ` Segher Boessenkool
2021-02-26 0:12 ` Daniel Axtens
0 siblings, 1 reply; 30+ messages in thread
From: Segher Boessenkool @ 2021-02-25 15:25 UTC (permalink / raw)
To: Daniel Axtens; +Cc: linuxppc-dev, llvmlinux
On Thu, Feb 25, 2021 at 02:10:02PM +1100, Daniel Axtens wrote:
> This is dumb but makes the llvm integrated assembler happy.
> https://github.com/ClangBuiltLinux/linux/issues/764
> -#define r0 %r0
> +#define r0 0
This is a big step back (compare 9a13a524ba37).
If you use a new enough GAS, you can use the -mregnames option and just
say "r0" directly (so not define it at all, or define it to itself).
===
addi 3,3,3
addi r3,r3,3
addi %r3,%r3,3
addi 3,3,3
addi r3,r3,r3
addi %r3,%r3,%r3
===
$ as t.s -o t.o -mregnames
t.s: Assembler messages:
t.s:6: Warning: invalid register expression
t.s:7: Warning: invalid register expression
Many people do not like bare numbers. It is a bit like not wearing
seatbelts (but so is all assembler code really: you just have to pay
attention). A better argument is that it is harder to read for people
not used to assembler code like this.
We used to have "#define r0 0" etc., and that was quite problematic.
Like that "addi r3,r3,r3" example, but also, people wrote "r0" where
only a plain 0 is allowed (like in "lwzx r3,0,r3": "r0" would be
misleading there!)
Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 4/8] powerpc/ppc_asm: use plain numbers for registers
2021-02-25 15:25 ` Segher Boessenkool
@ 2021-02-26 0:12 ` Daniel Axtens
2021-03-19 1:39 ` Nicholas Piggin
0 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-26 0:12 UTC (permalink / raw)
To: Segher Boessenkool; +Cc: linuxppc-dev, llvmlinux
Segher Boessenkool <segher@kernel.crashing.org> writes:
> On Thu, Feb 25, 2021 at 02:10:02PM +1100, Daniel Axtens wrote:
>> This is dumb but makes the llvm integrated assembler happy.
>> https://github.com/ClangBuiltLinux/linux/issues/764
>
>> -#define r0 %r0
>
>> +#define r0 0
>
> This is a big step back (compare 9a13a524ba37).
>
> If you use a new enough GAS, you can use the -mregnames option and just
> say "r0" directly (so not define it at all, or define it to itself).
>
> ===
> addi 3,3,3
> addi r3,r3,3
> addi %r3,%r3,3
>
> addi 3,3,3
> addi r3,r3,r3
> addi %r3,%r3,%r3
> ===
>
> $ as t.s -o t.o -mregnames
> t.s: Assembler messages:
> t.s:6: Warning: invalid register expression
> t.s:7: Warning: invalid register expression
>
>
> Many people do not like bare numbers. It is a bit like not wearing
> seatbelts (but so is all assembler code really: you just have to pay
> attention). A better argument is that it is harder to read for people
> not used to assembler code like this.
>
> We used to have "#define r0 0" etc., and that was quite problematic.
> Like that "addi r3,r3,r3" example, but also, people wrote "r0" where
> only a plain 0 is allowed (like in "lwzx r3,0,r3": "r0" would be
> misleading there!)
So an overarching comment on all of these patches is that they're not
intended to be ready to merge, nor are they necessarily what I think is
the best solution. I'm just swinging a big hammer to see how far towards
LLVM_IAS=1 I can get on powerpc, and I accept I'm going to have to come
back and clean things up.
Anyway, noted, I'll push harder on trying to get llvm to accept %rN:
there was a patch that went in after llvm-11 that should help.
Kind regards,
Daniel
>
>
> Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 4/8] powerpc/ppc_asm: use plain numbers for registers
2021-02-26 0:12 ` Daniel Axtens
@ 2021-03-19 1:39 ` Nicholas Piggin
0 siblings, 0 replies; 30+ messages in thread
From: Nicholas Piggin @ 2021-03-19 1:39 UTC (permalink / raw)
To: Daniel Axtens, Segher Boessenkool; +Cc: llvmlinux, linuxppc-dev
Excerpts from Daniel Axtens's message of February 26, 2021 10:12 am:
> Segher Boessenkool <segher@kernel.crashing.org> writes:
>
>> On Thu, Feb 25, 2021 at 02:10:02PM +1100, Daniel Axtens wrote:
>>> This is dumb but makes the llvm integrated assembler happy.
>>> https://github.com/ClangBuiltLinux/linux/issues/764
>>
>>> -#define r0 %r0
>>
>>> +#define r0 0
>>
>> This is a big step back (compare 9a13a524ba37).
>>
>> If you use a new enough GAS, you can use the -mregnames option and just
>> say "r0" directly (so not define it at all, or define it to itself).
>>
>> ===
>> addi 3,3,3
>> addi r3,r3,3
>> addi %r3,%r3,3
>>
>> addi 3,3,3
>> addi r3,r3,r3
>> addi %r3,%r3,%r3
>> ===
>>
>> $ as t.s -o t.o -mregnames
>> t.s: Assembler messages:
>> t.s:6: Warning: invalid register expression
>> t.s:7: Warning: invalid register expression
>>
>>
>> Many people do not like bare numbers. It is a bit like not wearing
>> seatbelts (but so is all assembler code really: you just have to pay
>> attention). A better argument is that it is harder to read for people
>> not used to assembler code like this.
>>
>> We used to have "#define r0 0" etc., and that was quite problematic.
>> Like that "addi r3,r3,r3" example, but also, people wrote "r0" where
>> only a plain 0 is allowed (like in "lwzx r3,0,r3": "r0" would be
>> misleading there!)
>
> So an overarching comment on all of these patches is that they're not
> intended to be ready to merge, nor are they necessarily what I think is
> the best solution. I'm just swinging a big hammer to see how far towards
> LLVM_IAS=1 I can get on powerpc, and I accept I'm going to have to come
> back and clean things up.
>
> Anyway, noted, I'll push harder on trying to get llvm to accept %rN:
> there was a patch that went in after llvm-11 that should help.
If you put it under ifdef CONFIG_CC_IS_CLANG in the meantime I think
that would be okay. Then we get error checking with gcc compiles and
llvm at least builds with its assembler which would be nice.
Thanks,
Nick
^ permalink raw reply [flat|nested] 30+ messages in thread
* [RFC PATCH 5/8] poweprc/lib/quad: Provide macros for lq/stq
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
` (3 preceding siblings ...)
2021-02-25 3:10 ` [RFC PATCH 4/8] powerpc/ppc_asm: use plain numbers for registers Daniel Axtens
@ 2021-02-25 3:10 ` Daniel Axtens
2021-02-25 15:44 ` Segher Boessenkool
2021-02-25 3:10 ` [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang Daniel Axtens
` (3 subsequent siblings)
8 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:10 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Daniel Axtens
For some reason the integrated assembler in clang-11 doesn't recognise
them. Eventually we should fix it there too.
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/include/asm/ppc-opcode.h | 4 ++++
arch/powerpc/lib/quad.S | 4 ++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h
index ed161ef2b3ca..a5249631cb83 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -339,11 +339,13 @@
#define PPC_RAW_DARN(t, l) (0x7c0005e6 | ___PPC_RT(t) | (((l) & 0x3) << 16))
#define PPC_RAW_DCBAL(a, b) (0x7c2005ec | __PPC_RA(a) | __PPC_RB(b))
#define PPC_RAW_DCBZL(a, b) (0x7c2007ec | __PPC_RA(a) | __PPC_RB(b))
+#define PPC_RAW_LQ(t, a, dq) (0xe0000000 | ___PPC_RT(t) | ___PPC_RA(a) | (((dq) & 0xfff) << 3))
#define PPC_RAW_LQARX(t, a, b, eh) (0x7c000228 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | __PPC_EH(eh))
#define PPC_RAW_LDARX(t, a, b, eh) (0x7c0000a8 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | __PPC_EH(eh))
#define PPC_RAW_LWARX(t, a, b, eh) (0x7c000028 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | __PPC_EH(eh))
#define PPC_RAW_PHWSYNC (0x7c8004ac)
#define PPC_RAW_PLWSYNC (0x7ca004ac)
+#define PPC_RAW_STQ(t, a, ds) (0xf8000002 | ___PPC_RT(t) | ___PPC_RA(a) | (((ds) & 0xfff) << 3))
#define PPC_RAW_STQCX(t, a, b) (0x7c00016d | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b))
#define PPC_RAW_MADDHD(t, a, b, c) (0x10000030 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | ___PPC_RC(c))
#define PPC_RAW_MADDHDU(t, a, b, c) (0x10000031 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | ___PPC_RC(c))
@@ -530,9 +532,11 @@
#define PPC_DCBZL(a, b) stringify_in_c(.long PPC_RAW_DCBZL(a, b))
#define PPC_DIVDE(t, a, b) stringify_in_c(.long PPC_RAW_DIVDE(t, a, b))
#define PPC_DIVDEU(t, a, b) stringify_in_c(.long PPC_RAW_DIVDEU(t, a, b))
+#define PPC_LQ(t, a, dq) stringify_in_c(.long PPC_RAW_LQ(t, a, dq))
#define PPC_LQARX(t, a, b, eh) stringify_in_c(.long PPC_RAW_LQARX(t, a, b, eh))
#define PPC_LDARX(t, a, b, eh) stringify_in_c(.long PPC_RAW_LDARX(t, a, b, eh))
#define PPC_LWARX(t, a, b, eh) stringify_in_c(.long PPC_RAW_LWARX(t, a, b, eh))
+#define PPC_STQ(t, a, ds) stringify_in_c(.long PPC_RAW_STQ(t, a, ds))
#define PPC_STQCX(t, a, b) stringify_in_c(.long PPC_RAW_STQCX(t, a, b))
#define PPC_MADDHD(t, a, b, c) stringify_in_c(.long PPC_RAW_MADDHD(t, a, b, c))
#define PPC_MADDHDU(t, a, b, c) stringify_in_c(.long PPC_RAW_MADDHDU(t, a, b, c))
diff --git a/arch/powerpc/lib/quad.S b/arch/powerpc/lib/quad.S
index da71760e50b5..de802a817992 100644
--- a/arch/powerpc/lib/quad.S
+++ b/arch/powerpc/lib/quad.S
@@ -15,7 +15,7 @@
/* do_lq(unsigned long ea, unsigned long *regs) */
_GLOBAL(do_lq)
-1: lq r6, 0(r3)
+1: PPC_LQ(6, 3, 0)
std r6, 0(r4)
std r7, 8(r4)
li r3, 0
@@ -26,7 +26,7 @@ _GLOBAL(do_lq)
/* do_stq(unsigned long ea, unsigned long val0, unsigned long val1) */
_GLOBAL(do_stq)
-1: stq r4, 0(r3)
+1: PPC_STQ(4, 3, 0)
li r3, 0
blr
2: li r3, -EFAULT
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 5/8] poweprc/lib/quad: Provide macros for lq/stq
2021-02-25 3:10 ` [RFC PATCH 5/8] poweprc/lib/quad: Provide macros for lq/stq Daniel Axtens
@ 2021-02-25 15:44 ` Segher Boessenkool
2021-02-26 0:13 ` Daniel Axtens
0 siblings, 1 reply; 30+ messages in thread
From: Segher Boessenkool @ 2021-02-25 15:44 UTC (permalink / raw)
To: Daniel Axtens; +Cc: linuxppc-dev, llvmlinux
On Thu, Feb 25, 2021 at 02:10:03PM +1100, Daniel Axtens wrote:
> +#define PPC_RAW_LQ(t, a, dq) (0xe0000000 | ___PPC_RT(t) | ___PPC_RA(a) | (((dq) & 0xfff) << 3))
Please keep the operand order the same as for the assembler insns? So
t,dq,a here.
It should be ((dq) & 0x0fff) << 4) .
> +#define PPC_RAW_STQ(t, a, ds) (0xf8000002 | ___PPC_RT(t) | ___PPC_RA(a) | (((ds) & 0xfff) << 3))
And t,ds,a here. (But it should use "s" instead of "t" preferably, and
use ___PPC_RS, because it is a source field, not a target).
It should be ((ds) & 0x3fff) << 2) as well.
Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 5/8] poweprc/lib/quad: Provide macros for lq/stq
2021-02-25 15:44 ` Segher Boessenkool
@ 2021-02-26 0:13 ` Daniel Axtens
0 siblings, 0 replies; 30+ messages in thread
From: Daniel Axtens @ 2021-02-26 0:13 UTC (permalink / raw)
To: Segher Boessenkool; +Cc: linuxppc-dev, llvmlinux
Segher Boessenkool <segher@kernel.crashing.org> writes:
> On Thu, Feb 25, 2021 at 02:10:03PM +1100, Daniel Axtens wrote:
>> +#define PPC_RAW_LQ(t, a, dq) (0xe0000000 | ___PPC_RT(t) | ___PPC_RA(a) | (((dq) & 0xfff) << 3))
>
> Please keep the operand order the same as for the assembler insns? So
> t,dq,a here.
>
> It should be ((dq) & 0x0fff) << 4) .
>
>> +#define PPC_RAW_STQ(t, a, ds) (0xf8000002 | ___PPC_RT(t) | ___PPC_RA(a) | (((ds) & 0xfff) << 3))
>
> And t,ds,a here. (But it should use "s" instead of "t" preferably, and
> use ___PPC_RS, because it is a source field, not a target).
>
> It should be ((ds) & 0x3fff) << 2) as well.
>
Ah, thank you. I'll fix this up.
Kind regards,
Daniel
>
> Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
` (4 preceding siblings ...)
2021-02-25 3:10 ` [RFC PATCH 5/8] poweprc/lib/quad: Provide macros for lq/stq Daniel Axtens
@ 2021-02-25 3:10 ` Daniel Axtens
2021-03-19 2:01 ` Nicholas Piggin
2021-02-25 3:10 ` [RFC PATCH 7/8] powerpc/purgatory: drop .machine specifier Daniel Axtens
` (2 subsequent siblings)
8 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:10 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Daniel Axtens
The llvm integrated assembler does not recognise the ISA 2.05 tlbiel
version. Eventually do this more smartly.
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/mm/book3s64/hash_native.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/arch/powerpc/mm/book3s64/hash_native.c b/arch/powerpc/mm/book3s64/hash_native.c
index 52e170bd95ae..c5937f69a452 100644
--- a/arch/powerpc/mm/book3s64/hash_native.c
+++ b/arch/powerpc/mm/book3s64/hash_native.c
@@ -267,9 +267,14 @@ static inline void __tlbiel(unsigned long vpn, int psize, int apsize, int ssize)
va |= ssize << 8;
sllp = get_sllp_encoding(apsize);
va |= sllp << 5;
+#if 0
asm volatile(ASM_FTR_IFSET("tlbiel %0", "tlbiel %0,0", %1)
: : "r" (va), "i" (CPU_FTR_ARCH_206)
: "memory");
+#endif
+ asm volatile("tlbiel %0"
+ : : "r" (va)
+ : "memory");
break;
default:
/* We need 14 to 14 + i bits of va */
@@ -286,9 +291,14 @@ static inline void __tlbiel(unsigned long vpn, int psize, int apsize, int ssize)
*/
va |= (vpn & 0xfe);
va |= 1; /* L */
+#if 0
asm volatile(ASM_FTR_IFSET("tlbiel %0", "tlbiel %0,1", %1)
: : "r" (va), "i" (CPU_FTR_ARCH_206)
: "memory");
+#endif
+ asm volatile("tlbiel %0"
+ : : "r" (va)
+ : "memory");
break;
}
trace_tlbie(0, 1, va, 0, 0, 0, 0);
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang
2021-02-25 3:10 ` [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang Daniel Axtens
@ 2021-03-19 2:01 ` Nicholas Piggin
2021-03-22 16:49 ` Christophe Leroy
0 siblings, 1 reply; 30+ messages in thread
From: Nicholas Piggin @ 2021-03-19 2:01 UTC (permalink / raw)
To: Daniel Axtens, linuxppc-dev, llvmlinux
Excerpts from Daniel Axtens's message of February 25, 2021 1:10 pm:
> The llvm integrated assembler does not recognise the ISA 2.05 tlbiel
> version. Eventually do this more smartly.
The whole thing with TLBIE and TLBIEL in this file seems a bit too
clever. We should have PPC_TLBIE* macros for all of them.
Thanks,
Nick
>
> Signed-off-by: Daniel Axtens <dja@axtens.net>
> ---
> arch/powerpc/mm/book3s64/hash_native.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/arch/powerpc/mm/book3s64/hash_native.c b/arch/powerpc/mm/book3s64/hash_native.c
> index 52e170bd95ae..c5937f69a452 100644
> --- a/arch/powerpc/mm/book3s64/hash_native.c
> +++ b/arch/powerpc/mm/book3s64/hash_native.c
> @@ -267,9 +267,14 @@ static inline void __tlbiel(unsigned long vpn, int psize, int apsize, int ssize)
> va |= ssize << 8;
> sllp = get_sllp_encoding(apsize);
> va |= sllp << 5;
> +#if 0
> asm volatile(ASM_FTR_IFSET("tlbiel %0", "tlbiel %0,0", %1)
> : : "r" (va), "i" (CPU_FTR_ARCH_206)
> : "memory");
> +#endif
> + asm volatile("tlbiel %0"
> + : : "r" (va)
> + : "memory");
> break;
> default:
> /* We need 14 to 14 + i bits of va */
> @@ -286,9 +291,14 @@ static inline void __tlbiel(unsigned long vpn, int psize, int apsize, int ssize)
> */
> va |= (vpn & 0xfe);
> va |= 1; /* L */
> +#if 0
> asm volatile(ASM_FTR_IFSET("tlbiel %0", "tlbiel %0,1", %1)
> : : "r" (va), "i" (CPU_FTR_ARCH_206)
> : "memory");
> +#endif
> + asm volatile("tlbiel %0"
> + : : "r" (va)
> + : "memory");
> break;
> }
> trace_tlbie(0, 1, va, 0, 0, 0, 0);
> --
> 2.27.0
>
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang
2021-03-19 2:01 ` Nicholas Piggin
@ 2021-03-22 16:49 ` Christophe Leroy
2021-03-22 18:11 ` Nicholas Piggin
0 siblings, 1 reply; 30+ messages in thread
From: Christophe Leroy @ 2021-03-22 16:49 UTC (permalink / raw)
To: Nicholas Piggin, Daniel Axtens, linuxppc-dev, llvmlinux
Le 19/03/2021 à 03:01, Nicholas Piggin a écrit :
> Excerpts from Daniel Axtens's message of February 25, 2021 1:10 pm:
>> The llvm integrated assembler does not recognise the ISA 2.05 tlbiel
>> version. Eventually do this more smartly.
>
> The whole thing with TLBIE and TLBIEL in this file seems a bit too
> clever. We should have PPC_TLBIE* macros for all of them.
I was expecting to drop PPC_* macros as much as possible taking into account the later binutils
support most of them (https://github.com/linuxppc/issues/issues/350). Was not expecting to go the
other direction.
See following series for an exemple of why we would want to get rid of PPC_* macros.
https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=231583&state=*
Christophe
>
> Thanks,
> Nick
>
>>
>> Signed-off-by: Daniel Axtens <dja@axtens.net>
>> ---
>> arch/powerpc/mm/book3s64/hash_native.c | 10 ++++++++++
>> 1 file changed, 10 insertions(+)
>>
>> diff --git a/arch/powerpc/mm/book3s64/hash_native.c b/arch/powerpc/mm/book3s64/hash_native.c
>> index 52e170bd95ae..c5937f69a452 100644
>> --- a/arch/powerpc/mm/book3s64/hash_native.c
>> +++ b/arch/powerpc/mm/book3s64/hash_native.c
>> @@ -267,9 +267,14 @@ static inline void __tlbiel(unsigned long vpn, int psize, int apsize, int ssize)
>> va |= ssize << 8;
>> sllp = get_sllp_encoding(apsize);
>> va |= sllp << 5;
>> +#if 0
>> asm volatile(ASM_FTR_IFSET("tlbiel %0", "tlbiel %0,0", %1)
>> : : "r" (va), "i" (CPU_FTR_ARCH_206)
>> : "memory");
>> +#endif
>> + asm volatile("tlbiel %0"
>> + : : "r" (va)
>> + : "memory");
>> break;
>> default:
>> /* We need 14 to 14 + i bits of va */
>> @@ -286,9 +291,14 @@ static inline void __tlbiel(unsigned long vpn, int psize, int apsize, int ssize)
>> */
>> va |= (vpn & 0xfe);
>> va |= 1; /* L */
>> +#if 0
>> asm volatile(ASM_FTR_IFSET("tlbiel %0", "tlbiel %0,1", %1)
>> : : "r" (va), "i" (CPU_FTR_ARCH_206)
>> : "memory");
>> +#endif
>> + asm volatile("tlbiel %0"
>> + : : "r" (va)
>> + : "memory");
>> break;
>> }
>> trace_tlbie(0, 1, va, 0, 0, 0, 0);
>> --
>> 2.27.0
>>
>>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang
2021-03-22 16:49 ` Christophe Leroy
@ 2021-03-22 18:11 ` Nicholas Piggin
2021-03-24 15:51 ` Segher Boessenkool
0 siblings, 1 reply; 30+ messages in thread
From: Nicholas Piggin @ 2021-03-22 18:11 UTC (permalink / raw)
To: Christophe Leroy, Daniel Axtens, linuxppc-dev, llvmlinux
Excerpts from Christophe Leroy's message of March 23, 2021 2:49 am:
>
>
> Le 19/03/2021 à 03:01, Nicholas Piggin a écrit :
>> Excerpts from Daniel Axtens's message of February 25, 2021 1:10 pm:
>>> The llvm integrated assembler does not recognise the ISA 2.05 tlbiel
>>> version. Eventually do this more smartly.
>>
>> The whole thing with TLBIE and TLBIEL in this file seems a bit too
>> clever. We should have PPC_TLBIE* macros for all of them.
>
> I was expecting to drop PPC_* macros as much as possible taking into account the later binutils
> support most of them (https://github.com/linuxppc/issues/issues/350). Was not expecting to go the
> other direction.
The problem in this file is we generate 3 different tlbie and tlbiel
instructions with the same mnemonic corresponding to different ISA
versions.
This might actually be the one good place to use .machine to make sure
the assembler generates the right thing. I'm not entirely sure it is
foolproof because some of the times the instruction variant is inferred
by the number of arguments it has yet arguments can be implicit. PPC_
define would be exactly explicit.
But if it can be made reasonably robust with .machine then I'd be okay
with that too.
Thanks,
Nick
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang
2021-03-22 18:11 ` Nicholas Piggin
@ 2021-03-24 15:51 ` Segher Boessenkool
2021-03-24 21:07 ` Segher Boessenkool
0 siblings, 1 reply; 30+ messages in thread
From: Segher Boessenkool @ 2021-03-24 15:51 UTC (permalink / raw)
To: Nicholas Piggin; +Cc: llvmlinux, linuxppc-dev, Daniel Axtens
Hi!
On Tue, Mar 23, 2021 at 04:11:10AM +1000, Nicholas Piggin wrote:
> The problem in this file is we generate 3 different tlbie and tlbiel
> instructions with the same mnemonic corresponding to different ISA
> versions.
>
> This might actually be the one good place to use .machine to make sure
> the assembler generates the right thing.
Yes, but then hide that in some macro.
(And "the one good place"? I protest!)
> I'm not entirely sure it is
> foolproof because some of the times the instruction variant is inferred
> by the number of arguments it has yet arguments can be implicit. PPC_
> define would be exactly explicit.
The variants with fewer operands have those coded as 0 in the
instruction. All of this is backwards compatible.
> But if it can be made reasonably robust with .machine then I'd be okay
> with that too.
Since you should do a macro (or inline) for it anyway, you could just
do .long, all the nastiness is in one place anyway then, it won't make
much difference what you do. It should be documented there as well :-)
Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang
2021-03-24 15:51 ` Segher Boessenkool
@ 2021-03-24 21:07 ` Segher Boessenkool
0 siblings, 0 replies; 30+ messages in thread
From: Segher Boessenkool @ 2021-03-24 21:07 UTC (permalink / raw)
To: Nicholas Piggin; +Cc: linuxppc-dev, llvmlinux, Daniel Axtens
On Wed, Mar 24, 2021 at 10:51:05AM -0500, Segher Boessenkool wrote:
> The variants with fewer operands have those coded as 0 in the
> instruction. All of this is backwards compatible.
I was reminded that this is broken in ISA 2.03. Ouch.
Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* [RFC PATCH 7/8] powerpc/purgatory: drop .machine specifier
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
` (5 preceding siblings ...)
2021-02-25 3:10 ` [RFC PATCH 6/8] powerpc/mm/book3s64/hash: drop pre 2.06 tlbiel for clang Daniel Axtens
@ 2021-02-25 3:10 ` Daniel Axtens
2021-02-25 15:58 ` Segher Boessenkool
2021-02-25 3:10 ` [RFC PATCH 8/8] powerpc/64/asm: don't reassign labels Daniel Axtens
2021-03-14 10:01 ` [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Michael Ellerman
8 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:10 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Daniel Axtens
It's ignored by future versions of llvm's integrated assembler (by not -11).
I'm not sure what it does for us in gas.
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/purgatory/trampoline_64.S | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/purgatory/trampoline_64.S b/arch/powerpc/purgatory/trampoline_64.S
index d956b8a35fd1..e6a2740a5da0 100644
--- a/arch/powerpc/purgatory/trampoline_64.S
+++ b/arch/powerpc/purgatory/trampoline_64.S
@@ -12,7 +12,7 @@
#include <asm/asm-compat.h>
#include <asm/crashdump-ppc64.h>
- .machine ppc64
+//upgrade clang, gets ignored .machine ppc64
.balign 256
.globl purgatory_start
purgatory_start:
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 7/8] powerpc/purgatory: drop .machine specifier
2021-02-25 3:10 ` [RFC PATCH 7/8] powerpc/purgatory: drop .machine specifier Daniel Axtens
@ 2021-02-25 15:58 ` Segher Boessenkool
2021-02-26 0:17 ` Daniel Axtens
2021-03-19 2:05 ` Nicholas Piggin
0 siblings, 2 replies; 30+ messages in thread
From: Segher Boessenkool @ 2021-02-25 15:58 UTC (permalink / raw)
To: Daniel Axtens; +Cc: linuxppc-dev, llvmlinux
On Thu, Feb 25, 2021 at 02:10:05PM +1100, Daniel Axtens wrote:
> It's ignored by future versions of llvm's integrated assembler (by not -11).
> I'm not sure what it does for us in gas.
It enables all insns that exist on 620 (the first 64-bit PowerPC CPU).
> --- a/arch/powerpc/purgatory/trampoline_64.S
> +++ b/arch/powerpc/purgatory/trampoline_64.S
> @@ -12,7 +12,7 @@
> #include <asm/asm-compat.h>
> #include <asm/crashdump-ppc64.h>
>
> - .machine ppc64
> +//upgrade clang, gets ignored .machine ppc64
Why delete it if it is ignored? Why add a cryptic comment?
Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 7/8] powerpc/purgatory: drop .machine specifier
2021-02-25 15:58 ` Segher Boessenkool
@ 2021-02-26 0:17 ` Daniel Axtens
2021-03-19 2:05 ` Nicholas Piggin
1 sibling, 0 replies; 30+ messages in thread
From: Daniel Axtens @ 2021-02-26 0:17 UTC (permalink / raw)
To: Segher Boessenkool; +Cc: linuxppc-dev, llvmlinux
Segher Boessenkool <segher@kernel.crashing.org> writes:
> On Thu, Feb 25, 2021 at 02:10:05PM +1100, Daniel Axtens wrote:
>> It's ignored by future versions of llvm's integrated assembler (by not -11).
>> I'm not sure what it does for us in gas.
>
> It enables all insns that exist on 620 (the first 64-bit PowerPC CPU).
>
>> --- a/arch/powerpc/purgatory/trampoline_64.S
>> +++ b/arch/powerpc/purgatory/trampoline_64.S
>> @@ -12,7 +12,7 @@
>> #include <asm/asm-compat.h>
>> #include <asm/crashdump-ppc64.h>
>>
>> - .machine ppc64
>> +//upgrade clang, gets ignored .machine ppc64
>
> Why delete it if it is ignored? Why add a cryptic comment?
Sorry, poor form on my part. I think I will give up on having llvm-11
work and target llvm HEAD, which means I can drop this.
>
>
> Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 7/8] powerpc/purgatory: drop .machine specifier
2021-02-25 15:58 ` Segher Boessenkool
2021-02-26 0:17 ` Daniel Axtens
@ 2021-03-19 2:05 ` Nicholas Piggin
2021-03-19 10:59 ` Michael Ellerman
1 sibling, 1 reply; 30+ messages in thread
From: Nicholas Piggin @ 2021-03-19 2:05 UTC (permalink / raw)
To: Daniel Axtens, Segher Boessenkool; +Cc: llvmlinux, linuxppc-dev
Excerpts from Segher Boessenkool's message of February 26, 2021 1:58 am:
> On Thu, Feb 25, 2021 at 02:10:05PM +1100, Daniel Axtens wrote:
>> It's ignored by future versions of llvm's integrated assembler (by not -11).
>> I'm not sure what it does for us in gas.
>
> It enables all insns that exist on 620 (the first 64-bit PowerPC CPU).
Same question for this, why do we have it at all?
Thanks,
Nick
^ permalink raw reply [flat|nested] 30+ messages in thread
* [RFC PATCH 8/8] powerpc/64/asm: don't reassign labels
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
` (6 preceding siblings ...)
2021-02-25 3:10 ` [RFC PATCH 7/8] powerpc/purgatory: drop .machine specifier Daniel Axtens
@ 2021-02-25 3:10 ` Daniel Axtens
2021-02-25 16:08 ` Segher Boessenkool
2021-03-14 10:01 ` [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Michael Ellerman
8 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-25 3:10 UTC (permalink / raw)
To: llvmlinux, linuxppc-dev; +Cc: Daniel Axtens
The assembler really does not like us reassigning things to the same
label:
<instantiation>:7:9: error: invalid reassignment of non-absolute variable 'fs_label'
This happens across a bunch of platforms:
https://github.com/ClangBuiltLinux/linux/issues/1043
https://github.com/ClangBuiltLinux/linux/issues/1008
https://github.com/ClangBuiltLinux/linux/issues/920
https://github.com/ClangBuiltLinux/linux/issues/1050
There is no hope of getting this fixed in LLVM, so if we want to build
with LLVM_IAS, we need to hack around it ourselves.
For us the big problem comes from this:
\#define USE_FIXED_SECTION(sname) \
fs_label = start_##sname; \
fs_start = sname##_start; \
use_ftsec sname;
\#define USE_TEXT_SECTION()
fs_label = start_text; \
fs_start = text_start; \
.text
and in particular fs_label.
I have tried to work around it by not setting those 'variables', and
requiring that users of the variables instead track for themselves
what section they are in. This isn't amazing, by any stretch, but it
gets us further in the compilation.
I'm still stuck with the following from head_64.S:
.balign 8
p_end: .8byte _end - copy_to_here
4:
/*
* Now copy the rest of the kernel up to _end, add
* _end - copy_to_here to the copy limit and run again.
*/
addis r8,r26,(ABS_ADDR(p_end, text))@ha
ld r8,(ABS_ADDR(p_end, text))@l(r8)
add r5,r5,r8
5: bl copy_and_flush /* copy the rest */
9: b start_here_multiplatform
Clang does not like this code - in particular it complains about the addis, saying
<unknown>:0: error: expected relocatable expression
I don't know what's special about p_end, because just above we do an
ABS_ADDR(4f, text) and that seems to work just fine.
Signed-off-by: Daniel Axtens <dja@axtens.net>
---
arch/powerpc/include/asm/head-64.h | 12 +++++------
arch/powerpc/kernel/exceptions-64s.S | 31 ++++++++++++++--------------
arch/powerpc/kernel/head_64.S | 16 +++++++-------
3 files changed, 29 insertions(+), 30 deletions(-)
diff --git a/arch/powerpc/include/asm/head-64.h b/arch/powerpc/include/asm/head-64.h
index 7d8ccab47e86..43849a777f91 100644
--- a/arch/powerpc/include/asm/head-64.h
+++ b/arch/powerpc/include/asm/head-64.h
@@ -98,13 +98,9 @@ linker_stub_catch: \
. = sname##_len;
#define USE_FIXED_SECTION(sname) \
- fs_label = start_##sname; \
- fs_start = sname##_start; \
use_ftsec sname;
#define USE_TEXT_SECTION() \
- fs_label = start_text; \
- fs_start = text_start; \
.text
#define CLOSE_FIXED_SECTION(sname) \
@@ -161,13 +157,15 @@ end_##sname:
* - ABS_ADDR is used to find the absolute address of any symbol, from within
* a fixed section.
*/
-#define DEFINE_FIXED_SYMBOL(label) \
- label##_absolute = (label - fs_label + fs_start)
+// define label as being _in_ sname
+#define DEFINE_FIXED_SYMBOL(label, sname) \
+ label##_absolute = (label - start_ ## sname + sname ## _start)
#define FIXED_SYMBOL_ABS_ADDR(label) \
(label##_absolute)
-#define ABS_ADDR(label) (label - fs_label + fs_start)
+// find label from _within_ sname
+#define ABS_ADDR(label, sname) (label - start_ ## sname + sname ## _start)
#endif /* __ASSEMBLY__ */
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 720fb9892745..295d90202665 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -64,7 +64,7 @@
.balign IFETCH_ALIGN_BYTES; \
.global name; \
_ASM_NOKPROBE_SYMBOL(name); \
- DEFINE_FIXED_SYMBOL(name); \
+ DEFINE_FIXED_SYMBOL(name, text); \
name:
#define TRAMP_REAL_BEGIN(name) \
@@ -92,18 +92,18 @@ name:
ld reg,PACAKBASE(r13); /* get high part of &label */ \
ori reg,reg,FIXED_SYMBOL_ABS_ADDR(label)
-#define __LOAD_HANDLER(reg, label) \
+#define __LOAD_HANDLER(reg, label, section) \
ld reg,PACAKBASE(r13); \
- ori reg,reg,(ABS_ADDR(label))@l
+ ori reg,reg,(ABS_ADDR(label, section))@l
/*
* Branches from unrelocated code (e.g., interrupts) to labels outside
* head-y require >64K offsets.
*/
-#define __LOAD_FAR_HANDLER(reg, label) \
+#define __LOAD_FAR_HANDLER(reg, label, section) \
ld reg,PACAKBASE(r13); \
- ori reg,reg,(ABS_ADDR(label))@l; \
- addis reg,reg,(ABS_ADDR(label))@h
+ ori reg,reg,(ABS_ADDR(label, section))@l; \
+ addis reg,reg,(ABS_ADDR(label, section))@h
/*
* Branch to label using its 0xC000 address. This results in instruction
@@ -113,8 +113,9 @@ name:
* This could set the 0xc bits for !RELOCATABLE as an immediate, rather than
* load KBASE for a slight optimisation.
*/
+// only called once
#define BRANCH_TO_C000(reg, label) \
- __LOAD_FAR_HANDLER(reg, label); \
+ __LOAD_FAR_HANDLER(reg, label, real_trampolines); \
mtctr reg; \
bctr
@@ -458,7 +459,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
* This switches to virtual mode and sets MSR[RI].
*/
.macro __GEN_COMMON_ENTRY name
-DEFINE_FIXED_SYMBOL(\name\()_common_real)
+DEFINE_FIXED_SYMBOL(\name\()_common_real, text)
\name\()_common_real:
.if IKVM_REAL
KVMTEST \name
@@ -481,7 +482,7 @@ DEFINE_FIXED_SYMBOL(\name\()_common_real)
.endif
.balign IFETCH_ALIGN_BYTES
-DEFINE_FIXED_SYMBOL(\name\()_common_virt)
+DEFINE_FIXED_SYMBOL(\name\()_common_virt, text)
\name\()_common_virt:
.if IKVM_VIRT
KVMTEST \name
@@ -495,7 +496,7 @@ DEFINE_FIXED_SYMBOL(\name\()_common_virt)
* want to run in real mode.
*/
.macro __GEN_REALMODE_COMMON_ENTRY name
-DEFINE_FIXED_SYMBOL(\name\()_common_real)
+DEFINE_FIXED_SYMBOL(\name\()_common_real, text)
\name\()_common_real:
.if IKVM_REAL
KVMTEST \name
@@ -856,12 +857,12 @@ EXC_VIRT_END(system_call_vectored, 0x3000, 0x1000)
#ifdef CONFIG_RELOCATABLE
TRAMP_VIRT_BEGIN(system_call_vectored_tramp)
- __LOAD_HANDLER(r10, system_call_vectored_common)
+ __LOAD_HANDLER(r10, system_call_vectored_common, virt_trampolines)
mtctr r10
bctr
TRAMP_VIRT_BEGIN(system_call_vectored_sigill_tramp)
- __LOAD_HANDLER(r10, system_call_vectored_sigill)
+ __LOAD_HANDLER(r10, system_call_vectored_sigill, virt_trampolines)
mtctr r10
bctr
#endif
@@ -1968,14 +1969,14 @@ END_FTR_SECTION_IFSET(CPU_FTR_REAL_LE)
HMT_MEDIUM
.if ! \virt
- __LOAD_HANDLER(r10, system_call_common_real)
+ __LOAD_HANDLER(r10, system_call_common_real, real_vectors)
mtctr r10
bctr
.else
li r10,MSR_RI
mtmsrd r10,1 /* Set RI (EE=0) */
#ifdef CONFIG_RELOCATABLE
- __LOAD_HANDLER(r10, system_call_common)
+ __LOAD_HANDLER(r10, system_call_common, virt_vectors)
mtctr r10
bctr
#else
@@ -3075,7 +3076,7 @@ USE_FIXED_SECTION(virt_trampolines)
.align 7
.globl __end_interrupts
__end_interrupts:
-DEFINE_FIXED_SYMBOL(__end_interrupts)
+DEFINE_FIXED_SYMBOL(__end_interrupts, virt_trampolines)
#ifdef CONFIG_PPC_970_NAP
/*
diff --git a/arch/powerpc/kernel/head_64.S b/arch/powerpc/kernel/head_64.S
index ece7f97bafff..3a10e9f55baa 100644
--- a/arch/powerpc/kernel/head_64.S
+++ b/arch/powerpc/kernel/head_64.S
@@ -156,7 +156,7 @@ __secondary_hold:
/* Tell the master cpu we're here */
/* Relocation is off & we are located at an address less */
/* than 0x100, so only need to grab low order offset. */
- std r24,(ABS_ADDR(__secondary_hold_acknowledge))(0)
+ std r24,(ABS_ADDR(__secondary_hold_acknowledge, first_256B))(0)
sync
li r26,0
@@ -164,7 +164,7 @@ __secondary_hold:
tovirt(r26,r26)
#endif
/* All secondary cpus wait here until told to start. */
-100: ld r12,(ABS_ADDR(__secondary_hold_spinloop))(r26)
+100: ld r12,(ABS_ADDR(__secondary_hold_spinloop, first_256B))(r26)
cmpdi 0,r12,0
beq 100b
@@ -646,15 +646,15 @@ __after_prom_start:
3:
#endif
/* # bytes of memory to copy */
- lis r5,(ABS_ADDR(copy_to_here))@ha
- addi r5,r5,(ABS_ADDR(copy_to_here))@l
+ lis r5,(ABS_ADDR(copy_to_here, text))@ha
+ addi r5,r5,(ABS_ADDR(copy_to_here, text))@l
bl copy_and_flush /* copy the first n bytes */
/* this includes the code being */
/* executed here. */
/* Jump to the copy of this code that we just made */
- addis r8,r3,(ABS_ADDR(4f))@ha
- addi r12,r8,(ABS_ADDR(4f))@l
+ addis r8,r3,(ABS_ADDR(4f, text))@ha
+ addi r12,r8,(ABS_ADDR(4f, text))@l
mtctr r12
bctr
@@ -666,8 +666,8 @@ p_end: .8byte _end - copy_to_here
* Now copy the rest of the kernel up to _end, add
* _end - copy_to_here to the copy limit and run again.
*/
- addis r8,r26,(ABS_ADDR(p_end))@ha
- ld r8,(ABS_ADDR(p_end))@l(r8)
+ addis r8,r26,(ABS_ADDR(p_end, text))@ha
+ ld r8,(ABS_ADDR(p_end, text))@l(r8)
add r5,r5,r8
5: bl copy_and_flush /* copy the rest */
--
2.27.0
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 8/8] powerpc/64/asm: don't reassign labels
2021-02-25 3:10 ` [RFC PATCH 8/8] powerpc/64/asm: don't reassign labels Daniel Axtens
@ 2021-02-25 16:08 ` Segher Boessenkool
2021-02-26 0:28 ` Daniel Axtens
0 siblings, 1 reply; 30+ messages in thread
From: Segher Boessenkool @ 2021-02-25 16:08 UTC (permalink / raw)
To: Daniel Axtens; +Cc: linuxppc-dev, llvmlinux
On Thu, Feb 25, 2021 at 02:10:06PM +1100, Daniel Axtens wrote:
> The assembler really does not like us reassigning things to the same
> label:
>
> <instantiation>:7:9: error: invalid reassignment of non-absolute variable 'fs_label'
>
> This happens across a bunch of platforms:
> https://github.com/ClangBuiltLinux/linux/issues/1043
> https://github.com/ClangBuiltLinux/linux/issues/1008
> https://github.com/ClangBuiltLinux/linux/issues/920
> https://github.com/ClangBuiltLinux/linux/issues/1050
>
> There is no hope of getting this fixed in LLVM, so if we want to build
> with LLVM_IAS, we need to hack around it ourselves.
>
> For us the big problem comes from this:
>
> \#define USE_FIXED_SECTION(sname) \
> fs_label = start_##sname; \
> fs_start = sname##_start; \
> use_ftsec sname;
>
> \#define USE_TEXT_SECTION()
> fs_label = start_text; \
> fs_start = text_start; \
> .text
>
> and in particular fs_label.
The "Setting Symbols" super short chapter reads:
"A symbol can be given an arbitrary value by writing a symbol, followed
by an equals sign '=', followed by an expression. This is equivalent
to using the '.set' directive."
And ".set" has
"Set the value of SYMBOL to EXPRESSION. This changes SYMBOL's value and
type to conform to EXPRESSION. If SYMBOL was flagged as external, it
remains flagged.
You may '.set' a symbol many times in the same assembly provided that
the values given to the symbol are constants. Values that are based on
expressions involving other symbols are allowed, but some targets may
restrict this to only being done once per assembly. This is because
those targets do not set the addresses of symbols at assembly time, but
rather delay the assignment until a final link is performed. This
allows the linker a chance to change the code in the files, changing the
location of, and the relative distance between, various different
symbols.
If you '.set' a global symbol, the value stored in the object file is
the last value stored into it."
So this really should be fixed in clang: it is basic assembler syntax.
Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 8/8] powerpc/64/asm: don't reassign labels
2021-02-25 16:08 ` Segher Boessenkool
@ 2021-02-26 0:28 ` Daniel Axtens
2021-03-19 2:15 ` Nicholas Piggin
0 siblings, 1 reply; 30+ messages in thread
From: Daniel Axtens @ 2021-02-26 0:28 UTC (permalink / raw)
To: Segher Boessenkool; +Cc: linuxppc-dev, llvmlinux
Segher Boessenkool <segher@kernel.crashing.org> writes:
> On Thu, Feb 25, 2021 at 02:10:06PM +1100, Daniel Axtens wrote:
>> The assembler really does not like us reassigning things to the same
>> label:
>>
>> <instantiation>:7:9: error: invalid reassignment of non-absolute variable 'fs_label'
>>
>> This happens across a bunch of platforms:
>> https://github.com/ClangBuiltLinux/linux/issues/1043
>> https://github.com/ClangBuiltLinux/linux/issues/1008
>> https://github.com/ClangBuiltLinux/linux/issues/920
>> https://github.com/ClangBuiltLinux/linux/issues/1050
>>
>> There is no hope of getting this fixed in LLVM, so if we want to build
>> with LLVM_IAS, we need to hack around it ourselves.
>>
>> For us the big problem comes from this:
>>
>> \#define USE_FIXED_SECTION(sname) \
>> fs_label = start_##sname; \
>> fs_start = sname##_start; \
>> use_ftsec sname;
>>
>> \#define USE_TEXT_SECTION()
>> fs_label = start_text; \
>> fs_start = text_start; \
>> .text
>>
>> and in particular fs_label.
>
> The "Setting Symbols" super short chapter reads:
>
> "A symbol can be given an arbitrary value by writing a symbol, followed
> by an equals sign '=', followed by an expression. This is equivalent
> to using the '.set' directive."
>
> And ".set" has
>
> "Set the value of SYMBOL to EXPRESSION. This changes SYMBOL's value and
> type to conform to EXPRESSION. If SYMBOL was flagged as external, it
> remains flagged.
>
> You may '.set' a symbol many times in the same assembly provided that
> the values given to the symbol are constants. Values that are based on
> expressions involving other symbols are allowed, but some targets may
> restrict this to only being done once per assembly. This is because
> those targets do not set the addresses of symbols at assembly time, but
> rather delay the assignment until a final link is performed. This
> allows the linker a chance to change the code in the files, changing the
> location of, and the relative distance between, various different
> symbols.
>
> If you '.set' a global symbol, the value stored in the object file is
> the last value stored into it."
>
> So this really should be fixed in clang: it is basic assembler syntax.
No doubt I have explained this poorly.
LLVM does allow some things, this builds fine for example:
.set foo, 8192
addi %r3, %r3, foo
.set foo, 1234
addi %r3, %r3, foo
However, this does not:
a:
.set foo, a
addi %r3, %r3, foo@l
b:
.set foo, b
addi %r3, %r3, foo-a
clang -target ppc64le -integrated-as foo.s -o foo.o -c
foo.s:5:11: error: invalid reassignment of non-absolute variable 'foo' in '.set' directive
.set foo, b
^
gas otoh, has no issues with reassignment:
$ powerpc64-linux-gnu-as foo.s -c -o foo.o
$ powerpc64-linux-gnu-objdump -dr foo.o
foo.o: file format elf64-powerpc
Disassembly of section .text:
0000000000000000 <a>:
0: 38 63 00 00 addi r3,r3,0
2: R_PPC64_ADDR16_LO .text
0000000000000004 <b>:
4: 38 63 00 04 addi r3,r3,4
It seems the llvm assembler only does a single pass, so they're not keen
on trying to support reassigning labels with non-absolute values.
Kind regards,
Daniel
>
> Segher
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 8/8] powerpc/64/asm: don't reassign labels
2021-02-26 0:28 ` Daniel Axtens
@ 2021-03-19 2:15 ` Nicholas Piggin
0 siblings, 0 replies; 30+ messages in thread
From: Nicholas Piggin @ 2021-03-19 2:15 UTC (permalink / raw)
To: Daniel Axtens, Segher Boessenkool; +Cc: llvmlinux, linuxppc-dev
Excerpts from Daniel Axtens's message of February 26, 2021 10:28 am:
> Segher Boessenkool <segher@kernel.crashing.org> writes:
>
>> On Thu, Feb 25, 2021 at 02:10:06PM +1100, Daniel Axtens wrote:
>>> The assembler really does not like us reassigning things to the same
>>> label:
>>>
>>> <instantiation>:7:9: error: invalid reassignment of non-absolute variable 'fs_label'
>>>
>>> This happens across a bunch of platforms:
>>> https://github.com/ClangBuiltLinux/linux/issues/1043
>>> https://github.com/ClangBuiltLinux/linux/issues/1008
>>> https://github.com/ClangBuiltLinux/linux/issues/920
>>> https://github.com/ClangBuiltLinux/linux/issues/1050
>>>
>>> There is no hope of getting this fixed in LLVM, so if we want to build
>>> with LLVM_IAS, we need to hack around it ourselves.
>>>
>>> For us the big problem comes from this:
>>>
>>> \#define USE_FIXED_SECTION(sname) \
>>> fs_label = start_##sname; \
>>> fs_start = sname##_start; \
>>> use_ftsec sname;
>>>
>>> \#define USE_TEXT_SECTION()
>>> fs_label = start_text; \
>>> fs_start = text_start; \
>>> .text
>>>
>>> and in particular fs_label.
>>
>> The "Setting Symbols" super short chapter reads:
>>
>> "A symbol can be given an arbitrary value by writing a symbol, followed
>> by an equals sign '=', followed by an expression. This is equivalent
>> to using the '.set' directive."
>>
>> And ".set" has
>>
>> "Set the value of SYMBOL to EXPRESSION. This changes SYMBOL's value and
>> type to conform to EXPRESSION. If SYMBOL was flagged as external, it
>> remains flagged.
>>
>> You may '.set' a symbol many times in the same assembly provided that
>> the values given to the symbol are constants. Values that are based on
>> expressions involving other symbols are allowed, but some targets may
>> restrict this to only being done once per assembly. This is because
>> those targets do not set the addresses of symbols at assembly time, but
>> rather delay the assignment until a final link is performed. This
>> allows the linker a chance to change the code in the files, changing the
>> location of, and the relative distance between, various different
>> symbols.
>>
>> If you '.set' a global symbol, the value stored in the object file is
>> the last value stored into it."
>>
>> So this really should be fixed in clang: it is basic assembler syntax.
>
> No doubt I have explained this poorly.
>
> LLVM does allow some things, this builds fine for example:
>
> .set foo, 8192
> addi %r3, %r3, foo
> .set foo, 1234
> addi %r3, %r3, foo
>
> However, this does not:
>
> a:
> .set foo, a
> addi %r3, %r3, foo@l
> b:
> .set foo, b
> addi %r3, %r3, foo-a
>
> clang -target ppc64le -integrated-as foo.s -o foo.o -c
> foo.s:5:11: error: invalid reassignment of non-absolute variable 'foo' in '.set' directive
> .set foo, b
> ^
So that does seem to be allowed by the specification.
I don't have a huge problem with the patch actually, doesn't seem too
bad.
Thanks,
Nick
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [RFC PATCH 0/8] WIP support for the LLVM integrated assembler
2021-02-25 3:09 [RFC PATCH 0/8] WIP support for the LLVM integrated assembler Daniel Axtens
` (7 preceding siblings ...)
2021-02-25 3:10 ` [RFC PATCH 8/8] powerpc/64/asm: don't reassign labels Daniel Axtens
@ 2021-03-14 10:01 ` Michael Ellerman
8 siblings, 0 replies; 30+ messages in thread
From: Michael Ellerman @ 2021-03-14 10:01 UTC (permalink / raw)
To: linuxppc-dev, llvmlinux, Daniel Axtens
On Thu, 25 Feb 2021 14:09:58 +1100, Daniel Axtens wrote:
> To support Clang's CFI we need LTO. For LTO, we need to be able to compile
> with the LLVM integrated assembler.
>
> Currently, we can't.
>
> This series gets us a bit closer, but I'm still stuck and I'm hoping
> someone can point me in the right direction.
>
> [...]
Patch 1 applied to powerpc/fixes.
[1/8] powerpc/64s/exception: Clean up a missed SRR specifier
https://git.kernel.org/powerpc/c/c080a173301ffc62cb6c76308c803c7fee05517a
cheers
^ permalink raw reply [flat|nested] 30+ messages in thread