Re: [PATCH 5/6] powerpc/mm/64s/hash: Add real-mode change_memory_range() for hash LPAR

[Michael Ellerman <mpe@ellerman.id.au>]

Nicholas Piggin <npiggin@gmail.com> writes:
> Excerpts from Michael Ellerman's message of March 16, 2021 4:40 pm:
>> Nicholas Piggin <npiggin@gmail.com> writes:
>>> Excerpts from Michael Ellerman's message of February 11, 2021 11:51 pm:
>>>> When we enabled STRICT_KERNEL_RWX we received some reports of boot
>>>> failures when using the Hash MMU and running under phyp. The crashes
>>>> are intermittent, and often exhibit as a completely unresponsive
>>>> system, or possibly an oops.
>> ...
>>>> 
>>>> diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c
>>>> index 3663d3cdffac..01de985df2c4 100644
>>>> --- a/arch/powerpc/mm/book3s64/hash_pgtable.c
>>>> +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c
>>>> @@ -414,6 +428,73 @@ static void change_memory_range(unsigned long start, unsigned long end,
>>>>  							mmu_kernel_ssize);
>>>>  }
>>>>  
>>>> +static int notrace chmem_secondary_loop(struct change_memory_parms *parms)
>>>> +{
>>>> +	unsigned long msr, tmp, flags;
>>>> +	int *p;
>>>> +
>>>> +	p = &parms->cpu_counter.counter;
>>>> +
>>>> +	local_irq_save(flags);
>>>> +	__hard_EE_RI_disable();
>>>> +
>>>> +	asm volatile (
>>>> +	// Switch to real mode and leave interrupts off
>>>> +	"mfmsr	%[msr]			;"
>>>> +	"li	%[tmp], %[MSR_IR_DR]	;"
>>>> +	"andc	%[tmp], %[msr], %[tmp]	;"
>>>> +	"mtmsrd %[tmp]			;"
>>>> +
>>>> +	// Tell the master we are in real mode
>>>> +	"1:				"
>>>> +	"lwarx	%[tmp], 0, %[p]		;"
>>>> +	"addic	%[tmp], %[tmp], -1	;"
>>>> +	"stwcx.	%[tmp], 0, %[p]		;"
>>>> +	"bne-	1b			;"
>>>> +
>>>> +	// Spin until the counter goes to zero
>>>> +	"2:				;"
>>>> +	"lwz	%[tmp], 0(%[p])		;"
>>>> +	"cmpwi	%[tmp], 0		;"
>>>> +	"bne-	2b			;"
>>>> +
>>>> +	// Switch back to virtual mode
>>>> +	"mtmsrd %[msr]			;"
>>>
>>> Pity we don't have something that can switch to emergency stack and
>>> so we can write this stuff in C.
>>>
>>> How's something like this suit you?
>> 
>> It looks like it would be really good for writing exploits :)
>
> Hmm. In that case maybe the callee function could be inlined into it 
> like the interrupt wrappers, and the asm real-mode entry/exit gets
> added around it rather than have this little exploit stub. So similar to 
> yours but with a stack switch as well so you can come back up in real 
> mode.

Yeah inlining as much as possible would reduce the risk.

>> I think at the very least we would want the asm part to load the SP
>> from the paca itself, rather than taking it as a parameter.
>> 
>> But I'm not sure writing these type of things in C is a big win, because
>> you have to be so careful about what you call anyway. It's almost better
>> in asm because it's so restrictive.
>> 
>> Obviously having said that, my first attempt got the IRQ save/restore
>> wrong, so maybe we should at least have some macros to help with it.
>> 
>> Did you have another user for this in mind? The only one that I can
>> think of at the moment is the subcore stuff.
>
> Possibly rtas entry/exit (although that has other issues). But I guess 
> it's not a huge amount of asm compared with what I'm dealing with.

Ah yep, I hadn't thought of RTAS.

> I'm okay if you just put your thing in at the moment, we might or might 
> not get keen and c-ify it later.

OK.

cheers