Re: [PATCHv2 2/4] x86_64: embrace KERNEL_STACK_OFFSET

[Denys Vlasenko <vda.linux@googlemail.com>]

On Wed, Jan 21, 2015 at 5:29 PM, Alexander van Heukelum
<heukelum@fastmail.fm> wrote:
> On Wed, Jan 21, 2015, at 14:44, Denys Vlasenko wrote:
>> On Sun, Jan 18, 2015 at 12:45 PM, Alexander van Heukelum
>> <heukelum@fastmail.fm> wrote:
>> > KERNEL_STACK_OFFSET is the offset from the top of the kernel stack
>> > page to the value of the kernel_stack percpu variable. This patch
>> > changes KERNEL_STACK_OFFSET to configure a reserved space of 16
>> > bytes above the user ptregs frame. KERNEL_STACK_OFFSET must be
>> > set to a multiple of 16 bytes due to the automatic stack alignment
>> > of interrupts, traps, and exceptions on x86_64.
>>
>> I propose to set kernel_stack percpu variable to point
>> to the top of kernel stack (obvious, isn't it?)
>> and eliminate KERNEL_STACK_OFFSET altogether.
>
> By "top of kernel stack", do you mean the page boundary or the
> top of struct pt_regs on the kernel stack? (is it really that obvious?)
> I think Borislav did the latter for x86_64 in his patchset.

Page boundary.

kernel_stack is currently initialized as follows:

        this_cpu_write(kernel_stack,
                  (unsigned long)task_stack_page(next_p) +
                  THREAD_SIZE - KERNEL_STACK_OFFSET);

i.e. it points KERNEL_STACK_OFFSET bytes below top-of-stack,
which is two pages above task_struct.

Why do we have KERNEL_STACK_OFFSET?

The original idea was that on SYSCALL instruction entry, which
does not create iret stack, we can eliminate one "sub $5*8,%rsp"
instruction. This idea currently does not work, because we
have such instruction anyway (it allocates pr_regs). Nothing is saved there.

And here, in 32-bit compat code:

ENTRY(ia32_sysenter_target)
        CFI_STARTPROC32 simple
        CFI_SIGNAL_FRAME
        CFI_DEF_CFA     rsp,0
        CFI_REGISTER    rsp,rbp
        SWAPGS_UNSAFE_STACK
        movq    PER_CPU_VAR(kernel_stack), %rsp
        addq    $(KERNEL_STACK_OFFSET),%rsp

we even need to _undo_ the "KERNEL_STACK_OFFSET optimization"
(last insn).

My patch "[PATCH 09/11] x86: get rid of KERNEL_STACK_OFFSET"
simply drops the KERNEL_STACK_OFFSET thing.