* Stack switching in a PV guest
@ 2013-12-16 13:31 Simon Martin
2013-12-16 13:36 ` Ian Campbell
0 siblings, 1 reply; 10+ messages in thread
From: Simon Martin @ 2013-12-16 13:31 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 635 bytes --]
Hi all,
I am currently trying to implement multitasking inside my little PV.
This is done by a a simple stack switch.
Whenever I touch the stack in the PV it crashes. I assume that this is
due to the iret at the end of the event handler returning control to the
Hypervisor. Can anyone point me in the right direction to sort this out
please.
Regards.
--
_ _ Debian GNU User Simon Martin
| | (_)_ __ _ ___ __ Project Manager
| | | | '_ \| | | \ \/ / Milliways
| |___| | | | | |_| |> < mailto: smartin@milliways.cl
|_____|_|_| |_|\__,_/_/\_\
Si Hoc Legere Scis Nimium Eruditionis Habes
[-- Attachment #1.2: Type: text/html, Size: 1772 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-16 13:31 Stack switching in a PV guest Simon Martin
@ 2013-12-16 13:36 ` Ian Campbell
2013-12-16 13:50 ` Simon Martin
0 siblings, 1 reply; 10+ messages in thread
From: Ian Campbell @ 2013-12-16 13:36 UTC (permalink / raw)
To: Simon Martin; +Cc: xen-devel
On Mon, 2013-12-16 at 13:31 +0000, Simon Martin wrote:
> Hi all,
>
> I am currently trying to implement multitasking inside my little PV.
> This is done by a a simple stack switch.
>
> Whenever I touch the stack in the PV it crashes. I assume that this is
> due to the iret at the end of the event handler returning control to
> the Hypervisor. Can anyone point me in the right direction to sort
> this out please.
Need more details of what you have done to advise properly I think.
Perhaps you need to be using HYPERVISOR_iret, or perhaps you need to
make sure that your stack frame actually returns you to the correct
context (which I expect hypervisor context is not).
Ian.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-16 13:36 ` Ian Campbell
@ 2013-12-16 13:50 ` Simon Martin
2013-12-16 14:45 ` Jan Beulich
0 siblings, 1 reply; 10+ messages in thread
From: Simon Martin @ 2013-12-16 13:50 UTC (permalink / raw)
To: Ian Campbell; +Cc: Simon Martin, xen-devel
Thanks Ian,
>> I am currently trying to implement multitasking inside my little PV.
>> This is done by a a simple stack switch.
>>
> Need more details of what you have done to advise properly I think.
For the moment I have just intercepted the main hypervisor callback.
I have simplified to the bare minimum what I am doing here (removed
the nested hypercall handling and storing the register file in a
static memory area). As long as I don't touch the stack pointer then
it works quite happily. As soon as point it to a different stack then
everything falls over.
> Perhaps you need to be using HYPERVISOR_iret, or perhaps you need to
> make sure that your stack frame actually returns you to the correct
> context (which I expect hypervisor context is not).
I have checked using gdb and when it gets to the iretq statement in
HYPERVISOR_iret the stack is as follows
%rsp -> valid instruction pointer
%rsp+8 -> same CS as the original stack frame.
%rsp+16 -> 0 (initial rFLAGS)
Regards.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-16 13:50 ` Simon Martin
@ 2013-12-16 14:45 ` Jan Beulich
2013-12-16 14:56 ` Simon Martin
0 siblings, 1 reply; 10+ messages in thread
From: Jan Beulich @ 2013-12-16 14:45 UTC (permalink / raw)
To: Simon Martin; +Cc: Simon Martin, Ian Campbell, xen-devel
>>> On 16.12.13 at 14:50, Simon Martin <furryfuttock@gmail.com> wrote:
> I have checked using gdb and when it gets to the iretq statement in
> HYPERVISOR_iret the stack is as follows
>
> %rsp -> valid instruction pointer
> %rsp+8 -> same CS as the original stack frame.
> %rsp+16 -> 0 (initial rFLAGS)
%rsp+24 -> ???
%rsp+32 -> ???
Those two are the crucial fields when you're having stack issues.
Furthermore, the guest's kernel stack wants to be switched via
__HYPERVISOR_stack_switch, or else the next interrupt/exception
would end up still using the old stack.
Jan
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-16 14:45 ` Jan Beulich
@ 2013-12-16 14:56 ` Simon Martin
2013-12-20 14:12 ` Simon Martin
0 siblings, 1 reply; 10+ messages in thread
From: Simon Martin @ 2013-12-16 14:56 UTC (permalink / raw)
To: Jan Beulich; +Cc: Simon Martin, Ian Campbell, xen-devel
Thanks Jan,
> %rsp+24 ->> ???
> %rsp+32 ->> ???
> Those two are the crucial fields when you're having stack issues.
>From the pt_regs description those looked like unused. I'll make sure
I fill them in now.
> Furthermore, the guest's kernel stack wants to be switched via
> __HYPERVISOR_stack_switch, or else the next interrupt/exception
> would end up still using the old stack.
Thank you very much. I'll investigate that.
--
Best regards,
Simon mailto:furryfuttock@gmail.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-16 14:56 ` Simon Martin
@ 2013-12-20 14:12 ` Simon Martin
2013-12-20 14:47 ` Jan Beulich
2013-12-21 0:51 ` Mukesh Rathor
0 siblings, 2 replies; 10+ messages in thread
From: Simon Martin @ 2013-12-20 14:12 UTC (permalink / raw)
To: Jan Beulich; +Cc: Ian Campbell, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 983 bytes --]
Hi Jan
>> Furthermore, the guest's kernel stack wants to be switched via
>> __HYPERVISOR_stack_switch, or else the next interrupt/exception
>> would end up still using the old stack.
I think I've finally got my head round using HYPERVISOR_stack_switch. It
seems to replace all my stack magic, and at least it's not crashing now.
However, I now have a different problem. I am running my context switch
off of a single shot timer. When the timer fires Xen transfers control
to my PV guest via the hypercall_callback.
The hypercall_callback handler stores the current CPU register file,
however the stored rsp always points to the hypercall stack, not the
stack that was active in my PV at the moment the timer event fired.
I need to store the PV stack pointer so I can recover it later when my
scheduler activates it. I have looked back up the stack and I don't
really see where my active stack pointer is. Any idea where I can get
it?
Regards.
[-- Attachment #1.2: Type: text/html, Size: 2357 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-20 14:12 ` Simon Martin
@ 2013-12-20 14:47 ` Jan Beulich
2013-12-20 15:10 ` Simon Martin
2013-12-21 0:51 ` Mukesh Rathor
1 sibling, 1 reply; 10+ messages in thread
From: Jan Beulich @ 2013-12-20 14:47 UTC (permalink / raw)
To: Simon Martin; +Cc: Ian Campbell, xen-devel
>>> On 20.12.13 at 15:12, "Simon Martin" <smartin@milliways.cl> wrote:
> However, I now have a different problem. I am running my context switch
> off of a single shot timer. When the timer fires Xen transfers control
> to my PV guest via the hypercall_callback.
>
> The hypercall_callback handler stores the current CPU register file,
> however the stored rsp always points to the hypercall stack, not the
> stack that was active in my PV at the moment the timer event fired.
I'm afraid I'm missing some context here. What is a "hypercall stack"?
> I need to store the PV stack pointer so I can recover it later when my
> scheduler activates it. I have looked back up the stack and I don't
> really see where my active stack pointer is. Any idea where I can get
> it?
The callback gets invoked much like other exceptions would be, with
a normal exception entry stack frame. The stack pointer would be
there (RCX, R11, RIP, CS, EFLAGS, RSP, SS) just as normal.
Jan
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-20 14:47 ` Jan Beulich
@ 2013-12-20 15:10 ` Simon Martin
2013-12-20 15:48 ` Jan Beulich
0 siblings, 1 reply; 10+ messages in thread
From: Simon Martin @ 2013-12-20 15:10 UTC (permalink / raw)
To: Jan Beulich; +Cc: Ian Campbell, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 783 bytes --]
>I'm afraid I'm missing some context here. What is a "hypercall stack"?
Hmm. When I look at the stack frame it's not what I expected, so I
assumed that the hypervisor was assigning a stack. Means I've got
another bug.
>
>The callback gets invoked much like other exceptions would be, with
>a normal exception entry stack frame. The stack pointer would be
>there (RCX, R11, RIP, CS, EFLAGS, RSP, SS) just as normal.
Probably a knock on from the problem I am seeing above then. I've
searched around for examples of HYPERVISOR_stack_switch usage, but I
can't find anything. I can find the declaration and handler in the Xen
source and however no example of making it work. Is there anything you
can point me to to see how it was intended to be used?
Thanks.
[-- Attachment #1.2: Type: text/html, Size: 1869 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-20 15:10 ` Simon Martin
@ 2013-12-20 15:48 ` Jan Beulich
0 siblings, 0 replies; 10+ messages in thread
From: Jan Beulich @ 2013-12-20 15:48 UTC (permalink / raw)
To: Simon Martin; +Cc: Ian Campbell, xen-devel
>>> On 20.12.13 at 16:10, "Simon Martin" <smartin@milliways.cl> wrote:
>>I'm afraid I'm missing some context here. What is a "hypercall stack"?
> Hmm. When I look at the stack frame it's not what I expected, so I
> assumed that the hypervisor was assigning a stack. Means I've got
> another bug.
>
>>The callback gets invoked much like other exceptions would be, with
>>a normal exception entry stack frame. The stack pointer would be
>>there (RCX, R11, RIP, CS, EFLAGS, RSP, SS) just as normal.
> Probably a knock on from the problem I am seeing above then. I've
> searched around for examples of HYPERVISOR_stack_switch usage, but I
> can't find anything. I can find the declaration and handler in the Xen
> source and however no example of making it work. Is there anything you
> can point me to to see how it was intended to be used?
As to the implementation - look for do_stack_switch() in
xen/arch/x86/x86_64/mm.c.
As for usage examples, see current Linux'es
{HYPERVISOR,MULTI}_stack_switch() in
arch/x86/include/asm/xen/hypercall.h or __switch_to() in
linux-2.6.18-xen.hg's arch/x86_64/kernel/process-xen.c.
Jan
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Stack switching in a PV guest
2013-12-20 14:12 ` Simon Martin
2013-12-20 14:47 ` Jan Beulich
@ 2013-12-21 0:51 ` Mukesh Rathor
1 sibling, 0 replies; 10+ messages in thread
From: Mukesh Rathor @ 2013-12-21 0:51 UTC (permalink / raw)
To: Simon Martin; +Cc: Ian Campbell, Jan Beulich, xen-devel
On Fri, 20 Dec 2013 14:12:55 +0000
"Simon Martin" <smartin@milliways.cl> wrote:
> Hi Jan
>
>
> >> Furthermore, the guest's kernel stack wants to be switched via
> >> __HYPERVISOR_stack_switch, or else the next interrupt/exception
> >> would end up still using the old stack.
> I think I've finally got my head round using HYPERVISOR_stack_switch.
> It seems to replace all my stack magic, and at least it's not
> crashing now.
>
> However, I now have a different problem. I am running my context
> switch off of a single shot timer. When the timer fires Xen transfers
> control to my PV guest via the hypercall_callback.
>
> The hypercall_callback handler stores the current CPU register file,
> however the stored rsp always points to the hypercall stack, not the
> stack that was active in my PV at the moment the timer event fired.
>
> I need to store the PV stack pointer so I can recover it later when
> my scheduler activates it. I have looked back up the stack and I
> don't really see where my active stack pointer is. Any idea where I
> can get it?
>
> Regards.
May I recommend taking a look at PVH, it would simplify things a
lot for you. Take a look at what linux PVH does, you could start with
linux PVH patches:
"[PATCH v11] PVH support for Linux kernel." email by konrad on
xen-devel.
If you have any questions, I can help, next year tho.
Mukesh
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-12-21 0:51 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-12-16 13:31 Stack switching in a PV guest Simon Martin
2013-12-16 13:36 ` Ian Campbell
2013-12-16 13:50 ` Simon Martin
2013-12-16 14:45 ` Jan Beulich
2013-12-16 14:56 ` Simon Martin
2013-12-20 14:12 ` Simon Martin
2013-12-20 14:47 ` Jan Beulich
2013-12-20 15:10 ` Simon Martin
2013-12-20 15:48 ` Jan Beulich
2013-12-21 0:51 ` Mukesh Rathor
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.