All of lore.kernel.org
 help / color / mirror / Atom feed
* Error booting Xen
@ 2016-01-18 18:03 Harmandeep Kaur
  2016-01-18 18:09 ` Andrew Cooper
  2016-01-19 14:07 ` Harmandeep Kaur
  0 siblings, 2 replies; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-18 18:03 UTC (permalink / raw)
  To: xen-devel; +Cc: Dario Faggioli

Hi,

I tried to boot Xen (cloned from git clone
git://xenbits.xen.org/xen.git), but it stucks
in a infinite loop. I got the log via serial.
You can inspect it at
http://paste2.org/YCpkzbpG

If anybody has encountered this issue
before, I will appreciate any help.

Regards,
Harmandeep Kaur
Outreachy Intern

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-18 18:03 Error booting Xen Harmandeep Kaur
@ 2016-01-18 18:09 ` Andrew Cooper
  2016-01-18 18:29   ` Dario Faggioli
  2016-01-19 14:07 ` Harmandeep Kaur
  1 sibling, 1 reply; 46+ messages in thread
From: Andrew Cooper @ 2016-01-18 18:09 UTC (permalink / raw)
  To: xen-devel

On 18/01/16 18:03, Harmandeep Kaur wrote:
> Hi,
>
> I tried to boot Xen (cloned from git clone
> git://xenbits.xen.org/xen.git), but it stucks
> in a infinite loop. I got the log via serial.
> You can inspect it at
> http://paste2.org/YCpkzbpG
>
> If anybody has encountered this issue
> before, I will appreciate any help.

Does that last line "(XEN) traps.c:3290: GPF (0000): ffff82d0801af585 ->
ffff82d080240e5a" repeat forever?

If so, it is a dom0 kernel bug not a Xen bug.

~Andrew

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-18 18:09 ` Andrew Cooper
@ 2016-01-18 18:29   ` Dario Faggioli
  2016-01-19 13:36     ` Jan Beulich
  0 siblings, 1 reply; 46+ messages in thread
From: Dario Faggioli @ 2016-01-18 18:29 UTC (permalink / raw)
  To: Andrew Cooper, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1347 bytes --]

On Mon, 2016-01-18 at 18:09 +0000, Andrew Cooper wrote:
> On 18/01/16 18:03, Harmandeep Kaur wrote:
> > Hi,
> > 
> > I tried to boot Xen (cloned from git clone
> > git://xenbits.xen.org/xen.git), but it stucks
> > in a infinite loop. I got the log via serial.
> > You can inspect it at
> > http://paste2.org/YCpkzbpG
> > 
> > If anybody has encountered this issue
> > before, I will appreciate any help.
> 
> Does that last line "(XEN) traps.c:3290: GPF (0000): ffff82d0801af585
> ->
> ffff82d080240e5a" repeat forever?
> 
It does, as far as Harmandeep told me.

Also, that same Linux kernel binary boots as baremetal.

ISTR (but, Harmandeep, correct me if I'm wrong) that it was booting
fine as Dom0, if the Ubuntu packaged version of Xen was used (which I
think is 4.5).

> If so, it is a dom0 kernel bug not a Xen bug.
> 
Yeah, but again, it was booting as dom0 with another Xen... doesn't
that mean that Xen at least has a role in exposing it?

Also, what would you think it's better to try next... try another
kernel?

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-18 18:29   ` Dario Faggioli
@ 2016-01-19 13:36     ` Jan Beulich
  2016-01-19 14:51       ` Dario Faggioli
  0 siblings, 1 reply; 46+ messages in thread
From: Jan Beulich @ 2016-01-19 13:36 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Andrew Cooper, xen-devel

>>> On 18.01.16 at 19:29, <dario.faggioli@citrix.com> wrote:
> On Mon, 2016-01-18 at 18:09 +0000, Andrew Cooper wrote:
>> On 18/01/16 18:03, Harmandeep Kaur wrote:
>> > I tried to boot Xen (cloned from git clone
>> > git://xenbits.xen.org/xen.git), but it stucks
>> > in a infinite loop. I got the log via serial.
>> > You can inspect it at
>> > http://paste2.org/YCpkzbpG 
>> > 
>> > If anybody has encountered this issue
>> > before, I will appreciate any help.
>> 
>> Does that last line "(XEN) traps.c:3290: GPF (0000): ffff82d0801af585
>> ->
>> ffff82d080240e5a" repeat forever?
>> 
> It does, as far as Harmandeep told me.
> 
> Also, that same Linux kernel binary boots as baremetal.
> 
> ISTR (but, Harmandeep, correct me if I'm wrong) that it was booting
> fine as Dom0, if the Ubuntu packaged version of Xen was used (which I
> think is 4.5).
> 
>> If so, it is a dom0 kernel bug not a Xen bug.
>> 
> Yeah, but again, it was booting as dom0 with another Xen... doesn't
> that mean that Xen at least has a role in exposing it?
> 
> Also, what would you think it's better to try next... try another
> kernel?

Figure out what the GP fault that gets recovered from is being
caused by. Then - if indeed it worked on another hypervisor
build - figure out the difference(s) in behavior.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-18 18:03 Error booting Xen Harmandeep Kaur
  2016-01-18 18:09 ` Andrew Cooper
@ 2016-01-19 14:07 ` Harmandeep Kaur
  1 sibling, 0 replies; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-19 14:07 UTC (permalink / raw)
  To: xen-devel; +Cc: Dario Faggioli

> On Mon, 2016-01-18 at 18:09 +0000, Andrew Cooper wrote:
> > On 18/01/16 18:03, Harmandeep Kaur wrote:
> > > Hi,
> > >
> > > I tried to boot Xen (cloned from git clone
> > > git://xenbits.xen.org/xen.git), but it stucks
> > > in a infinite loop. I got the log via serial.
> > > You can inspect it at
> > > http://paste2.org/YCpkzbpG
> > >
> > > If anybody has encountered this issue
> > > before, I will appreciate any help.
> >
> > Does that last line "(XEN) traps.c:3290: GPF (0000): ffff82d0801af585
> > ->
> > ffff82d080240e5a" repeat forever?
> >
> It does, as far as Harmandeep told me.
>
> Also, that same Linux kernel binary boots as baremetal.
>
> ISTR (but, Harmandeep, correct me if I'm wrong) that it was booting
> fine as Dom0, if the Ubuntu packaged version of Xen was used (which I
> think is 4.5).
>

Yes, you're right Dario. Even 4.7-unstable used to run (around till 2
weeks ago).

Maybe kernel has been changed since then.

> > If so, it is a dom0 kernel bug not a Xen bug.
> >
> Yeah, but again, it was booting as dom0 with another Xen... doesn't
> that mean that Xen at least has a role in exposing it?
>
> Also, what would you think it's better to try next... try another
> kernel?
>
> Thanks and Regards,
> Dario

On Mon, Jan 18, 2016 at 11:33 PM, Harmandeep Kaur
<write.harmandeep@gmail.com> wrote:
> Hi,
>
> I tried to boot Xen (cloned from git clone
> git://xenbits.xen.org/xen.git), but it stucks
> in a infinite loop. I got the log via serial.
> You can inspect it at
> http://paste2.org/YCpkzbpG
>
> If anybody has encountered this issue
> before, I will appreciate any help.
>
> Regards,
> Harmandeep Kaur
> Outreachy Intern

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 13:36     ` Jan Beulich
@ 2016-01-19 14:51       ` Dario Faggioli
  2016-01-19 15:07         ` Andrew Cooper
  0 siblings, 1 reply; 46+ messages in thread
From: Dario Faggioli @ 2016-01-19 14:51 UTC (permalink / raw)
  To: Jan Beulich, Harmandeep Kaur; +Cc: Andrew Cooper, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1526 bytes --]

On Tue, 2016-01-19 at 06:36 -0700, Jan Beulich wrote:
> > > > On 18.01.16 at 19:29, <dario.faggioli@citrix.com> wrote:
> > 
> > Yeah, but again, it was booting as dom0 with another Xen... doesn't
> > that mean that Xen at least has a role in exposing it?
> > 
> > Also, what would you think it's better to try next... try another
> > kernel?
> 
> Figure out what the GP fault that gets recovered from is being
> caused by. 
>
Yep, thanks.

Harmandeep is looking at (learning how to do) this already. If you feel
like sharing any knowledge/giving any advice on how to best achieve
that, I'm sure she'll appreciate (but if you can't or don't have time,
no problem :-D).

> Then - if indeed it worked on another hypervisor
> build - figure out the difference(s) in behavior.
> 
Right. Harmandeep, you can try this rather easily!.

You have a clone of the xen.git already. I think you just have to:
 - `make uninstall' (I'm not sure it will work, but it at least should 
    not harm!)
 - clean the sources
 - checkout, e.g., in a branch, one of the tags of a previous release, 
   say RELEASE-4.6.0 or RELEASE-4.5.0
 - build again
 - install
 - update the bootloader
 - reboot and report the result.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 14:51       ` Dario Faggioli
@ 2016-01-19 15:07         ` Andrew Cooper
  2016-01-19 16:27           ` Harmandeep Kaur
  0 siblings, 1 reply; 46+ messages in thread
From: Andrew Cooper @ 2016-01-19 15:07 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Dario Faggioli, Jan Beulich, xen-devel

On 19/01/16 14:51, Dario Faggioli wrote:
> On Tue, 2016-01-19 at 06:36 -0700, Jan Beulich wrote:
>>>>> On 18.01.16 at 19:29, <dario.faggioli@citrix.com> wrote:
>>>  
>>> Yeah, but again, it was booting as dom0 with another Xen... doesn't
>>> that mean that Xen at least has a role in exposing it?
>>>
>>> Also, what would you think it's better to try next... try another
>>> kernel?
>> Figure out what the GP fault that gets recovered from is being
>> caused by. 
>>
> Yep, thanks.
>
> Harmandeep is looking at (learning how to do) this already. If you feel
> like sharing any knowledge/giving any advice on how to best achieve
> that, I'm sure she'll appreciate (but if you can't or don't have time,
> no problem :-D).

First, locate exactly where the fault is

`addr2line -e xen-syms ffff82d0801af585`

will give you a file/line reference.  xen-syms must be from the same
build of the hypervisor that you booted.

~Andrew

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 15:07         ` Andrew Cooper
@ 2016-01-19 16:27           ` Harmandeep Kaur
  2016-01-19 16:47             ` Jan Beulich
  0 siblings, 1 reply; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-19 16:27 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Dario Faggioli, Jan Beulich, xen-devel

Hi,

Adding 'xsave=0' is working for now. Thank you
all for your help :)

Regards,
Harman

On Tue, Jan 19, 2016 at 8:37 PM, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 19/01/16 14:51, Dario Faggioli wrote:
>> On Tue, 2016-01-19 at 06:36 -0700, Jan Beulich wrote:
>>>>>> On 18.01.16 at 19:29, <dario.faggioli@citrix.com> wrote:
>>>>
>>>> Yeah, but again, it was booting as dom0 with another Xen... doesn't
>>>> that mean that Xen at least has a role in exposing it?
>>>>
>>>> Also, what would you think it's better to try next... try another
>>>> kernel?
>>> Figure out what the GP fault that gets recovered from is being
>>> caused by.
>>>
>> Yep, thanks.
>>
>> Harmandeep is looking at (learning how to do) this already. If you feel
>> like sharing any knowledge/giving any advice on how to best achieve
>> that, I'm sure she'll appreciate (but if you can't or don't have time,
>> no problem :-D).
>
> First, locate exactly where the fault is
>
> `addr2line -e xen-syms ffff82d0801af585`
>
> will give you a file/line reference.  xen-syms must be from the same
> build of the hypervisor that you booted.
>
> ~Andrew

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 16:27           ` Harmandeep Kaur
@ 2016-01-19 16:47             ` Jan Beulich
  2016-01-19 17:02               ` Andrew Cooper
  0 siblings, 1 reply; 46+ messages in thread
From: Jan Beulich @ 2016-01-19 16:47 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel

>>> On 19.01.16 at 17:27, <write.harmandeep@gmail.com> wrote:
> Adding 'xsave=0' is working for now. Thank you
> all for your help :)

But that means we actually should get to the bottom of your problem!

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 16:47             ` Jan Beulich
@ 2016-01-19 17:02               ` Andrew Cooper
  2016-01-19 17:08                 ` Dario Faggioli
  0 siblings, 1 reply; 46+ messages in thread
From: Andrew Cooper @ 2016-01-19 17:02 UTC (permalink / raw)
  To: Jan Beulich, Harmandeep Kaur; +Cc: Dario Faggioli, xen-devel

On 19/01/16 16:47, Jan Beulich wrote:
>>>> On 19.01.16 at 17:27, <write.harmandeep@gmail.com> wrote:
>> Adding 'xsave=0' is working for now. Thank you
>> all for your help :)
> But that means we actually should get to the bottom of your problem!

There was some discussion on IRC.  `xrstror` was repeatedly taking the
same fault; i.e. the fixup code wasn't fixing up suitably.

As a first candidate, I expect 83ae0bb2 is a likely candidate. 
Harmandeep is using a Skylake processor.

~Andrew

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 17:02               ` Andrew Cooper
@ 2016-01-19 17:08                 ` Dario Faggioli
  2016-01-19 17:09                   ` Andrew Cooper
  2016-01-19 19:55                   ` Harmandeep Kaur
  0 siblings, 2 replies; 46+ messages in thread
From: Dario Faggioli @ 2016-01-19 17:08 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Harmandeep Kaur, Jan Beulich, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1079 bytes --]

On Tue, 2016-01-19 at 17:02 +0000, Andrew Cooper wrote:
> On 19/01/16 16:47, Jan Beulich wrote:
> > > > > On 19.01.16 at 17:27, <write.harmandeep@gmail.com> wrote:
> > > Adding 'xsave=0' is working for now. Thank you
> > > all for your help :)
> > But that means we actually should get to the bottom of your
> > problem!
> 
> There was some discussion on IRC.  `xrstror` was repeatedly taking
> the
> same fault; i.e. the fixup code wasn't fixing up suitably.
> 
Yes, indeed.
 
> As a first candidate, I expect 83ae0bb2 is a likely candidate. 
> Harmandeep is using a Skylake processor.
> 
Yes, she is on Skylake. But she's also using master, so,
AFAICT, 83ae0bb2 is not there.

She will be trying to use staging (and kill xsave=0) soon, and will let
us know.

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 17:08                 ` Dario Faggioli
@ 2016-01-19 17:09                   ` Andrew Cooper
  2016-01-19 19:55                   ` Harmandeep Kaur
  1 sibling, 0 replies; 46+ messages in thread
From: Andrew Cooper @ 2016-01-19 17:09 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Harmandeep Kaur, Jan Beulich, xen-devel

On 19/01/16 17:08, Dario Faggioli wrote:
> On Tue, 2016-01-19 at 17:02 +0000, Andrew Cooper wrote:
>> On 19/01/16 16:47, Jan Beulich wrote:
>>>>>> On 19.01.16 at 17:27, <write.harmandeep@gmail.com> wrote:
>>>> Adding 'xsave=0' is working for now. Thank you
>>>> all for your help :)
>>> But that means we actually should get to the bottom of your
>>> problem!
>> There was some discussion on IRC.  `xrstror` was repeatedly taking
>> the
>> same fault; i.e. the fixup code wasn't fixing up suitably.
>>
> Yes, indeed.
>  
>> As a first candidate, I expect 83ae0bb2 is a likely candidate. 
>> Harmandeep is using a Skylake processor.
>>
> Yes, she is on Skylake. But she's also using master, so,
> AFAICT, 83ae0bb2 is not there.
>
> She will be trying to use staging (and kill xsave=0) soon, and will let
> us know.

We have some Skylake kit in XenServer.  I will investigate when it
becomes available.

~Andrew

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 17:08                 ` Dario Faggioli
  2016-01-19 17:09                   ` Andrew Cooper
@ 2016-01-19 19:55                   ` Harmandeep Kaur
  2016-01-20 10:06                     ` Jan Beulich
  1 sibling, 1 reply; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-19 19:55 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Andrew Cooper, Jan Beulich, xen-devel

On Tue, Jan 19, 2016 at 10:38 PM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> On Tue, 2016-01-19 at 17:02 +0000, Andrew Cooper wrote:
>> On 19/01/16 16:47, Jan Beulich wrote:
>> > > > > On 19.01.16 at 17:27, <write.harmandeep@gmail.com> wrote:
>> > > Adding 'xsave=0' is working for now. Thank you
>> > > all for your help :)
>> > But that means we actually should get to the bottom of your
>> > problem!
>>
>> There was some discussion on IRC.  `xrstror` was repeatedly taking
>> the
>> same fault; i.e. the fixup code wasn't fixing up suitably.
>>
> Yes, indeed.
>
>> As a first candidate, I expect 83ae0bb2 is a likely candidate.
>> Harmandeep is using a Skylake processor.
>>
> Yes, she is on Skylake. But she's also using master, so,
> AFAICT, 83ae0bb2 is not there.
>
> She will be trying to use staging (and kill xsave=0) soon, and will let
> us know.

I tried booting staging branch but results were identical. Following
line repeats endlessly.
(XEN) traps.c:3290: GPF (0000): ffff82d0801c1cce -> ffff82d080252e5c

$ 'addr2line -e xen-syms ffff82d0801c1cce' returns
'xen/xen/arch/x86/xstate.c:387' which again points to
xsave. Also, adding 'xsave=0' makes it boot just fine.

Full boot log here, http://paste2.org/1DCge9Fb

Thanks and regards,
Harmandeep

> Thanks and Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-19 19:55                   ` Harmandeep Kaur
@ 2016-01-20 10:06                     ` Jan Beulich
  2016-01-21 15:14                       ` Dario Faggioli
                                         ` (2 more replies)
  0 siblings, 3 replies; 46+ messages in thread
From: Jan Beulich @ 2016-01-20 10:06 UTC (permalink / raw)
  To: Harmandeep Kaur, Shuai Ruan; +Cc: Andrew Cooper, Dario Faggioli, xen-devel

>>> On 19.01.16 at 20:55, <write.harmandeep@gmail.com> wrote:
> I tried booting staging branch but results were identical. Following
> line repeats endlessly.
> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cce -> ffff82d080252e5c
> 
> $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
> 'xen/xen/arch/x86/xstate.c:387' which again points to
> xsave. Also, adding 'xsave=0' makes it boot just fine.

Ah, I think I see the issue: We're zeroing the entire save area in
the fixup code, which will make XRSTORS fault unconditionally.
Shuai, would you please look into possible ways of fixing this?
Just setting the compressed flag may not be enough, and falling
back to plain XRSTOR doesn't seem to be an option either - I'm in
particular worried that the current model of zeroing everything
is bogus with the growing number of different components which
get loaded here.

But of course another question then is why the XRSTORS faults
in the first place. I guess we'll need a debugging patch to dump
the full state to understand that.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-20 10:06                     ` Jan Beulich
@ 2016-01-21 15:14                       ` Dario Faggioli
  2016-01-21 15:24                         ` Harmandeep Kaur
  2016-01-25 13:42                         ` Jan Beulich
  2016-02-02  4:43                       ` Shuai Ruan
       [not found]                       ` <20160202044349.GA3036@shuai.ruan@linux.intel.com>
  2 siblings, 2 replies; 46+ messages in thread
From: Dario Faggioli @ 2016-01-21 15:14 UTC (permalink / raw)
  To: Jan Beulich, Harmandeep Kaur, Shuai Ruan; +Cc: Andrew Cooper, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1385 bytes --]

On Wed, 2016-01-20 at 03:06 -0700, Jan Beulich wrote:
> > > > On 19.01.16 at 20:55, <write.harmandeep@gmail.com> wrote:
> > 
> > $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
> > 'xen/xen/arch/x86/xstate.c:387' which again points to
> > xsave. Also, adding 'xsave=0' makes it boot just fine.
> 
> Ah, I think I see the issue: We're zeroing the entire save area in
> the fixup code, which will make XRSTORS fault unconditionally.
> Shuai, would you please look into possible ways of fixing this?
> Just setting the compressed flag may not be enough, and falling
> back to plain XRSTOR doesn't seem to be an option either - I'm in
> particular worried that the current model of zeroing everything
> is bogus with the growing number of different components which
> get loaded here.
> 
> But of course another question then is why the XRSTORS faults
> in the first place. I guess we'll need a debugging patch to dump
> the full state to understand that.
> 
If someone can produce and send such patch, I'm sure Harmandeep will be
happy to give it a try on her hardware.

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-21 15:14                       ` Dario Faggioli
@ 2016-01-21 15:24                         ` Harmandeep Kaur
  2016-01-25 13:42                         ` Jan Beulich
  1 sibling, 0 replies; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-21 15:24 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Andrew Cooper, xen-devel, Jan Beulich, Shuai Ruan

On Thu, Jan 21, 2016 at 8:44 PM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> On Wed, 2016-01-20 at 03:06 -0700, Jan Beulich wrote:
>> > > > On 19.01.16 at 20:55, <write.harmandeep@gmail.com> wrote:
>> >
>> > $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
>> > 'xen/xen/arch/x86/xstate.c:387' which again points to
>> > xsave. Also, adding 'xsave=0' makes it boot just fine.
>>
>> Ah, I think I see the issue: We're zeroing the entire save area in
>> the fixup code, which will make XRSTORS fault unconditionally.
>> Shuai, would you please look into possible ways of fixing this?
>> Just setting the compressed flag may not be enough, and falling
>> back to plain XRSTOR doesn't seem to be an option either - I'm in
>> particular worried that the current model of zeroing everything
>> is bogus with the growing number of different components which
>> get loaded here.
>>
>> But of course another question then is why the XRSTORS faults
>> in the first place. I guess we'll need a debugging patch to dump
>> the full state to understand that.
>>
> If someone can produce and send such patch, I'm sure Harmandeep will be
> happy to give it a try on her hardware.
>

Yes, sure. I will be more than happy to do this :)
Thank you Dario.

Regards,
Harman
> Thanks and Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-21 15:14                       ` Dario Faggioli
  2016-01-21 15:24                         ` Harmandeep Kaur
@ 2016-01-25 13:42                         ` Jan Beulich
  2016-01-26 11:58                           ` Dario Faggioli
  2016-01-26 12:51                           ` Harmandeep Kaur
  1 sibling, 2 replies; 46+ messages in thread
From: Jan Beulich @ 2016-01-25 13:42 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

[-- Attachment #1: Type: text/plain, Size: 1488 bytes --]

>>> On 21.01.16 at 16:14, <dario.faggioli@citrix.com> wrote:
> On Wed, 2016-01-20 at 03:06 -0700, Jan Beulich wrote:
>> > > > On 19.01.16 at 20:55, <write.harmandeep@gmail.com> wrote:
>> > 
>> > $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
>> > 'xen/xen/arch/x86/xstate.c:387' which again points to
>> > xsave. Also, adding 'xsave=0' makes it boot just fine.
>> 
>> Ah, I think I see the issue: We're zeroing the entire save area in
>> the fixup code, which will make XRSTORS fault unconditionally.
>> Shuai, would you please look into possible ways of fixing this?
>> Just setting the compressed flag may not be enough, and falling
>> back to plain XRSTOR doesn't seem to be an option either - I'm in
>> particular worried that the current model of zeroing everything
>> is bogus with the growing number of different components which
>> get loaded here.
>> 
>> But of course another question then is why the XRSTORS faults
>> in the first place. I guess we'll need a debugging patch to dump
>> the full state to understand that.
>> 
> If someone can produce and send such patch, I'm sure Harmandeep will be
> happy to give it a try on her hardware.

So here you go. Instead of a debugging one, I hope I have at
once fixed the issue in a suitable way. Whether we'd like to keep
the debugging output we can decide later on.

Both patches need to be applied; while the order shouldn't matter,
the alignment one is a prereq to the actual change.

Jan


[-- Attachment #2: x86-xrstors-fault.patch --]
[-- Type: text/plain, Size: 7042 bytes --]

x86/xstate: fix fault behavior on XRSTORS

XRSTORS unconditionally faults when xcomp_bv has bit 63 clear. Instead
of just fixing this issue, overhaul the fault recovery code, which -
one of the many mistakes made when xstate support got introduced - was
blindly mirroring that accompanying FXRSTOR, neglecting the fact that
XRSTOR{,S} aren't all-or-nothing instructions. The new code, first of
all, does all the recovery actions in C, simplifying the inline
assembly used. And it does its work in a multi-stage fashion: Upon
first seeing a fault, state fixups get applied strictly based on what
architecturally may cause #GP. When seeing another fault despite the
fixups done, state gets fully reset. A third fault would then lead to
crashing the domain (instead of hanging the hypervisor in an infinite
loop of recurring faults).

Reported-by: Harmandeep Kaur <write.harmandeep@gmail.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/xstate.c	2016-01-25 09:35:12.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-25 12:34:48.000000000 +0100
@@ -29,6 +29,8 @@ unsigned int *__read_mostly xstate_sizes
 static unsigned int __read_mostly xstate_features;
 static unsigned int __read_mostly xstate_comp_offsets[sizeof(xfeature_mask)*8];
 
+static uint32_t __read_mostly mxcsr_mask = MXCSR_DEFAULT;
+
 /* Cached xcr0 for fast read */
 static DEFINE_PER_CPU(uint64_t, xcr0);
 
@@ -342,6 +344,7 @@ void xrstor(struct vcpu *v, uint64_t mas
     uint32_t hmask = mask >> 32;
     uint32_t lmask = mask;
     struct xsave_struct *ptr = v->arch.xsave_area;
+    unsigned int faults, prev_faults;
 
     /*
      * AMD CPUs don't save/restore FDP/FIP/FOP unless an exception
@@ -361,35 +364,79 @@ void xrstor(struct vcpu *v, uint64_t mas
     /*
      * XRSTOR can fault if passed a corrupted data block. We handle this
      * possibility, which may occur if the block was passed to us by control
-     * tools or through VCPUOP_initialise, by silently clearing the block.
+     * tools or through VCPUOP_initialise, by silently adjusting state.
      */
-    switch ( __builtin_expect(ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET], 8) )
+    for ( prev_faults = faults = 0; ; prev_faults = faults )
     {
+        switch ( __builtin_expect(ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET], 8) )
+        {
 #define XRSTOR(pfx) \
         alternative_io("1: .byte " pfx "0x0f,0xae,0x2f\n" \
+                       "3:\n" \
                        "   .section .fixup,\"ax\"\n" \
-                       "2: mov %[size],%%ecx\n" \
-                       "   xor %[lmask_out],%[lmask_out]\n" \
-                       "   rep stosb\n" \
-                       "   lea %[mem],%[ptr]\n" \
-                       "   mov %[lmask_in],%[lmask_out]\n" \
-                       "   jmp 1b\n" \
+                       "2: inc%z[faults] %[faults]\n" \
+                       "   jmp 3b\n" \
                        "   .previous\n" \
                        _ASM_EXTABLE(1b, 2b), \
                        ".byte " pfx "0x0f,0xc7,0x1f\n", \
                        X86_FEATURE_XSAVES, \
-                       ASM_OUTPUT2([ptr] "+&D" (ptr), [lmask_out] "+&a" (lmask)), \
-                       [mem] "m" (*ptr), [lmask_in] "g" (lmask), \
-                       [hmask] "d" (hmask), [size] "m" (xsave_cntxt_size) \
-                       : "ecx")
-
-    default:
-        XRSTOR("0x48,");
-        break;
-    case 4: case 2:
-        XRSTOR("");
-        break;
+                       ASM_OUTPUT2([mem] "+m" (*ptr), [faults] "+g" (faults)), \
+                       [lmask] "a" (lmask), [hmask] "d" (hmask), \
+                       [ptr] "D" (ptr))
+
+        default:
+            XRSTOR("0x48,");
+            break;
+        case 4: case 2:
+            XRSTOR("");
+            break;
 #undef XRSTOR
+        }
+        if ( likely(faults == prev_faults) )
+            break;
+#ifndef NDEBUG
+        gprintk(XENLOG_WARNING, "fault#%u: mxcsr=%08x\n",
+                faults, ptr->fpu_sse.mxcsr);
+        gprintk(XENLOG_WARNING, "xs=%016lx xc=%016lx\n",
+                ptr->xsave_hdr.xstate_bv, ptr->xsave_hdr.xcomp_bv);
+        gprintk(XENLOG_WARNING, "r0=%016lx r1=%016lx\n",
+                ptr->xsave_hdr.reserved[0], ptr->xsave_hdr.reserved[1]);
+        gprintk(XENLOG_WARNING, "r2=%016lx r3=%016lx\n",
+                ptr->xsave_hdr.reserved[2], ptr->xsave_hdr.reserved[3]);
+        gprintk(XENLOG_WARNING, "r4=%016lx r5=%016lx\n",
+                ptr->xsave_hdr.reserved[4], ptr->xsave_hdr.reserved[5]);
+#endif
+        switch ( faults )
+        {
+        case 1:
+            if ( (ptr->fpu_sse.mxcsr & ~mxcsr_mask) &&
+                 ((mask & XSTATE_SSE) ||
+                  ((mask & XSTATE_YMM) &&
+                   !(ptr->xsave_hdr.xcomp_bv & XSTATE_COMPACTION_ENABLED))) )
+                ptr->fpu_sse.mxcsr &= mxcsr_mask;
+            ptr->xsave_hdr.xstate_bv &= ~mask;
+            if ( cpu_has_xsaves || cpu_has_xsavec )
+            {
+                ptr->xsave_hdr.xcomp_bv &= this_cpu(xcr0) | this_cpu(xss);
+                ptr->xsave_hdr.xstate_bv &= ptr->xsave_hdr.xcomp_bv;
+            }
+            else
+            {
+                ptr->xsave_hdr.xstate_bv &= this_cpu(xcr0);
+                ptr->xsave_hdr.xcomp_bv = 0;
+            }
+            memset(ptr->xsave_hdr.reserved, 0, sizeof(ptr->xsave_hdr.reserved));
+            continue;
+        case 2:
+            ptr->fpu_sse.mxcsr = MXCSR_DEFAULT;
+            ptr->xsave_hdr.xstate_bv = 0;
+            ptr->xsave_hdr.xcomp_bv = cpu_has_xsaves
+                                      ? XSTATE_COMPACTION_ENABLED : 0;
+            continue;
+        default:
+            domain_crash(current->domain);
+            break;
+        }
     }
 }
 
@@ -414,7 +461,7 @@ int xstate_alloc_save_area(struct vcpu *
     BUG_ON(xsave_cntxt_size < XSTATE_AREA_MIN_SIZE);
 
     /* XSAVE/XRSTOR requires the save area be 64-byte-boundary aligned. */
-    BUILD_BUG_ON(__alignof(*save_area) < 64);
+    BUILD_BUG_ON(__alignof(*v->arch.xsave_area) < 64);
     save_area = _xzalloc(xsave_cntxt_size, __alignof(*save_area));
     if ( save_area == NULL )
         return -ENOMEM;
@@ -496,6 +543,8 @@ void xstate_init(struct cpuinfo_x86 *c)
 
     if ( bsp )
     {
+        static typeof(current->arch.xsave_area->fpu_sse) __initdata ctxt;
+
         xfeature_mask = feature_mask;
         /*
          * xsave_cntxt_size is the max size required by enabled features.
@@ -504,6 +553,10 @@ void xstate_init(struct cpuinfo_x86 *c)
         xsave_cntxt_size = _xstate_ctxt_size(feature_mask);
         printk("%s: using cntxt_size: %#x and states: %#"PRIx64"\n",
             __func__, xsave_cntxt_size, xfeature_mask);
+
+        asm ( "fxsave %0" : "=m" (ctxt) );
+        if ( ctxt.mxcsr_mask )
+            mxcsr_mask = ctxt.mxcsr_mask;
     }
     else
     {

[-- Attachment #3: x86-xstate-align.patch --]
[-- Type: text/plain, Size: 2384 bytes --]

x86: adjust xsave structure attributes

The packed attribute was pointlessly used here - there are no
misaligned fields, and hence even if the attribute took effect, it
would at best lead to the compiler generating worse code.

At the same time specify the required alignment of the fpu_sse sub-
structure, such that the various typeof() uses on that field obtain
pointers to properly aligned memory (knowledge which a compiler may
want to make use of).

Also add suitable build-time checks.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/i387.c	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/arch/x86/i387.c	2016-01-25 09:35:36.000000000 +0100
@@ -277,7 +277,9 @@ int vcpu_init_fpu(struct vcpu *v)
     }
     else
     {
-        v->arch.fpu_ctxt = _xzalloc(sizeof(v->arch.xsave_area->fpu_sse), 16);
+        BUILD_BUG_ON(__alignof(v->arch.xsave_area->fpu_sse) < 16);
+        v->arch.fpu_ctxt = _xzalloc(sizeof(v->arch.xsave_area->fpu_sse),
+                                    __alignof(v->arch.xsave_area->fpu_sse));
         if ( v->arch.fpu_ctxt )
         {
             typeof(v->arch.xsave_area->fpu_sse) *fpu_sse = v->arch.fpu_ctxt;
--- unstable.orig/xen/arch/x86/xstate.c	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-25 09:35:12.000000000 +0100
@@ -414,7 +414,8 @@ int xstate_alloc_save_area(struct vcpu *
     BUG_ON(xsave_cntxt_size < XSTATE_AREA_MIN_SIZE);
 
     /* XSAVE/XRSTOR requires the save area be 64-byte-boundary aligned. */
-    save_area = _xzalloc(xsave_cntxt_size, 64);
+    BUILD_BUG_ON(__alignof(*save_area) < 64);
+    save_area = _xzalloc(xsave_cntxt_size, __alignof(*save_area));
     if ( save_area == NULL )
         return -ENOMEM;
 
--- unstable.orig/xen/include/asm-x86/xstate.h	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/include/asm-x86/xstate.h	2016-01-25 11:33:20.000000000 +0100
@@ -48,9 +48,9 @@ extern u64 xfeature_mask;
 extern unsigned int *xstate_sizes;
 
 /* extended state save area */
-struct __packed __attribute__((aligned (64))) xsave_struct
+struct __attribute__((aligned (64))) xsave_struct
 {
-    union {                                  /* FPU/MMX, SSE */
+    union __attribute__((aligned(16))) {     /* FPU/MMX, SSE */
         char x[512];
         struct {
             uint16_t fcw;

[-- Attachment #4: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-25 13:42                         ` Jan Beulich
@ 2016-01-26 11:58                           ` Dario Faggioli
  2016-01-26 12:51                           ` Harmandeep Kaur
  1 sibling, 0 replies; 46+ messages in thread
From: Dario Faggioli @ 2016-01-26 11:58 UTC (permalink / raw)
  To: Jan Beulich, Harmandeep Kaur; +Cc: Andrew Cooper, xen-devel, Shuai Ruan


[-- Attachment #1.1: Type: text/plain, Size: 1252 bytes --]

On Mon, 2016-01-25 at 06:42 -0700, Jan Beulich wrote:
> > > > On 21.01.16 at 16:14, <dario.faggioli@citrix.com> wrote:
> > On Wed, 2016-01-20 at 03:06 -0700, Jan Beulich wrote:

> > > But of course another question then is why the XRSTORS faults
> > > in the first place. I guess we'll need a debugging patch to dump
> > > the full state to understand that.
> > > 
> > If someone can produce and send such patch, I'm sure Harmandeep
> > will be
> > happy to give it a try on her hardware.
> 
> So here you go. Instead of a debugging one, I hope I have at
> once fixed the issue in a suitable way. Whether we'd like to keep
> the debugging output we can decide later on.
> 
Great, thanks!

> Both patches need to be applied; while the order shouldn't matter,
> the alignment one is a prereq to the actual change.
> 
Ok. Harmandeep, can you give these patches a try when you get a chance?
(contact me, by email or on IRC, if you need help doing so).

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-25 13:42                         ` Jan Beulich
  2016-01-26 11:58                           ` Dario Faggioli
@ 2016-01-26 12:51                           ` Harmandeep Kaur
  2016-01-26 13:13                             ` Harmandeep Kaur
  2016-01-26 13:22                             ` Jan Beulich
  1 sibling, 2 replies; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-26 12:51 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

Hi guys,

I tried the patch and I am very happy to inform you
all that the patch has solved my problem. Now I am
able to boot Xen without disabling XSAVE. I have
full log of boot at http://paste2.org/gVW0Z9nm (if
any one is interested. also "XXX Hello, this is my
first mod :)" is printed by my mod, so ignore that
one).

Thank you guys for your help.
Regards,
Harman

On Mon, Jan 25, 2016 at 7:12 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 21.01.16 at 16:14, <dario.faggioli@citrix.com> wrote:
>> On Wed, 2016-01-20 at 03:06 -0700, Jan Beulich wrote:
>>> > > > On 19.01.16 at 20:55, <write.harmandeep@gmail.com> wrote:
>>> >
>>> > $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
>>> > 'xen/xen/arch/x86/xstate.c:387' which again points to
>>> > xsave. Also, adding 'xsave=0' makes it boot just fine.
>>>
>>> Ah, I think I see the issue: We're zeroing the entire save area in
>>> the fixup code, which will make XRSTORS fault unconditionally.
>>> Shuai, would you please look into possible ways of fixing this?
>>> Just setting the compressed flag may not be enough, and falling
>>> back to plain XRSTOR doesn't seem to be an option either - I'm in
>>> particular worried that the current model of zeroing everything
>>> is bogus with the growing number of different components which
>>> get loaded here.
>>>
>>> But of course another question then is why the XRSTORS faults
>>> in the first place. I guess we'll need a debugging patch to dump
>>> the full state to understand that.
>>>
>> If someone can produce and send such patch, I'm sure Harmandeep will be
>> happy to give it a try on her hardware.
>
> So here you go. Instead of a debugging one, I hope I have at
> once fixed the issue in a suitable way. Whether we'd like to keep
> the debugging output we can decide later on.
>
> Both patches need to be applied; while the order shouldn't matter,
> the alignment one is a prereq to the actual change.
>
> Jan
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 12:51                           ` Harmandeep Kaur
@ 2016-01-26 13:13                             ` Harmandeep Kaur
  2016-01-26 13:23                               ` Jan Beulich
  2016-01-26 13:22                             ` Jan Beulich
  1 sibling, 1 reply; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-26 13:13 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

The patch as I already said is letting me boot
into the Xen, but the system is now resetting
stating XSAVE as the cause. I have attached
links to two cases where system was reset as
the result. I don't think that problem is fully
solved yet.
http://paste2.org/Ky56Z92g
http://paste2.org/3hcbG6L7

Regards,
Harman

On Tue, Jan 26, 2016 at 6:21 PM, Harmandeep Kaur
<write.harmandeep@gmail.com> wrote:
> Hi guys,
>
> I tried the patch and I am very happy to inform you
> all that the patch has solved my problem. Now I am
> able to boot Xen without disabling XSAVE. I have
> full log of boot at http://paste2.org/gVW0Z9nm (if
> any one is interested. also "XXX Hello, this is my
> first mod :)" is printed by my mod, so ignore that
> one).
>
> Thank you guys for your help.
> Regards,
> Harman
>
> On Mon, Jan 25, 2016 at 7:12 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 21.01.16 at 16:14, <dario.faggioli@citrix.com> wrote:
>>> On Wed, 2016-01-20 at 03:06 -0700, Jan Beulich wrote:
>>>> > > > On 19.01.16 at 20:55, <write.harmandeep@gmail.com> wrote:
>>>> >
>>>> > $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
>>>> > 'xen/xen/arch/x86/xstate.c:387' which again points to
>>>> > xsave. Also, adding 'xsave=0' makes it boot just fine.
>>>>
>>>> Ah, I think I see the issue: We're zeroing the entire save area in
>>>> the fixup code, which will make XRSTORS fault unconditionally.
>>>> Shuai, would you please look into possible ways of fixing this?
>>>> Just setting the compressed flag may not be enough, and falling
>>>> back to plain XRSTOR doesn't seem to be an option either - I'm in
>>>> particular worried that the current model of zeroing everything
>>>> is bogus with the growing number of different components which
>>>> get loaded here.
>>>>
>>>> But of course another question then is why the XRSTORS faults
>>>> in the first place. I guess we'll need a debugging patch to dump
>>>> the full state to understand that.
>>>>
>>> If someone can produce and send such patch, I'm sure Harmandeep will be
>>> happy to give it a try on her hardware.
>>
>> So here you go. Instead of a debugging one, I hope I have at
>> once fixed the issue in a suitable way. Whether we'd like to keep
>> the debugging output we can decide later on.
>>
>> Both patches need to be applied; while the order shouldn't matter,
>> the alignment one is a prereq to the actual change.
>>
>> Jan
>>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 12:51                           ` Harmandeep Kaur
  2016-01-26 13:13                             ` Harmandeep Kaur
@ 2016-01-26 13:22                             ` Jan Beulich
  1 sibling, 0 replies; 46+ messages in thread
From: Jan Beulich @ 2016-01-26 13:22 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

>>> On 26.01.16 at 13:51, <write.harmandeep@gmail.com> wrote:
> I tried the patch and I am very happy to inform you
> all that the patch has solved my problem. Now I am
> able to boot Xen without disabling XSAVE. I have
> full log of boot at http://paste2.org/gVW0Z9nm (if
> any one is interested. also "XXX Hello, this is my
> first mod :)" is printed by my mod, so ignore that
> one).

Thanks for trying it out, but while I'm glad it helped I'm afraid
we're not done here. With (for every vCPU)

(XEN) traps.c:3290: GPF (0000): ffff82d0801c1d08 -> ffff82d080252e5c
(XEN) d0v1 fault#1: mxcsr=00001f80
(XEN) d0v1 xs=0000000000000000 xc=0000000000000000
(XEN) d0v1 r0=0000000000000000 r1=0000000000000000
(XEN) d0v1 r2=0000000000000000 r3=0000000000000000
(XEN) d0v1 r4=0000000000000000 r5=0000000000000000
(XEN) traps.c:3290: GPF (0000): ffff82d0801c1d08 -> ffff82d080252e5c
(XEN) d0v1 fault#2: mxcsr=00001f80
(XEN) d0v1 xs=0000000000000000 xc=0000000000000000
(XEN) d0v1 r0=0000000000000000 r1=0000000000000000
(XEN) d0v1 r2=0000000000000000 r3=0000000000000000
(XEN) d0v1 r4=0000000000000000 r5=0000000000000000

it continues to be unclear why bit 63 in the value printed as
xc= isn't set from the beginning. Or wait, I think I see where
the problem is. Here's a 3rd patch, to try together with the
other two. The expectation would be for the above log
messages to then disappear altogether. (And then the patch
should also work on its own, i.e. with the other two removed
again.) Please let us know.

Thanks, Jan

x86/xstate: don't unintentionally clear compaction bit

When the VGCF_I387_VALID flag is clear in arch_set_info_guest()'s input
we must not clear the compaction bit when using XSAVES/XRSTORS. Split
initialization of xcomp_bv from the other FPU/SSE/AVX related state
setup in this function.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/domain.c
+++ unstable/xen/arch/x86/domain.c
@@ -922,15 +922,10 @@ int arch_set_info_guest(
     {
         memcpy(v->arch.fpu_ctxt, &c.nat->fpu_ctxt, sizeof(c.nat->fpu_ctxt));
         if ( v->arch.xsave_area )
-        {
             v->arch.xsave_area->xsave_hdr.xstate_bv = XSTATE_FP_SSE;
-            v->arch.xsave_area->xsave_hdr.xcomp_bv =
-                cpu_has_xsaves ? XSTATE_COMPACTION_ENABLED : 0;
-        }
     }
     else if ( v->arch.xsave_area )
-        memset(&v->arch.xsave_area->xsave_hdr, 0,
-               sizeof(v->arch.xsave_area->xsave_hdr));
+        v->arch.xsave_area->xsave_hdr.xstate_bv = 0;
     else
     {
         typeof(v->arch.xsave_area->fpu_sse) *fpu_sse = v->arch.fpu_ctxt;
@@ -939,6 +934,11 @@ int arch_set_info_guest(
         fpu_sse->fcw = FCW_DEFAULT;
         fpu_sse->mxcsr = MXCSR_DEFAULT;
     }
+    if ( cpu_has_xsaves )
+    {
+        ASSERT(v->arch.xsave_area);
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = XSTATE_COMPACTION_ENABLED;
+    }
 
     if ( !compat )
     {

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 13:13                             ` Harmandeep Kaur
@ 2016-01-26 13:23                               ` Jan Beulich
  2016-01-26 17:01                                 ` Harmandeep Kaur
  0 siblings, 1 reply; 46+ messages in thread
From: Jan Beulich @ 2016-01-26 13:23 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

>>> On 26.01.16 at 14:13, <write.harmandeep@gmail.com> wrote:
> The patch as I already said is letting me boot
> into the Xen, but the system is now resetting
> stating XSAVE as the cause. I have attached
> links to two cases where system was reset as
> the result. I don't think that problem is fully
> solved yet.
> http://paste2.org/Ky56Z92g 
> http://paste2.org/3hcbG6L7 

I guess this would also get resolved by the 3rd patch just sent.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 13:23                               ` Jan Beulich
@ 2016-01-26 17:01                                 ` Harmandeep Kaur
  2016-01-26 17:23                                   ` Jan Beulich
  0 siblings, 1 reply; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-26 17:01 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

I tried 3rd patch together with earlier two. I'm
afraid the problem is not solved completely.
Full log goes here, http://paste2.org/KEAetMHb

Regards,
Harmandeep

On Tue, Jan 26, 2016 at 6:53 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 26.01.16 at 14:13, <write.harmandeep@gmail.com> wrote:
>> The patch as I already said is letting me boot
>> into the Xen, but the system is now resetting
>> stating XSAVE as the cause. I have attached
>> links to two cases where system was reset as
>> the result. I don't think that problem is fully
>> solved yet.
>> http://paste2.org/Ky56Z92g
>> http://paste2.org/3hcbG6L7
>
> I guess this would also get resolved by the 3rd patch just sent.
>
> Jan
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 17:01                                 ` Harmandeep Kaur
@ 2016-01-26 17:23                                   ` Jan Beulich
  2016-01-26 18:02                                     ` Harmandeep Kaur
  0 siblings, 1 reply; 46+ messages in thread
From: Jan Beulich @ 2016-01-26 17:23 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

>>> On 26.01.16 at 18:01, <write.harmandeep@gmail.com> wrote:
> I tried 3rd patch together with earlier two. I'm
> afraid the problem is not solved completely.
> Full log goes here, http://paste2.org/KEAetMHb 

Well, wait - we can't solve all problems at once. The crash here
is in the context of do_domctl(), i.e. makes me assume that the
system booted up fine (and the problematic log messages are
gone). So the original issue is fixed. Now for this second issue,
would you mind first of all telling us what exactly it is you're
doing when the crash occurs? Because, as you hopefully
understand, such information often helps understanding where
and/or why things are going wrong.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 17:23                                   ` Jan Beulich
@ 2016-01-26 18:02                                     ` Harmandeep Kaur
  2016-01-26 18:28                                       ` Dario Faggioli
                                                         ` (2 more replies)
  0 siblings, 3 replies; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-26 18:02 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

Last time, I did absolutely nothing. System was idle
and it crashed just after the login. Now, I booted the
system again and this time, there is no reset. But,
performance of the system is very slow. Browser
(Mozilla Firefox) freezes a lot. Also, before applying
patches, when I used to disabe xsave it resulted in
same kind of performance issues. And the following
is still present in the log.

(XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
(XEN) d1v1 fault#1: mxcsr=00001f80
(XEN) d1v1 xs=0000000000000003 xc=8000000000000000
(XEN) d1v1 r0=0000000000000000 r1=0000000000000000
(XEN) d1v1 r2=0000000000000000 r3=0000000000000000
(XEN) d1v1 r4=0000000000000000 r5=0000000000000000
(XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
(XEN) d1v1 fault#2: mxcsr=00001f80
(XEN) d1v1 xs=0000000000000000 xc=0000000000000000
(XEN) d1v1 r0=0000000000000000 r1=0000000000000000
(XEN) d1v1 r2=0000000000000000 r3=0000000000000000
(XEN) d1v1 r4=0000000000000000 r5=0000000000000000

Full log here: http://paste2.org/C8WpyKOg

Regards,
Harmandeep

On Tue, Jan 26, 2016 at 10:53 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 26.01.16 at 18:01, <write.harmandeep@gmail.com> wrote:
>> I tried 3rd patch together with earlier two. I'm
>> afraid the problem is not solved completely.
>> Full log goes here, http://paste2.org/KEAetMHb
>
> Well, wait - we can't solve all problems at once. The crash here
> is in the context of do_domctl(), i.e. makes me assume that the
> system booted up fine (and the problematic log messages are
> gone). So the original issue is fixed. Now for this second issue,
> would you mind first of all telling us what exactly it is you're
> doing when the crash occurs? Because, as you hopefully
> understand, such information often helps understanding where
> and/or why things are going wrong.
>
> Jan
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 18:02                                     ` Harmandeep Kaur
@ 2016-01-26 18:28                                       ` Dario Faggioli
  2016-01-26 18:36                                         ` Harmandeep Kaur
  2016-01-27  8:24                                       ` Jan Beulich
  2016-01-27 13:12                                       ` Jan Beulich
  2 siblings, 1 reply; 46+ messages in thread
From: Dario Faggioli @ 2016-01-26 18:28 UTC (permalink / raw)
  To: Harmandeep Kaur, Jan Beulich; +Cc: Andrew Cooper, xen-devel, Shuai Ruan


[-- Attachment #1.1: Type: text/plain, Size: 3131 bytes --]

On Tue, 2016-01-26 at 23:32 +0530, Harmandeep Kaur wrote:
> Last time, I did absolutely nothing. System was idle
> and it crashed just after the login. Now, I booted the
> system again and this time, there is no reset. But,
> performance of the system is very slow. Browser
> (Mozilla Firefox) freezes a lot. Also, before applying
> patches, when I used to disabe xsave it resulted in
> same kind of performance issues. 
>
Mmm... are you sure the performance is actually affected by "xsave=0",
and/or by Jan's patch? It's hard to check, as without at least one of
them the box does not boot, but I don't think the things (e.g., Firefox
freezing or not starting) are necessarily related.

In particular, you have in your Xen boot parameters list, this item:

 "dom0_mem=1024M,max:1024M"

This means that, in dom0, you will "only" have 1GB of RAM available.
And if you just login after boot and start Firefox, dom0 is where
Firefox is going to be running... and 1G, that Firefox will have to
share with the rest of Linux running as dom0, may be too few RAM for
it. And in fact, in your last log, we see this (from dom0, not from
Xen!):

[  851.644443] Out of memory: Kill process 1945 (firefox) score 325 or sacrifice child
[  851.644461] Killed process 1945 (firefox) total-vm:1228008kB, anon-rss:305536kB, file-rss:0kB

If you want to run a graphical environment on that test box, and browse
with Firefox, then you should increase the amount of RAM you allow dom0
to use. When I suggested you to use 1024, I was assuming (given how
your work environment is setup) you were not going to do any such
thing.

> And the following
> is still present in the log.
> 
> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
> (XEN) d1v1 fault#1: mxcsr=00001f80
> (XEN) d1v1 xs=0000000000000003 xc=8000000000000000
> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
> (XEN) d1v1 fault#2: mxcsr=00001f80
> (XEN) d1v1 xs=0000000000000000 xc=0000000000000000
> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
> 
Mmm... and this is with all Jan's patches applied?

So, just to make sure we understand each others, you're saying that,
again with all patches applied, and with you not doing anything
significantly different between a) and b) below, the system either:

 a) crashes right after login, like this: http://paste2.org/KEAetMHb

 b) does not crash (you're even able to try starting Firefox), but Xen
    produces the following output: http://paste2.org/C8WpyKOg

Is this correct?

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 18:28                                       ` Dario Faggioli
@ 2016-01-26 18:36                                         ` Harmandeep Kaur
  0 siblings, 0 replies; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-26 18:36 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Andrew Cooper, xen-devel, Jan Beulich, Shuai Ruan

On Tue, Jan 26, 2016 at 11:58 PM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> On Tue, 2016-01-26 at 23:32 +0530, Harmandeep Kaur wrote:
>> Last time, I did absolutely nothing. System was idle
>> and it crashed just after the login. Now, I booted the
>> system again and this time, there is no reset. But,
>> performance of the system is very slow. Browser
>> (Mozilla Firefox) freezes a lot. Also, before applying
>> patches, when I used to disabe xsave it resulted in
>> same kind of performance issues.
>>
> Mmm... are you sure the performance is actually affected by "xsave=0",
> and/or by Jan's patch? It's hard to check, as without at least one of
> them the box does not boot, but I don't think the things (e.g., Firefox
> freezing or not starting) are necessarily related.
>
> In particular, you have in your Xen boot parameters list, this item:
>
>  "dom0_mem=1024M,max:1024M"
>
> This means that, in dom0, you will "only" have 1GB of RAM available.
> And if you just login after boot and start Firefox, dom0 is where
> Firefox is going to be running... and 1G, that Firefox will have to
> share with the rest of Linux running as dom0, may be too few RAM for
> it. And in fact, in your last log, we see this (from dom0, not from
> Xen!):
>
> [  851.644443] Out of memory: Kill process 1945 (firefox) score 325 or sacrifice child
> [  851.644461] Killed process 1945 (firefox) total-vm:1228008kB, anon-rss:305536kB, file-rss:0kB
>
> If you want to run a graphical environment on that test box, and browse
> with Firefox, then you should increase the amount of RAM you allow dom0
> to use. When I suggested you to use 1024, I was assuming (given how
> your work environment is setup) you were not going to do any such
> thing.
>
Actually I didn't notice this performance issue before this xsave
bug, maybe we added this line later on. Anyways I can now
check this by increasing the memory.

>> And the following
>> is still present in the log.
>>
>> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
>> (XEN) d1v1 fault#1: mxcsr=00001f80
>> (XEN) d1v1 xs=0000000000000003 xc=8000000000000000
>> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
>> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
>> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
>> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
>> (XEN) d1v1 fault#2: mxcsr=00001f80
>> (XEN) d1v1 xs=0000000000000000 xc=0000000000000000
>> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
>> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
>> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
>>
> Mmm... and this is with all Jan's patches applied?

Yes, all three patches applied.

> So, just to make sure we understand each others, you're saying that,
> again with all patches applied, and with you not doing anything
> significantly different between a) and b) below, the system either:
>
>  a) crashes right after login, like this: http://paste2.org/KEAetMHb
>
>  b) does not crash (you're even able to try starting Firefox), but Xen
>     produces the following output: http://paste2.org/C8WpyKOg
>
> Is this correct?

Yes. And I tried to boot several times and around 70-80% of
times system crashes right after the login as in case 'a'.

Regards,
Harmandeep

> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 18:02                                     ` Harmandeep Kaur
  2016-01-26 18:28                                       ` Dario Faggioli
@ 2016-01-27  8:24                                       ` Jan Beulich
  2016-01-27 13:12                                       ` Jan Beulich
  2 siblings, 0 replies; 46+ messages in thread
From: Jan Beulich @ 2016-01-27  8:24 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

>>> On 26.01.16 at 19:02, <write.harmandeep@gmail.com> wrote:
> Last time, I did absolutely nothing. System was idle
> and it crashed just after the login. Now, I booted the
> system again and this time, there is no reset. But,
> performance of the system is very slow. Browser
> (Mozilla Firefox) freezes a lot. Also, before applying
> patches, when I used to disabe xsave it resulted in
> same kind of performance issues. And the following
> is still present in the log.
> 
> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
> (XEN) d1v1 fault#1: mxcsr=00001f80
> (XEN) d1v1 xs=0000000000000003 xc=8000000000000000
> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
> (XEN) d1v1 fault#2: mxcsr=00001f80
> (XEN) d1v1 xs=0000000000000000 xc=0000000000000000
> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
> 
> Full log here: http://paste2.org/C8WpyKOg 

This is not _still_ present, but that's another, different instance,
now during guest creation. Please try to be precise in your
wording. Namely, in this case, you again fail to mention what
kind of guest you're now creating that causes this to re-appear.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-26 18:02                                     ` Harmandeep Kaur
  2016-01-26 18:28                                       ` Dario Faggioli
  2016-01-27  8:24                                       ` Jan Beulich
@ 2016-01-27 13:12                                       ` Jan Beulich
  2016-01-27 13:28                                         ` Harmandeep Kaur
  2 siblings, 1 reply; 46+ messages in thread
From: Jan Beulich @ 2016-01-27 13:12 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

[-- Attachment #1: Type: text/plain, Size: 1986 bytes --]

>>> On 26.01.16 at 19:02, <write.harmandeep@gmail.com> wrote:
> Last time, I did absolutely nothing. System was idle
> and it crashed just after the login. Now, I booted the
> system again and this time, there is no reset. But,
> performance of the system is very slow. Browser
> (Mozilla Firefox) freezes a lot. Also, before applying
> patches, when I used to disabe xsave it resulted in
> same kind of performance issues. And the following
> is still present in the log.
> 
> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
> (XEN) d1v1 fault#1: mxcsr=00001f80
> (XEN) d1v1 xs=0000000000000003 xc=8000000000000000
> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
> (XEN) d1v1 fault#2: mxcsr=00001f80
> (XEN) d1v1 xs=0000000000000000 xc=0000000000000000
> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
> 
> Full log here: http://paste2.org/C8WpyKOg 

This together with ...

> On Tue, Jan 26, 2016 at 10:53 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 26.01.16 at 18:01, <write.harmandeep@gmail.com> wrote:
>>> I tried 3rd patch together with earlier two. I'm
>>> afraid the problem is not solved completely.
>>> Full log goes here, http://paste2.org/KEAetMHb 

... this, and both being apparently the same build makes me suspect
uninitialized data to get passed in from the tool stack. But that's a
secondary issue for now. For the immediate problem here are four
patches replacing the three earlier ones (I think only one of them is
unchanged, so be sure to remove the old ones first).

Their intended ordering is:
x86-xsaves-init.patch
x86-xstate-align.patch
x86-xrstors-fault.patch
x86-xstate-validate.patch

Jan


[-- Attachment #2: x86-xrstors-fault.patch --]
[-- Type: text/plain, Size: 6925 bytes --]

x86/xstate: fix fault behavior on XRSTORS

XRSTORS unconditionally faults when xcomp_bv has bit 63 clear. Instead
of just fixing this issue, overhaul the fault recovery code, which -
one of the many mistakes made when xstate support got introduced - was
blindly mirroring that accompanying FXRSTOR, neglecting the fact that
XRSTOR{,S} aren't all-or-nothing instructions. The new code, first of
all, does all the recovery actions in C, simplifying the inline
assembly used. And it does its work in a multi-stage fashion: Upon
first seeing a fault, state fixups get applied strictly based on what
architecturally may cause #GP. When seeing another fault despite the
fixups done, state gets fully reset. A third fault would then lead to
crashing the domain (instead of hanging the hypervisor in an infinite
loop of recurring faults).

Reported-by: Harmandeep Kaur <write.harmandeep@gmail.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/xstate.c	2016-01-25 09:35:12.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-27 10:23:06.000000000 +0100
@@ -29,6 +29,8 @@ unsigned int *__read_mostly xstate_sizes
 static unsigned int __read_mostly xstate_features;
 static unsigned int __read_mostly xstate_comp_offsets[sizeof(xfeature_mask)*8];
 
+static uint32_t __read_mostly mxcsr_mask = MXCSR_DEFAULT;
+
 /* Cached xcr0 for fast read */
 static DEFINE_PER_CPU(uint64_t, xcr0);
 
@@ -342,6 +344,7 @@ void xrstor(struct vcpu *v, uint64_t mas
     uint32_t hmask = mask >> 32;
     uint32_t lmask = mask;
     struct xsave_struct *ptr = v->arch.xsave_area;
+    unsigned int faults, prev_faults;
 
     /*
      * AMD CPUs don't save/restore FDP/FIP/FOP unless an exception
@@ -361,35 +364,85 @@ void xrstor(struct vcpu *v, uint64_t mas
     /*
      * XRSTOR can fault if passed a corrupted data block. We handle this
      * possibility, which may occur if the block was passed to us by control
-     * tools or through VCPUOP_initialise, by silently clearing the block.
+     * tools or through VCPUOP_initialise, by silently adjusting state.
      */
-    switch ( __builtin_expect(ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET], 8) )
+    for ( prev_faults = faults = 0; ; prev_faults = faults )
     {
+        switch ( __builtin_expect(ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET], 8) )
+        {
 #define XRSTOR(pfx) \
         alternative_io("1: .byte " pfx "0x0f,0xae,0x2f\n" \
+                       "3:\n" \
                        "   .section .fixup,\"ax\"\n" \
-                       "2: mov %[size],%%ecx\n" \
-                       "   xor %[lmask_out],%[lmask_out]\n" \
-                       "   rep stosb\n" \
-                       "   lea %[mem],%[ptr]\n" \
-                       "   mov %[lmask_in],%[lmask_out]\n" \
-                       "   jmp 1b\n" \
+                       "2: inc%z[faults] %[faults]\n" \
+                       "   jmp 3b\n" \
                        "   .previous\n" \
                        _ASM_EXTABLE(1b, 2b), \
                        ".byte " pfx "0x0f,0xc7,0x1f\n", \
                        X86_FEATURE_XSAVES, \
-                       ASM_OUTPUT2([ptr] "+&D" (ptr), [lmask_out] "+&a" (lmask)), \
-                       [mem] "m" (*ptr), [lmask_in] "g" (lmask), \
-                       [hmask] "d" (hmask), [size] "m" (xsave_cntxt_size) \
-                       : "ecx")
-
-    default:
-        XRSTOR("0x48,");
-        break;
-    case 4: case 2:
-        XRSTOR("");
-        break;
+                       ASM_OUTPUT2([mem] "+m" (*ptr), [faults] "+g" (faults)), \
+                       [lmask] "a" (lmask), [hmask] "d" (hmask), \
+                       [ptr] "D" (ptr))
+
+        default:
+            XRSTOR("0x48,");
+            break;
+        case 4: case 2:
+            XRSTOR("");
+            break;
 #undef XRSTOR
+        }
+        if ( likely(faults == prev_faults) )
+            break;
+#ifndef NDEBUG
+        gprintk(XENLOG_WARNING, "fault#%u: mxcsr=%08x\n",
+                faults, ptr->fpu_sse.mxcsr);
+        gprintk(XENLOG_WARNING, "xs=%016lx xc=%016lx\n",
+                ptr->xsave_hdr.xstate_bv, ptr->xsave_hdr.xcomp_bv);
+        gprintk(XENLOG_WARNING, "r0=%016lx r1=%016lx\n",
+                ptr->xsave_hdr.reserved[0], ptr->xsave_hdr.reserved[1]);
+        gprintk(XENLOG_WARNING, "r2=%016lx r3=%016lx\n",
+                ptr->xsave_hdr.reserved[2], ptr->xsave_hdr.reserved[3]);
+        gprintk(XENLOG_WARNING, "r4=%016lx r5=%016lx\n",
+                ptr->xsave_hdr.reserved[4], ptr->xsave_hdr.reserved[5]);
+#endif
+        switch ( faults )
+        {
+        case 1:
+            /* Stage 1: Reset state to be loaded. */
+            ptr->xsave_hdr.xstate_bv &= ~mask;
+            /*
+             * Also try to eliminate fault reasons, even if this shouldn't be
+             * needed here (other code should ensure the sanity of the data).
+             */
+            if ( ((mask & XSTATE_SSE) ||
+                  ((mask & XSTATE_YMM) &&
+                   !(ptr->xsave_hdr.xcomp_bv & XSTATE_COMPACTION_ENABLED))) )
+                ptr->fpu_sse.mxcsr &= mxcsr_mask;
+            if ( cpu_has_xsaves || cpu_has_xsavec )
+            {
+                ptr->xsave_hdr.xcomp_bv &= this_cpu(xcr0) | this_cpu(xss);
+                ptr->xsave_hdr.xstate_bv &= ptr->xsave_hdr.xcomp_bv;
+                ptr->xsave_hdr.xcomp_bv |= XSTATE_COMPACTION_ENABLED;
+            }
+            else
+            {
+                ptr->xsave_hdr.xstate_bv &= this_cpu(xcr0);
+                ptr->xsave_hdr.xcomp_bv = 0;
+            }
+            memset(ptr->xsave_hdr.reserved, 0, sizeof(ptr->xsave_hdr.reserved));
+            continue;
+        case 2:
+            /* Stage 2: Reset all state. */
+            ptr->fpu_sse.mxcsr = MXCSR_DEFAULT;
+            ptr->xsave_hdr.xstate_bv = 0;
+            ptr->xsave_hdr.xcomp_bv = cpu_has_xsaves
+                                      ? XSTATE_COMPACTION_ENABLED : 0;
+            continue;
+        default:
+            domain_crash(current->domain);
+            break;
+        }
     }
 }
 
@@ -496,6 +549,8 @@ void xstate_init(struct cpuinfo_x86 *c)
 
     if ( bsp )
     {
+        static typeof(current->arch.xsave_area->fpu_sse) __initdata ctxt;
+
         xfeature_mask = feature_mask;
         /*
          * xsave_cntxt_size is the max size required by enabled features.
@@ -504,6 +559,10 @@ void xstate_init(struct cpuinfo_x86 *c)
         xsave_cntxt_size = _xstate_ctxt_size(feature_mask);
         printk("%s: using cntxt_size: %#x and states: %#"PRIx64"\n",
             __func__, xsave_cntxt_size, xfeature_mask);
+
+        asm ( "fxsave %0" : "=m" (ctxt) );
+        if ( ctxt.mxcsr_mask )
+            mxcsr_mask = ctxt.mxcsr_mask;
     }
     else
     {

[-- Attachment #3: x86-xsaves-init.patch --]
[-- Type: text/plain, Size: 3255 bytes --]

x86/xstate: fix xcomp_bv initialization

We must not clear the compaction bit when using XSAVES/XRSTORS. And
we need to guarantee that xcomp_bv never has any bits clear which
are set in xstate_bv (which requires partly undoing commit 83ae0bb226
["x86/xsave: simplify xcomp_bv initialization"]). Split initialization
of xcomp_bv from the other FPU/SSE/AVX related state setup in
arch_set_info_guest() and hvm_load_cpu_ctxt().

Reported-by: Harmandeep Kaur <write.harmandeep@gmail.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/domain.c	2016-01-27 09:29:50.000000000 +0100
+++ unstable/xen/arch/x86/domain.c	2016-01-27 09:52:37.000000000 +0100
@@ -922,15 +922,10 @@ int arch_set_info_guest(
     {
         memcpy(v->arch.fpu_ctxt, &c.nat->fpu_ctxt, sizeof(c.nat->fpu_ctxt));
         if ( v->arch.xsave_area )
-        {
             v->arch.xsave_area->xsave_hdr.xstate_bv = XSTATE_FP_SSE;
-            v->arch.xsave_area->xsave_hdr.xcomp_bv =
-                cpu_has_xsaves ? XSTATE_COMPACTION_ENABLED : 0;
-        }
     }
     else if ( v->arch.xsave_area )
-        memset(&v->arch.xsave_area->xsave_hdr, 0,
-               sizeof(v->arch.xsave_area->xsave_hdr));
+        v->arch.xsave_area->xsave_hdr.xstate_bv = 0;
     else
     {
         typeof(v->arch.xsave_area->fpu_sse) *fpu_sse = v->arch.fpu_ctxt;
@@ -939,6 +934,14 @@ int arch_set_info_guest(
         fpu_sse->fcw = FCW_DEFAULT;
         fpu_sse->mxcsr = MXCSR_DEFAULT;
     }
+    if ( cpu_has_xsaves )
+    {
+        ASSERT(v->arch.xsave_area);
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = XSTATE_COMPACTION_ENABLED |
+            v->arch.xsave_area->xsave_hdr.xstate_bv;
+    }
+    else if ( v->arch.xsave_area )
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = 0;
 
     if ( !compat )
     {
--- unstable.orig/xen/arch/x86/hvm/hvm.c	2015-12-18 12:22:20.000000000 +0100
+++ unstable/xen/arch/x86/hvm/hvm.c	2016-01-27 09:52:26.000000000 +0100
@@ -2094,11 +2094,17 @@ static int hvm_load_cpu_ctxt(struct doma
 
         memcpy(v->arch.xsave_area, ctxt.fpu_regs, sizeof(ctxt.fpu_regs));
         xsave_area->xsave_hdr.xstate_bv = XSTATE_FP_SSE;
-        xsave_area->xsave_hdr.xcomp_bv =
-            cpu_has_xsaves ? XSTATE_COMPACTION_ENABLED : 0;
     }
     else
         memcpy(v->arch.fpu_ctxt, ctxt.fpu_regs, sizeof(ctxt.fpu_regs));
+    if ( cpu_has_xsaves )
+    {
+        ASSERT(v->arch.xsave_area);
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = XSTATE_COMPACTION_ENABLED |
+            v->arch.xsave_area->xsave_hdr.xstate_bv;
+    }
+    else if ( v->arch.xsave_area )
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = 0;
 
     v->arch.user_regs.eax = ctxt.rax;
     v->arch.user_regs.ebx = ctxt.rbx;
@@ -5488,8 +5494,8 @@ void hvm_vcpu_reset_state(struct vcpu *v
     if ( v->arch.xsave_area )
     {
         v->arch.xsave_area->xsave_hdr.xstate_bv = XSTATE_FP;
-        v->arch.xsave_area->xsave_hdr.xcomp_bv =
-            cpu_has_xsaves ? XSTATE_COMPACTION_ENABLED : 0;
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = cpu_has_xsaves
+            ? XSTATE_COMPACTION_ENABLED | XSTATE_FP : 0;
     }
 
     v->arch.vgc_flags = VGCF_online;

[-- Attachment #4: x86-xstate-align.patch --]
[-- Type: text/plain, Size: 2384 bytes --]

x86: adjust xsave structure attributes

The packed attribute was pointlessly used here - there are no
misaligned fields, and hence even if the attribute took effect, it
would at best lead to the compiler generating worse code.

At the same time specify the required alignment of the fpu_sse sub-
structure, such that the various typeof() uses on that field obtain
pointers to properly aligned memory (knowledge which a compiler may
want to make use of).

Also add suitable build-time checks.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/i387.c	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/arch/x86/i387.c	2016-01-25 09:35:36.000000000 +0100
@@ -277,7 +277,9 @@ int vcpu_init_fpu(struct vcpu *v)
     }
     else
     {
-        v->arch.fpu_ctxt = _xzalloc(sizeof(v->arch.xsave_area->fpu_sse), 16);
+        BUILD_BUG_ON(__alignof(v->arch.xsave_area->fpu_sse) < 16);
+        v->arch.fpu_ctxt = _xzalloc(sizeof(v->arch.xsave_area->fpu_sse),
+                                    __alignof(v->arch.xsave_area->fpu_sse));
         if ( v->arch.fpu_ctxt )
         {
             typeof(v->arch.xsave_area->fpu_sse) *fpu_sse = v->arch.fpu_ctxt;
--- unstable.orig/xen/arch/x86/xstate.c	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-25 09:35:12.000000000 +0100
@@ -414,7 +414,8 @@ int xstate_alloc_save_area(struct vcpu *
     BUG_ON(xsave_cntxt_size < XSTATE_AREA_MIN_SIZE);
 
     /* XSAVE/XRSTOR requires the save area be 64-byte-boundary aligned. */
-    save_area = _xzalloc(xsave_cntxt_size, 64);
+    BUILD_BUG_ON(__alignof(*save_area) < 64);
+    save_area = _xzalloc(xsave_cntxt_size, __alignof(*save_area));
     if ( save_area == NULL )
         return -ENOMEM;
 
--- unstable.orig/xen/include/asm-x86/xstate.h	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/include/asm-x86/xstate.h	2016-01-25 11:33:20.000000000 +0100
@@ -48,9 +48,9 @@ extern u64 xfeature_mask;
 extern unsigned int *xstate_sizes;
 
 /* extended state save area */
-struct __packed __attribute__((aligned (64))) xsave_struct
+struct __attribute__((aligned (64))) xsave_struct
 {
-    union {                                  /* FPU/MMX, SSE */
+    union __attribute__((aligned(16))) {     /* FPU/MMX, SSE */
         char x[512];
         struct {
             uint16_t fcw;

[-- Attachment #5: x86-xstate-validate.patch --]
[-- Type: text/plain, Size: 4969 bytes --]

x86/xstate: extend validation to cover full header

Since we never hand out compacted state, at least for now we're also
not going to accept such.

Reported-by: Harmandeep Kaur <write.harmandeep@gmail.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/domctl.c	2016-01-27 10:54:16.000000000 +0100
+++ unstable/xen/arch/x86/domctl.c	2016-01-27 10:44:52.000000000 +0100
@@ -958,7 +958,7 @@ long arch_do_domctl(
             {
                 if ( evc->size >= 2 * sizeof(uint64_t) + XSTATE_AREA_MIN_SIZE )
                     ret = validate_xstate(_xcr0, _xcr0_accum,
-                                          _xsave_area->xsave_hdr.xstate_bv);
+                                          &_xsave_area->xsave_hdr);
             }
             else if ( !_xcr0 )
                 ret = 0;
--- unstable.orig/xen/arch/x86/hvm/hvm.c	2016-01-27 09:52:26.000000000 +0100
+++ unstable/xen/arch/x86/hvm/hvm.c	2016-01-27 11:09:44.000000000 +0100
@@ -2178,6 +2178,19 @@ static int hvm_save_cpu_xsave_states(str
     return 0;
 }
 
+/*
+ * Structure layout conformity checks, documenting correctness of the cast in
+ * the invocation of validate_xstate() below.
+ * Leverage CONFIG_COMPAT machinery to perform this.
+ */
+#define xen_xsave_hdr xsave_hdr
+#define compat_xsave_hdr hvm_hw_cpu_xsave_hdr
+CHECK_FIELD_(struct, xsave_hdr, xstate_bv);
+CHECK_FIELD_(struct, xsave_hdr, xcomp_bv);
+CHECK_FIELD_(struct, xsave_hdr, reserved);
+#undef compat_xsave_hdr
+#undef xen_xsave_hdr
+
 static int hvm_load_cpu_xsave_states(struct domain *d, hvm_domain_context_t *h)
 {
     unsigned int vcpuid, size;
@@ -2233,7 +2246,7 @@ static int hvm_load_cpu_xsave_states(str
     h->cur += desc->length;
 
     err = validate_xstate(ctxt->xcr0, ctxt->xcr0_accum,
-                          ctxt->save_area.xsave_hdr.xstate_bv);
+                          (const void *)&ctxt->save_area.xsave_hdr);
     if ( err )
     {
         printk(XENLOG_G_WARNING
--- unstable.orig/xen/arch/x86/xstate.c	2016-01-27 10:23:06.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-27 10:48:22.000000000 +0100
@@ -614,17 +614,24 @@ static bool_t valid_xcr0(u64 xcr0)
     return !(xcr0 & XSTATE_BNDREGS) == !(xcr0 & XSTATE_BNDCSR);
 }
 
-int validate_xstate(u64 xcr0, u64 xcr0_accum, u64 xstate_bv)
+int validate_xstate(u64 xcr0, u64 xcr0_accum, const struct xsave_hdr *hdr)
 {
-    if ( (xstate_bv & ~xcr0_accum) ||
+    unsigned int i;
+
+    if ( (hdr->xstate_bv & ~xcr0_accum) ||
          (xcr0 & ~xcr0_accum) ||
          !valid_xcr0(xcr0) ||
          !valid_xcr0(xcr0_accum) )
         return -EINVAL;
 
-    if ( xcr0_accum & ~xfeature_mask )
+    if ( (xcr0_accum & ~xfeature_mask) ||
+         hdr->xcomp_bv )
         return -EOPNOTSUPP;
 
+    for ( i = 0; i < ARRAY_SIZE(hdr->reserved); ++i )
+        if ( hdr->reserved[i] )
+            return -EIO;
+
     return 0;
 }
 
--- unstable.orig/xen/include/asm-x86/xstate.h	2016-01-25 11:33:20.000000000 +0100
+++ unstable/xen/include/asm-x86/xstate.h	2016-01-27 10:57:54.000000000 +0100
@@ -72,14 +72,13 @@ struct __attribute__((aligned (64))) xsa
         };
     } fpu_sse;
 
-    struct {
+    struct xsave_hdr {
         u64 xstate_bv;
         u64 xcomp_bv;
         u64 reserved[6];
     } xsave_hdr;                             /* The 64-byte header */
 
-    struct { char x[XSTATE_YMM_SIZE]; } ymm; /* YMM */
-    char   data[];                           /* Future new states */
+    char data[];                             /* Variable layout states */
 };
 
 /* extended state operations */
@@ -90,7 +89,8 @@ uint64_t get_msr_xss(void);
 void xsave(struct vcpu *v, uint64_t mask);
 void xrstor(struct vcpu *v, uint64_t mask);
 bool_t xsave_enabled(const struct vcpu *v);
-int __must_check validate_xstate(u64 xcr0, u64 xcr0_accum, u64 xstate_bv);
+int __must_check validate_xstate(u64 xcr0, u64 xcr0_accum,
+                                 const struct xsave_hdr *);
 int __must_check handle_xsetbv(u32 index, u64 new_bv);
 void expand_xsave_states(struct vcpu *v, void *dest, unsigned int size);
 void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size);
--- unstable.orig/xen/include/public/arch-x86/hvm/save.h	2016-01-13 07:56:27.000000000 +0100
+++ unstable/xen/include/public/arch-x86/hvm/save.h	2016-01-27 11:09:20.000000000 +0100
@@ -550,12 +550,11 @@ struct hvm_hw_cpu_xsave {
     struct {
         struct { char x[512]; } fpu_sse;
 
-        struct {
+        struct hvm_hw_cpu_xsave_hdr {
             uint64_t xstate_bv;         /* Updated by XRSTOR */
-            uint64_t reserved[7];
+            uint64_t xcomp_bv;          /* Updated by XRSTOR{C,S} */
+            uint64_t reserved[6];
         } xsave_hdr;                    /* The 64-byte header */
-
-        struct { char x[0]; } ymm;    /* YMM */
     } save_area;
 };
 

[-- Attachment #6: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-27 13:12                                       ` Jan Beulich
@ 2016-01-27 13:28                                         ` Harmandeep Kaur
  2016-01-27 14:28                                           ` Jan Beulich
  0 siblings, 1 reply; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-27 13:28 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

I tried to apply your patches but it seems
to have some merge conflicts with latest
staging branch.

~/xen$ git apply ~/Downloads/x86-xsaves-init.patch
error: patch failed: xen/arch/x86/hvm/hvm.c:2094
error: xen/arch/x86/hvm/hvm.c: patch does not apply

Do you mind having a look ?

Regards,
Harmandeep

On Wed, Jan 27, 2016 at 6:42 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 26.01.16 at 19:02, <write.harmandeep@gmail.com> wrote:
>> Last time, I did absolutely nothing. System was idle
>> and it crashed just after the login. Now, I booted the
>> system again and this time, there is no reset. But,
>> performance of the system is very slow. Browser
>> (Mozilla Firefox) freezes a lot. Also, before applying
>> patches, when I used to disabe xsave it resulted in
>> same kind of performance issues. And the following
>> is still present in the log.
>>
>> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
>> (XEN) d1v1 fault#1: mxcsr=00001f80
>> (XEN) d1v1 xs=0000000000000003 xc=8000000000000000
>> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
>> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
>> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
>> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cea -> ffff82d080252e5c
>> (XEN) d1v1 fault#2: mxcsr=00001f80
>> (XEN) d1v1 xs=0000000000000000 xc=0000000000000000
>> (XEN) d1v1 r0=0000000000000000 r1=0000000000000000
>> (XEN) d1v1 r2=0000000000000000 r3=0000000000000000
>> (XEN) d1v1 r4=0000000000000000 r5=0000000000000000
>>
>> Full log here: http://paste2.org/C8WpyKOg
>
> This together with ...
>
>> On Tue, Jan 26, 2016 at 10:53 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>>> On 26.01.16 at 18:01, <write.harmandeep@gmail.com> wrote:
>>>> I tried 3rd patch together with earlier two. I'm
>>>> afraid the problem is not solved completely.
>>>> Full log goes here, http://paste2.org/KEAetMHb
>
> ... this, and both being apparently the same build makes me suspect
> uninitialized data to get passed in from the tool stack. But that's a
> secondary issue for now. For the immediate problem here are four
> patches replacing the three earlier ones (I think only one of them is
> unchanged, so be sure to remove the old ones first).
>
> Their intended ordering is:
> x86-xsaves-init.patch
> x86-xstate-align.patch
> x86-xrstors-fault.patch
> x86-xstate-validate.patch
>
> Jan
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-27 13:28                                         ` Harmandeep Kaur
@ 2016-01-27 14:28                                           ` Jan Beulich
  2016-01-28 19:17                                             ` Harmandeep Kaur
  0 siblings, 1 reply; 46+ messages in thread
From: Jan Beulich @ 2016-01-27 14:28 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

[-- Attachment #1: Type: text/plain, Size: 382 bytes --]

>>> On 27.01.16 at 14:28, <write.harmandeep@gmail.com> wrote:
> I tried to apply your patches but it seems
> to have some merge conflicts with latest
> staging branch.
> 
> ~/xen$ git apply ~/Downloads/x86-xsaves-init.patch
> error: patch failed: xen/arch/x86/hvm/hvm.c:2094
> error: xen/arch/x86/hvm/hvm.c: patch does not apply

Oh, indeed. Here's a better set.

Jan


[-- Attachment #2: x86-xrstors-fault.patch --]
[-- Type: text/plain, Size: 6925 bytes --]

x86/xstate: fix fault behavior on XRSTORS

XRSTORS unconditionally faults when xcomp_bv has bit 63 clear. Instead
of just fixing this issue, overhaul the fault recovery code, which -
one of the many mistakes made when xstate support got introduced - was
blindly mirroring that accompanying FXRSTOR, neglecting the fact that
XRSTOR{,S} aren't all-or-nothing instructions. The new code, first of
all, does all the recovery actions in C, simplifying the inline
assembly used. And it does its work in a multi-stage fashion: Upon
first seeing a fault, state fixups get applied strictly based on what
architecturally may cause #GP. When seeing another fault despite the
fixups done, state gets fully reset. A third fault would then lead to
crashing the domain (instead of hanging the hypervisor in an infinite
loop of recurring faults).

Reported-by: Harmandeep Kaur <write.harmandeep@gmail.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/xstate.c	2016-01-25 09:35:12.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-27 10:23:06.000000000 +0100
@@ -29,6 +29,8 @@ unsigned int *__read_mostly xstate_sizes
 static unsigned int __read_mostly xstate_features;
 static unsigned int __read_mostly xstate_comp_offsets[sizeof(xfeature_mask)*8];
 
+static uint32_t __read_mostly mxcsr_mask = MXCSR_DEFAULT;
+
 /* Cached xcr0 for fast read */
 static DEFINE_PER_CPU(uint64_t, xcr0);
 
@@ -342,6 +344,7 @@ void xrstor(struct vcpu *v, uint64_t mas
     uint32_t hmask = mask >> 32;
     uint32_t lmask = mask;
     struct xsave_struct *ptr = v->arch.xsave_area;
+    unsigned int faults, prev_faults;
 
     /*
      * AMD CPUs don't save/restore FDP/FIP/FOP unless an exception
@@ -361,35 +364,85 @@ void xrstor(struct vcpu *v, uint64_t mas
     /*
      * XRSTOR can fault if passed a corrupted data block. We handle this
      * possibility, which may occur if the block was passed to us by control
-     * tools or through VCPUOP_initialise, by silently clearing the block.
+     * tools or through VCPUOP_initialise, by silently adjusting state.
      */
-    switch ( __builtin_expect(ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET], 8) )
+    for ( prev_faults = faults = 0; ; prev_faults = faults )
     {
+        switch ( __builtin_expect(ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET], 8) )
+        {
 #define XRSTOR(pfx) \
         alternative_io("1: .byte " pfx "0x0f,0xae,0x2f\n" \
+                       "3:\n" \
                        "   .section .fixup,\"ax\"\n" \
-                       "2: mov %[size],%%ecx\n" \
-                       "   xor %[lmask_out],%[lmask_out]\n" \
-                       "   rep stosb\n" \
-                       "   lea %[mem],%[ptr]\n" \
-                       "   mov %[lmask_in],%[lmask_out]\n" \
-                       "   jmp 1b\n" \
+                       "2: inc%z[faults] %[faults]\n" \
+                       "   jmp 3b\n" \
                        "   .previous\n" \
                        _ASM_EXTABLE(1b, 2b), \
                        ".byte " pfx "0x0f,0xc7,0x1f\n", \
                        X86_FEATURE_XSAVES, \
-                       ASM_OUTPUT2([ptr] "+&D" (ptr), [lmask_out] "+&a" (lmask)), \
-                       [mem] "m" (*ptr), [lmask_in] "g" (lmask), \
-                       [hmask] "d" (hmask), [size] "m" (xsave_cntxt_size) \
-                       : "ecx")
-
-    default:
-        XRSTOR("0x48,");
-        break;
-    case 4: case 2:
-        XRSTOR("");
-        break;
+                       ASM_OUTPUT2([mem] "+m" (*ptr), [faults] "+g" (faults)), \
+                       [lmask] "a" (lmask), [hmask] "d" (hmask), \
+                       [ptr] "D" (ptr))
+
+        default:
+            XRSTOR("0x48,");
+            break;
+        case 4: case 2:
+            XRSTOR("");
+            break;
 #undef XRSTOR
+        }
+        if ( likely(faults == prev_faults) )
+            break;
+#ifndef NDEBUG
+        gprintk(XENLOG_WARNING, "fault#%u: mxcsr=%08x\n",
+                faults, ptr->fpu_sse.mxcsr);
+        gprintk(XENLOG_WARNING, "xs=%016lx xc=%016lx\n",
+                ptr->xsave_hdr.xstate_bv, ptr->xsave_hdr.xcomp_bv);
+        gprintk(XENLOG_WARNING, "r0=%016lx r1=%016lx\n",
+                ptr->xsave_hdr.reserved[0], ptr->xsave_hdr.reserved[1]);
+        gprintk(XENLOG_WARNING, "r2=%016lx r3=%016lx\n",
+                ptr->xsave_hdr.reserved[2], ptr->xsave_hdr.reserved[3]);
+        gprintk(XENLOG_WARNING, "r4=%016lx r5=%016lx\n",
+                ptr->xsave_hdr.reserved[4], ptr->xsave_hdr.reserved[5]);
+#endif
+        switch ( faults )
+        {
+        case 1:
+            /* Stage 1: Reset state to be loaded. */
+            ptr->xsave_hdr.xstate_bv &= ~mask;
+            /*
+             * Also try to eliminate fault reasons, even if this shouldn't be
+             * needed here (other code should ensure the sanity of the data).
+             */
+            if ( ((mask & XSTATE_SSE) ||
+                  ((mask & XSTATE_YMM) &&
+                   !(ptr->xsave_hdr.xcomp_bv & XSTATE_COMPACTION_ENABLED))) )
+                ptr->fpu_sse.mxcsr &= mxcsr_mask;
+            if ( cpu_has_xsaves || cpu_has_xsavec )
+            {
+                ptr->xsave_hdr.xcomp_bv &= this_cpu(xcr0) | this_cpu(xss);
+                ptr->xsave_hdr.xstate_bv &= ptr->xsave_hdr.xcomp_bv;
+                ptr->xsave_hdr.xcomp_bv |= XSTATE_COMPACTION_ENABLED;
+            }
+            else
+            {
+                ptr->xsave_hdr.xstate_bv &= this_cpu(xcr0);
+                ptr->xsave_hdr.xcomp_bv = 0;
+            }
+            memset(ptr->xsave_hdr.reserved, 0, sizeof(ptr->xsave_hdr.reserved));
+            continue;
+        case 2:
+            /* Stage 2: Reset all state. */
+            ptr->fpu_sse.mxcsr = MXCSR_DEFAULT;
+            ptr->xsave_hdr.xstate_bv = 0;
+            ptr->xsave_hdr.xcomp_bv = cpu_has_xsaves
+                                      ? XSTATE_COMPACTION_ENABLED : 0;
+            continue;
+        default:
+            domain_crash(current->domain);
+            break;
+        }
     }
 }
 
@@ -496,6 +549,8 @@ void xstate_init(struct cpuinfo_x86 *c)
 
     if ( bsp )
     {
+        static typeof(current->arch.xsave_area->fpu_sse) __initdata ctxt;
+
         xfeature_mask = feature_mask;
         /*
          * xsave_cntxt_size is the max size required by enabled features.
@@ -504,6 +559,10 @@ void xstate_init(struct cpuinfo_x86 *c)
         xsave_cntxt_size = _xstate_ctxt_size(feature_mask);
         printk("%s: using cntxt_size: %#x and states: %#"PRIx64"\n",
             __func__, xsave_cntxt_size, xfeature_mask);
+
+        asm ( "fxsave %0" : "=m" (ctxt) );
+        if ( ctxt.mxcsr_mask )
+            mxcsr_mask = ctxt.mxcsr_mask;
     }
     else
     {

[-- Attachment #3: x86-xsaves-init.patch --]
[-- Type: text/plain, Size: 4052 bytes --]

x86/xstate: fix xcomp_bv initialization

We must not clear the compaction bit when using XSAVES/XRSTORS. And
we need to guarantee that xcomp_bv never has any bits clear which
are set in xstate_bv (which requires partly undoing commit 83ae0bb226
["x86/xsave: simplify xcomp_bv initialization"]). Split initialization
of xcomp_bv from the other FPU/SSE/AVX related state setup in
arch_set_info_guest() and hvm_load_cpu_ctxt().

Reported-by: Harmandeep Kaur <write.harmandeep@gmail.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/domain.c	2015-12-18 12:21:58.000000000 +0100
+++ unstable/xen/arch/x86/domain.c	2016-01-27 09:52:37.000000000 +0100
@@ -922,15 +922,10 @@ int arch_set_info_guest(
     {
         memcpy(v->arch.fpu_ctxt, &c.nat->fpu_ctxt, sizeof(c.nat->fpu_ctxt));
         if ( v->arch.xsave_area )
-        {
             v->arch.xsave_area->xsave_hdr.xstate_bv = XSTATE_FP_SSE;
-            v->arch.xsave_area->xsave_hdr.xcomp_bv =
-                cpu_has_xsaves ? XSTATE_COMPACTION_ENABLED : 0;
-        }
     }
     else if ( v->arch.xsave_area )
-        memset(&v->arch.xsave_area->xsave_hdr, 0,
-               sizeof(v->arch.xsave_area->xsave_hdr));
+        v->arch.xsave_area->xsave_hdr.xstate_bv = 0;
     else
     {
         typeof(v->arch.xsave_area->fpu_sse) *fpu_sse = v->arch.fpu_ctxt;
@@ -939,6 +934,14 @@ int arch_set_info_guest(
         fpu_sse->fcw = FCW_DEFAULT;
         fpu_sse->mxcsr = MXCSR_DEFAULT;
     }
+    if ( cpu_has_xsaves )
+    {
+        ASSERT(v->arch.xsave_area);
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = XSTATE_COMPACTION_ENABLED |
+            v->arch.xsave_area->xsave_hdr.xstate_bv;
+    }
+    else if ( v->arch.xsave_area )
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = 0;
 
     if ( !compat )
     {
--- unstable.orig/xen/arch/x86/hvm/hvm.c	2015-12-18 12:22:20.000000000 +0100
+++ unstable/xen/arch/x86/hvm/hvm.c	2016-01-27 15:25:13.000000000 +0100
@@ -1974,6 +1974,7 @@ static int hvm_load_cpu_ctxt(struct doma
     struct hvm_hw_cpu ctxt;
     struct segment_register seg;
     const char *errstr;
+    struct xsave_struct *xsave_area;
 
     /* Which vcpu is this? */
     vcpuid = hvm_load_instance(h);
@@ -2100,20 +2101,24 @@ static int hvm_load_cpu_ctxt(struct doma
     seg.attr.bytes = ctxt.ldtr_arbytes;
     hvm_set_segment_register(v, x86_seg_ldtr, &seg);
 
+    /* Cover xsave-absent save file restoration on xsave-capable host. */
+    xsave_area = xsave_enabled(v) ? NULL : v->arch.xsave_area;
+
     v->fpu_initialised = !!(ctxt.flags & XEN_X86_FPU_INITIALISED);
     if ( v->fpu_initialised )
     {
         memcpy(v->arch.fpu_ctxt, ctxt.fpu_regs, sizeof(ctxt.fpu_regs));
-        /* In case xsave-absent save file is restored on a xsave-capable host */
-        if ( cpu_has_xsave && !xsave_enabled(v) )
-        {
-            struct xsave_struct *xsave_area = v->arch.xsave_area;
-
+        if ( xsave_area )
             xsave_area->xsave_hdr.xstate_bv = XSTATE_FP_SSE;
-            xsave_area->xsave_hdr.xcomp_bv =
-                cpu_has_xsaves ? XSTATE_COMPACTION_ENABLED : 0;
-        }
     }
+    else if ( xsave_area )
+    {
+        xsave_area->xsave_hdr.xstate_bv = 0;
+        xsave_area->fpu_sse.mxcsr = MXCSR_DEFAULT;
+    }
+    if ( cpu_has_xsaves && xsave_area )
+        xsave_area->xsave_hdr.xcomp_bv = XSTATE_COMPACTION_ENABLED |
+            xsave_area->xsave_hdr.xstate_bv;
 
     v->arch.user_regs.eax = ctxt.rax;
     v->arch.user_regs.ebx = ctxt.rbx;
@@ -5502,8 +5507,8 @@ void hvm_vcpu_reset_state(struct vcpu *v
     if ( v->arch.xsave_area )
     {
         v->arch.xsave_area->xsave_hdr.xstate_bv = XSTATE_FP;
-        v->arch.xsave_area->xsave_hdr.xcomp_bv =
-            cpu_has_xsaves ? XSTATE_COMPACTION_ENABLED : 0;
+        v->arch.xsave_area->xsave_hdr.xcomp_bv = cpu_has_xsaves
+            ? XSTATE_COMPACTION_ENABLED | XSTATE_FP : 0;
     }
 
     v->arch.vgc_flags = VGCF_online;

[-- Attachment #4: x86-xstate-align.patch --]
[-- Type: text/plain, Size: 2384 bytes --]

x86: adjust xsave structure attributes

The packed attribute was pointlessly used here - there are no
misaligned fields, and hence even if the attribute took effect, it
would at best lead to the compiler generating worse code.

At the same time specify the required alignment of the fpu_sse sub-
structure, such that the various typeof() uses on that field obtain
pointers to properly aligned memory (knowledge which a compiler may
want to make use of).

Also add suitable build-time checks.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/i387.c	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/arch/x86/i387.c	2016-01-25 09:35:36.000000000 +0100
@@ -277,7 +277,9 @@ int vcpu_init_fpu(struct vcpu *v)
     }
     else
     {
-        v->arch.fpu_ctxt = _xzalloc(sizeof(v->arch.xsave_area->fpu_sse), 16);
+        BUILD_BUG_ON(__alignof(v->arch.xsave_area->fpu_sse) < 16);
+        v->arch.fpu_ctxt = _xzalloc(sizeof(v->arch.xsave_area->fpu_sse),
+                                    __alignof(v->arch.xsave_area->fpu_sse));
         if ( v->arch.fpu_ctxt )
         {
             typeof(v->arch.xsave_area->fpu_sse) *fpu_sse = v->arch.fpu_ctxt;
--- unstable.orig/xen/arch/x86/xstate.c	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-25 09:35:12.000000000 +0100
@@ -414,7 +414,8 @@ int xstate_alloc_save_area(struct vcpu *
     BUG_ON(xsave_cntxt_size < XSTATE_AREA_MIN_SIZE);
 
     /* XSAVE/XRSTOR requires the save area be 64-byte-boundary aligned. */
-    save_area = _xzalloc(xsave_cntxt_size, 64);
+    BUILD_BUG_ON(__alignof(*save_area) < 64);
+    save_area = _xzalloc(xsave_cntxt_size, __alignof(*save_area));
     if ( save_area == NULL )
         return -ENOMEM;
 
--- unstable.orig/xen/include/asm-x86/xstate.h	2016-01-25 11:30:11.000000000 +0100
+++ unstable/xen/include/asm-x86/xstate.h	2016-01-25 11:33:20.000000000 +0100
@@ -48,9 +48,9 @@ extern u64 xfeature_mask;
 extern unsigned int *xstate_sizes;
 
 /* extended state save area */
-struct __packed __attribute__((aligned (64))) xsave_struct
+struct __attribute__((aligned (64))) xsave_struct
 {
-    union {                                  /* FPU/MMX, SSE */
+    union __attribute__((aligned(16))) {     /* FPU/MMX, SSE */
         char x[512];
         struct {
             uint16_t fcw;

[-- Attachment #5: x86-xstate-validate.patch --]
[-- Type: text/plain, Size: 4969 bytes --]

x86/xstate: extend validation to cover full header

Since we never hand out compacted state, at least for now we're also
not going to accept such.

Reported-by: Harmandeep Kaur <write.harmandeep@gmail.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- unstable.orig/xen/arch/x86/domctl.c	2016-01-27 14:56:10.000000000 +0100
+++ unstable/xen/arch/x86/domctl.c	2016-01-27 10:44:52.000000000 +0100
@@ -958,7 +958,7 @@ long arch_do_domctl(
             {
                 if ( evc->size >= 2 * sizeof(uint64_t) + XSTATE_AREA_MIN_SIZE )
                     ret = validate_xstate(_xcr0, _xcr0_accum,
-                                          _xsave_area->xsave_hdr.xstate_bv);
+                                          &_xsave_area->xsave_hdr);
             }
             else if ( !_xcr0 )
                 ret = 0;
--- unstable.orig/xen/arch/x86/hvm/hvm.c	2016-01-27 15:25:13.000000000 +0100
+++ unstable/xen/arch/x86/hvm/hvm.c	2016-01-27 15:26:15.000000000 +0100
@@ -2191,6 +2191,19 @@ static int hvm_save_cpu_xsave_states(str
     return 0;
 }
 
+/*
+ * Structure layout conformity checks, documenting correctness of the cast in
+ * the invocation of validate_xstate() below.
+ * Leverage CONFIG_COMPAT machinery to perform this.
+ */
+#define xen_xsave_hdr xsave_hdr
+#define compat_xsave_hdr hvm_hw_cpu_xsave_hdr
+CHECK_FIELD_(struct, xsave_hdr, xstate_bv);
+CHECK_FIELD_(struct, xsave_hdr, xcomp_bv);
+CHECK_FIELD_(struct, xsave_hdr, reserved);
+#undef compat_xsave_hdr
+#undef xen_xsave_hdr
+
 static int hvm_load_cpu_xsave_states(struct domain *d, hvm_domain_context_t *h)
 {
     unsigned int vcpuid, size;
@@ -2246,7 +2259,7 @@ static int hvm_load_cpu_xsave_states(str
     h->cur += desc->length;
 
     err = validate_xstate(ctxt->xcr0, ctxt->xcr0_accum,
-                          ctxt->save_area.xsave_hdr.xstate_bv);
+                          (const void *)&ctxt->save_area.xsave_hdr);
     if ( err )
     {
         printk(XENLOG_G_WARNING
--- unstable.orig/xen/arch/x86/xstate.c	2016-01-27 10:23:06.000000000 +0100
+++ unstable/xen/arch/x86/xstate.c	2016-01-27 10:48:22.000000000 +0100
@@ -614,17 +614,24 @@ static bool_t valid_xcr0(u64 xcr0)
     return !(xcr0 & XSTATE_BNDREGS) == !(xcr0 & XSTATE_BNDCSR);
 }
 
-int validate_xstate(u64 xcr0, u64 xcr0_accum, u64 xstate_bv)
+int validate_xstate(u64 xcr0, u64 xcr0_accum, const struct xsave_hdr *hdr)
 {
-    if ( (xstate_bv & ~xcr0_accum) ||
+    unsigned int i;
+
+    if ( (hdr->xstate_bv & ~xcr0_accum) ||
          (xcr0 & ~xcr0_accum) ||
          !valid_xcr0(xcr0) ||
          !valid_xcr0(xcr0_accum) )
         return -EINVAL;
 
-    if ( xcr0_accum & ~xfeature_mask )
+    if ( (xcr0_accum & ~xfeature_mask) ||
+         hdr->xcomp_bv )
         return -EOPNOTSUPP;
 
+    for ( i = 0; i < ARRAY_SIZE(hdr->reserved); ++i )
+        if ( hdr->reserved[i] )
+            return -EIO;
+
     return 0;
 }
 
--- unstable.orig/xen/include/asm-x86/xstate.h	2016-01-25 11:33:20.000000000 +0100
+++ unstable/xen/include/asm-x86/xstate.h	2016-01-27 10:57:54.000000000 +0100
@@ -72,14 +72,13 @@ struct __attribute__((aligned (64))) xsa
         };
     } fpu_sse;
 
-    struct {
+    struct xsave_hdr {
         u64 xstate_bv;
         u64 xcomp_bv;
         u64 reserved[6];
     } xsave_hdr;                             /* The 64-byte header */
 
-    struct { char x[XSTATE_YMM_SIZE]; } ymm; /* YMM */
-    char   data[];                           /* Future new states */
+    char data[];                             /* Variable layout states */
 };
 
 /* extended state operations */
@@ -90,7 +89,8 @@ uint64_t get_msr_xss(void);
 void xsave(struct vcpu *v, uint64_t mask);
 void xrstor(struct vcpu *v, uint64_t mask);
 bool_t xsave_enabled(const struct vcpu *v);
-int __must_check validate_xstate(u64 xcr0, u64 xcr0_accum, u64 xstate_bv);
+int __must_check validate_xstate(u64 xcr0, u64 xcr0_accum,
+                                 const struct xsave_hdr *);
 int __must_check handle_xsetbv(u32 index, u64 new_bv);
 void expand_xsave_states(struct vcpu *v, void *dest, unsigned int size);
 void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size);
--- unstable.orig/xen/include/public/arch-x86/hvm/save.h	2016-01-27 14:58:51.000000000 +0100
+++ unstable/xen/include/public/arch-x86/hvm/save.h	2016-01-27 15:26:25.000000000 +0100
@@ -564,12 +564,11 @@ struct hvm_hw_cpu_xsave {
     struct {
         struct { char x[512]; } fpu_sse;
 
-        struct {
+        struct hvm_hw_cpu_xsave_hdr {
             uint64_t xstate_bv;         /* Updated by XRSTOR */
-            uint64_t reserved[7];
+            uint64_t xcomp_bv;          /* Updated by XRSTOR{C,S} */
+            uint64_t reserved[6];
         } xsave_hdr;                    /* The 64-byte header */
-
-        struct { char x[0]; } ymm;    /* YMM */
     } save_area;
 };
 

[-- Attachment #6: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-27 14:28                                           ` Jan Beulich
@ 2016-01-28 19:17                                             ` Harmandeep Kaur
  2016-01-29 10:13                                               ` Jan Beulich
  0 siblings, 1 reply; 46+ messages in thread
From: Harmandeep Kaur @ 2016-01-28 19:17 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

Latest set looks good. No boot issues. No resets.
Full log at http://paste2.org/NxHNW4vn

Sorry I don't know much about source of last few
lines. I was just tracing in xen when these came.
I was unable to create them again. I will inform
you if I get these again.
Regards,
Harmandeep

On Wed, Jan 27, 2016 at 7:58 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 27.01.16 at 14:28, <write.harmandeep@gmail.com> wrote:
>> I tried to apply your patches but it seems
>> to have some merge conflicts with latest
>> staging branch.
>>
>> ~/xen$ git apply ~/Downloads/x86-xsaves-init.patch
>> error: patch failed: xen/arch/x86/hvm/hvm.c:2094
>> error: xen/arch/x86/hvm/hvm.c: patch does not apply
>
> Oh, indeed. Here's a better set.
>
> Jan
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-28 19:17                                             ` Harmandeep Kaur
@ 2016-01-29 10:13                                               ` Jan Beulich
  2016-02-03  8:23                                                 ` Dario Faggioli
  0 siblings, 1 reply; 46+ messages in thread
From: Jan Beulich @ 2016-01-29 10:13 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

>>> On 28.01.16 at 20:17, <write.harmandeep@gmail.com> wrote:
> Latest set looks good. No boot issues. No resets.
> Full log at http://paste2.org/NxHNW4vn 

Thanks!

> Sorry I don't know much about source of last few
> lines. I was just tracing in xen when these came.
> I was unable to create them again. I will inform
> you if I get these again.

The WRMSR ones are of no concern. What I'm curious about are the
five instances of

(XEN) traps.c:3290: GPF (0000): ffff82d08019cb87 -> ffff82d080242e15

Could you check what these addresses correspond to in source?

Also did you meanwhile clarify for yourself what domain(s) are
being started here? So far you've said you don't start any, but
the logs clearly show otherwise. Are there perhaps any that
get auto-started? In which case I'd be interested in you
confirming that these come up fine.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-20 10:06                     ` Jan Beulich
  2016-01-21 15:14                       ` Dario Faggioli
@ 2016-02-02  4:43                       ` Shuai Ruan
       [not found]                       ` <20160202044349.GA3036@shuai.ruan@linux.intel.com>
  2 siblings, 0 replies; 46+ messages in thread
From: Shuai Ruan @ 2016-02-02  4:43 UTC (permalink / raw)
  To: Jan Beulich; +Cc: Andrew Cooper, Dario Faggioli, Harmandeep Kaur, xen-devel

On Wed, Jan 20, 2016 at 03:06:09AM -0700, Jan Beulich wrote:
> >>> On 19.01.16 at 20:55, <write.harmandeep@gmail.com> wrote:
Sorry for late response. I am away from the mail list for a couple
weeks.

> > I tried booting staging branch but results were identical. Following
> > line repeats endlessly.
> > (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cce -> ffff82d080252e5c
> > 
> > $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
> > 'xen/xen/arch/x86/xstate.c:387' which again points to
> > xsave. Also, adding 'xsave=0' makes it boot just fine.
> 
> Ah, I think I see the issue: We're zeroing the entire save area in
> the fixup code, which will make XRSTORS fault unconditionally.
> Shuai, would you please look into possible ways of fixing this?
Ok. I will look into this problem.
> Just setting the compressed flag may not be enough, and falling
> back to plain XRSTOR doesn't seem to be an option either - I'm in
> particular worried that the current model of zeroing everything
> is bogus with the growing number of different components which
> get loaded here.
> 
> But of course another question then is why the XRSTORS faults
> in the first place. I guess we'll need a debugging patch to dump
> the full state to understand that.
> 
> Jan
> 

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
       [not found]                       ` <20160202044349.GA3036@shuai.ruan@linux.intel.com>
@ 2016-02-02 10:39                         ` Andrew Cooper
  2016-02-03  0:56                           ` Shuai Ruan
  2016-02-03  7:56                           ` Shuai Ruan
  0 siblings, 2 replies; 46+ messages in thread
From: Andrew Cooper @ 2016-02-02 10:39 UTC (permalink / raw)
  To: Shuai Ruan; +Cc: Dario Faggioli, Harmandeep Kaur, Jan Beulich, xen-devel

On 02/02/16 04:43, Shuai Ruan wrote:
>
>>> I tried booting staging branch but results were identical. Following
>>> line repeats endlessly.
>>> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cce -> ffff82d080252e5c
>>>
>>> $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
>>> 'xen/xen/arch/x86/xstate.c:387' which again points to
>>> xsave. Also, adding 'xsave=0' makes it boot just fine.
>> Ah, I think I see the issue: We're zeroing the entire save area in
>> the fixup code, which will make XRSTORS fault unconditionally.
>> Shuai, would you please look into possible ways of fixing this?
> Ok. I will look into this problem.

Fixes for this have been committed already.  changesets 104a409^..d570d82

It would be useful if you could review them to see if there are any
issues Jan and myself missed.

~Andrew

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-02 10:39                         ` Andrew Cooper
@ 2016-02-03  0:56                           ` Shuai Ruan
  2016-02-03  7:56                           ` Shuai Ruan
  1 sibling, 0 replies; 46+ messages in thread
From: Shuai Ruan @ 2016-02-03  0:56 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Dario Faggioli, Harmandeep Kaur, Jan Beulich, xen-devel

On Tue, Feb 02, 2016 at 10:39:09AM +0000, Andrew Cooper wrote:
> On 02/02/16 04:43, Shuai Ruan wrote:
> >
> >>> I tried booting staging branch but results were identical. Following
> >>> line repeats endlessly.
> >>> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cce -> ffff82d080252e5c
> >>>
> >>> $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
> >>> 'xen/xen/arch/x86/xstate.c:387' which again points to
> >>> xsave. Also, adding 'xsave=0' makes it boot just fine.
> >> Ah, I think I see the issue: We're zeroing the entire save area in
> >> the fixup code, which will make XRSTORS fault unconditionally.
> >> Shuai, would you please look into possible ways of fixing this?
> > Ok. I will look into this problem.
> 
> Fixes for this have been committed already.  changesets 104a409^..d570d82
> 
> It would be useful if you could review them to see if there are any
> issues Jan and myself missed.
Ok. I will review them and test them. 

Thanks
> 
> ~Andrew
> 

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-02 10:39                         ` Andrew Cooper
  2016-02-03  0:56                           ` Shuai Ruan
@ 2016-02-03  7:56                           ` Shuai Ruan
  1 sibling, 0 replies; 46+ messages in thread
From: Shuai Ruan @ 2016-02-03  7:56 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Dario Faggioli, Harmandeep Kaur, Jan Beulich, xen-devel

On Tue, Feb 02, 2016 at 10:39:09AM +0000, Andrew Cooper wrote:
> On 02/02/16 04:43, Shuai Ruan wrote:
> >
> >>> I tried booting staging branch but results were identical. Following
> >>> line repeats endlessly.
> >>> (XEN) traps.c:3290: GPF (0000): ffff82d0801c1cce -> ffff82d080252e5c
> >>>
> >>> $ 'addr2line -e xen-syms ffff82d0801c1cce' returns
> >>> 'xen/xen/arch/x86/xstate.c:387' which again points to
> >>> xsave. Also, adding 'xsave=0' makes it boot just fine.
> >> Ah, I think I see the issue: We're zeroing the entire save area in
> >> the fixup code, which will make XRSTORS fault unconditionally.
> >> Shuai, would you please look into possible ways of fixing this?
> > Ok. I will look into this problem.
> 
> Fixes for this have been committed already.  changesets 104a409^..d570d82
> 
> It would be useful if you could review them to see if there are any
> issues Jan and myself missed.
The patch is ok to me. 
And I test it on the skylake machine (with xsaves support), no boot error happen.

Thanks
> 
> ~Andrew
> 

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-01-29 10:13                                               ` Jan Beulich
@ 2016-02-03  8:23                                                 ` Dario Faggioli
  2016-02-03 11:35                                                   ` Harmandeep Kaur
  0 siblings, 1 reply; 46+ messages in thread
From: Dario Faggioli @ 2016-02-03  8:23 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, xen-devel, Jan Beulich, Shuai Ruan


[-- Attachment #1.1: Type: text/plain, Size: 1303 bytes --]

Hey Harmandeep,

On Fri, 2016-01-29 at 03:13 -0700, Jan Beulich wrote:
> > > > On 28.01.16 at 20:17, <write.harmandeep@gmail.com> wrote:
> > Latest set looks good. No boot issues. No resets.
> > Full log at http://paste2.org/NxHNW4vn 
> 
> > Sorry I don't know much about source of last few
> > lines. I was just tracing in xen when these came.
> > I was unable to create them again. I will inform
> > you if I get these again.
> 
> The WRMSR ones are of no concern. What I'm curious about are the
> five instances of
> 
> (XEN) traps.c:3290: GPF (0000): ffff82d08019cb87 -> ffff82d080242e15
> 
> Could you check what these addresses correspond to in source?
> 
I guess you just happened to forgot to follow up on this question from
Jan?

> So far you've said you don't start any, but
> the logs clearly show otherwise. Are there perhaps any that
> get auto-started? In which case I'd be interested in you
> confirming that these come up fine.
> 
And this too?

Can you please do that? :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03  8:23                                                 ` Dario Faggioli
@ 2016-02-03 11:35                                                   ` Harmandeep Kaur
  2016-02-03 12:49                                                     ` Jan Beulich
  2016-02-03 12:55                                                     ` Dario Faggioli
  0 siblings, 2 replies; 46+ messages in thread
From: Harmandeep Kaur @ 2016-02-03 11:35 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Andrew Cooper, xen-devel, Jan Beulich, Shuai Ruan

On Wed, Feb 3, 2016 at 1:53 PM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> Hey Harmandeep,
>
> On Fri, 2016-01-29 at 03:13 -0700, Jan Beulich wrote:
>> > > > On 28.01.16 at 20:17, <write.harmandeep@gmail.com> wrote:
>> > Latest set looks good. No boot issues. No resets.
>> > Full log at http://paste2.org/NxHNW4vn
>>
>> > Sorry I don't know much about source of last few
>> > lines. I was just tracing in xen when these came.
>> > I was unable to create them again. I will inform
>> > you if I get these again.
>>
>> The WRMSR ones are of no concern. What I'm curious about are the
>> five instances of
>>
>> (XEN) traps.c:3290: GPF (0000): ffff82d08019cb87 -> ffff82d080242e15
>>
>> Could you check what these addresses correspond to in source?
>>
> I guess you just happened to forgot to follow up on this question from
> Jan?

Sorry for responding so late.

$ addr2line -e xen-syms ffff82d08019cb87
xen/xen/arch/x86/traps.c:2820

>> So far you've said you don't start any, but
>> the logs clearly show otherwise. Are there perhaps any that
>> get auto-started? In which case I'd be interested in you
>> confirming that these come up fine.
>>

Maybe I failed to shutdown a guest which was showing up
on next boot. But there are no auto-starting guests.
Following link goes to latest serial log
http://paste2.org/5PDz9bP1 and then I created a guest with
config (http://paste2.org/azvj25Hg) and
http://paste2.org/cgKna0j1 log was presented at terminal.

I hope this helps.

Thanks and regards,
Harmandeep Kaur

> And this too?
>
> Can you please do that? :-)
>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03 11:35                                                   ` Harmandeep Kaur
@ 2016-02-03 12:49                                                     ` Jan Beulich
  2016-02-03 12:55                                                     ` Dario Faggioli
  1 sibling, 0 replies; 46+ messages in thread
From: Jan Beulich @ 2016-02-03 12:49 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, Dario Faggioli, xen-devel, Shuai Ruan

>>> On 03.02.16 at 12:35, <write.harmandeep@gmail.com> wrote:
> On Wed, Feb 3, 2016 at 1:53 PM, Dario Faggioli
> <dario.faggioli@citrix.com> wrote:
>> Hey Harmandeep,
>>
>> On Fri, 2016-01-29 at 03:13 -0700, Jan Beulich wrote:
>>> > > > On 28.01.16 at 20:17, <write.harmandeep@gmail.com> wrote:
>>> > Latest set looks good. No boot issues. No resets.
>>> > Full log at http://paste2.org/NxHNW4vn 
>>>
>>> > Sorry I don't know much about source of last few
>>> > lines. I was just tracing in xen when these came.
>>> > I was unable to create them again. I will inform
>>> > you if I get these again.
>>>
>>> The WRMSR ones are of no concern. What I'm curious about are the
>>> five instances of
>>>
>>> (XEN) traps.c:3290: GPF (0000): ffff82d08019cb87 -> ffff82d080242e15
>>>
>>> Could you check what these addresses correspond to in source?
>>>
>> I guess you just happened to forgot to follow up on this question from
>> Jan?
> 
> Sorry for responding so late.

Thanks for getting back.

> $ addr2line -e xen-syms ffff82d08019cb87
> xen/xen/arch/x86/traps.c:2820

That's the RDMSR one, so (luckily) not of interest here.

>>> So far you've said you don't start any, but
>>> the logs clearly show otherwise. Are there perhaps any that
>>> get auto-started? In which case I'd be interested in you
>>> confirming that these come up fine.
>>>
> 
> Maybe I failed to shutdown a guest which was showing up
> on next boot. But there are no auto-starting guests.
> Following link goes to latest serial log
> http://paste2.org/5PDz9bP1 and then I created a guest with

But this _again_ shows the presence of a (running) Dom1, ...

> config (http://paste2.org/azvj25Hg) and
> http://paste2.org/cgKna0j1 log was presented at terminal.

... and you leave us guessing whether the first of these logs was
indeed captured before you started the guest (as one may imply
from your wording) or after (as one may imply from your claim of
there not being any auto-started guests). Please try to be more
precise in your statements.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03 11:35                                                   ` Harmandeep Kaur
  2016-02-03 12:49                                                     ` Jan Beulich
@ 2016-02-03 12:55                                                     ` Dario Faggioli
  2016-02-03 13:10                                                       ` Andrew Cooper
  2016-02-03 13:40                                                       ` Harmandeep Kaur
  1 sibling, 2 replies; 46+ messages in thread
From: Dario Faggioli @ 2016-02-03 12:55 UTC (permalink / raw)
  To: Harmandeep Kaur; +Cc: Andrew Cooper, xen-devel, Jan Beulich, Shuai Ruan


[-- Attachment #1.1: Type: text/plain, Size: 3252 bytes --]

On Wed, 2016-02-03 at 17:05 +0530, Harmandeep Kaur wrote:
> On Wed, Feb 3, 2016 at 1:53 PM, Dario Faggioli
> <dario.faggioli@citrix.com> wrote:
> > 
> Maybe I failed to shutdown a guest which was showing up
> on next boot. But there are no auto-starting guests.
> Following link goes to latest serial log
> http://paste2.org/5PDz9bP1 
>
No, sorry, I probably am not getting. Are you saying that, just booting
Xen and *not* doing anything else --either from SSH, from the keyboard
of that box, from serial connection... anything at all-- and without
any guest configured to auto start itself, results in these lines to
appear on your serial console?

tbox login: (d1) mapping kernel into physical memory
(d1) about to get started...
(XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000081 from 0xe023e00800000000 to 0x0023001000000000.
(XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000082 from 0xffff82d0bfffe080 to 0xffffffff817ef990.
(XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000083 from 0xffff82d0bfffe0a0 to 0xffffffff817f1f60.
(XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000174 from 0x000000000000e008 to 0x0000000000000010.
(XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000175 from 0xffff83026d40ffc0 to 0x0000000000000000.
(XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000176 from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
(XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000084 from 0x0000000000074700 to 0x0000000000047700.
(XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000081 from 0xe023e00800000000 to 0x0023001000000000.
(XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000082 from 0xffff82d0bffff000 to 0xffffffff817ef990.
(XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000083 from 0xffff82d0bffff020 to 0xffffffff817f1f60.
(XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000174 from 0x000000000000e008 to 0x0000000000000010.
(XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000175 from 0xffff8300866fffc0 to 0x0000000000000000.
(XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000176 from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
(XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000084 from 0x0000000000074700 to 0x0000000000047700.

Because this is what you just said above, and that's... well... just
impossible?!?! :-O

What's the output then, if you login (either via serial, ssh or regular
keyboad+monitor) and run `xl list' and `xl vcpu-list' as the very first
thing?  What's inside `ls -l /var/log/xen/' (or `ls -l
/var/local/log/xen')?

> and then I created a guest with
> config (http://paste2.org/azvj25Hg) and
> http://paste2.org/cgKna0j1 log was presented at terminal.
> 
Well, it looks like the guest boot did not indeed go well, but that is
probably because of other thing... can you try uncommenting the line
'extra = "root=/dev/xvda1"' from the config?

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03 12:55                                                     ` Dario Faggioli
@ 2016-02-03 13:10                                                       ` Andrew Cooper
  2016-02-03 13:17                                                         ` Dario Faggioli
  2016-02-03 13:40                                                       ` Harmandeep Kaur
  1 sibling, 1 reply; 46+ messages in thread
From: Andrew Cooper @ 2016-02-03 13:10 UTC (permalink / raw)
  To: Dario Faggioli, Harmandeep Kaur; +Cc: xen-devel, Jan Beulich, Shuai Ruan

On 03/02/16 12:55, Dario Faggioli wrote:
> On Wed, 2016-02-03 at 17:05 +0530, Harmandeep Kaur wrote:
>> On Wed, Feb 3, 2016 at 1:53 PM, Dario Faggioli
>> <dario.faggioli@citrix.com> wrote:
>>>  
>> Maybe I failed to shutdown a guest which was showing up
>> on next boot. But there are no auto-starting guests.
>> Following link goes to latest serial log
>> http://paste2.org/5PDz9bP1 
>>
> No, sorry, I probably am not getting. Are you saying that, just booting
> Xen and *not* doing anything else --either from SSH, from the keyboard
> of that box, from serial connection... anything at all-- and without
> any guest configured to auto start itself, results in these lines to
> appear on your serial console?
>
> tbox login: (d1) mapping kernel into physical memory
> (d1) about to get started...
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000081 from 0xe023e00800000000 to 0x0023001000000000.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000082 from 0xffff82d0bfffe080 to 0xffffffff817ef990.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000083 from 0xffff82d0bfffe0a0 to 0xffffffff817f1f60.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000174 from 0x000000000000e008 to 0x0000000000000010.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000175 from 0xffff83026d40ffc0 to 0x0000000000000000.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000176 from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000084 from 0x0000000000074700 to 0x0000000000047700.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000081 from 0xe023e00800000000 to 0x0023001000000000.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000082 from 0xffff82d0bffff000 to 0xffffffff817ef990.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000083 from 0xffff82d0bffff020 to 0xffffffff817f1f60.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000174 from 0x000000000000e008 to 0x0000000000000010.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000175 from 0xffff8300866fffc0 to 0x0000000000000000.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000176 from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000084 from 0x0000000000074700 to 0x0000000000047700.
>
> Because this is what you just said above, and that's... well... just
> impossible?!?! :-O

This is a Linux PV trying to set up the SYSCALL/SYSENTER MSRs when it
shouldn't.  There is a fix upstream, but this specifically is harmless
noise.

~Andrew

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03 13:10                                                       ` Andrew Cooper
@ 2016-02-03 13:17                                                         ` Dario Faggioli
  2016-02-03 16:57                                                           ` Konrad Rzeszutek Wilk
  0 siblings, 1 reply; 46+ messages in thread
From: Dario Faggioli @ 2016-02-03 13:17 UTC (permalink / raw)
  To: Andrew Cooper, Harmandeep Kaur; +Cc: xen-devel, Jan Beulich, Shuai Ruan


[-- Attachment #1.1: Type: text/plain, Size: 2623 bytes --]

On Wed, 2016-02-03 at 13:10 +0000, Andrew Cooper wrote:
> On 03/02/16 12:55, Dario Faggioli wrote:
> > 
> > tbox login: (d1) mapping kernel into physical memory
> > (d1) about to get started...
> > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000081
> > from 0xe023e00800000000 to 0x0023001000000000.
> > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000082
> > from 0xffff82d0bfffe080 to 0xffffffff817ef990.
> > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000083
> > from 0xffff82d0bfffe0a0 to 0xffffffff817f1f60.
> > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000174
> > from 0x000000000000e008 to 0x0000000000000010.
> > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000175
> > from 0xffff83026d40ffc0 to 0x0000000000000000.
> > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000176
> > from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000084
> > from 0x0000000000074700 to 0x0000000000047700.
> > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000081
> > from 0xe023e00800000000 to 0x0023001000000000.
> > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000082
> > from 0xffff82d0bffff000 to 0xffffffff817ef990.
> > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000083
> > from 0xffff82d0bffff020 to 0xffffffff817f1f60.
> > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000174
> > from 0x000000000000e008 to 0x0000000000000010.
> > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000175
> > from 0xffff8300866fffc0 to 0x0000000000000000.
> > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000176
> > from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000084
> > from 0x0000000000074700 to 0x0000000000047700.
> > 
> > Because this is what you just said above, and that's... well...
> > just
> > impossible?!?! :-O
> 
> This is a Linux PV trying to set up the SYSCALL/SYSENTER MSRs when it
> shouldn't.  There is a fix upstream, but this specifically is
> harmless
> noise.
> 
That's ok... What we're arguing is that is happens as a consequence of
a domain being created, not just "by itself, after boot, without
starting any domain" !!!! :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03 12:55                                                     ` Dario Faggioli
  2016-02-03 13:10                                                       ` Andrew Cooper
@ 2016-02-03 13:40                                                       ` Harmandeep Kaur
  2016-02-03 14:18                                                         ` Jan Beulich
  1 sibling, 1 reply; 46+ messages in thread
From: Harmandeep Kaur @ 2016-02-03 13:40 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Andrew Cooper, xen-devel, Jan Beulich, Shuai Ruan

On Wed, Feb 3, 2016 at 6:25 PM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> On Wed, 2016-02-03 at 17:05 +0530, Harmandeep Kaur wrote:
>> On Wed, Feb 3, 2016 at 1:53 PM, Dario Faggioli
>> <dario.faggioli@citrix.com> wrote:
>> >
>> Maybe I failed to shutdown a guest which was showing up
>> on next boot. But there are no auto-starting guests.
>> Following link goes to latest serial log
>> http://paste2.org/5PDz9bP1
>>
> No, sorry, I probably am not getting. Are you saying that, just booting
> Xen and *not* doing anything else --either from SSH, from the keyboard
> of that box, from serial connection... anything at all-- and without
> any guest configured to auto start itself, results in these lines to
> appear on your serial console?
>
> tbox login: (d1) mapping kernel into physical memory
> (d1) about to get started...
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000081 from 0xe023e00800000000 to 0x0023001000000000.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000082 from 0xffff82d0bfffe080 to 0xffffffff817ef990.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000083 from 0xffff82d0bfffe0a0 to 0xffffffff817f1f60.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000174 from 0x000000000000e008 to 0x0000000000000010.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000175 from 0xffff83026d40ffc0 to 0x0000000000000000.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000176 from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000084 from 0x0000000000074700 to 0x0000000000047700.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000081 from 0xe023e00800000000 to 0x0023001000000000.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000082 from 0xffff82d0bffff000 to 0xffffffff817ef990.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000083 from 0xffff82d0bffff020 to 0xffffffff817f1f60.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000174 from 0x000000000000e008 to 0x0000000000000010.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000175 from 0xffff8300866fffc0 to 0x0000000000000000.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000176 from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000084 from 0x0000000000074700 to 0x0000000000047700.
>
> Because this is what you just said above, and that's... well... just
> impossible?!?! :-O
>
> What's the output then, if you login (either via serial, ssh or regular
> keyboad+monitor) and run `xl list'

$ sudo xl list
Name                                        ID   Mem VCPUs    State    Time(s)
Domain-0                                     0  4096     4     r-----      37.6

> and `xl vcpu-list' as the very first
> thing?

$ sudo xl vcpu-list
Name                                ID  VCPU   CPU State   Time(s)
Affinity (Hard / Soft)
Domain-0                             0     0    3   r--      13.3  all / all
Domain-0                             0     1    0   -b-       7.0  all / all
Domain-0                             0     2    1   -b-       5.4  all / all
Domain-0                             0     3    2   r--      13.5  all / all


> What's inside `ls -l /var/log/xen/' (or `ls -l
> /var/local/log/xen')?

$ ls -l /var/log/xen/
total 48
-rw-r--r-- 1 root adm  28 Jan 23 00:14 xen-hotplug.log
-rw-r--r-- 1 root adm  91 Feb  3 17:14 xl-ubuntu.pvlinux.log
-rw-r--r-- 1 root adm  91 Jan 29 00:54 xl-ubuntu.pvlinux.log.1
-rw-r--r-- 1 root adm 144 Jan 26 23:50 xl-ubuntu.pvlinux.log.10
-rw-r--r-- 1 root adm  91 Jan 29 00:41 xl-ubuntu.pvlinux.log.2
-rw-r--r-- 1 root adm  91 Jan 29 00:32 xl-ubuntu.pvlinux.log.3
-rw-r--r-- 1 root adm  91 Jan 29 00:24 xl-ubuntu.pvlinux.log.4
-rw-r--r-- 1 root adm  91 Jan 28 22:22 xl-ubuntu.pvlinux.log.5
-rw-r--r-- 1 root adm  62 Jan 27 19:09 xl-ubuntu.pvlinux.log.6
-rw-r--r-- 1 root adm  91 Jan 27 19:07 xl-ubuntu.pvlinux.log.7
-rw-r--r-- 1 root adm 222 Jan 27 18:41 xl-ubuntu.pvlinux.log.8
-rw-r--r-- 1 root adm  62 Jan 27 15:03 xl-ubuntu.pvlinux.log.9

>
>> and then I created a guest with
>> config (http://paste2.org/azvj25Hg) and
>> http://paste2.org/cgKna0j1 log was presented at terminal.
>>
> Well, it looks like the guest boot did not indeed go well, but that is
> probably because of other thing... can you try uncommenting the line
> 'extra = "root=/dev/xvda1"' from the config?

Trying this suggestion.

>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03 13:40                                                       ` Harmandeep Kaur
@ 2016-02-03 14:18                                                         ` Jan Beulich
  0 siblings, 0 replies; 46+ messages in thread
From: Jan Beulich @ 2016-02-03 14:18 UTC (permalink / raw)
  To: Dario Faggioli, Harmandeep Kaur; +Cc: Andrew Cooper, xen-devel, Shuai Ruan

>>> On 03.02.16 at 14:40, <write.harmandeep@gmail.com> wrote:
> $ ls -l /var/log/xen/
> total 48
> -rw-r--r-- 1 root adm  28 Jan 23 00:14 xen-hotplug.log
> -rw-r--r-- 1 root adm  91 Feb  3 17:14 xl-ubuntu.pvlinux.log

This date looks suspiciously new.

Jan

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Error booting Xen
  2016-02-03 13:17                                                         ` Dario Faggioli
@ 2016-02-03 16:57                                                           ` Konrad Rzeszutek Wilk
  0 siblings, 0 replies; 46+ messages in thread
From: Konrad Rzeszutek Wilk @ 2016-02-03 16:57 UTC (permalink / raw)
  To: Dario Faggioli
  Cc: Andrew Cooper, Shuai Ruan, Harmandeep Kaur, Jan Beulich, xen-devel

On Wed, Feb 03, 2016 at 02:17:26PM +0100, Dario Faggioli wrote:
> On Wed, 2016-02-03 at 13:10 +0000, Andrew Cooper wrote:
> > On 03/02/16 12:55, Dario Faggioli wrote:
> > > 
> > > tbox login: (d1) mapping kernel into physical memory
> > > (d1) about to get started...
> > > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000081
> > > from 0xe023e00800000000 to 0x0023001000000000.
> > > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000082
> > > from 0xffff82d0bfffe080 to 0xffffffff817ef990.
> > > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000083
> > > from 0xffff82d0bfffe0a0 to 0xffffffff817f1f60.
> > > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000174
> > > from 0x000000000000e008 to 0x0000000000000010.
> > > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000175
> > > from 0xffff83026d40ffc0 to 0x0000000000000000.
> > > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 0000000000000176
> > > from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> > > (XEN) traps.c:2684:d1v0 Domain attempted WRMSR 00000000c0000084
> > > from 0x0000000000074700 to 0x0000000000047700.
> > > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000081
> > > from 0xe023e00800000000 to 0x0023001000000000.
> > > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000082
> > > from 0xffff82d0bffff000 to 0xffffffff817ef990.
> > > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000083
> > > from 0xffff82d0bffff020 to 0xffffffff817f1f60.
> > > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000174
> > > from 0x000000000000e008 to 0x0000000000000010.
> > > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000175
> > > from 0xffff8300866fffc0 to 0x0000000000000000.
> > > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 0000000000000176
> > > from 0xffff82d08023eaf0 to 0xffffffff817f1d60.
> > > (XEN) traps.c:2684:d1v1 Domain attempted WRMSR 00000000c0000084
> > > from 0x0000000000074700 to 0x0000000000047700.
> > > 
> > > Because this is what you just said above, and that's... well...
> > > just
> > > impossible?!?! :-O
> > 
> > This is a Linux PV trying to set up the SYSCALL/SYSENTER MSRs when it
> > shouldn't.  There is a fix upstream, but this specifically is
> > harmless
> > noise.
> > 
> That's ok... What we're arguing is that is happens as a consequence of
> a domain being created, not just "by itself, after boot, without
> starting any domain" !!!! :-)

It has booted. It says 'about to get started...' which is in the early
code of the Linux pvops.

^ permalink raw reply	[flat|nested] 46+ messages in thread

end of thread, other threads:[~2016-02-03 16:57 UTC | newest]

Thread overview: 46+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-18 18:03 Error booting Xen Harmandeep Kaur
2016-01-18 18:09 ` Andrew Cooper
2016-01-18 18:29   ` Dario Faggioli
2016-01-19 13:36     ` Jan Beulich
2016-01-19 14:51       ` Dario Faggioli
2016-01-19 15:07         ` Andrew Cooper
2016-01-19 16:27           ` Harmandeep Kaur
2016-01-19 16:47             ` Jan Beulich
2016-01-19 17:02               ` Andrew Cooper
2016-01-19 17:08                 ` Dario Faggioli
2016-01-19 17:09                   ` Andrew Cooper
2016-01-19 19:55                   ` Harmandeep Kaur
2016-01-20 10:06                     ` Jan Beulich
2016-01-21 15:14                       ` Dario Faggioli
2016-01-21 15:24                         ` Harmandeep Kaur
2016-01-25 13:42                         ` Jan Beulich
2016-01-26 11:58                           ` Dario Faggioli
2016-01-26 12:51                           ` Harmandeep Kaur
2016-01-26 13:13                             ` Harmandeep Kaur
2016-01-26 13:23                               ` Jan Beulich
2016-01-26 17:01                                 ` Harmandeep Kaur
2016-01-26 17:23                                   ` Jan Beulich
2016-01-26 18:02                                     ` Harmandeep Kaur
2016-01-26 18:28                                       ` Dario Faggioli
2016-01-26 18:36                                         ` Harmandeep Kaur
2016-01-27  8:24                                       ` Jan Beulich
2016-01-27 13:12                                       ` Jan Beulich
2016-01-27 13:28                                         ` Harmandeep Kaur
2016-01-27 14:28                                           ` Jan Beulich
2016-01-28 19:17                                             ` Harmandeep Kaur
2016-01-29 10:13                                               ` Jan Beulich
2016-02-03  8:23                                                 ` Dario Faggioli
2016-02-03 11:35                                                   ` Harmandeep Kaur
2016-02-03 12:49                                                     ` Jan Beulich
2016-02-03 12:55                                                     ` Dario Faggioli
2016-02-03 13:10                                                       ` Andrew Cooper
2016-02-03 13:17                                                         ` Dario Faggioli
2016-02-03 16:57                                                           ` Konrad Rzeszutek Wilk
2016-02-03 13:40                                                       ` Harmandeep Kaur
2016-02-03 14:18                                                         ` Jan Beulich
2016-01-26 13:22                             ` Jan Beulich
2016-02-02  4:43                       ` Shuai Ruan
     [not found]                       ` <20160202044349.GA3036@shuai.ruan@linux.intel.com>
2016-02-02 10:39                         ` Andrew Cooper
2016-02-03  0:56                           ` Shuai Ruan
2016-02-03  7:56                           ` Shuai Ruan
2016-01-19 14:07 ` Harmandeep Kaur

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.