Re: Xen-unstable PVHdom0: Assertion 'IS_ALIGNED(dfn_x(dfn), (1ul << page_order))' failed at iommu.c:324

["Roger Pau Monné" <roger.pau@citrix.com>]

On Thu, Jan 24, 2019 at 12:55:06PM +0100, Sander Eikelenboom wrote:
> On 24/01/2019 11:11, Roger Pau Monné wrote:
> > On Thu, Jan 24, 2019 at 10:25:33AM +0100, Sander Eikelenboom wrote:
> >> On 24/01/2019 08:50, Roger Pau Monné wrote:
> >>> On Wed, Jan 23, 2019 at 08:56:48PM +0100, Sander Eikelenboom wrote:
> >>>> On 23/01/2019 19:25, Roger Pau Monné wrote:
> >>>>> On Wed, Jan 23, 2019 at 12:39:21AM +0100, Sander Eikelenboom wrote:
> >>>>>> On 22/01/2019 17:14, Roger Pau Monné wrote:
> >>>>>>> On Sun, Jan 20, 2019 at 11:09:25PM +0100, Sander Eikelenboom wrote:
> >>>>>>>> On 18/01/2019 18:56, Roger Pau Monné wrote:
> >>>>>>>>> On Fri, Jan 18, 2019 at 03:17:57PM +0100, Sander Eikelenboom wrote:
> >>>>>>>>>> On 18/01/2019 13:50, Roger Pau Monné wrote:
> >>>>>>>>>>> On Fri, Jan 18, 2019 at 01:03:04PM +0100, Sander Eikelenboom wrote:
> >>>>>>>>>>>> Hi Roger,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I gave PVH dom0 a spin, see how far I would get.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>>
> >>>>>>>>>>>> With current xen-unstable unfortunately not that far, i got the splat below.
> >>>>>>>>>>>
> >>>>>>>>>>> Yes, this was already reported:
> >>>>>>>>>>>
> >>>>>>>>>>> https://lists.xenproject.org/archives/html/xen-devel/2019-01/msg01030.html
> >>>>>>>>>>>> If you need more info, would like me to test a patch (or some other git tree/branch), 
> >>>>>>>>>>>> I will be happy to give it a spin !
> >>>>>>>>>>>
> >>>>>>>>>>> Paul is working on a fix, but in the meantime just removing the
> >>>>>>>>>>> assertions should be fine:
> >>>>>>>>>>>
> >>>>>>>>>>> ---8<---
> >>>>>>>>>>> diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
> >>>>>>>>>>> index bd1af35a13..98e6fc35e2 100644
> >>>>>>>>>>> --- a/xen/drivers/passthrough/iommu.c
> >>>>>>>>>>> +++ b/xen/drivers/passthrough/iommu.c
> >>>>>>>>>>> @@ -321,9 +321,6 @@ int iommu_map(struct domain *d, dfn_t dfn, mfn_t mfn,
> >>>>>>>>>>>      if ( !iommu_enabled || !hd->platform_ops )
> >>>>>>>>>>>          return 0;
> >>>>>>>>>>>  
> >>>>>>>>>>> -    ASSERT(IS_ALIGNED(dfn_x(dfn), (1ul << page_order)));
> >>>>>>>>>>> -    ASSERT(IS_ALIGNED(mfn_x(mfn), (1ul << page_order)));
> >>>>>>>>>>> -
> >>>>>>>>>>>      for ( i = 0; i < (1ul << page_order); i++ )
> >>>>>>>>>>>      {
> >>>>>>>>>>>          rc = hd->platform_ops->map_page(d, dfn_add(dfn, i), mfn_add(mfn, i),
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> I gave that a spin and i now get a seemingly endless stream of IO_PAGE_FAULTs
> >>>>>>>>>
> >>>>>>>>> You shouldn't get those page faults since they are for addresses that
> >>>>>>>>> belong to a reserved region, and that should be mapped into the p2m.
> >>>>>>>>> I've just tested on my AMD box and I'm also seeing errors (albeit
> >>>>>>>>> different ones), so I guess something broke since I last fixed PVH
> >>>>>>>>> Dom0 to boot on AMD hardware.
> >>>>>>>>>
> >>>>>>>>> I've also tested commit:
> >>>>>>>>>
> >>>>>>>>> commit fad6ba64a8c98bebb9374f390cc255fac05237ab (HEAD)
> >>>>>>>>> Author: Roger Pau Monné <roger.pau@citrix.com>
> >>>>>>>>> Date:   Fri Nov 30 12:10:00 2018 +0100
> >>>>>>>>> amd/iommu: skip host bridge devices when updating IOMMU page tables
> >>>>>>>>>
> >>>>>>>>> And it works on my AMD box and I'm able to boot as a PVH Dom0. Can you
> >>>>>>>>> give this commit a spin?
> >>>>>>>>>
> >>>>>>>>> Thanks, Roger.
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> Hi Roger,
> >>>>>>>>
> >>>>>>>> Tested that commit, but that didn't help.
> >>>>>>>
> >>>>>>> Thanks! Sorry for the delay, I got sidetracked with something else.
> >>>>>>
> >>>>>> No problem, it's not too urgent and probably a busy time with the remaining 4.12 stuff.
> >>>>>>  
> >>>>>>> Can you please post the serial log when using the above commit?
> >>>>>>
> >>>>>> Sure, I attached a log of:
> >>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PVH unsuccesful boot
> >>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PV    succesful boot
> >>>>>
> >>>>> Thanks. So you get the same IO page faults.
> >>>>>
> >>>>> I don't seem to be able to reproduce this behaviour on my AMD box, but
> >>>>> that might be just luck. I've been finding some issues today related
> >>>>> to the IOMMU, could you give the following patch a spin and paste the
> >>>>> serial log that you get.
> >>>>
> >>>> Hi Roger,
> >>>>
> >>>> Sure, on top of what ?
> >>>> - fad6ba64a8c98bebb9374f390cc255fac05237ab ?
> >>>> - xen-unstable ?
> >>>> - xen-unstable + Paul's patch ?
> >>>
> >>> Hello,
> >>>
> >>> Sorry for not proving the right context, let's try on top of
> >>> xen-unstable + Paul's patch.
> >>>
> >>> Thanks, Roger.
> >>>
> >>
> >> Seems to be giving the same result (stream of IO_PAGE_FAULTs), serial
> >> log attached.
> > 
> > Thanks, I think I've figured out what's wrong. I've prepared a git
> > branch for you to test:
> > 
> > git://xenbits.xen.org/people/royger/xen.git iommu-fixes
> > 
> > Could you give this a try?
> > 
> > Thanks, Roger.
> 
> Hi Roger,
> 
> The good news is, with that branch, dom0 boots on both PVH and PV !
> The bad news is, I can't boot up a PVH guest (but perhaps this warrants a separate mail thread / change of subject):
> 
> xc: error: panic: xc_dom_boot.c:159: xc_dom_boot_domU_map: failed to mmap domU pages 0x1000+0x2426 [mmap, errno=22 (Invalid argument)]: Internal error
> libxl: error: libxl_dom.c:760:libxl__build_dom: xc_dom_build_image failed: Invalid argument
> libxl: error: libxl_create.c:1286:domcreate_rebuild_done: Domain 1:cannot (re-)build domain: -3
> libxl: error: libxl_domain.c:1038:libxl__destroy_domid: Domain 1:Non-existant domain
> libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 1:Unable to destroy guest
> libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 1:Destruction of domain failed
> 
> There is more in the attached serial.log
> 
> Thanks so far !

Thank you for the testing!

Let me clean the branch a little bit and prepare a new patch.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel