I worked with Roman to do several more tests and here is an update on
the situation. We don't know why my patch didn't work when Boris' patch
[1] worked.  Both of them should have worked the same way.

Anyway, we continued with Boris patch to debug the new mmc error which
looks like this:

[    3.084464] mmc0: unrecognised SCR structure version 15
[    3.089176] mmc0: error -22 whilst initialising SD card

I asked to add a lot of trancing, see attached diff. We discovered a bug
in xen_swiotlb_init: if io_tlb_start != 0 at the beginning of
xen_swiotlb_init, start_dma_addr is not set correctly. This oneline
patch fixes it:

diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 0a40ac332a4c..b75c43356eba 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -191,6 +191,7 @@ int __ref xen_swiotlb_init(int verbose, bool early)
         * IO TLB memory already allocated. Just use it.
         */
        if (io_tlb_start != 0) {
+               start_dma_addr = io_tlb_start;
                xen_io_tlb_start = phys_to_virt(io_tlb_start);
                goto end;
        }

Unfortunately it doesn't solve the mmc0 error.


As you might notice from the logs, none of the other interesting printks
printed anything, which seems to mean that the memory allocated by
xen_swiotlb_alloc_coherent and mapped by xen_swiotlb_map_page should be
just fine.

I am starting to be out of ideas. Do you guys have any suggestions on
what could be the issue or what could be done to discover it?

Cheers,

Stefano


On Wed, 6 May 2020, Roman Shaposhnik wrote:
> > So, to recap we have 2 issues as far as I can tell:
> >
> > - virt_to_page not working in some cases on ARM, leading to a crash
> > - WARN_ON for range_straddles_page_boundary which is normal on ARM
> >
> > The appended patch addresses them by:
> >
> > - using pfn_to_page instead virt_to_page
> > - moving the WARN_ON under a #ifdef (Juergen might have a better
> >   suggestion on how to rework the WARN_ON)
> >
> > Please let me know if this patch works!
> >
> > Cheers,
> >
> > Stefano
> >
> >
> > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> > index b6d27762c6f8..0a40ac332a4c 100644
> > --- a/drivers/xen/swiotlb-xen.c
> > +++ b/drivers/xen/swiotlb-xen.c
> > @@ -322,7 +322,7 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
> >                         xen_free_coherent_pages(hwdev, size, ret, (dma_addr_t)phys, attrs);
> >                         return NULL;
> >                 }
> > -               SetPageXenRemapped(virt_to_page(ret));
> > +               SetPageXenRemapped(pfn_to_page(PFN_DOWN(phys)));
> >         }
> >         memset(ret, 0, size);
> >         return ret;
> > @@ -346,9 +346,14 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t size, void *vaddr,
> >         /* Convert the size to actually allocated. */
> >         size = 1UL << (order + XEN_PAGE_SHIFT);
> >
> > -       if (!WARN_ON((dev_addr + size - 1 > dma_mask) ||
> > -                    range_straddles_page_boundary(phys, size)) &&
> > -           TestClearPageXenRemapped(virt_to_page(vaddr)))
> > +#ifdef CONFIG_X86
> > +       if (WARN_ON(dev_addr + size - 1 > dma_mask) ||
> > +                   range_straddles_page_boundary(phys, size)) {
> > +           return;
> > +       }
> > +#endif
> > +
> > +       if (TestClearPageXenRemapped(pfn_to_page(PFN_DOWN(phys))))
> >                 xen_destroy_contiguous_region(phys, order);
> >
> >         xen_free_coherent_pages(hwdev, size, vaddr, (dma_addr_t)phys, attrs);
> 
> Stefano, with your patch applied, I'm still getting:
> 
> [    0.590705] Unable to handle kernel paging request at virtual
> address fffffe0003700000
> 
> However, Boris' patch seems to get us much closer. It would be awesome if you
> can take a look at that (plus the additional DMA issue that seems to
> be dependent
> on how much memory I allocate to Dom0).