From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23CD7C433E0 for ; Wed, 13 May 2020 19:50:24 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ABDC3205ED for ; Wed, 13 May 2020 19:50:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="wWUYIolu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABDC3205ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jYxO7-0005ZP-Qg; Wed, 13 May 2020 19:49:55 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jYxO7-0005ZK-1z for xen-devel@lists.xenproject.org; Wed, 13 May 2020 19:49:55 +0000 X-Inumbo-ID: ea0569a0-9552-11ea-a401-12813bfff9fa Received: from mail.kernel.org (unknown [198.145.29.99]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id ea0569a0-9552-11ea-a401-12813bfff9fa; Wed, 13 May 2020 19:49:53 +0000 (UTC) Received: from localhost (c-67-164-102-47.hsd1.ca.comcast.net [67.164.102.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6A42720659; Wed, 13 May 2020 19:49:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589399393; bh=A6OIMY+x0cwZr6eTl0RCx+93TS1h+MYJs+vrefIb5dc=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=wWUYIolus5RDYmnCxmhhw7kPQGmhmpVQHdXkquXs7/iBMdeyGq7tU/wHM8hg4Z4Hi lJ6jJrAr/TtDDrjar4RcOrcC39O//2Wp9fIEouWsykBc5K177Ev2rgaPYoCEURl/VF gw5zut3YqgaGFnrGf5TCG9J9+YpM/T4/vCoa49AQ= Date: Wed, 13 May 2020 12:49:51 -0700 (PDT) From: Stefano Stabellini X-X-Sender: sstabellini@sstabellini-ThinkPad-T480s To: Julien Grall Subject: Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU In-Reply-To: Message-ID: References: <9ee0fb6f-98df-d993-c42e-f47270bf2eaa@suse.com> <2187cd7c-4d48-986b-77d6-4428e8178404@oracle.com> <42253259-9663-67e8-117f-8ba631243585@xen.org> User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: jgross@suse.com, Peng Fan , Stefano Stabellini , "minyard@acm.org" , roman@zededa.com, "jeff.kubascik@dornerworks.com" , Julien Grall , Nataliya Korovkina , "xen-devel@lists.xenproject.org" , boris.ostrovsky@oracle.com, Stefano Stabellini Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" On Wed, 13 May 2020, Julien Grall wrote: > On 13/05/2020 16:11, Stefano Stabellini wrote: > > On Wed, 13 May 2020, Julien Grall wrote: > > > Hi, > > > > > > On 13/05/2020 01:33, Stefano Stabellini wrote: > > > > I worked with Roman to do several more tests and here is an update on > > > > the situation. We don't know why my patch didn't work when Boris' patch > > > > [1] worked. Both of them should have worked the same way. > > > > > > > > Anyway, we continued with Boris patch to debug the new mmc error which > > > > looks like this: > > > > > > > > [ 3.084464] mmc0: unrecognised SCR structure version 15 > > > > [ 3.089176] mmc0: error -22 whilst initialising SD card > > > > > > > > I asked to add a lot of trancing, see attached diff. > > > > > > Please avoid attachment on mailing list and use pastebin for diff. > > > > > > > We discovered a bug > > > > in xen_swiotlb_init: if io_tlb_start != 0 at the beginning of > > > > xen_swiotlb_init, start_dma_addr is not set correctly. This oneline > > > > patch fixes it: > > > > > > > > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c > > > > index 0a40ac332a4c..b75c43356eba 100644 > > > > --- a/drivers/xen/swiotlb-xen.c > > > > +++ b/drivers/xen/swiotlb-xen.c > > > > @@ -191,6 +191,7 @@ int __ref xen_swiotlb_init(int verbose, bool early) > > > > * IO TLB memory already allocated. Just use it. > > > > */ > > > > if (io_tlb_start != 0) { > > > > + start_dma_addr = io_tlb_start; > > > > xen_io_tlb_start = phys_to_virt(io_tlb_start); > > > > goto end; > > > > } > > > > > > > > Unfortunately it doesn't solve the mmc0 error. > > > > > > > > > > > > As you might notice from the logs, none of the other interesting printks > > > > printed anything, which seems to mean that the memory allocated by > > > > xen_swiotlb_alloc_coherent and mapped by xen_swiotlb_map_page should be > > > > just fine. > > > > > > > > I am starting to be out of ideas. Do you guys have any suggestions on > > > > what could be the issue or what could be done to discover it? > > > > > > Sorry if my suggestions are going to be obvious, but I can't confirm > > > whether > > > this was already done: > > > 1) Does the kernel boot on baremetal (i.e without Xen)? This should > > > help > > > to confirm whether the bug is Xen is related. > > > > Yes it boots > > > > > 2) Swiotlb should not be necessary for basic dom0 boot on Arm. Did > > > you try > > > to disable it? This should help to confirm whether swiotlb is the problem > > > or > > > not. > > > > It boots disabling swiotlb-xen > > Thank you for the answer! swiotlb-xen should basically be a NOP for dom0. So > this suggests swiotlb is doing some transformation on the DMA address. Yes, but couldn't spot where yet > I have an idea what may have gone wrong. AFAICT, xen-swiotlb seems to assume > the DMA address space and CPU address space is the same. Is RPI using the same > address space? I thought about it too but we didn't add an explicit check for this yet. My understanding is that it is not supposed to happen on arm64 boards but at this point it is worth checking it for sure. Thanks for the suggestion.