From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29906C433ED for ; Fri, 14 May 2021 13:41:53 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 936D0611CE for ; Fri, 14 May 2021 13:41:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 936D0611CE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eBEx0CCX2R+x1bE77BcJu6fvtYx0u4bathtXXFhankk=; b=QWJM0VdBLRdl024OeF8OeE3FO yRreHvvfkH2+Xyn0KQLX+xoretl02/76txJ2XD3aJMnCgXCnSAaeyXTh3GP9Ro5a1dytQn1rVN5TC Ei6k6cif1cNU+JJeb9WHepAEKzymE9+uju8mXaX8xpe4EqSou2PEhtKrRbjuD6AJ/aZSBUUg8xPYw fbLDP+8WrkDghATGcKrnksDfDG0cRPq3e7BHW4KfZmxmIlgzKEuR5s6xnfSPSKCRm3h05n9NsMdRZ PxHbPwS0owzGPvuH11VyCwe1zzLQShACcHKH7unR26Coc6qQ/PZypxgBkyx8HZYxQA7ZUhL51dn/p GYBagxzag==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lhY2e-0089pq-Jz; Fri, 14 May 2021 13:39:48 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lhY2c-0089pM-IU for linux-arm-kernel@desiato.infradead.org; Fri, 14 May 2021 13:39:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=fw8jRpLJUUiDLFl+R/OeLe/+vcSlILAW8JDLv7Pnlgw=; b=zIjUVOfw+dWXDC6k9+iagU/Qi6 2El/n5s0cab7KbDQYWRumxdUUJJWRNn2NEL++Wws+Bh/lmpxH0rGwykTGGuAUCRGqjRuvMP6KMyN6 76olUCLJmCKUDJu3GDNcDkkVRIbdh822g5boHEu/612qZWsoGcJqc5ieJEhtE7nMZ8L2Zho28sv5a QFm5wC3KRxAL1hxksFT2uji8kOVINxRetoNZASkn7QUAjuNfdH+DUaZ53x9+xPcJStEyVS7z0c/J+ 2WrQdAo6qEPEBgF8pv1cyBhqVsyVf8/+Nq2wAhhluGYYjrGC/R+5ubin+HlBvH7M8GeXcVXeXicSz PEJ0bONw==; Received: from mail-qt1-x82b.google.com ([2607:f8b0:4864:20::82b]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lhY2Z-00C0S5-Kv for linux-arm-kernel@lists.infradead.org; Fri, 14 May 2021 13:39:45 +0000 Received: by mail-qt1-x82b.google.com with SMTP id k19so2787012qta.2 for ; Fri, 14 May 2021 06:39:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=fw8jRpLJUUiDLFl+R/OeLe/+vcSlILAW8JDLv7Pnlgw=; b=ZA9iIphKPPHvcLSkBu2C/QMxUDhXyUp7Ql0A7zhq+gl8tX2VVw4d6MWbL3YGt3zMia eovdKfEgpKv/Kc6JgMp9UCw9S8MZR5nPFduJNr7x/z0XbVlGgNLXVftzfUVKBu0b6ABm RinPPw/eYvAo5xq2FjGO1ry+PBwldBVbyqtEvWJumFPyH6ThWymQ9gXFH0NAf+gQaVpn M9srCHz/FBZ/gl8Q7AQ70j9xSGfCbxZD9KMqMhWJ8ejqfvSLeS76TlpPfl+r5p2DMKcd O1K+SZm8GDVksZO0NYGjA4OCifqgsBaX7QsLUNiJf774sdooSmqhK4/lbsOJfcz7ocEi FQ+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=fw8jRpLJUUiDLFl+R/OeLe/+vcSlILAW8JDLv7Pnlgw=; b=Nl8g0v4Kmh+4g0jWuE1/JBbnLVJbc0kl9MeMyEwdDWiSgF6y5gA0DmmQBn7tXOi71O 9IxJwaoCP6BM+jV1cNWxiE/eBuOkS102N/bkHp8kLMAhZzvJR/zU8Lqd9iX0SjlSx6rH 09bhghhHPRn5keJQqJw/w+LjFb4Snc1cNIyE2I4KyYOy+ucdh9FCFgO/EJ7cNTZEK/fA d1VcAn1Kxa7pxJ4JfUWbqVS7fjSshnzheP0RlS97VfnRdN+FbD91ZP2/1ktRsmk5BOzF r2AaMD+wJoSM4L/AmlADruHSEfHNqchaJq4y/ENf35amvTS3Y3xtwZSo/j7yagz0btQT XxFg== X-Gm-Message-State: AOAM533wCAPEfRySajGkph5a3HQEFfhzMDP+yI/ZV+aQXgtBPeLKLPdl orQ8NjFQvBcIbnl3wt7YsBbqvw== X-Google-Smtp-Source: ABdhPJyHkmiobTC20DDQbl/CkwGCuAAsPOfAEooNlfis4Grfk/a3JX140lwF9zGRuivL/kulThANeQ== X-Received: by 2002:a05:622a:5c9:: with SMTP id d9mr14474203qtb.177.1620999582009; Fri, 14 May 2021 06:39:42 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-47-55-113-94.dhcp-dynamic.fibreop.ns.bellaliant.net. [47.55.113.94]) by smtp.gmail.com with ESMTPSA id 11sm4839279qkk.31.2021.05.14.06.39.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 May 2021 06:39:41 -0700 (PDT) Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lhY2V-007RfW-Pl; Fri, 14 May 2021 10:39:39 -0300 Date: Fri, 14 May 2021 10:39:39 -0300 From: Jason Gunthorpe To: "Tian, Kevin" Cc: Christoph Hellwig , Joerg Roedel , Alex Williamson , David Woodhouse , Lu Baolu , Will Deacon , Kirti Wankhede , "linux-arm-kernel@lists.infradead.org" , "iommu@lists.linux-foundation.org" , "kvm@vger.kernel.org" Subject: Re: [PATCH 3/6] vfio: remove the unused mdev iommu hook Message-ID: <20210514133939.GL1096940@ziepe.ca> References: <20210510065405.2334771-1-hch@lst.de> <20210510065405.2334771-4-hch@lst.de> <20210510155454.GA1096940@ziepe.ca> <20210513120058.GG1096940@ziepe.ca> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210514_063943_715643_D8F1BE4F X-CRM114-Status: GOOD ( 30.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, May 14, 2021 at 01:17:23PM +0000, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Thursday, May 13, 2021 8:01 PM > > > > On Thu, May 13, 2021 at 03:28:52AM +0000, Tian, Kevin wrote: > > > > > Are you specially concerned about this iommu_device hack which > > > directly connects mdev_device to iommu layer or the entire removed > > > logic including the aux domain concept? For the former we are now > > > following up the referred thread to find a clean way. But for the latter > > > we feel it's still necessary regardless of how iommu interface is redesigned > > > to support device connection from the upper level driver. The reason is > > > that with mdev or subdevice one physical device could be attached to > > > multiple domains now. there could be a primary domain with DOMAIN_ > > > DMA type for DMA_API use by parent driver itself, and multiple auxiliary > > > domains with DOMAIN_UNMANAGED types for subdevices assigned to > > > different VMs. > > > > Why do we need more domains than just the physical domain for the > > parent? How does auxdomain appear in /dev/ioasid? > > > > Another simple reason. Say you have 4 mdevs each from a different > parent are attached to an ioasid. If only using physical domain of the > parent + PASID it means there are 4 domains (thus 4 page tables) under > this IOASID thus every dma map operation must be replicated in all > 4 domains which is really unnecessary. Having the domain created > with ioasid and allow a device attaching to multiple domains is much > cleaner for the upper-layer drivers to work with iommu interface. Eh? That sounds messed up. The IOASID is the page table. If you have one IOASID and you attach it to 4 IOMMU routings (be it pasid, rid, whatever) then there should only ever by one page table. The iommu driver should just point the iommu HW at the shared page table for each of the 4 routes and be done with it. At worst it has to replicate invalidates, but that is very HW dependent. Domain is just a half-completed-ioasid concept. It is today not flexible enough to be a true IOASID, but it still does hold the io page table. Basically your data structure is an IOASID. It holds a single HW specific page table. The IOASID has a list of RID and (RID,PASID) that are authorized to use it. The IOMMU HW is programed to match the RID/(RID,PASID) list and search the single page table. When the page table is changed the IOMMU is told to dump caches, however that works. When a device driver wants to use an IOASID it tells the iommu to setup the route, either RID or (RID,PASID). Setting the route causes the IOMMU driver to update the HW. The struct device has enough context to provide the RID and the IOMMU driver connection when operating on the IOASID. Jason _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel