From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86BDAC3A5A6 for ; Thu, 19 Sep 2019 15:18:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4BEC620665 for ; Thu, 19 Sep 2019 15:18:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Yqn2hP5/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404005AbfISPSu (ORCPT ); Thu, 19 Sep 2019 11:18:50 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:42109 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389171AbfISPSu (ORCPT ); Thu, 19 Sep 2019 11:18:50 -0400 Received: by mail-ed1-f66.google.com with SMTP id y91so3533663ede.9 for ; Thu, 19 Sep 2019 08:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=Yqn2hP5/7LoUE96rW6LgXvDeRFDoEgSIYLWO331yu/t8KvfcDLYs0svMMqhPl0nEKU 3028G05kj86vZEYdH0BT8B4Snvw/8TzsFj/6z3cDmgcLrYqAf9CTKaxFxWz6TetvRVWD JaOhJxFoN6Lysw3uc3I7TCd+bGbTAYEQdtrh530GBo5Aqg2kjpsReLMDlMYPva23+thK 3paKUHNXhfjBIBDEcc9d7OpRoL4M9YEswI1G3XEqU9hMkBTILBRZAfv3lk1GyAApc7kF uBVf89FvXfuH3jiXShL1/vpa79fW+aGEKUGlC597SIRD1tjHojMUhcUCeA7TA1Um/ffs UzGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=JTiSXK0YDQ6PJ0UzAuUCZ7IORqNVCH7gQ3DAKf87Cj/evHlUVBRcEk5ITxBBZGLJCN nOqOxY921OhqLO170S9NdmbvokoHEAIz7OeuG0vAz2Yeg801NsVaWmQAZ00Ah/DkzqVs EyfjTOgY+K2gJptiTOT7gaUSZiE6NVhSOhN5pbh8lj48usC31+iCY4n4rEP8EC9BSZda 3UistVnimseXcvpHR4yrYVB/bpP+pv8lg84hwTHcjAkjFDhvQPuiI54n8Ky5WpDffbIw z6csO2RuurpokXws5yaMn4XEhdJAHqmYgUpz7FL+5o318siIbLoZbJWGMmVMlL2qRJKx qGaA== X-Gm-Message-State: APjAAAUjFk5FNyJZklyBVPr2/8NG0O1mcgNcFQViQLeh03VSE/v5l/qo Ha5HhBYZcPQP60R5X+7YefdH6Q== X-Google-Smtp-Source: APXvYqyjE5Z/ExpKjAFeUwQaYOlsDCBhURZQ6GgY1ZAERUi1xVf6/dGQvcGXJsfycKEfjO1ON7WJpA== X-Received: by 2002:a17:906:6a92:: with SMTP id p18mr14982887ejr.253.1568906327557; Thu, 19 Sep 2019 08:18:47 -0700 (PDT) Received: from lophozonia ([85.195.192.192]) by smtp.gmail.com with ESMTPSA id x4sm763681eds.87.2019.09.19.08.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Sep 2019 08:18:46 -0700 (PDT) Date: Thu, 19 Sep 2019 17:18:44 +0200 From: Jean-Philippe Brucker To: Guo Ren Cc: Will Deacon , Catalin Marinas , Palmer Dabbelt , christoffer.dall@arm.com, Atish Patra , Julien Grall , gary@garyguo.net, linux-riscv@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Mike Rapoport , aou@eecs.berkeley.edu, Arnd Bergmann , suzuki.poulose@arm.com, Marc Zyngier , Paul Walmsley , linux-arm-kernel@lists.infradead.org, Anup Patel , Linux Kernel Mailing List , iommu@lists.linux-foundation.org, james.morse@arm.com Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Message-ID: <20190919151844.GG1013538@lophozonia> References: <20190619123939.GF7767@fuggles.cambridge.arm.com> <20190624104006.lvm32nahemaqklxc@willie-the-truck> <20190912140256.fwbutgmadpjbjnab@willie-the-truck> <057a0af3-93f7-271c-170e-4b31e6894c3c@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Sep 19, 2019 at 09:07:15PM +0800, Guo Ren wrote: > > The solution I had to this problem is pinning the ASID [1] used by the > > IOMMU, to prevent the CPU from recycling the ASID on rollover. This way > > the CPU doesn't have to wait for IOMMU invalidations to complete, when > > scheduling a task that might not even have anything to do with the IOMMU. > > > > > In the Arm SMMU, ASID and IOASID (PASID) are separate identifiers. IOASID > > indexes an entry in the context descriptor table, which contains the ASID. > > So with unpinned shared ASID you don't need to invalidate the ATC on > > rollover, since the IOASID doesn't change, but you do need to modify the > > context descriptor and invalidate cached versions of it. > The terminology confused me a lot. I perfer use PASID for IOMMU and > ASID is for CPU. > Arm's entry of the context descriptor table contains a "IOASID" The terminology I've been using so far is different: * IOASID is PASID * The entry in the context descriptor table contains an ASID, which is either "shared" with CPUs or "private" to the SMMU (the SMMU spec says "shared" or "non-shared"). * So the CPU and SMMU TLBs use ASIDs, and the PCI ATC uses IOASID > IOASID != ASID for CPU_TLB and IOMMU_TLB. > > When you say "since the IOASID doesn't change",Is it PASID or my IOASID ? -_*! I was talking about PASID. Maybe we can drop "IOASID" and talk only about ASID and PASID :) > PASID in PCI-sig was used to determine transfer address space. > For intel, the entry which is indexed by PASID also contain S1/S2.PGD > and DID(VMID). > For arm, the entry which is indexed by PASID only contain S1.PGD and > IOASID. Compare to Intel Vt-d Scalable mode, arm's design can't > support PCI Virtual Function. The SMMU does support PCI Virtual Function - an hypervisor can assign a VF to a guest, and let that guest partition the VF into smaller contexts by using PASID. What it can't support is assigning partitions of a PCI function (VF or PF) to multiple Virtual Machines, since there is a single S2 PGD per function (in the Stream Table Entry), rather than one S2 PGD per PASID context. Thanks, Jean > > Once you have pinned ASIDs, you could also declare that IOASID = ASID. I > > don't remember finding an argument to strictly forbid it, even though ASID > > and IOASID have different sizes on Arm (respectively 8/16 and 20 bits). > ASID and IOASID are hard to keep the same between CPU system and IOMMU > system. So I introduce S1/S2.PGD.PPN as a bridge between CPUs and > IOMMUs. > See my proposal [1] > > 1: https://lore.kernel.org/linux-csky/1568896556-28769-1-git-send-email-guoren@kernel.org/T/#u > -- > Best Regards > Guo Ren > > ML: https://lore.kernel.org/linux-csky/ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEFABC49ED7 for ; Thu, 19 Sep 2019 15:18:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 94FE920665 for ; Thu, 19 Sep 2019 15:18:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="o9wxNjXf"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Yqn2hP5/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 94FE920665 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+infradead-linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=im6A2UWtNzGMuhro3SYmFNsZ6SR9azUj7/SoGJO8u/k=; b=o9wxNjXfJt7pNQ UipTleyd7lHS5ZSZPyFHIBP0+7KleMBHEFgMPP4LP3OOjea99EDSNQYnvTV2YzBcqZ4m6BxAtuX0f Gl+MViVdTbpEIPpfhFDzed2SXFywqIxuigcXJ2QDUtMtWLY4x807AIU15xoOxxAJUl1cZ61C2TlNr GXx1C3Hds/x7RLDtX2/2WIzG7Yd+mVeA2g0iG/rIdwil33pqNRqvHmKpIgKKmr9Su+EUD3en9/fR1 VDmClouEq1OtXwU7d0pHNwP7dsxbLCZz9jSrF4+XqHmZGdtnrAxBBjE9+HTP33LUU1wSo5j8T6p0/ UDICB3Hp10z4XP0RdRCw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1iAyCr-0001Jq-Rd; Thu, 19 Sep 2019 15:18:53 +0000 Received: from mail-ed1-x544.google.com ([2a00:1450:4864:20::544]) by bombadil.infradead.org with esmtps (Exim 4.92.2 #3 (Red Hat Linux)) id 1iAyCn-0001J2-4C for linux-riscv@lists.infradead.org; Thu, 19 Sep 2019 15:18:51 +0000 Received: by mail-ed1-x544.google.com with SMTP id c4so3597153edl.0 for ; Thu, 19 Sep 2019 08:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=Yqn2hP5/7LoUE96rW6LgXvDeRFDoEgSIYLWO331yu/t8KvfcDLYs0svMMqhPl0nEKU 3028G05kj86vZEYdH0BT8B4Snvw/8TzsFj/6z3cDmgcLrYqAf9CTKaxFxWz6TetvRVWD JaOhJxFoN6Lysw3uc3I7TCd+bGbTAYEQdtrh530GBo5Aqg2kjpsReLMDlMYPva23+thK 3paKUHNXhfjBIBDEcc9d7OpRoL4M9YEswI1G3XEqU9hMkBTILBRZAfv3lk1GyAApc7kF uBVf89FvXfuH3jiXShL1/vpa79fW+aGEKUGlC597SIRD1tjHojMUhcUCeA7TA1Um/ffs UzGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=HfNW1CxL0G1Ob9BAeOgJpV72i6M3xyYyXejhOvU093i+QIIQ0y2uE6gKTg6uBMZPz2 0YYwrctwwwNAs1R8cdrGG6J7pPA4T6BHQK+jUirlUIGPvkJiBMbcxBsRRvSV2kCTHFqt rRaviSkwrMa8B7vdJZKAIdvlh92COe57rexBfFiJCCj0TL3lryGsY8u9ndHq/3/2BfLY u4LHzabJFOR7XE1GTYKrHLLa60HDlLf2o8OZXF3JnOZQMnQOSFvUPkYhEdQiz30XvlUv RqcdmqwIUrX6GAH13Li5WSwFvXXSRbL4FyG6MaDuyMqt/rPXbVCeUrg32WxDCFu30Oay ZFEw== X-Gm-Message-State: APjAAAW6iWaNTOEyhmBAd5iQrq7siY3GVL3z08dOBilMOXM9UEWLi7NO XA8sEM2vfdqZFQkEeoSM5/iAFA== X-Google-Smtp-Source: APXvYqyjE5Z/ExpKjAFeUwQaYOlsDCBhURZQ6GgY1ZAERUi1xVf6/dGQvcGXJsfycKEfjO1ON7WJpA== X-Received: by 2002:a17:906:6a92:: with SMTP id p18mr14982887ejr.253.1568906327557; Thu, 19 Sep 2019 08:18:47 -0700 (PDT) Received: from lophozonia ([85.195.192.192]) by smtp.gmail.com with ESMTPSA id x4sm763681eds.87.2019.09.19.08.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Sep 2019 08:18:46 -0700 (PDT) Date: Thu, 19 Sep 2019 17:18:44 +0200 From: Jean-Philippe Brucker To: Guo Ren Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Message-ID: <20190919151844.GG1013538@lophozonia> References: <20190619123939.GF7767@fuggles.cambridge.arm.com> <20190624104006.lvm32nahemaqklxc@willie-the-truck> <20190912140256.fwbutgmadpjbjnab@willie-the-truck> <057a0af3-93f7-271c-170e-4b31e6894c3c@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190919_081849_191272_FF5F6F95 X-CRM114-Status: GOOD ( 21.02 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aou@eecs.berkeley.edu, Linux Kernel Mailing List , Arnd Bergmann , suzuki.poulose@arm.com, Marc Zyngier , Catalin Marinas , Palmer Dabbelt , christoffer.dall@arm.com, iommu@lists.linux-foundation.org, Mike Rapoport , Anup Patel , Atish Patra , Julien Grall , james.morse@arm.com, gary@garyguo.net, Paul Walmsley , linux-riscv@lists.infradead.org, Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+infradead-linux-riscv=archiver.kernel.org@lists.infradead.org On Thu, Sep 19, 2019 at 09:07:15PM +0800, Guo Ren wrote: > > The solution I had to this problem is pinning the ASID [1] used by the > > IOMMU, to prevent the CPU from recycling the ASID on rollover. This way > > the CPU doesn't have to wait for IOMMU invalidations to complete, when > > scheduling a task that might not even have anything to do with the IOMMU. > > > > > In the Arm SMMU, ASID and IOASID (PASID) are separate identifiers. IOASID > > indexes an entry in the context descriptor table, which contains the ASID. > > So with unpinned shared ASID you don't need to invalidate the ATC on > > rollover, since the IOASID doesn't change, but you do need to modify the > > context descriptor and invalidate cached versions of it. > The terminology confused me a lot. I perfer use PASID for IOMMU and > ASID is for CPU. > Arm's entry of the context descriptor table contains a "IOASID" The terminology I've been using so far is different: * IOASID is PASID * The entry in the context descriptor table contains an ASID, which is either "shared" with CPUs or "private" to the SMMU (the SMMU spec says "shared" or "non-shared"). * So the CPU and SMMU TLBs use ASIDs, and the PCI ATC uses IOASID > IOASID != ASID for CPU_TLB and IOMMU_TLB. > > When you say "since the IOASID doesn't change",Is it PASID or my IOASID ? -_*! I was talking about PASID. Maybe we can drop "IOASID" and talk only about ASID and PASID :) > PASID in PCI-sig was used to determine transfer address space. > For intel, the entry which is indexed by PASID also contain S1/S2.PGD > and DID(VMID). > For arm, the entry which is indexed by PASID only contain S1.PGD and > IOASID. Compare to Intel Vt-d Scalable mode, arm's design can't > support PCI Virtual Function. The SMMU does support PCI Virtual Function - an hypervisor can assign a VF to a guest, and let that guest partition the VF into smaller contexts by using PASID. What it can't support is assigning partitions of a PCI function (VF or PF) to multiple Virtual Machines, since there is a single S2 PGD per function (in the Stream Table Entry), rather than one S2 PGD per PASID context. Thanks, Jean > > Once you have pinned ASIDs, you could also declare that IOASID = ASID. I > > don't remember finding an argument to strictly forbid it, even though ASID > > and IOASID have different sizes on Arm (respectively 8/16 and 20 bits). > ASID and IOASID are hard to keep the same between CPU system and IOMMU > system. So I introduce S1/S2.PGD.PPN as a bridge between CPUs and > IOMMUs. > See my proposal [1] > > 1: https://lore.kernel.org/linux-csky/1568896556-28769-1-git-send-email-guoren@kernel.org/T/#u > -- > Best Regards > Guo Ren > > ML: https://lore.kernel.org/linux-csky/ _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA230C4CEC4 for ; Thu, 19 Sep 2019 15:18:51 +0000 (UTC) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7D25A21907 for ; Thu, 19 Sep 2019 15:18:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Yqn2hP5/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D25A21907 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 7030ACCC; Thu, 19 Sep 2019 15:18:50 +0000 (UTC) Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id C7A80C3A for ; Thu, 19 Sep 2019 15:18:49 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id E49BF711 for ; Thu, 19 Sep 2019 15:18:48 +0000 (UTC) Received: by mail-ed1-f67.google.com with SMTP id h2so3575485edn.3 for ; Thu, 19 Sep 2019 08:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=Yqn2hP5/7LoUE96rW6LgXvDeRFDoEgSIYLWO331yu/t8KvfcDLYs0svMMqhPl0nEKU 3028G05kj86vZEYdH0BT8B4Snvw/8TzsFj/6z3cDmgcLrYqAf9CTKaxFxWz6TetvRVWD JaOhJxFoN6Lysw3uc3I7TCd+bGbTAYEQdtrh530GBo5Aqg2kjpsReLMDlMYPva23+thK 3paKUHNXhfjBIBDEcc9d7OpRoL4M9YEswI1G3XEqU9hMkBTILBRZAfv3lk1GyAApc7kF uBVf89FvXfuH3jiXShL1/vpa79fW+aGEKUGlC597SIRD1tjHojMUhcUCeA7TA1Um/ffs UzGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=Zs76UHLT2MheUsTdtLarbrLcqrhzQA+1tcLJNgnigNeaVeFzDlsyrbpReudNjXciys J7oMZyQ54UHn6Ij7+6Ku3AOOFjqlb3yr/5CrL3KmixEweecIUrnFnPC/HbNIJHCeysOO N+UEAVUANiSAeYRq0di6BFDbR4Hw2EzWMpTaZYSJUNC7z+tbtCAbamZpMW8hRPy/LIe7 6hku6An4x1M7adVhbgTGcG9N1540rqTRXXvaFmrr6zpuLP6O0qiLhySy0M46YOX54ctr f7rdTImkj/pIt9CHtkbNDellwnerV0WQxJ4Bgena0sxI81nL/wlqpZhqYQ8WMLqcWRsm ZTcw== X-Gm-Message-State: APjAAAULi7S7Ga2ri5R7ZD5utQN20A3t6EF7mdn0C4Kf7CAC4ZJAhf06 2jxSuRLaJyXUEKQXZcsKIGCLfg== X-Google-Smtp-Source: APXvYqyjE5Z/ExpKjAFeUwQaYOlsDCBhURZQ6GgY1ZAERUi1xVf6/dGQvcGXJsfycKEfjO1ON7WJpA== X-Received: by 2002:a17:906:6a92:: with SMTP id p18mr14982887ejr.253.1568906327557; Thu, 19 Sep 2019 08:18:47 -0700 (PDT) Received: from lophozonia ([85.195.192.192]) by smtp.gmail.com with ESMTPSA id x4sm763681eds.87.2019.09.19.08.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Sep 2019 08:18:46 -0700 (PDT) Date: Thu, 19 Sep 2019 17:18:44 +0200 From: Jean-Philippe Brucker To: Guo Ren Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Message-ID: <20190919151844.GG1013538@lophozonia> References: <20190619123939.GF7767@fuggles.cambridge.arm.com> <20190624104006.lvm32nahemaqklxc@willie-the-truck> <20190912140256.fwbutgmadpjbjnab@willie-the-truck> <057a0af3-93f7-271c-170e-4b31e6894c3c@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) Cc: aou@eecs.berkeley.edu, Linux Kernel Mailing List , Arnd Bergmann , suzuki.poulose@arm.com, Marc Zyngier , Catalin Marinas , Palmer Dabbelt , christoffer.dall@arm.com, iommu@lists.linux-foundation.org, Mike Rapoport , Anup Patel , Atish Patra , Julien Grall , james.morse@arm.com, gary@garyguo.net, Paul Walmsley , linux-riscv@lists.infradead.org, Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: iommu-bounces@lists.linux-foundation.org Errors-To: iommu-bounces@lists.linux-foundation.org On Thu, Sep 19, 2019 at 09:07:15PM +0800, Guo Ren wrote: > > The solution I had to this problem is pinning the ASID [1] used by the > > IOMMU, to prevent the CPU from recycling the ASID on rollover. This way > > the CPU doesn't have to wait for IOMMU invalidations to complete, when > > scheduling a task that might not even have anything to do with the IOMMU. > > > > > In the Arm SMMU, ASID and IOASID (PASID) are separate identifiers. IOASID > > indexes an entry in the context descriptor table, which contains the ASID. > > So with unpinned shared ASID you don't need to invalidate the ATC on > > rollover, since the IOASID doesn't change, but you do need to modify the > > context descriptor and invalidate cached versions of it. > The terminology confused me a lot. I perfer use PASID for IOMMU and > ASID is for CPU. > Arm's entry of the context descriptor table contains a "IOASID" The terminology I've been using so far is different: * IOASID is PASID * The entry in the context descriptor table contains an ASID, which is either "shared" with CPUs or "private" to the SMMU (the SMMU spec says "shared" or "non-shared"). * So the CPU and SMMU TLBs use ASIDs, and the PCI ATC uses IOASID > IOASID != ASID for CPU_TLB and IOMMU_TLB. > > When you say "since the IOASID doesn't change",Is it PASID or my IOASID ? -_*! I was talking about PASID. Maybe we can drop "IOASID" and talk only about ASID and PASID :) > PASID in PCI-sig was used to determine transfer address space. > For intel, the entry which is indexed by PASID also contain S1/S2.PGD > and DID(VMID). > For arm, the entry which is indexed by PASID only contain S1.PGD and > IOASID. Compare to Intel Vt-d Scalable mode, arm's design can't > support PCI Virtual Function. The SMMU does support PCI Virtual Function - an hypervisor can assign a VF to a guest, and let that guest partition the VF into smaller contexts by using PASID. What it can't support is assigning partitions of a PCI function (VF or PF) to multiple Virtual Machines, since there is a single S2 PGD per function (in the Stream Table Entry), rather than one S2 PGD per PASID context. Thanks, Jean > > Once you have pinned ASIDs, you could also declare that IOASID = ASID. I > > don't remember finding an argument to strictly forbid it, even though ASID > > and IOASID have different sizes on Arm (respectively 8/16 and 20 bits). > ASID and IOASID are hard to keep the same between CPU system and IOMMU > system. So I introduce S1/S2.PGD.PPN as a bridge between CPUs and > IOMMUs. > See my proposal [1] > > 1: https://lore.kernel.org/linux-csky/1568896556-28769-1-git-send-email-guoren@kernel.org/T/#u > -- > Best Regards > Guo Ren > > ML: https://lore.kernel.org/linux-csky/ _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3FE6C4CECE for ; Thu, 19 Sep 2019 15:18:52 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 1C372218AF for ; Thu, 19 Sep 2019 15:18:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Yqn2hP5/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C372218AF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 717824A6CE; Thu, 19 Sep 2019 11:18:51 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@linaro.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lv5XilD19Egk; Thu, 19 Sep 2019 11:18:50 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 416554A6A6; Thu, 19 Sep 2019 11:18:50 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id EE5EA4A69F for ; Thu, 19 Sep 2019 11:18:49 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G0W8Sn-wYOxq for ; Thu, 19 Sep 2019 11:18:48 -0400 (EDT) Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id A75654A688 for ; Thu, 19 Sep 2019 11:18:48 -0400 (EDT) Received: by mail-ed1-f65.google.com with SMTP id v8so3579445eds.2 for ; Thu, 19 Sep 2019 08:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=Yqn2hP5/7LoUE96rW6LgXvDeRFDoEgSIYLWO331yu/t8KvfcDLYs0svMMqhPl0nEKU 3028G05kj86vZEYdH0BT8B4Snvw/8TzsFj/6z3cDmgcLrYqAf9CTKaxFxWz6TetvRVWD JaOhJxFoN6Lysw3uc3I7TCd+bGbTAYEQdtrh530GBo5Aqg2kjpsReLMDlMYPva23+thK 3paKUHNXhfjBIBDEcc9d7OpRoL4M9YEswI1G3XEqU9hMkBTILBRZAfv3lk1GyAApc7kF uBVf89FvXfuH3jiXShL1/vpa79fW+aGEKUGlC597SIRD1tjHojMUhcUCeA7TA1Um/ffs UzGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=jWf19Ec5UGiQXUhnKztu2pvbDDohrYz/kqX5nGtYctwKmcAQT2t376pvFvFjKWnXO2 M5fXgOMf8z0pF4Ea4jLFARUz+hrjWON2qJQFi7Y6HCKcNup/75QsfP5hNP1UUShB+I09 dOJ+WdiM/EJmunXFRe4TKXE30pRZxrQU94MusDxtrrGpLFTkJGzz6acVGs8sZNiJYrXR rOqYHrxYu0wmsBOFgrZkZwFbRv7ruloYmOHCNM3EO3DaOhXPSZSP3DujGsFv38QwnmQq k7m9FpTC2Egr7SUhFuZtX49cAom/WBqPrJkTiU2ZwUqqE9gXEWNiwD82okmCj2LSqSQH /mBg== X-Gm-Message-State: APjAAAUKzezHB48cIDM8ImNYuADXXNIPSNxFEBhjWsZk+pdcNjzIgNFe XmAosDQwcubcbLZ0Ce0RLNyyRQ== X-Google-Smtp-Source: APXvYqyjE5Z/ExpKjAFeUwQaYOlsDCBhURZQ6GgY1ZAERUi1xVf6/dGQvcGXJsfycKEfjO1ON7WJpA== X-Received: by 2002:a17:906:6a92:: with SMTP id p18mr14982887ejr.253.1568906327557; Thu, 19 Sep 2019 08:18:47 -0700 (PDT) Received: from lophozonia ([85.195.192.192]) by smtp.gmail.com with ESMTPSA id x4sm763681eds.87.2019.09.19.08.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Sep 2019 08:18:46 -0700 (PDT) Date: Thu, 19 Sep 2019 17:18:44 +0200 From: Jean-Philippe Brucker To: Guo Ren Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Message-ID: <20190919151844.GG1013538@lophozonia> References: <20190619123939.GF7767@fuggles.cambridge.arm.com> <20190624104006.lvm32nahemaqklxc@willie-the-truck> <20190912140256.fwbutgmadpjbjnab@willie-the-truck> <057a0af3-93f7-271c-170e-4b31e6894c3c@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) Cc: aou@eecs.berkeley.edu, Linux Kernel Mailing List , Arnd Bergmann , Marc Zyngier , Catalin Marinas , Palmer Dabbelt , iommu@lists.linux-foundation.org, Mike Rapoport , Anup Patel , Atish Patra , Julien Grall , gary@garyguo.net, Paul Walmsley , linux-riscv@lists.infradead.org, Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Thu, Sep 19, 2019 at 09:07:15PM +0800, Guo Ren wrote: > > The solution I had to this problem is pinning the ASID [1] used by the > > IOMMU, to prevent the CPU from recycling the ASID on rollover. This way > > the CPU doesn't have to wait for IOMMU invalidations to complete, when > > scheduling a task that might not even have anything to do with the IOMMU. > > > > > In the Arm SMMU, ASID and IOASID (PASID) are separate identifiers. IOASID > > indexes an entry in the context descriptor table, which contains the ASID. > > So with unpinned shared ASID you don't need to invalidate the ATC on > > rollover, since the IOASID doesn't change, but you do need to modify the > > context descriptor and invalidate cached versions of it. > The terminology confused me a lot. I perfer use PASID for IOMMU and > ASID is for CPU. > Arm's entry of the context descriptor table contains a "IOASID" The terminology I've been using so far is different: * IOASID is PASID * The entry in the context descriptor table contains an ASID, which is either "shared" with CPUs or "private" to the SMMU (the SMMU spec says "shared" or "non-shared"). * So the CPU and SMMU TLBs use ASIDs, and the PCI ATC uses IOASID > IOASID != ASID for CPU_TLB and IOMMU_TLB. > > When you say "since the IOASID doesn't change",Is it PASID or my IOASID ? -_*! I was talking about PASID. Maybe we can drop "IOASID" and talk only about ASID and PASID :) > PASID in PCI-sig was used to determine transfer address space. > For intel, the entry which is indexed by PASID also contain S1/S2.PGD > and DID(VMID). > For arm, the entry which is indexed by PASID only contain S1.PGD and > IOASID. Compare to Intel Vt-d Scalable mode, arm's design can't > support PCI Virtual Function. The SMMU does support PCI Virtual Function - an hypervisor can assign a VF to a guest, and let that guest partition the VF into smaller contexts by using PASID. What it can't support is assigning partitions of a PCI function (VF or PF) to multiple Virtual Machines, since there is a single S2 PGD per function (in the Stream Table Entry), rather than one S2 PGD per PASID context. Thanks, Jean > > Once you have pinned ASIDs, you could also declare that IOASID = ASID. I > > don't remember finding an argument to strictly forbid it, even though ASID > > and IOASID have different sizes on Arm (respectively 8/16 and 20 bits). > ASID and IOASID are hard to keep the same between CPU system and IOMMU > system. So I introduce S1/S2.PGD.PPN as a bridge between CPUs and > IOMMUs. > See my proposal [1] > > 1: https://lore.kernel.org/linux-csky/1568896556-28769-1-git-send-email-guoren@kernel.org/T/#u > -- > Best Regards > Guo Ren > > ML: https://lore.kernel.org/linux-csky/ _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92D1AC3A5A6 for ; Thu, 19 Sep 2019 15:19:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6321520665 for ; Thu, 19 Sep 2019 15:19:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="OdH+fjoY"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Yqn2hP5/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6321520665 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+jhr3DKNeEDve46IaI65MQcBNtpFmOubDhEg7QC0fQw=; b=OdH+fjoY/3LN4g 7/Fm5v+7qTNIbtZq61YISKFJFBVTOTYyemN+kuNobQ3THZ5RLCJGATi6RqFrjMC5uSHWda4EH+RhM 4kW9rptpb1MNAbvINSCBb4mPYjWt4TuyUh95u7UkQsq+nQldS2/BWTmAezfsmVXWBW1sC6jkxraKE 8386SJ9YrTdyvSJd/C6NkKwCJJZyPHleS4YEf8L5Ai6Z9JA9j7HCMq+Sg2WXtn2m1r63UNJs9wkPz e9GQFwummOr/qZmW4BNyXdulH5PT9UYgdG8bwtVEelkibXgqIi36euIDANjDQ3J6i49GH9mE7+gvJ gr1H372tuY2KKi7ngnJg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1iAyD5-0001Mb-RI; Thu, 19 Sep 2019 15:19:07 +0000 Received: from mail-ed1-x542.google.com ([2a00:1450:4864:20::542]) by bombadil.infradead.org with esmtps (Exim 4.92.2 #3 (Red Hat Linux)) id 1iAyCn-0001J3-4D for linux-arm-kernel@lists.infradead.org; Thu, 19 Sep 2019 15:18:51 +0000 Received: by mail-ed1-x542.google.com with SMTP id r4so3574120edy.4 for ; Thu, 19 Sep 2019 08:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=Yqn2hP5/7LoUE96rW6LgXvDeRFDoEgSIYLWO331yu/t8KvfcDLYs0svMMqhPl0nEKU 3028G05kj86vZEYdH0BT8B4Snvw/8TzsFj/6z3cDmgcLrYqAf9CTKaxFxWz6TetvRVWD JaOhJxFoN6Lysw3uc3I7TCd+bGbTAYEQdtrh530GBo5Aqg2kjpsReLMDlMYPva23+thK 3paKUHNXhfjBIBDEcc9d7OpRoL4M9YEswI1G3XEqU9hMkBTILBRZAfv3lk1GyAApc7kF uBVf89FvXfuH3jiXShL1/vpa79fW+aGEKUGlC597SIRD1tjHojMUhcUCeA7TA1Um/ffs UzGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=eJ0UWMRY33TSZcE2wFwrgcT5zmWwOXQ+xlDh/x5Rbok=; b=mORxDM2TZ57DDQr84CJl/bHet82SNdyh+KBKRw4duVqWMYmkdfzoMs3zGHq557LJIG V07QxhHP3SjsQvWfAIp9WbreShAO2kJ7jmdoiVgfKkSlCL8Woe2PlfBRHzyOq7BZKdmK 2waQYZ7rVAKtuI89zPd+8uQHk7SCGXob+0XkpiOtR9vIv8FYrMI4jGg/DTrEGmsu8Evp DsjSLWzfxmzhM5eea1dME+HGAc8hZXhxlD9sk1EJLY0d4azNeK3QSqWAabfN0ex3bUAv Sdp8rytCRzJPQR+RzhiSFx81v4Bi78jlshlDoS2zBNT6yzpKzXz063wjmLmVUceVs3gs B9xQ== X-Gm-Message-State: APjAAAUv8zxtwcWN/No+MYVTGn/jzL547VbzwwbaOvO905qmdPU16tjG ytecnps56FHCyhK36yzF0L4a9Q== X-Google-Smtp-Source: APXvYqyjE5Z/ExpKjAFeUwQaYOlsDCBhURZQ6GgY1ZAERUi1xVf6/dGQvcGXJsfycKEfjO1ON7WJpA== X-Received: by 2002:a17:906:6a92:: with SMTP id p18mr14982887ejr.253.1568906327557; Thu, 19 Sep 2019 08:18:47 -0700 (PDT) Received: from lophozonia ([85.195.192.192]) by smtp.gmail.com with ESMTPSA id x4sm763681eds.87.2019.09.19.08.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Sep 2019 08:18:46 -0700 (PDT) Date: Thu, 19 Sep 2019 17:18:44 +0200 From: Jean-Philippe Brucker To: Guo Ren Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file Message-ID: <20190919151844.GG1013538@lophozonia> References: <20190619123939.GF7767@fuggles.cambridge.arm.com> <20190624104006.lvm32nahemaqklxc@willie-the-truck> <20190912140256.fwbutgmadpjbjnab@willie-the-truck> <057a0af3-93f7-271c-170e-4b31e6894c3c@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190919_081849_191541_5AC7D8CD X-CRM114-Status: GOOD ( 22.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aou@eecs.berkeley.edu, Linux Kernel Mailing List , Arnd Bergmann , suzuki.poulose@arm.com, Marc Zyngier , Catalin Marinas , Palmer Dabbelt , christoffer.dall@arm.com, iommu@lists.linux-foundation.org, Mike Rapoport , Anup Patel , Atish Patra , Julien Grall , james.morse@arm.com, gary@garyguo.net, Paul Walmsley , linux-riscv@lists.infradead.org, Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Sep 19, 2019 at 09:07:15PM +0800, Guo Ren wrote: > > The solution I had to this problem is pinning the ASID [1] used by the > > IOMMU, to prevent the CPU from recycling the ASID on rollover. This way > > the CPU doesn't have to wait for IOMMU invalidations to complete, when > > scheduling a task that might not even have anything to do with the IOMMU. > > > > > In the Arm SMMU, ASID and IOASID (PASID) are separate identifiers. IOASID > > indexes an entry in the context descriptor table, which contains the ASID. > > So with unpinned shared ASID you don't need to invalidate the ATC on > > rollover, since the IOASID doesn't change, but you do need to modify the > > context descriptor and invalidate cached versions of it. > The terminology confused me a lot. I perfer use PASID for IOMMU and > ASID is for CPU. > Arm's entry of the context descriptor table contains a "IOASID" The terminology I've been using so far is different: * IOASID is PASID * The entry in the context descriptor table contains an ASID, which is either "shared" with CPUs or "private" to the SMMU (the SMMU spec says "shared" or "non-shared"). * So the CPU and SMMU TLBs use ASIDs, and the PCI ATC uses IOASID > IOASID != ASID for CPU_TLB and IOMMU_TLB. > > When you say "since the IOASID doesn't change",Is it PASID or my IOASID ? -_*! I was talking about PASID. Maybe we can drop "IOASID" and talk only about ASID and PASID :) > PASID in PCI-sig was used to determine transfer address space. > For intel, the entry which is indexed by PASID also contain S1/S2.PGD > and DID(VMID). > For arm, the entry which is indexed by PASID only contain S1.PGD and > IOASID. Compare to Intel Vt-d Scalable mode, arm's design can't > support PCI Virtual Function. The SMMU does support PCI Virtual Function - an hypervisor can assign a VF to a guest, and let that guest partition the VF into smaller contexts by using PASID. What it can't support is assigning partitions of a PCI function (VF or PF) to multiple Virtual Machines, since there is a single S2 PGD per function (in the Stream Table Entry), rather than one S2 PGD per PASID context. Thanks, Jean > > Once you have pinned ASIDs, you could also declare that IOASID = ASID. I > > don't remember finding an argument to strictly forbid it, even though ASID > > and IOASID have different sizes on Arm (respectively 8/16 and 20 bits). > ASID and IOASID are hard to keep the same between CPU system and IOMMU > system. So I introduce S1/S2.PGD.PPN as a bridge between CPUs and > IOMMUs. > See my proposal [1] > > 1: https://lore.kernel.org/linux-csky/1568896556-28769-1-git-send-email-guoren@kernel.org/T/#u > -- > Best Regards > Guo Ren > > ML: https://lore.kernel.org/linux-csky/ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel