From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BB8CC3A5A6 for ; Thu, 19 Sep 2019 15:07:23 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3746121907 for ; Thu, 19 Sep 2019 15:07:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="LNnzGGE9"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="sm1kYe9P" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3746121907 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nU7+fYrxUp+FBKu+1gwlMOYFhMt4GRX6JblrMunVsZs=; b=LNnzGGE9E4jPQF dwAsBw5KL4WoiF2XSCRdwSmUSNT9ZcWqBPgXERv7Ds/CAYM3NfDWPFaYIx3yWKnSjE0J0t0t2y6Vd UihnWbt6BvHPeBpTmBdL+v4ftJJRFs9cChUBMCGnluFn1dO8HUDx4oSWwZF7F+j0lvTx12II44aUf BjtnlBbBW6pYwWlSkJZy23vtSXDkwU/jEyhDz3ZYqMMOhyMz0omVoW8PkAuDy+etYEIevNRtZ3i9a JmIuAsIgQk3poilCNj08no/FgXkxmJPLQw2XAAGg6n1NkjT7g7nPQR2iO4/43ltM3ivE+QD6ijwMQ hxQ2uYQ26UTcjPGcEYQg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1iAy1V-0005AM-LR; Thu, 19 Sep 2019 15:07:09 +0000 Received: from mail-ed1-x541.google.com ([2a00:1450:4864:20::541]) by bombadil.infradead.org with esmtps (Exim 4.92.2 #3 (Red Hat Linux)) id 1iAy0z-0004tH-Mz for linux-arm-kernel@lists.infradead.org; Thu, 19 Sep 2019 15:06:39 +0000 Received: by mail-ed1-x541.google.com with SMTP id a23so3525939edv.5 for ; Thu, 19 Sep 2019 08:06:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=oUQUKfGGLyoufGiv0FoEyuW3RCfv5lyXYP4LRA9yUnI=; b=sm1kYe9P5pTViCKFjNk/d6ebqMWyJNooZxf91TSUePYb/5i7/VaEYPsMPIc2wfZVi8 P/ooqN0dZaXdOGopagPyYfk47lN5uurdLwK5TuwLmyhYW1FITstvFCYOZDjE+2XBbEGs DVxBofb6ZPhP/lrm+qE4xpNtlFMAcMZAd1Ca6mP8mwuQVxEQi5Rsqjk+fTKinMoccU00 yhhDKLGlRrtw55fnv6F+DBvplwGlSB8NUvnA7oH0UD9rik/SsDa/qisHAdtFxMcNHJ5E poqcqtsjOyIKhSbiF+IAS/z0wbnfXeBUqDE1jU6R7i186QYsRgMIkQLl1pqPo1r1IhhB MT4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=oUQUKfGGLyoufGiv0FoEyuW3RCfv5lyXYP4LRA9yUnI=; b=kiciElU3TouYS+NLp+ftxZ0wyOliW5b/fITBIGNgyI2Xhmqp6L5WcvpPRvC4Zo5kf+ DRAU8XI/YJX0kK3uE12ch3mK8LGEymgDkUyHCCrq1MWPh5G4a12LicnCGyzrKPx2ixIc 0Oewlh0wjHEBbGEkThFTttViqNbl2r5beTe7zJD7Bu7vyIIpmGhqI+tpN3YvQKFtt/of n8YfvYxI3kAkLlAkz+Fgg7yk70/cCz03+OuIPqm6DDEMxy7kp8X1d3STIPZmouhXBLwe zjr5BqCjEnz6ZV2YcG7qRMPyMH+VFCT7FBVeYrumJ/zmM1YuuAVJCCyZL+z+HOQR+93H gZIg== X-Gm-Message-State: APjAAAWwdBcV1MQK7kx43BMrkGaKwrbRUf/8OBTSMjDzALkpIwcBQBfg b0oulFJso41q8ZcJo26ZNorl+g== X-Google-Smtp-Source: APXvYqyS7bkpasvboOin36PhpvvHG70FNN59FYjP0q8duVqRlwWW9FcPkvkPkK/8rnvVy/QAXxBLOw== X-Received: by 2002:aa7:d295:: with SMTP id w21mr9488517edq.302.1568905596363; Thu, 19 Sep 2019 08:06:36 -0700 (PDT) Received: from lophozonia ([85.195.192.192]) by smtp.gmail.com with ESMTPSA id a19sm1374424edy.37.2019.09.19.08.06.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Sep 2019 08:06:35 -0700 (PDT) Date: Thu, 19 Sep 2019 17:06:34 +0200 From: Jean-Philippe Brucker To: Will Deacon Subject: Re: [PATCH 6/8] iommu/arm-smmu-v3: Support auxiliary domains Message-ID: <20190919150634.GE1013538@lophozonia> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> <20190610184714.6786-7-jean-philippe.brucker@arm.com> <20190626175959.ubxvb2qn4taclact@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190626175959.ubxvb2qn4taclact@willie-the-truck> User-Agent: Mutt/1.12.1 (2019-06-15) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190919_080637_785299_C18355D7 X-CRM114-Status: GOOD ( 19.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, Jean-Philippe Brucker , joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Jun 26, 2019 at 06:59:59PM +0100, Will Deacon wrote: > > @@ -666,8 +668,14 @@ struct arm_smmu_domain { > > > > struct iommu_domain domain; > > > > + /* Unused in aux domains */ > > struct list_head devices; > > spinlock_t devices_lock; > > + > > + /* Auxiliary domain stuff */ > > + struct arm_smmu_domain *parent; > > + ioasid_t ssid; > > + unsigned long aux_nr_devs; > > Maybe use a union to avoid comments about what is used/unused? OK > > +static void arm_smmu_aux_detach_dev(struct iommu_domain *domain, struct device *dev) > > +{ > > + struct iommu_domain *parent_domain; > > + struct arm_smmu_domain *parent_smmu_domain; > > + struct arm_smmu_master *master = dev_to_master(dev); > > + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); > > + > > + if (!arm_smmu_dev_feature_enabled(dev, IOMMU_DEV_FEAT_AUX)) > > + return; > > + > > + parent_domain = iommu_get_domain_for_dev(dev); > > + if (!parent_domain) > > + return; > > + parent_smmu_domain = to_smmu_domain(parent_domain); > > + > > + mutex_lock(&smmu_domain->init_mutex); > > + if (!smmu_domain->aux_nr_devs) > > + goto out_unlock; > > + > > + if (!--smmu_domain->aux_nr_devs) { > > + arm_smmu_write_ctx_desc(parent_smmu_domain, smmu_domain->ssid, > > + NULL); > > + /* > > + * TLB doesn't need invalidation since accesses from the device > > + * can't use this domain's ASID once the CD is clear. > > + * > > + * Sadly that doesn't apply to ATCs, which are PASID tagged. > > + * Invalidate all other devices as well, because even though > > + * they weren't 'officially' attached to the auxiliary domain, > > + * they could have formed ATC entries. > > + */ > > + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); > > I've been struggling to understand the locking here, since both > arm_smmu_write_ctx_desc and arm_smmu_atc_inv_domain take and release the > devices_lock for the domain. Is there not a problem with devices coming and > going in-between the two calls? Yes, I need to think about this more. I bet there are plenty more issues like this. For example I don't think I currently prevent the parent domain from disappearing while auxiliary domains are attached. > > static struct iommu_ops arm_smmu_ops = { > > .capable = arm_smmu_capable, > > .domain_alloc = arm_smmu_domain_alloc, > > @@ -2539,6 +2772,13 @@ static struct iommu_ops arm_smmu_ops = { > > .of_xlate = arm_smmu_of_xlate, > > .get_resv_regions = arm_smmu_get_resv_regions, > > .put_resv_regions = arm_smmu_put_resv_regions, > > + .dev_has_feat = arm_smmu_dev_has_feature, > > + .dev_feat_enabled = arm_smmu_dev_feature_enabled, > > + .dev_enable_feat = arm_smmu_dev_enable_feature, > > + .dev_disable_feat = arm_smmu_dev_disable_feature, > > Why can't we use the existing ->capable and ->dev_{get,set}_attr callbacks > for this? ->capable isn't very useful because it applies to all SMMUs in the system. The existing ->{get,set}_attr callbacks apply to an iommu_domain. I think the main reason for doing it on endpoints was that it would be tedious to keep track of capabilities when attaching and detaching devices to a domain, especially for drivers that allow multiple IOMMUs per domain [1]. There were more discussions, and in the end we agreed on this API for device attributes [2]. Thanks, Jean [1] https://lore.kernel.org/lkml/aa1ff748-c2ec-acc0-f1d9-cdff2b131e58@linux.intel.com/ [2] https://lore.kernel.org/linux-iommu/20181207102926.GM16835@8bytes.org/ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel