From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 198A6C3F2D2 for ; Fri, 28 Feb 2020 14:44:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E5D8A246AE for ; Fri, 28 Feb 2020 14:44:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="JCRrLIEo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726860AbgB1Oo3 (ORCPT ); Fri, 28 Feb 2020 09:44:29 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:36518 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727031AbgB1Oo3 (ORCPT ); Fri, 28 Feb 2020 09:44:29 -0500 Received: by mail-wm1-f66.google.com with SMTP id g83so1261380wme.1 for ; Fri, 28 Feb 2020 06:44:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=EaNst+hWQycMu0KGsnv4CxZ9Si7qKvgKxC1mvggZU4w=; b=JCRrLIEo6Jpe8dHIBFGmuTXIFtETxczvLzWf5uy8Y4Rm9R7wgxKO+2afcz/GNnH0Y3 q0tqjcxhgffbbsl10LWkrDKFYFGroNVipFdobpiY+ORfr6InAZrnVRnUUJdKxAKtACEM dTxklEFgeMyTkRclu9MCF6ibbvsenPwWRDYZLFrWvg+m8RzwfWoY6XELfqEtrbGkmQ5q TgpYmhbgZkm0bFXzLd85KOOeaNDPUJQuCrD90Ya9ovii8HTS+dVU1eWttG0owswwOm8u MFcNQWUAz9eoCClWH9QiIu0LftzF5ZzkomXsU1JPNJCynNEi1BYGHsYrU8xBnhT/OP6K 8hMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=EaNst+hWQycMu0KGsnv4CxZ9Si7qKvgKxC1mvggZU4w=; b=Ty34LfSXZgjFOF8gV9RekbZDrqz+I8t6mlXGPZzcbFX7NuMhQgCeMGNpZkRrv8mGgP ZXcDIE8fdWtNdPQ9bPlSfs8S+1G7jU2YFzMJ6Ay5QeOkuwhnJe7qBy00NsGPUTY9kAgT XsnqPBvhDHQatvnmBSpyuHfOlJ51QAW3evC5aO+Dt+W2C6WT9ozaaBR1UlfJMhe87ixr /hFxbtqPdaWsyiwocdOJ36AONE2mvX9a1V0l5dxJAg8/oIC4/yZ64afjFtDh9uD1GkMS xVuOuIxBWPTBSw32gUfvUieuqFbqY7VZ+qEl8c3JlJnDOnEqYgIc2yxtLusMiE+CARgY iueg== X-Gm-Message-State: APjAAAVsdcL3H6Gq3a50bO/aXEmwZCXZUVs6tuoTnVemswAWTlftaSGY ZtUk9ZVwwhFwMzWjxKliO+dSYQ== X-Google-Smtp-Source: APXvYqwmzNWF57lJGnnMPvJ7KLRmrlfEQjMUS3cUSNh6ONuPqWLfuhP1DtkFcJz13ydjNSUlBxXNAQ== X-Received: by 2002:a7b:c4d0:: with SMTP id g16mr3359111wmk.3.1582901067287; Fri, 28 Feb 2020 06:44:27 -0800 (PST) Received: from myrica ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id n11sm2001521wrq.40.2020.02.28.06.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 06:44:26 -0800 (PST) Date: Fri, 28 Feb 2020 15:44:20 +0100 From: Jean-Philippe Brucker To: Jacob Pan Cc: iommu@lists.linux-foundation.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, joro@8bytes.org, robh+dt@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, will@kernel.org, robin.murphy@arm.com, kevin.tian@intel.com, baolu.lu@linux.intel.com, Jonathan.Cameron@huawei.com, christian.koenig@amd.com, yi.l.liu@intel.com, zhangfei.gao@linaro.org, Jean-Philippe Brucker Subject: Re: [PATCH v4 06/26] iommu/sva: Register page fault handler Message-ID: <20200228144420.GE2156@myrica> References: <20200224182401.353359-1-jean-philippe@linaro.org> <20200224182401.353359-7-jean-philippe@linaro.org> <20200226113959.62621098@jacob-builder> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200226113959.62621098@jacob-builder> Sender: devicetree-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org On Wed, Feb 26, 2020 at 11:39:59AM -0800, Jacob Pan wrote: > > @@ -442,11 +444,20 @@ static void iommu_sva_unbind_locked(struct > > iommu_bond *bond) > > void iommu_sva_unbind_generic(struct iommu_sva *handle) > > { > > + int pasid; > > struct iommu_param *param = handle->dev->iommu_param; > > > > if (WARN_ON(!param)) > > return; > > > > + /* > > + * Caller stopped the device from issuing PASIDs, now make > > sure they are > > + * out of the fault queue. > > + */ > > + pasid = iommu_sva_get_pasid_generic(handle); > > + if (pasid != IOMMU_PASID_INVALID) > > + iopf_queue_flush_dev(handle->dev, pasid); > > + > I have an ordering concern. > The caller can only stop the device issuing page request but there will > be in-flight request inside the IOMMU. If we flush here before clearing > the PASID context, there might be new request coming in before the > detach. The goal of this flush is also to clear the IOMMU PRI queue. It calls the IOMMU's flush() callback before flushing the workqueue. So when this returns, there shouldn't be any more pending fault. Thanks, Jean > How about detach first then flush? Then anything come after the detach > would be faults. Flush will be clean. > > > mutex_lock(¶m->sva_lock); > > mutex_lock(&iommu_sva_lock); > > iommu_sva_unbind_locked(to_iommu_bond(handle)); > > @@ -484,6 +495,10 @@ int iommu_sva_enable(struct device *dev, struct > > iommu_sva_param *sva_param) goto err_unlock; > > } > > > > + ret = iommu_register_device_fault_handler(dev, > > iommu_queue_iopf, dev); > > + if (ret) > > + goto err_unlock; > > + > > dev->iommu_param->sva_param = new_param; > > mutex_unlock(¶m->sva_lock); > > return 0; > > @@ -521,6 +536,7 @@ int iommu_sva_disable(struct device *dev) > > goto out_unlock; > > } > > > > + iommu_unregister_device_fault_handler(dev); > > kfree(param->sva_param); > > param->sva_param = NULL; > > out_unlock: > > [Jacob Pan] From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80AADC3F2D2 for ; Fri, 28 Feb 2020 14:44:38 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4AF6E246A3 for ; Fri, 28 Feb 2020 14:44:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="JCRrLIEo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4AF6E246A3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 12416833C6; Fri, 28 Feb 2020 14:44:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ePF1EYujoi5B; Fri, 28 Feb 2020 14:44:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 9D2E6873B8; Fri, 28 Feb 2020 14:44:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 84299C08A0; Fri, 28 Feb 2020 14:44:33 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0D506C0177 for ; Fri, 28 Feb 2020 14:44:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id EBB9D2267B for ; Fri, 28 Feb 2020 14:44:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7u+qpnqguyne for ; Fri, 28 Feb 2020 14:44:30 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by silver.osuosl.org (Postfix) with ESMTPS id CF28B2075B for ; Fri, 28 Feb 2020 14:44:28 +0000 (UTC) Received: by mail-wm1-f65.google.com with SMTP id d138so1856681wmd.5 for ; Fri, 28 Feb 2020 06:44:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=EaNst+hWQycMu0KGsnv4CxZ9Si7qKvgKxC1mvggZU4w=; b=JCRrLIEo6Jpe8dHIBFGmuTXIFtETxczvLzWf5uy8Y4Rm9R7wgxKO+2afcz/GNnH0Y3 q0tqjcxhgffbbsl10LWkrDKFYFGroNVipFdobpiY+ORfr6InAZrnVRnUUJdKxAKtACEM dTxklEFgeMyTkRclu9MCF6ibbvsenPwWRDYZLFrWvg+m8RzwfWoY6XELfqEtrbGkmQ5q TgpYmhbgZkm0bFXzLd85KOOeaNDPUJQuCrD90Ya9ovii8HTS+dVU1eWttG0owswwOm8u MFcNQWUAz9eoCClWH9QiIu0LftzF5ZzkomXsU1JPNJCynNEi1BYGHsYrU8xBnhT/OP6K 8hMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=EaNst+hWQycMu0KGsnv4CxZ9Si7qKvgKxC1mvggZU4w=; b=q+6F/03/Ov25EFuz7sWGZQvxfVLkeyU2zYEgS2oNV96o/cU5kgPDzY+Md6A0G/5TTq lhatUkU1OBXQpuklMaBMXfvB6F/hSWwSTZspPOI70VaOBEF9o9wf8PFwob1huo56rKZl Il2uKoqkuvslt2wtZmq9EvWO6WUPOGkEiNB+I9+2GuV/umUcCik3pnteRxTCwUkF5zgG Gxtq0otKI/2KFv7cheItKITiXtpMV41L/0k4hEyHRaukFA8PU5/8TEgR6upVWDIITzmv zurCjAC03JX1GNjh3i++qWosudagE56wrv2BIQxP5dPoQCxDVFi5FN/F6qxurDP627YC SsxQ== X-Gm-Message-State: APjAAAUdpTpe60a7eLweJorqI5+X/vbzr0zfyH1IuP2hEQKpLGJkZp6S hTft9LfL6UHfCKkJKcs0vWM+Qw== X-Google-Smtp-Source: APXvYqwmzNWF57lJGnnMPvJ7KLRmrlfEQjMUS3cUSNh6ONuPqWLfuhP1DtkFcJz13ydjNSUlBxXNAQ== X-Received: by 2002:a7b:c4d0:: with SMTP id g16mr3359111wmk.3.1582901067287; Fri, 28 Feb 2020 06:44:27 -0800 (PST) Received: from myrica ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id n11sm2001521wrq.40.2020.02.28.06.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 06:44:26 -0800 (PST) Date: Fri, 28 Feb 2020 15:44:20 +0100 From: Jean-Philippe Brucker To: Jacob Pan Subject: Re: [PATCH v4 06/26] iommu/sva: Register page fault handler Message-ID: <20200228144420.GE2156@myrica> References: <20200224182401.353359-1-jean-philippe@linaro.org> <20200224182401.353359-7-jean-philippe@linaro.org> <20200226113959.62621098@jacob-builder> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200226113959.62621098@jacob-builder> Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, kevin.tian@intel.com, Jean-Philippe Brucker , linux-pci@vger.kernel.org, robin.murphy@arm.com, linux-mm@kvack.org, iommu@lists.linux-foundation.org, robh+dt@kernel.org, catalin.marinas@arm.com, zhangfei.gao@linaro.org, will@kernel.org, christian.koenig@amd.com, linux-arm-kernel@lists.infradead.org X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On Wed, Feb 26, 2020 at 11:39:59AM -0800, Jacob Pan wrote: > > @@ -442,11 +444,20 @@ static void iommu_sva_unbind_locked(struct > > iommu_bond *bond) > > void iommu_sva_unbind_generic(struct iommu_sva *handle) > > { > > + int pasid; > > struct iommu_param *param = handle->dev->iommu_param; > > > > if (WARN_ON(!param)) > > return; > > > > + /* > > + * Caller stopped the device from issuing PASIDs, now make > > sure they are > > + * out of the fault queue. > > + */ > > + pasid = iommu_sva_get_pasid_generic(handle); > > + if (pasid != IOMMU_PASID_INVALID) > > + iopf_queue_flush_dev(handle->dev, pasid); > > + > I have an ordering concern. > The caller can only stop the device issuing page request but there will > be in-flight request inside the IOMMU. If we flush here before clearing > the PASID context, there might be new request coming in before the > detach. The goal of this flush is also to clear the IOMMU PRI queue. It calls the IOMMU's flush() callback before flushing the workqueue. So when this returns, there shouldn't be any more pending fault. Thanks, Jean > How about detach first then flush? Then anything come after the detach > would be faults. Flush will be clean. > > > mutex_lock(¶m->sva_lock); > > mutex_lock(&iommu_sva_lock); > > iommu_sva_unbind_locked(to_iommu_bond(handle)); > > @@ -484,6 +495,10 @@ int iommu_sva_enable(struct device *dev, struct > > iommu_sva_param *sva_param) goto err_unlock; > > } > > > > + ret = iommu_register_device_fault_handler(dev, > > iommu_queue_iopf, dev); > > + if (ret) > > + goto err_unlock; > > + > > dev->iommu_param->sva_param = new_param; > > mutex_unlock(¶m->sva_lock); > > return 0; > > @@ -521,6 +536,7 @@ int iommu_sva_disable(struct device *dev) > > goto out_unlock; > > } > > > > + iommu_unregister_device_fault_handler(dev); > > kfree(param->sva_param); > > param->sva_param = NULL; > > out_unlock: > > [Jacob Pan] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 024FAC3F2CD for ; Fri, 28 Feb 2020 14:44:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A61FC246A3 for ; Fri, 28 Feb 2020 14:44:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="XTcxkiMb"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="JCRrLIEo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A61FC246A3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=CNkQySiZ430sd4fC+tZSW2XZrrocl+Ybv7GPjGu2vOA=; b=XTcxkiMbyzG2dQ 17DXkWQ5T5a1C9vm6HdLMvKzBjNECEkSwAN0cCIDojr/2JFzASHI5P8yTq4zK9oPQOEl8jdWSrqha srvgo3HoODlS+ZFzLEZyM4IQLi+UallDIJ2jmlHS8GEvM+1LegfolaYHGAeSITYl6X3JKy2wC9s7w GWH1LakpxrKO++JmdfGXJ9hPiGkAA95d2q1YbOPKbL3fRwSFUlh5wKUzF+7VnSEpVbx9dK2LEK6k0 uQREIWNsFA//9d4ithCpCGNQASoCUpQPp/spOXC1rH1/kU6c5XsJQs0TsQVcFjq2XFGeImK20VFBn mxlNxxtaAyf0/vTEE3cg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j7gsV-0001dh-RV; Fri, 28 Feb 2020 14:44:35 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1j7gsP-0001Wj-6B for linux-arm-kernel@lists.infradead.org; Fri, 28 Feb 2020 14:44:30 +0000 Received: by mail-wm1-x341.google.com with SMTP id m3so3478803wmi.0 for ; Fri, 28 Feb 2020 06:44:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=EaNst+hWQycMu0KGsnv4CxZ9Si7qKvgKxC1mvggZU4w=; b=JCRrLIEo6Jpe8dHIBFGmuTXIFtETxczvLzWf5uy8Y4Rm9R7wgxKO+2afcz/GNnH0Y3 q0tqjcxhgffbbsl10LWkrDKFYFGroNVipFdobpiY+ORfr6InAZrnVRnUUJdKxAKtACEM dTxklEFgeMyTkRclu9MCF6ibbvsenPwWRDYZLFrWvg+m8RzwfWoY6XELfqEtrbGkmQ5q TgpYmhbgZkm0bFXzLd85KOOeaNDPUJQuCrD90Ya9ovii8HTS+dVU1eWttG0owswwOm8u MFcNQWUAz9eoCClWH9QiIu0LftzF5ZzkomXsU1JPNJCynNEi1BYGHsYrU8xBnhT/OP6K 8hMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=EaNst+hWQycMu0KGsnv4CxZ9Si7qKvgKxC1mvggZU4w=; b=Xn38Yf+wFhe7GqTGFri0/+RVGZu/Tt2zAhTCtOx7RflD04obdYGgl0kzez2hO3RkWL FRLWmvdbi6I5y9J2OsXrnRVl12F/Mi2kmqeS/3xCpDlevxUNCz3ixI/xVfaSWgKUGrCW VI4wmf57JU21iKP9PS2ZGObwL1QZU5zA7jciPh9j4KwMSFQRXjb4UOQt1Scu1YCxMO/C K3UJqBHdEuTx+rbEME8HU/zjR9V8/bXAuK1l1A2E85ZzysRDkBc0rQxLiLznRLXjshjL HnballWeWylcK4Hgzzx9wzrTniWFAm4g3kczuTZnpdeYOugJc8nYf9Pecc0bq5FNQPwU b5wg== X-Gm-Message-State: APjAAAUJWi01wVhAzrJu1aqj8chzXPWwMY0iNVgPcJRVNn8u5fY03vVM ScNouyXKFpL3ClBwhldHnLJhZA== X-Google-Smtp-Source: APXvYqwmzNWF57lJGnnMPvJ7KLRmrlfEQjMUS3cUSNh6ONuPqWLfuhP1DtkFcJz13ydjNSUlBxXNAQ== X-Received: by 2002:a7b:c4d0:: with SMTP id g16mr3359111wmk.3.1582901067287; Fri, 28 Feb 2020 06:44:27 -0800 (PST) Received: from myrica ([2001:171b:c9a8:fbc0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id n11sm2001521wrq.40.2020.02.28.06.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 06:44:26 -0800 (PST) Date: Fri, 28 Feb 2020 15:44:20 +0100 From: Jean-Philippe Brucker To: Jacob Pan Subject: Re: [PATCH v4 06/26] iommu/sva: Register page fault handler Message-ID: <20200228144420.GE2156@myrica> References: <20200224182401.353359-1-jean-philippe@linaro.org> <20200224182401.353359-7-jean-philippe@linaro.org> <20200226113959.62621098@jacob-builder> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200226113959.62621098@jacob-builder> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200228_064429_360297_10AA7FAB X-CRM114-Status: GOOD ( 16.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, kevin.tian@intel.com, yi.l.liu@intel.com, Jean-Philippe Brucker , linux-pci@vger.kernel.org, joro@8bytes.org, Jonathan.Cameron@huawei.com, robin.murphy@arm.com, linux-mm@kvack.org, iommu@lists.linux-foundation.org, robh+dt@kernel.org, catalin.marinas@arm.com, zhangfei.gao@linaro.org, will@kernel.org, christian.koenig@amd.com, linux-arm-kernel@lists.infradead.org, baolu.lu@linux.intel.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Feb 26, 2020 at 11:39:59AM -0800, Jacob Pan wrote: > > @@ -442,11 +444,20 @@ static void iommu_sva_unbind_locked(struct > > iommu_bond *bond) > > void iommu_sva_unbind_generic(struct iommu_sva *handle) > > { > > + int pasid; > > struct iommu_param *param = handle->dev->iommu_param; > > > > if (WARN_ON(!param)) > > return; > > > > + /* > > + * Caller stopped the device from issuing PASIDs, now make > > sure they are > > + * out of the fault queue. > > + */ > > + pasid = iommu_sva_get_pasid_generic(handle); > > + if (pasid != IOMMU_PASID_INVALID) > > + iopf_queue_flush_dev(handle->dev, pasid); > > + > I have an ordering concern. > The caller can only stop the device issuing page request but there will > be in-flight request inside the IOMMU. If we flush here before clearing > the PASID context, there might be new request coming in before the > detach. The goal of this flush is also to clear the IOMMU PRI queue. It calls the IOMMU's flush() callback before flushing the workqueue. So when this returns, there shouldn't be any more pending fault. Thanks, Jean > How about detach first then flush? Then anything come after the detach > would be faults. Flush will be clean. > > > mutex_lock(¶m->sva_lock); > > mutex_lock(&iommu_sva_lock); > > iommu_sva_unbind_locked(to_iommu_bond(handle)); > > @@ -484,6 +495,10 @@ int iommu_sva_enable(struct device *dev, struct > > iommu_sva_param *sva_param) goto err_unlock; > > } > > > > + ret = iommu_register_device_fault_handler(dev, > > iommu_queue_iopf, dev); > > + if (ret) > > + goto err_unlock; > > + > > dev->iommu_param->sva_param = new_param; > > mutex_unlock(¶m->sva_lock); > > return 0; > > @@ -521,6 +536,7 @@ int iommu_sva_disable(struct device *dev) > > goto out_unlock; > > } > > > > + iommu_unregister_device_fault_handler(dev); > > kfree(param->sva_param); > > param->sva_param = NULL; > > out_unlock: > > [Jacob Pan] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel