From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BB9BC3A59F for ; Thu, 29 Aug 2019 07:58:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1993E205C9 for ; Thu, 29 Aug 2019 07:58:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727007AbfH2H6O (ORCPT ); Thu, 29 Aug 2019 03:58:14 -0400 Received: from mga11.intel.com ([192.55.52.93]:60991 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725807AbfH2H6O (ORCPT ); Thu, 29 Aug 2019 03:58:14 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2019 00:58:14 -0700 X-IronPort-AV: E=Sophos;i="5.64,442,1559545200"; d="scan'208";a="171821817" Received: from jkrzyszt-desk.igk.intel.com (HELO jkrzyszt-desk.ger.corp.intel.com) ([172.22.244.17]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2019 00:58:12 -0700 From: Janusz Krzysztofik To: Lu Baolu Cc: David Woodhouse , Joerg Roedel , iommu@lists.linux-foundation.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, =?utf-8?B?TWljaGHFgg==?= Wajdeczko Subject: Re: [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug Date: Thu, 29 Aug 2019 09:58:05 +0200 Message-ID: <3255251.C7nBVfOIaa@jkrzyszt-desk.ger.corp.intel.com> Organization: Intel Technology Poland sp. z o.o. - ul. Slowackiego 173, 80-298 Gdansk - KRS 101882 - NIP 957-07-52-316 In-Reply-To: <0cf4e930-1132-1e7f-815b-57a08a1fe5de@linux.intel.com> References: <20190822142922.31526-1-janusz.krzysztofik@linux.intel.com> <3275480.HMaYE7B3nd@jkrzyszt-desk.ger.corp.intel.com> <0cf4e930-1132-1e7f-815b-57a08a1fe5de@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Baolu, On Thursday, August 29, 2019 3:43:31 AM CEST Lu Baolu wrote: > Hi Janusz, > > On 8/28/19 10:17 PM, Janusz Krzysztofik wrote: > >> We should avoid kernel panic when a intel_unmap() is called against > >> a non-existent domain. > > Does that mean you suggest to replace > > BUG_ON(!domain); > > with something like > > if (WARN_ON(!domain)) > > return; > > and to not care of orphaned mappings left allocated? Is there a way to inform > > users that their active DMA mappings are no longer valid and they shouldn't > > call dma_unmap_*()? > > > >> But we shouldn't expect the IOMMU driver not > >> cleaning up the domain info when a device remove notification comes and > >> wait until all file descriptors being closed, right? > > Shouldn't then the IOMMU driver take care of cleaning up resources still > > allocated on device remove before it invalidates and forgets their pointers? > > > > You are right. We need to wait until all allocated resources (iova and > mappings) to be released. > > How about registering a callback for BUS_NOTIFY_UNBOUND_DRIVER, and > removing the domain info when the driver detachment completes? Device core calls BUS_NOTIFY_UNBOUND_DRIVER on each driver unbind, regardless of a device being removed or not. As long as the device is not unplugged and the BUS_NOTIFY_REMOVED_DEVICE notification not generated, an unbound driver is not a problem here. Morever, BUS_NOTIFY_UNBOUND_DRIVER is called even before BUS_NOTIFY_REMOVED_DEVICE so that wouldn't help anyway. Last but not least, bus events are independent of the IOMMU driver use via DMA-API it exposes. If keeping data for unplugged devices and reusing it on device re-plug is not acceptable then maybe the IOMMU driver should perform reference counting of its internal resources occupied by DMA-API users and perform cleanups on last release? Thanks, Janusz > > Thanks, > > Janusz > > Best regards, > Baolu > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95B14C3A59F for ; Thu, 29 Aug 2019 07:58:28 +0000 (UTC) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61BE2205C9 for ; Thu, 29 Aug 2019 07:58:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61BE2205C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 1BED54576; Thu, 29 Aug 2019 07:58:28 +0000 (UTC) Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 13754428A for ; Thu, 29 Aug 2019 07:58:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A8690E6 for ; Thu, 29 Aug 2019 07:58:14 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2019 00:58:14 -0700 X-IronPort-AV: E=Sophos;i="5.64,442,1559545200"; d="scan'208";a="171821817" Received: from jkrzyszt-desk.igk.intel.com (HELO jkrzyszt-desk.ger.corp.intel.com) ([172.22.244.17]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2019 00:58:12 -0700 From: Janusz Krzysztofik To: Lu Baolu Subject: Re: [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug Date: Thu, 29 Aug 2019 09:58:05 +0200 Message-ID: <3255251.C7nBVfOIaa@jkrzyszt-desk.ger.corp.intel.com> Organization: Intel Technology Poland sp. z o.o. - ul. Slowackiego 173, 80-298 Gdansk - KRS 101882 - NIP 957-07-52-316 In-Reply-To: <0cf4e930-1132-1e7f-815b-57a08a1fe5de@linux.intel.com> References: <20190822142922.31526-1-janusz.krzysztofik@linux.intel.com> <3275480.HMaYE7B3nd@jkrzyszt-desk.ger.corp.intel.com> <0cf4e930-1132-1e7f-815b-57a08a1fe5de@linux.intel.com> MIME-Version: 1.0 Cc: linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, David Woodhouse , intel-gfx@lists.freedesktop.org, =?utf-8?B?TWljaGHFgg==?= Wajdeczko X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: iommu-bounces@lists.linux-foundation.org Errors-To: iommu-bounces@lists.linux-foundation.org Hi Baolu, On Thursday, August 29, 2019 3:43:31 AM CEST Lu Baolu wrote: > Hi Janusz, > > On 8/28/19 10:17 PM, Janusz Krzysztofik wrote: > >> We should avoid kernel panic when a intel_unmap() is called against > >> a non-existent domain. > > Does that mean you suggest to replace > > BUG_ON(!domain); > > with something like > > if (WARN_ON(!domain)) > > return; > > and to not care of orphaned mappings left allocated? Is there a way to inform > > users that their active DMA mappings are no longer valid and they shouldn't > > call dma_unmap_*()? > > > >> But we shouldn't expect the IOMMU driver not > >> cleaning up the domain info when a device remove notification comes and > >> wait until all file descriptors being closed, right? > > Shouldn't then the IOMMU driver take care of cleaning up resources still > > allocated on device remove before it invalidates and forgets their pointers? > > > > You are right. We need to wait until all allocated resources (iova and > mappings) to be released. > > How about registering a callback for BUS_NOTIFY_UNBOUND_DRIVER, and > removing the domain info when the driver detachment completes? Device core calls BUS_NOTIFY_UNBOUND_DRIVER on each driver unbind, regardless of a device being removed or not. As long as the device is not unplugged and the BUS_NOTIFY_REMOVED_DEVICE notification not generated, an unbound driver is not a problem here. Morever, BUS_NOTIFY_UNBOUND_DRIVER is called even before BUS_NOTIFY_REMOVED_DEVICE so that wouldn't help anyway. Last but not least, bus events are independent of the IOMMU driver use via DMA-API it exposes. If keeping data for unplugged devices and reusing it on device re-plug is not acceptable then maybe the IOMMU driver should perform reference counting of its internal resources occupied by DMA-API users and perform cleanups on last release? Thanks, Janusz > > Thanks, > > Janusz > > Best regards, > Baolu > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 From: Janusz Krzysztofik Subject: Re: [RFC PATCH] iommu/vt-d: Fix IOMMU field not populated on device hot re-plug Date: Thu, 29 Aug 2019 09:58:05 +0200 Message-ID: <3255251.C7nBVfOIaa@jkrzyszt-desk.ger.corp.intel.com> References: <20190822142922.31526-1-janusz.krzysztofik@linux.intel.com> <3275480.HMaYE7B3nd@jkrzyszt-desk.ger.corp.intel.com> <0cf4e930-1132-1e7f-815b-57a08a1fe5de@linux.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4BCA189FCC for ; Thu, 29 Aug 2019 07:58:14 +0000 (UTC) In-Reply-To: <0cf4e930-1132-1e7f-815b-57a08a1fe5de@linux.intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: Lu Baolu Cc: Joerg Roedel , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, David Woodhouse , intel-gfx@lists.freedesktop.org List-Id: intel-gfx@lists.freedesktop.org SGkgQmFvbHUsCgpPbiBUaHVyc2RheSwgQXVndXN0IDI5LCAyMDE5IDM6NDM6MzEgQU0gQ0VTVCBM dSBCYW9sdSB3cm90ZToKPiBIaSBKYW51c3osCj4gCj4gT24gOC8yOC8xOSAxMDoxNyBQTSwgSmFu dXN6IEtyenlzenRvZmlrIHdyb3RlOgo+ID4+IFdlIHNob3VsZCBhdm9pZCBrZXJuZWwgcGFuaWMg d2hlbiBhIGludGVsX3VubWFwKCkgaXMgY2FsbGVkIGFnYWluc3QKPiA+PiBhIG5vbi1leGlzdGVu dCBkb21haW4uCj4gPiBEb2VzIHRoYXQgbWVhbiB5b3Ugc3VnZ2VzdCB0byByZXBsYWNlCj4gPiAJ QlVHX09OKCFkb21haW4pOwo+ID4gd2l0aCBzb21ldGhpbmcgbGlrZQo+ID4gCWlmIChXQVJOX09O KCFkb21haW4pKQo+ID4gCQlyZXR1cm47Cj4gPiBhbmQgdG8gbm90IGNhcmUgb2Ygb3JwaGFuZWQg bWFwcGluZ3MgbGVmdCBhbGxvY2F0ZWQ/ICBJcyB0aGVyZSBhIHdheSB0byAKaW5mb3JtCj4gPiB1 c2VycyB0aGF0IHRoZWlyIGFjdGl2ZSBETUEgbWFwcGluZ3MgYXJlIG5vIGxvbmdlciB2YWxpZCBh bmQgdGhleSAKc2hvdWxkbid0Cj4gPiBjYWxsIGRtYV91bm1hcF8qKCk/Cj4gPiAKPiA+PiBCdXQg d2Ugc2hvdWxkbid0IGV4cGVjdCB0aGUgSU9NTVUgZHJpdmVyIG5vdAo+ID4+IGNsZWFuaW5nIHVw IHRoZSBkb21haW4gaW5mbyB3aGVuIGEgZGV2aWNlIHJlbW92ZSBub3RpZmljYXRpb24gY29tZXMg YW5kCj4gPj4gd2FpdCB1bnRpbCBhbGwgZmlsZSBkZXNjcmlwdG9ycyBiZWluZyBjbG9zZWQsIHJp Z2h0Pwo+ID4gU2hvdWxkbid0IHRoZW4gdGhlIElPTU1VIGRyaXZlciB0YWtlIGNhcmUgb2YgY2xl YW5pbmcgdXAgcmVzb3VyY2VzIHN0aWxsCj4gPiBhbGxvY2F0ZWQgb24gZGV2aWNlIHJlbW92ZSBi ZWZvcmUgaXQgaW52YWxpZGF0ZXMgYW5kIGZvcmdldHMgdGhlaXIgCnBvaW50ZXJzPwo+ID4gCj4g Cj4gWW91IGFyZSByaWdodC4gV2UgbmVlZCB0byB3YWl0IHVudGlsIGFsbCBhbGxvY2F0ZWQgcmVz b3VyY2VzIChpb3ZhIGFuZAo+IG1hcHBpbmdzKSB0byBiZSByZWxlYXNlZC4KPiAKPiBIb3cgYWJv dXQgcmVnaXN0ZXJpbmcgYSBjYWxsYmFjayBmb3IgQlVTX05PVElGWV9VTkJPVU5EX0RSSVZFUiwg YW5kCj4gcmVtb3ZpbmcgdGhlIGRvbWFpbiBpbmZvIHdoZW4gdGhlIGRyaXZlciBkZXRhY2htZW50 IGNvbXBsZXRlcz8KCkRldmljZSBjb3JlIGNhbGxzIEJVU19OT1RJRllfVU5CT1VORF9EUklWRVIg b24gZWFjaCBkcml2ZXIgdW5iaW5kLCByZWdhcmRsZXNzIApvZiBhIGRldmljZSBiZWluZyByZW1v dmVkIG9yIG5vdC4gIEFzIGxvbmcgYXMgdGhlIGRldmljZSBpcyBub3QgdW5wbHVnZ2VkIGFuZCAK dGhlIEJVU19OT1RJRllfUkVNT1ZFRF9ERVZJQ0Ugbm90aWZpY2F0aW9uIG5vdCBnZW5lcmF0ZWQs IGFuIHVuYm91bmQgZHJpdmVyIGlzIApub3QgYSBwcm9ibGVtIGhlcmUuCk1vcmV2ZXIsIEJVU19O T1RJRllfVU5CT1VORF9EUklWRVIgIGlzIGNhbGxlZCBldmVuIGJlZm9yZSAKQlVTX05PVElGWV9S RU1PVkVEX0RFVklDRSBzbyB0aGF0IHdvdWxkbid0IGhlbHAgYW55d2F5LgpMYXN0IGJ1dCBub3Qg bGVhc3QsIGJ1cyBldmVudHMgYXJlIGluZGVwZW5kZW50IG9mIHRoZSBJT01NVSBkcml2ZXIgdXNl IHZpYSAKRE1BLUFQSSBpdCBleHBvc2VzLgoKSWYga2VlcGluZyBkYXRhIGZvciB1bnBsdWdnZWQg ZGV2aWNlcyBhbmQgcmV1c2luZyBpdCBvbiBkZXZpY2UgcmUtcGx1ZyBpcyBub3QgCmFjY2VwdGFi bGUgdGhlbiBtYXliZSB0aGUgSU9NTVUgZHJpdmVyIHNob3VsZCBwZXJmb3JtIHJlZmVyZW5jZSBj b3VudGluZyBvZiAKaXRzIGludGVybmFsIHJlc291cmNlcyBvY2N1cGllZCBieSBETUEtQVBJIHVz ZXJzIGFuZCBwZXJmb3JtIGNsZWFudXBzIG9uIGxhc3QgCnJlbGVhc2U/CgpUaGFua3MsCkphbnVz egoKCj4gPiBUaGFua3MsCj4gPiBKYW51c3oKPiAKPiBCZXN0IHJlZ2FyZHMsCj4gQmFvbHUKPiAK CgoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkludGVs LWdmeCBtYWlsaW5nIGxpc3QKSW50ZWwtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczov L2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ludGVsLWdmeA==