From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,PDS_BAD_THREAD_QP_64, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B0ECC433B4 for ; Tue, 18 May 2021 14:02:44 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 599676117A for ; Tue, 18 May 2021 14:02:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 599676117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 55E636E883; Tue, 18 May 2021 14:02:43 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2066.outbound.protection.outlook.com [40.107.244.66]) by gabe.freedesktop.org (Postfix) with ESMTPS id A60A46E883; Tue, 18 May 2021 14:02:42 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X3wLEDa2j3cQXkBa+fSzjxQLEKWJ8JFF9dn2JuMifmEidQcpxftdKz9mts2ev9y+3XfV/Fvj9WipxaS7QtA43QobfFaUYXvztKNFsJo2wBtbrrcLkG5HoAbOVpwsGJRgv1qi3urAdSreZtr+gI9DGtKmKU9R/0rMuTQYWuyg5QkgKuFjm7V7+FRJ2vjaMNUn7j00MnuSVWUd+Lf9AXbknoIV/6ksauBzvwW5en3l1z54yKsW3TIEAcPGwaQth7xMdXIg0q24KcBBHpwl6Pqrn8s3KIYjCCZtw5vp4tr+W07qhJ3l2brO3ltGK565Ltni6Wtj635BZgl9DAYOxMpp2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7vxA9p1DyLoldSmU7UIF2BaSpNbKy5LB8PjyD/neGho=; b=nSg+nbFWJ/nqfgelNyExkkUQdeZw9t7n1p5eSqfzsJMg5GniJZjm8Mpr2mmpLoDPDYNKkoeuXjOE6EVJRZSfiTwAaCb/iebrw1ZRwnIH++mSan2YCR/wpjz1Dn69fm2vOqXkjFEXiqdnTQWK7ewRCE/LYJ586DOsiWIBV2LQRtzJ1aqHoXsH5HgOPE6hhKWohZqFs1VwdAbszUUCDQA21qAvhMZkHad1i//rURtOWTpYhw7BhDsn1YKr/QaOnVCLojsA7SPJ6q6dgSn+EsqMYLnq/GSnp6djPbLjhgjdNM1ku3HmRyZHsprCGf4FpzonlHPyYQUoYoxER4lODtr5ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7vxA9p1DyLoldSmU7UIF2BaSpNbKy5LB8PjyD/neGho=; b=wWkaA5Qr+vXPDciXWxpLTD33Uxh33ASJraLDHQInpHYgHrB3w7k7SGa610TLNyJZJoZ1uznep9h9q2DiLi2TJiqVVkuuMJQJdzbiiFSPhY/ynXxnnl2R0Jcaldj6ptWN+Q1wvH8bWEFLJZAfzTIUvIouWhOZ2ebQ/IzLNPsVnkM= Received: from MN2PR12MB4488.namprd12.prod.outlook.com (2603:10b6:208:24e::19) by BL0PR12MB2402.namprd12.prod.outlook.com (2603:10b6:207:45::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.28; Tue, 18 May 2021 14:02:40 +0000 Received: from MN2PR12MB4488.namprd12.prod.outlook.com ([fe80::3d98:cefb:476c:c36e]) by MN2PR12MB4488.namprd12.prod.outlook.com ([fe80::3d98:cefb:476c:c36e%8]) with mapi id 15.20.4129.031; Tue, 18 May 2021 14:02:40 +0000 From: "Deucher, Alexander" To: "Grodzovsky, Andrey" , "dri-devel@lists.freedesktop.org" , "amd-gfx@lists.freedesktop.org" , "linux-pci@vger.kernel.org" , "ckoenig.leichtzumerken@gmail.com" , "daniel.vetter@ffwll.ch" , "Wentland, Harry" Subject: Re: [PATCH] drm/amdgpu: Unmap all MMIO mappings Thread-Topic: [PATCH] drm/amdgpu: Unmap all MMIO mappings Thread-Index: AQHXS1M6N8cxiamYe0ef5VQK9FP0tarpRgYAgAAAJi0= Date: Tue, 18 May 2021 14:02:40 +0000 Message-ID: References: <20210517193105.491461-1-andrey.grodzovsky@amd.com>, <491692a8-a7e4-66db-9cde-f660158ad256@amd.com> In-Reply-To: <491692a8-a7e4-66db-9cde-f660158ad256@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_Enabled=True; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_SiteId=3dd8961f-e488-4e60-8e11-a82d994e183d; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_SetDate=2021-05-18T14:02:39.809Z; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_Name=Public; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_ContentBits=0; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_Method=Privileged; authentication-results: amd.com; dkim=none (message not signed) header.d=none;amd.com; dmarc=none action=none header.from=amd.com; x-originating-ip: [204.111.139.213] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7dfd3727-1fc9-45b6-76a8-08d91a0599b9 x-ms-traffictypediagnostic: BL0PR12MB2402: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2043; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Wdbv27GQENIDgVha2DQ8kqKOvjIXz1f5gYfF6njRirNcEVcpxftG+Rk2L2T9YoSqmjsDIGdehxnUc30vjLETqo6ovwcou66DQFns0mddT2YvYHXJ/4Y2AgaOzxjYiq8gPlMlFQPkvhby91e5MfX9lL/h7TaEH1NrcCJQf7GGh160THK5jN2U3MCSPBzCtNORjYsCh3WQhPAp2bLBgYndP997Vr/VGJpbm3RfnORP4F0W2/TctSAXUVmW9gX83lROMtLJp1soi2bPEa0lsRvXgG+0bGt0a6BIn3cFZDpbIUnc6/y23SCeyOkwJo5SaTIq4Wtie+rho41ayVz3Ba3ECMl93jh3tHmgf3yatf4Z9/BfE71DpQ/SrPkkDj6ziBwClCB2mfhkOIc0YAZ889egnO4Nhff0TbO3Oi+ugP7G4R/p0NrS676YeKUv3S8XCuPVELREfd0eSha9EkiYg7bMtPTmgLEpT4e5AtaB4LNfDHuMJCf1b97+B3LESQrkdDPDnZRNMJ+mPs89sRp88NIFoOcvnFyvQp3WO3Ys45HIgo1zMyxhQmqDoaSQDcT+ejnIH1hnb2x6znoUl0IIzgpioWTIx0Juvja/mYJSNkfMvyM= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB4488.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(396003)(376002)(346002)(39860400002)(136003)(4326008)(86362001)(5660300002)(186003)(316002)(6506007)(110136005)(66446008)(122000001)(54906003)(33656002)(71200400001)(53546011)(52536014)(26005)(83380400001)(9686003)(76116006)(8676002)(8936002)(55016002)(7696005)(64756008)(2906002)(66946007)(66556008)(66476007)(478600001)(19627405001)(6636002)(38100700002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?piuIGabgMi6OGEbjxYYIv73Y1Luj+SIZ2XrdNM4TdQHUdL63/Z2N6AqU/BAx?= =?us-ascii?Q?K8kzyv22RT+y8FCXn5KioKHPo6wxiahU88kr9ZjQQqb9AzAJ1xRc0RottJ/s?= =?us-ascii?Q?+78xQQ+eiANh/1H9I/bCKG4iX7Sxxet/9iHaGiIUNKD5FxrY09iKUo72bhU4?= =?us-ascii?Q?887hy8MAN0gVAhzsTbZ9Yv1JrerpCx0T4PCn9O40606JAGSaY7ToNcYycmHX?= =?us-ascii?Q?xLWFQb2Ee8RrsIF9q6WsKjR3SOZwG5RNoz7bUFSPg6wH46RokParDFyjUfKu?= =?us-ascii?Q?I2xI9KKBHQ9HR63xFkVK6yo8+EqzB2xVMKN/qc7S6r0YF6pYbQrH5w5cNF2R?= =?us-ascii?Q?xYGL4a41SrXP8B4i1odR+32M9sydTmaOJbfSZWHUTR+AWP2lUeBRZxBQE/Ir?= =?us-ascii?Q?uv/Tyf0ZHVu/NAwm8QcP93kLrIa1JHC5ZeeO3n6ovO60KY6PXn+QBSK028FB?= =?us-ascii?Q?FRCrQwN9paXUPvvPw6nj+jq667OgTR24FelAziiCQ6atSIMxeuZY1GRfAx4x?= =?us-ascii?Q?nHDMBMVn3b42ztwMafjs5vM9928yFw/iOlcKT/qyvU9aDx4jgR1xGojvJd3u?= =?us-ascii?Q?KbpljICTOmTFz8+NquZbBypfWKTGUz9IYZ7EEnF5lhNgLfXZ7IE/Ldywq+wC?= =?us-ascii?Q?rNDKftFzBK7SH0X4CUp8hzNpwcKqO/XFtco3Uo8LAOflOsC9LDQKkABvhNMR?= =?us-ascii?Q?4jfbFSHKDT1uMwZ6NMFtqRSuo9MlTnYBC0vVAoNvDC6pvrEWyr7tIngiKOH7?= =?us-ascii?Q?aagTDVJyadqat9pJfL1gmiwkCEyrKPRKT2P4/A89crVCHAjTrz+JJl3KBc3w?= =?us-ascii?Q?Aqj+iZnCixKPMo/EmTTlHTbtkk/8vk7bccqax+uKrGt49H9h7qOFqwe3KRo3?= =?us-ascii?Q?Ldr1mysYYBLXPx4sjW3y48bmaotmz4He2J2PrQBDydvvO5yz3Y7+rTVO9uvh?= =?us-ascii?Q?GJEZGX5CSieZZ3WC2rQUEKW/OspDFE21usBf4djr5Z51xyVG1MwXQlPMIASJ?= =?us-ascii?Q?EeApeQc57hQfXaXLinx+KAsxVSDTQp/kJZs3GorsAXGY4nKuBk7Pj2FuAXID?= =?us-ascii?Q?DiK2gfNKKKuneRdk+cqlyvf0wseN8tPCWwIpjF1jAXIOSoiF1LtD7wYi9w+5?= =?us-ascii?Q?88Tl1Wq/IEh52tPYm/R5emVm6HffcRqLX9jvgQfGNgJbalfwpvUER8ol0/8Z?= =?us-ascii?Q?8Y8NA0b7DOjqeIigYQq3dv/X28sLgeAcJ3CJjjv0LVOBC9pOaWn8jqCjtaro?= =?us-ascii?Q?5B/0tJcIGM4DDMwZLh6ecTMGzkG/UnoZU1E0nXTmvemqBtEJNG8OrdDYL50U?= =?us-ascii?Q?nvKmfAg3QX5iEhkL3R7hAsov?= Content-Type: multipart/alternative; boundary="_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_" MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4488.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7dfd3727-1fc9-45b6-76a8-08d91a0599b9 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 May 2021 14:02:40.4260 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1A1X29/Lc4GQVlyR3vzwV0YrK73wt7KLVw84FsnQ7amfRgBl6ECGb5NxtlsWcdP3ZaytXQuIVxMOY6eiBnw1dA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2402 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "gregkh@linuxfoundation.org" , "helgaas@kernel.org" , "Kuehling, Felix" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" --_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable [Public] Reviewed-by: Alex Deucher ________________________________ From: Grodzovsky, Andrey Sent: Tuesday, May 18, 2021 10:01 AM To: dri-devel@lists.freedesktop.org ; amd-= gfx@lists.freedesktop.org ; linux-pci@vger.k= ernel.org ; ckoenig.leichtzumerken@gmail.com ; daniel.vetter@ffwll.ch ; Wentland, Harry Cc: ppaalanen@gmail.com ; Deucher, Alexander ; gregkh@linuxfoundation.org = ; helgaas@kernel.org ; Kuehling, Felix Subject: Re: [PATCH] drm/amdgpu: Unmap all MMIO mappings Ping Andrey On 2021-05-17 3:31 p.m., Andrey Grodzovsky wrote: > Access to those must be prevented post pci_remove > > v6: Drop BOs list, unampping VRAM BAR is enough. > v8: > Add condition of xgmi.connected_to_cpu to MTTR > handling and remove MTTR handling from the old place. > > Signed-off-by: Andrey Grodzovsky > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++++++--- > drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 4 ---- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 4 ---- > 3 files changed, 23 insertions(+), 11 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_device.c > index f7cca25c0fa0..8b50315d1fe1 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > @@ -3666,6 +3666,27 @@ int amdgpu_device_init(struct amdgpu_device *adev, > return r; > } > > +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) > +{ > + /* Clear all CPU mappings pointing to this device */ > + unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1); > + > + /* Unmap all mapped bars - Doorbell, registers and VRAM */ > + amdgpu_device_doorbell_fini(adev); > + > + iounmap(adev->rmmio); > + adev->rmmio =3D NULL; > + if (adev->mman.aper_base_kaddr) > + iounmap(adev->mman.aper_base_kaddr); > + adev->mman.aper_base_kaddr =3D NULL; > + > + /* Memory manager related */ > + if (!adev->gmc.xgmi.connected_to_cpu) { > + arch_phys_wc_del(adev->gmc.vram_mtrr); > + arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper= _size); > + } > +} > + > /** > * amdgpu_device_fini - tear down the driver > * > @@ -3712,6 +3733,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *ad= ev) > amdgpu_device_ip_fini_early(adev); > > amdgpu_gart_dummy_page_fini(adev); > + > + amdgpu_device_unmap_mmio(adev); > } > > void amdgpu_device_fini_sw(struct amdgpu_device *adev) > @@ -3739,9 +3762,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *ad= ev) > } > if ((adev->pdev->class >> 8) =3D=3D PCI_CLASS_DISPLAY_VGA) > vga_client_register(adev->pdev, NULL, NULL, NULL); > - iounmap(adev->rmmio); > - adev->rmmio =3D NULL; > - amdgpu_device_doorbell_fini(adev); > > if (IS_ENABLED(CONFIG_PERF_EVENTS)) > amdgpu_pmu_fini(adev); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_object.c > index 0adffcace326..8eabe3c9ad17 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > @@ -1107,10 +1107,6 @@ int amdgpu_bo_init(struct amdgpu_device *adev) > void amdgpu_bo_fini(struct amdgpu_device *adev) > { > amdgpu_ttm_fini(adev); > - if (!adev->gmc.xgmi.connected_to_cpu) { > - arch_phys_wc_del(adev->gmc.vram_mtrr); > - arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper= _size); > - } > } > > /** > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/am= d/amdgpu/amdgpu_ttm.c > index 0d54e70278ca..58ad2fecc9e3 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > @@ -1841,10 +1841,6 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) > amdgpu_bo_free_kernel(&adev->mman.discovery_memory, NULL, NULL); > amdgpu_ttm_fw_reserve_vram_fini(adev); > > - if (adev->mman.aper_base_kaddr) > - iounmap(adev->mman.aper_base_kaddr); > - adev->mman.aper_base_kaddr =3D NULL; > - > amdgpu_vram_mgr_fini(adev); > amdgpu_gtt_mgr_fini(adev); > ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_GDS); > --_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

[Public]


Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
From: Grodzovsky, Andrey &l= t;Andrey.Grodzovsky@amd.com>
Sent: Tuesday, May 18, 2021 10:01 AM
To: dri-devel@lists.freedesktop.org <dri-devel@lists.freedesktop.= org>; amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>= ;; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>; ckoenig.lei= chtzumerken@gmail.com <ckoenig.leichtzumerken@gmail.com>; daniel.vetter@ffwll.ch <daniel.vetter@ffwll.ch>; Wentland, Harry <= ;Harry.Wentland@amd.com>
Cc: ppaalanen@gmail.com <ppaalanen@gmail.com>; Deucher, Alexan= der <Alexander.Deucher@amd.com>; gregkh@linuxfoundation.org <gregk= h@linuxfoundation.org>; helgaas@kernel.org <helgaas@kernel.org>; K= uehling, Felix <Felix.Kuehling@amd.com>
Subject: Re: [PATCH] drm/amdgpu: Unmap all MMIO mappings
 
Ping

Andrey

On 2021-05-17 3:31 p.m., Andrey Grodzovsky wrote:
> Access to those must be prevented post pci_remove
>
> v6: Drop BOs list, unampping VRAM BAR is enough.
> v8:
> Add condition of xgmi.connected_to_cpu to MTTR
> handling and remove MTTR handling from the old place.
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 ++++++++++= +++++++++---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c |  4 ----<= br> >   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    = |  4 ----
>   3 files changed, 23 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/= drm/amd/amdgpu/amdgpu_device.c
> index f7cca25c0fa0..8b50315d1fe1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -3666,6 +3666,27 @@ int amdgpu_device_init(struct amdgpu_device *ad= ev,
>        return r;
>   }
>  
> +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
> +{
> +     /* Clear all CPU mappings pointing to this d= evice */
> +     unmap_mapping_range(adev->ddev.anon_inode= ->i_mapping, 0, 0, 1);
> +
> +     /* Unmap all mapped bars - Doorbell, registe= rs and VRAM */
> +     amdgpu_device_doorbell_fini(adev);
> +
> +     iounmap(adev->rmmio);
> +     adev->rmmio =3D NULL;
> +     if (adev->mman.aper_base_kaddr)
> +           &nb= sp; iounmap(adev->mman.aper_base_kaddr);
> +     adev->mman.aper_base_kaddr =3D NULL;
> +
> +     /* Memory manager related */
> +     if (!adev->gmc.xgmi.connected_to_cpu) { > +           &nb= sp; arch_phys_wc_del(adev->gmc.vram_mtrr);
> +           &nb= sp; arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size)= ;
> +     }
> +}
> +
>   /**
>    * amdgpu_device_fini - tear down the driver
>    *
> @@ -3712,6 +3733,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device = *adev)
>        amdgpu_device_ip_fini_early(= adev);
>  
>        amdgpu_gart_dummy_page_fini(= adev);
> +
> +     amdgpu_device_unmap_mmio(adev);
>   }
>  
>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> @@ -3739,9 +3762,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device = *adev)
>        }
>        if ((adev->pdev->class= >> 8) =3D=3D PCI_CLASS_DISPLAY_VGA)
>            = ;    vga_client_register(adev->pdev, NULL, NULL, NULL); > -     iounmap(adev->rmmio);
> -     adev->rmmio =3D NULL;
> -     amdgpu_device_doorbell_fini(adev);
>  
>        if (IS_ENABLED(CONFIG_PERF_E= VENTS))
>            = ;    amdgpu_pmu_fini(adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/= drm/amd/amdgpu/amdgpu_object.c
> index 0adffcace326..8eabe3c9ad17 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> @@ -1107,10 +1107,6 @@ int amdgpu_bo_init(struct amdgpu_device *adev)<= br> >   void amdgpu_bo_fini(struct amdgpu_device *adev)
>   {
>        amdgpu_ttm_fini(adev);
> -     if (!adev->gmc.xgmi.connected_to_cpu) { > -           &nb= sp; arch_phys_wc_del(adev->gmc.vram_mtrr);
> -           &nb= sp; arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size)= ;
> -     }
>   }
>  
>   /**
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_ttm.c
> index 0d54e70278ca..58ad2fecc9e3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -1841,10 +1841,6 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev= )
>        amdgpu_bo_free_kernel(&a= dev->mman.discovery_memory, NULL, NULL);
>        amdgpu_ttm_fw_reserve_vram_f= ini(adev);
>  
> -     if (adev->mman.aper_base_kaddr)
> -           &nb= sp; iounmap(adev->mman.aper_base_kaddr);
> -     adev->mman.aper_base_kaddr =3D NULL;
> -
>        amdgpu_vram_mgr_fini(adev);<= br> >        amdgpu_gtt_mgr_fini(adev); >        ttm_range_man_fini(&adev= ->mman.bdev, AMDGPU_PL_GDS);
>
--_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,PDS_BAD_THREAD_QP_64,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7ECEAC433ED for ; Tue, 18 May 2021 14:02:44 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3646161185 for ; Tue, 18 May 2021 14:02:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3646161185 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 99B3D6E884; Tue, 18 May 2021 14:02:43 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2066.outbound.protection.outlook.com [40.107.244.66]) by gabe.freedesktop.org (Postfix) with ESMTPS id A60A46E883; Tue, 18 May 2021 14:02:42 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X3wLEDa2j3cQXkBa+fSzjxQLEKWJ8JFF9dn2JuMifmEidQcpxftdKz9mts2ev9y+3XfV/Fvj9WipxaS7QtA43QobfFaUYXvztKNFsJo2wBtbrrcLkG5HoAbOVpwsGJRgv1qi3urAdSreZtr+gI9DGtKmKU9R/0rMuTQYWuyg5QkgKuFjm7V7+FRJ2vjaMNUn7j00MnuSVWUd+Lf9AXbknoIV/6ksauBzvwW5en3l1z54yKsW3TIEAcPGwaQth7xMdXIg0q24KcBBHpwl6Pqrn8s3KIYjCCZtw5vp4tr+W07qhJ3l2brO3ltGK565Ltni6Wtj635BZgl9DAYOxMpp2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7vxA9p1DyLoldSmU7UIF2BaSpNbKy5LB8PjyD/neGho=; b=nSg+nbFWJ/nqfgelNyExkkUQdeZw9t7n1p5eSqfzsJMg5GniJZjm8Mpr2mmpLoDPDYNKkoeuXjOE6EVJRZSfiTwAaCb/iebrw1ZRwnIH++mSan2YCR/wpjz1Dn69fm2vOqXkjFEXiqdnTQWK7ewRCE/LYJ586DOsiWIBV2LQRtzJ1aqHoXsH5HgOPE6hhKWohZqFs1VwdAbszUUCDQA21qAvhMZkHad1i//rURtOWTpYhw7BhDsn1YKr/QaOnVCLojsA7SPJ6q6dgSn+EsqMYLnq/GSnp6djPbLjhgjdNM1ku3HmRyZHsprCGf4FpzonlHPyYQUoYoxER4lODtr5ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7vxA9p1DyLoldSmU7UIF2BaSpNbKy5LB8PjyD/neGho=; b=wWkaA5Qr+vXPDciXWxpLTD33Uxh33ASJraLDHQInpHYgHrB3w7k7SGa610TLNyJZJoZ1uznep9h9q2DiLi2TJiqVVkuuMJQJdzbiiFSPhY/ynXxnnl2R0Jcaldj6ptWN+Q1wvH8bWEFLJZAfzTIUvIouWhOZ2ebQ/IzLNPsVnkM= Received: from MN2PR12MB4488.namprd12.prod.outlook.com (2603:10b6:208:24e::19) by BL0PR12MB2402.namprd12.prod.outlook.com (2603:10b6:207:45::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.28; Tue, 18 May 2021 14:02:40 +0000 Received: from MN2PR12MB4488.namprd12.prod.outlook.com ([fe80::3d98:cefb:476c:c36e]) by MN2PR12MB4488.namprd12.prod.outlook.com ([fe80::3d98:cefb:476c:c36e%8]) with mapi id 15.20.4129.031; Tue, 18 May 2021 14:02:40 +0000 From: "Deucher, Alexander" To: "Grodzovsky, Andrey" , "dri-devel@lists.freedesktop.org" , "amd-gfx@lists.freedesktop.org" , "linux-pci@vger.kernel.org" , "ckoenig.leichtzumerken@gmail.com" , "daniel.vetter@ffwll.ch" , "Wentland, Harry" Subject: Re: [PATCH] drm/amdgpu: Unmap all MMIO mappings Thread-Topic: [PATCH] drm/amdgpu: Unmap all MMIO mappings Thread-Index: AQHXS1M6N8cxiamYe0ef5VQK9FP0tarpRgYAgAAAJi0= Date: Tue, 18 May 2021 14:02:40 +0000 Message-ID: References: <20210517193105.491461-1-andrey.grodzovsky@amd.com>, <491692a8-a7e4-66db-9cde-f660158ad256@amd.com> In-Reply-To: <491692a8-a7e4-66db-9cde-f660158ad256@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_Enabled=True; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_SiteId=3dd8961f-e488-4e60-8e11-a82d994e183d; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_SetDate=2021-05-18T14:02:39.809Z; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_Name=Public; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_ContentBits=0; MSIP_Label_d4243a53-6221-4f75-8154-e4b33a5707a1_Method=Privileged; authentication-results: amd.com; dkim=none (message not signed) header.d=none;amd.com; dmarc=none action=none header.from=amd.com; x-originating-ip: [204.111.139.213] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7dfd3727-1fc9-45b6-76a8-08d91a0599b9 x-ms-traffictypediagnostic: BL0PR12MB2402: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2043; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Wdbv27GQENIDgVha2DQ8kqKOvjIXz1f5gYfF6njRirNcEVcpxftG+Rk2L2T9YoSqmjsDIGdehxnUc30vjLETqo6ovwcou66DQFns0mddT2YvYHXJ/4Y2AgaOzxjYiq8gPlMlFQPkvhby91e5MfX9lL/h7TaEH1NrcCJQf7GGh160THK5jN2U3MCSPBzCtNORjYsCh3WQhPAp2bLBgYndP997Vr/VGJpbm3RfnORP4F0W2/TctSAXUVmW9gX83lROMtLJp1soi2bPEa0lsRvXgG+0bGt0a6BIn3cFZDpbIUnc6/y23SCeyOkwJo5SaTIq4Wtie+rho41ayVz3Ba3ECMl93jh3tHmgf3yatf4Z9/BfE71DpQ/SrPkkDj6ziBwClCB2mfhkOIc0YAZ889egnO4Nhff0TbO3Oi+ugP7G4R/p0NrS676YeKUv3S8XCuPVELREfd0eSha9EkiYg7bMtPTmgLEpT4e5AtaB4LNfDHuMJCf1b97+B3LESQrkdDPDnZRNMJ+mPs89sRp88NIFoOcvnFyvQp3WO3Ys45HIgo1zMyxhQmqDoaSQDcT+ejnIH1hnb2x6znoUl0IIzgpioWTIx0Juvja/mYJSNkfMvyM= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB4488.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(396003)(376002)(346002)(39860400002)(136003)(4326008)(86362001)(5660300002)(186003)(316002)(6506007)(110136005)(66446008)(122000001)(54906003)(33656002)(71200400001)(53546011)(52536014)(26005)(83380400001)(9686003)(76116006)(8676002)(8936002)(55016002)(7696005)(64756008)(2906002)(66946007)(66556008)(66476007)(478600001)(19627405001)(6636002)(38100700002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?piuIGabgMi6OGEbjxYYIv73Y1Luj+SIZ2XrdNM4TdQHUdL63/Z2N6AqU/BAx?= =?us-ascii?Q?K8kzyv22RT+y8FCXn5KioKHPo6wxiahU88kr9ZjQQqb9AzAJ1xRc0RottJ/s?= =?us-ascii?Q?+78xQQ+eiANh/1H9I/bCKG4iX7Sxxet/9iHaGiIUNKD5FxrY09iKUo72bhU4?= =?us-ascii?Q?887hy8MAN0gVAhzsTbZ9Yv1JrerpCx0T4PCn9O40606JAGSaY7ToNcYycmHX?= =?us-ascii?Q?xLWFQb2Ee8RrsIF9q6WsKjR3SOZwG5RNoz7bUFSPg6wH46RokParDFyjUfKu?= =?us-ascii?Q?I2xI9KKBHQ9HR63xFkVK6yo8+EqzB2xVMKN/qc7S6r0YF6pYbQrH5w5cNF2R?= =?us-ascii?Q?xYGL4a41SrXP8B4i1odR+32M9sydTmaOJbfSZWHUTR+AWP2lUeBRZxBQE/Ir?= =?us-ascii?Q?uv/Tyf0ZHVu/NAwm8QcP93kLrIa1JHC5ZeeO3n6ovO60KY6PXn+QBSK028FB?= =?us-ascii?Q?FRCrQwN9paXUPvvPw6nj+jq667OgTR24FelAziiCQ6atSIMxeuZY1GRfAx4x?= =?us-ascii?Q?nHDMBMVn3b42ztwMafjs5vM9928yFw/iOlcKT/qyvU9aDx4jgR1xGojvJd3u?= =?us-ascii?Q?KbpljICTOmTFz8+NquZbBypfWKTGUz9IYZ7EEnF5lhNgLfXZ7IE/Ldywq+wC?= =?us-ascii?Q?rNDKftFzBK7SH0X4CUp8hzNpwcKqO/XFtco3Uo8LAOflOsC9LDQKkABvhNMR?= =?us-ascii?Q?4jfbFSHKDT1uMwZ6NMFtqRSuo9MlTnYBC0vVAoNvDC6pvrEWyr7tIngiKOH7?= =?us-ascii?Q?aagTDVJyadqat9pJfL1gmiwkCEyrKPRKT2P4/A89crVCHAjTrz+JJl3KBc3w?= =?us-ascii?Q?Aqj+iZnCixKPMo/EmTTlHTbtkk/8vk7bccqax+uKrGt49H9h7qOFqwe3KRo3?= =?us-ascii?Q?Ldr1mysYYBLXPx4sjW3y48bmaotmz4He2J2PrQBDydvvO5yz3Y7+rTVO9uvh?= =?us-ascii?Q?GJEZGX5CSieZZ3WC2rQUEKW/OspDFE21usBf4djr5Z51xyVG1MwXQlPMIASJ?= =?us-ascii?Q?EeApeQc57hQfXaXLinx+KAsxVSDTQp/kJZs3GorsAXGY4nKuBk7Pj2FuAXID?= =?us-ascii?Q?DiK2gfNKKKuneRdk+cqlyvf0wseN8tPCWwIpjF1jAXIOSoiF1LtD7wYi9w+5?= =?us-ascii?Q?88Tl1Wq/IEh52tPYm/R5emVm6HffcRqLX9jvgQfGNgJbalfwpvUER8ol0/8Z?= =?us-ascii?Q?8Y8NA0b7DOjqeIigYQq3dv/X28sLgeAcJ3CJjjv0LVOBC9pOaWn8jqCjtaro?= =?us-ascii?Q?5B/0tJcIGM4DDMwZLh6ecTMGzkG/UnoZU1E0nXTmvemqBtEJNG8OrdDYL50U?= =?us-ascii?Q?nvKmfAg3QX5iEhkL3R7hAsov?= MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4488.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7dfd3727-1fc9-45b6-76a8-08d91a0599b9 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 May 2021 14:02:40.4260 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1A1X29/Lc4GQVlyR3vzwV0YrK73wt7KLVw84FsnQ7amfRgBl6ECGb5NxtlsWcdP3ZaytXQuIVxMOY6eiBnw1dA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2402 X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "gregkh@linuxfoundation.org" , "ppaalanen@gmail.com" , "helgaas@kernel.org" , "Kuehling, Felix" Content-Type: multipart/mixed; boundary="===============0029053651==" Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" --===============0029053651== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_" --_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable [Public] Reviewed-by: Alex Deucher ________________________________ From: Grodzovsky, Andrey Sent: Tuesday, May 18, 2021 10:01 AM To: dri-devel@lists.freedesktop.org ; amd-= gfx@lists.freedesktop.org ; linux-pci@vger.k= ernel.org ; ckoenig.leichtzumerken@gmail.com ; daniel.vetter@ffwll.ch ; Wentland, Harry Cc: ppaalanen@gmail.com ; Deucher, Alexander ; gregkh@linuxfoundation.org = ; helgaas@kernel.org ; Kuehling, Felix Subject: Re: [PATCH] drm/amdgpu: Unmap all MMIO mappings Ping Andrey On 2021-05-17 3:31 p.m., Andrey Grodzovsky wrote: > Access to those must be prevented post pci_remove > > v6: Drop BOs list, unampping VRAM BAR is enough. > v8: > Add condition of xgmi.connected_to_cpu to MTTR > handling and remove MTTR handling from the old place. > > Signed-off-by: Andrey Grodzovsky > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++++++--- > drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 4 ---- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 4 ---- > 3 files changed, 23 insertions(+), 11 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_device.c > index f7cca25c0fa0..8b50315d1fe1 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > @@ -3666,6 +3666,27 @@ int amdgpu_device_init(struct amdgpu_device *adev, > return r; > } > > +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) > +{ > + /* Clear all CPU mappings pointing to this device */ > + unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1); > + > + /* Unmap all mapped bars - Doorbell, registers and VRAM */ > + amdgpu_device_doorbell_fini(adev); > + > + iounmap(adev->rmmio); > + adev->rmmio =3D NULL; > + if (adev->mman.aper_base_kaddr) > + iounmap(adev->mman.aper_base_kaddr); > + adev->mman.aper_base_kaddr =3D NULL; > + > + /* Memory manager related */ > + if (!adev->gmc.xgmi.connected_to_cpu) { > + arch_phys_wc_del(adev->gmc.vram_mtrr); > + arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper= _size); > + } > +} > + > /** > * amdgpu_device_fini - tear down the driver > * > @@ -3712,6 +3733,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *ad= ev) > amdgpu_device_ip_fini_early(adev); > > amdgpu_gart_dummy_page_fini(adev); > + > + amdgpu_device_unmap_mmio(adev); > } > > void amdgpu_device_fini_sw(struct amdgpu_device *adev) > @@ -3739,9 +3762,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *ad= ev) > } > if ((adev->pdev->class >> 8) =3D=3D PCI_CLASS_DISPLAY_VGA) > vga_client_register(adev->pdev, NULL, NULL, NULL); > - iounmap(adev->rmmio); > - adev->rmmio =3D NULL; > - amdgpu_device_doorbell_fini(adev); > > if (IS_ENABLED(CONFIG_PERF_EVENTS)) > amdgpu_pmu_fini(adev); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_object.c > index 0adffcace326..8eabe3c9ad17 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > @@ -1107,10 +1107,6 @@ int amdgpu_bo_init(struct amdgpu_device *adev) > void amdgpu_bo_fini(struct amdgpu_device *adev) > { > amdgpu_ttm_fini(adev); > - if (!adev->gmc.xgmi.connected_to_cpu) { > - arch_phys_wc_del(adev->gmc.vram_mtrr); > - arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper= _size); > - } > } > > /** > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/am= d/amdgpu/amdgpu_ttm.c > index 0d54e70278ca..58ad2fecc9e3 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > @@ -1841,10 +1841,6 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) > amdgpu_bo_free_kernel(&adev->mman.discovery_memory, NULL, NULL); > amdgpu_ttm_fw_reserve_vram_fini(adev); > > - if (adev->mman.aper_base_kaddr) > - iounmap(adev->mman.aper_base_kaddr); > - adev->mman.aper_base_kaddr =3D NULL; > - > amdgpu_vram_mgr_fini(adev); > amdgpu_gtt_mgr_fini(adev); > ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_GDS); > --_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

[Public]


Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
From: Grodzovsky, Andrey &l= t;Andrey.Grodzovsky@amd.com>
Sent: Tuesday, May 18, 2021 10:01 AM
To: dri-devel@lists.freedesktop.org <dri-devel@lists.freedesktop.= org>; amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>= ;; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>; ckoenig.lei= chtzumerken@gmail.com <ckoenig.leichtzumerken@gmail.com>; daniel.vetter@ffwll.ch <daniel.vetter@ffwll.ch>; Wentland, Harry <= ;Harry.Wentland@amd.com>
Cc: ppaalanen@gmail.com <ppaalanen@gmail.com>; Deucher, Alexan= der <Alexander.Deucher@amd.com>; gregkh@linuxfoundation.org <gregk= h@linuxfoundation.org>; helgaas@kernel.org <helgaas@kernel.org>; K= uehling, Felix <Felix.Kuehling@amd.com>
Subject: Re: [PATCH] drm/amdgpu: Unmap all MMIO mappings
 
Ping

Andrey

On 2021-05-17 3:31 p.m., Andrey Grodzovsky wrote:
> Access to those must be prevented post pci_remove
>
> v6: Drop BOs list, unampping VRAM BAR is enough.
> v8:
> Add condition of xgmi.connected_to_cpu to MTTR
> handling and remove MTTR handling from the old place.
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 ++++++++++= +++++++++---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c |  4 ----<= br> >   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    = |  4 ----
>   3 files changed, 23 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/= drm/amd/amdgpu/amdgpu_device.c
> index f7cca25c0fa0..8b50315d1fe1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -3666,6 +3666,27 @@ int amdgpu_device_init(struct amdgpu_device *ad= ev,
>        return r;
>   }
>  
> +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
> +{
> +     /* Clear all CPU mappings pointing to this d= evice */
> +     unmap_mapping_range(adev->ddev.anon_inode= ->i_mapping, 0, 0, 1);
> +
> +     /* Unmap all mapped bars - Doorbell, registe= rs and VRAM */
> +     amdgpu_device_doorbell_fini(adev);
> +
> +     iounmap(adev->rmmio);
> +     adev->rmmio =3D NULL;
> +     if (adev->mman.aper_base_kaddr)
> +           &nb= sp; iounmap(adev->mman.aper_base_kaddr);
> +     adev->mman.aper_base_kaddr =3D NULL;
> +
> +     /* Memory manager related */
> +     if (!adev->gmc.xgmi.connected_to_cpu) { > +           &nb= sp; arch_phys_wc_del(adev->gmc.vram_mtrr);
> +           &nb= sp; arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size)= ;
> +     }
> +}
> +
>   /**
>    * amdgpu_device_fini - tear down the driver
>    *
> @@ -3712,6 +3733,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device = *adev)
>        amdgpu_device_ip_fini_early(= adev);
>  
>        amdgpu_gart_dummy_page_fini(= adev);
> +
> +     amdgpu_device_unmap_mmio(adev);
>   }
>  
>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> @@ -3739,9 +3762,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device = *adev)
>        }
>        if ((adev->pdev->class= >> 8) =3D=3D PCI_CLASS_DISPLAY_VGA)
>            = ;    vga_client_register(adev->pdev, NULL, NULL, NULL); > -     iounmap(adev->rmmio);
> -     adev->rmmio =3D NULL;
> -     amdgpu_device_doorbell_fini(adev);
>  
>        if (IS_ENABLED(CONFIG_PERF_E= VENTS))
>            = ;    amdgpu_pmu_fini(adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/= drm/amd/amdgpu/amdgpu_object.c
> index 0adffcace326..8eabe3c9ad17 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> @@ -1107,10 +1107,6 @@ int amdgpu_bo_init(struct amdgpu_device *adev)<= br> >   void amdgpu_bo_fini(struct amdgpu_device *adev)
>   {
>        amdgpu_ttm_fini(adev);
> -     if (!adev->gmc.xgmi.connected_to_cpu) { > -           &nb= sp; arch_phys_wc_del(adev->gmc.vram_mtrr);
> -           &nb= sp; arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size)= ;
> -     }
>   }
>  
>   /**
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_ttm.c
> index 0d54e70278ca..58ad2fecc9e3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -1841,10 +1841,6 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev= )
>        amdgpu_bo_free_kernel(&a= dev->mman.discovery_memory, NULL, NULL);
>        amdgpu_ttm_fw_reserve_vram_f= ini(adev);
>  
> -     if (adev->mman.aper_base_kaddr)
> -           &nb= sp; iounmap(adev->mman.aper_base_kaddr);
> -     adev->mman.aper_base_kaddr =3D NULL;
> -
>        amdgpu_vram_mgr_fini(adev);<= br> >        amdgpu_gtt_mgr_fini(adev); >        ttm_range_man_fini(&adev= ->mman.bdev, AMDGPU_PL_GDS);
>
--_000_MN2PR12MB4488EBDF561AEED5F0C265D0F72C9MN2PR12MB4488namp_-- --===============0029053651== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx --===============0029053651==--