From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C288EC433B4 for ; Thu, 20 May 2021 03:21:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A450C61186 for ; Thu, 20 May 2021 03:21:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229978AbhETDWf (ORCPT ); Wed, 19 May 2021 23:22:35 -0400 Received: from mail-co1nam11on2080.outbound.protection.outlook.com ([40.107.220.80]:19040 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S229598AbhETDWe (ORCPT ); Wed, 19 May 2021 23:22:34 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YGs2ARB+FP8pdegzsalwbq5XDnEB+23yL425/0HxwHIIWQWt/Z8IS1FyhfO6+6E1X82bGoHOknYh1gh9Rv8+bXXtZZZeus7VTDUehE7XjDB2AylJGflVizPIGxIPennNTT/EnUcqDROfKwCQe0e57xWFPbkLt6MAhry4wDue4JF+c1eDDt3Nx7kLlR3ahhuNd3ZCZW0xKdQTUpz/GnV6f088StjU14ASVwQhEi77lVbiw1Vr6nEOovV9BrGALNbkdbcdu7i/ZWlHCDqQWItsATkjf5yxQvNrbLfNcgByWw7+A8rOL8l+reAb4x88SKHHUrsMiXNGPHHLa7Hrr0n84g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFuDU6XxOnvDBO9eG4wfkvbyFAjPnORcxBq1DCHGBuk=; b=dVjWPZqQxu1WAj/FBJ3FSSPPPiNGM7vGz39UdJYOLGpvu5ontfh1/xCpMI8Oon7TnzH9yCXmuLiowFoTpm1fnu3vYF/wlCtNURT5XJdv7us5QxHF7yuCroqZn9k6vfYLmEXLJA9tfkhdJsU4Sx1HMClH36iZzcBscRX1Nvk0PWee1dE9jxzci21mUn5G8uywCRj/j7c4m6zqEpkC7NnVG7fcYxzQNyOJfp3z7qvEZkm7t6US7kD2xIzL4tHEa/QxQroRl3ud7DxKK44tOZGoBmJ7Zz4INH2bQIo5Jar+e+sIIbESlJKf+vFvqgi9VakEOz81+18FmiTMPabxZPzorw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFuDU6XxOnvDBO9eG4wfkvbyFAjPnORcxBq1DCHGBuk=; b=wf4X/6de3V45nRMmW2V6agAgFz5UGkMhKxQncreaJzYbtOLrvXuA+7Nl67+CnUYOCpUbRLrGVsS5Xn+1jRx6hnhh5hsBOvGQGB1hkckoBTzosz+ypZx+PrNVkH0fmn0EyWKlQBn3xB6pfKoLRhmPe3tzMzjkAdyyFVWqX/iTnoM= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4541.namprd12.prod.outlook.com (2603:10b6:806:9e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.26; Thu, 20 May 2021 03:21:13 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.033; Thu, 20 May 2021 03:21:12 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com, Felix.Kuehling@amd.com Cc: ppaalanen@gmail.com, Alexander.Deucher@amd.com, gregkh@linuxfoundation.org, helgaas@kernel.org, Andrey Grodzovsky , =?UTF-8?q?Christian=20K=C3=B6nig?= Subject: [PATCH] drm/amdgpu: Add early fini callback Date: Wed, 19 May 2021 23:20:57 -0400 Message-Id: <20210520032057.497334-1-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Originating-IP: [2607:fea8:3edf:49b0:9c48:15d0:c2cd:c6e0] X-ClientProxiedBy: YT1PR01CA0043.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2e::12) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:9c48:15d0:c2cd:c6e0) by YT1PR01CA0043.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Thu, 20 May 2021 03:21:11 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aa13e93b-284b-45ab-30dc-08d91b3e51d8 X-MS-TrafficTypeDiagnostic: SA0PR12MB4541: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:551; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CXCaiveV9+oEa46rjkOIDAf8qy3dmAXme9BGaCqRiADDGd1sYH/QesXdPHxcWU8pjLxNIyV85GcUcaidrwEcZ24cr1d1CGlQ0f4PKkEkHqvslBuK1TJwQ6Fixdxnd84f0SPiYkb62T8WOWEhetjocpSx3ZJkk0TdeHeg3Ldcbx+3FcPRhOI9WS7oXdI4/UBQ0zTlhGdIBMg94vy1qc7k5SGimF4ytlQ3uDvGj2gzEvXJwos9gcJRl8r1+Ag+166zi3CBNILA/rt2p1Tun7gLPmlAvJYgCDSko3eXC1eN8R0jmuTrkdnLbNAsQvUChnWHV18iOQi1FbxFWt2jNlfJ8Z5oMp2eidNvfCeHUR9G8i4nMgbm2ozBQVd929eTsFO2DMjgVAj8qzeonLsZ7w79Cc26+K2NiL7TvNzJqvhzIodkpGOPAZDHvb3n35I3Txnz7fQmlmzmQoz/sx6tVqiajbIvJOHuX7PAOJnNWQBN5w0VmPf0N3+8tcqblBsPimSo756gXq678SO64JGHP2Yh8+SFg0o8ZbPTMM9UXZX5O+8ZAUXZPdrXEiUeyEvksjnfeaMYQairvyoeDoQQy+hvVg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB4623.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(376002)(39860400002)(346002)(366004)(136003)(396003)(2906002)(316002)(16526019)(6666004)(186003)(2616005)(86362001)(8676002)(6636002)(36756003)(54906003)(66946007)(478600001)(1076003)(4326008)(66556008)(6512007)(66476007)(66574015)(6486002)(5660300002)(83380400001)(6506007)(44832011)(52116002)(8936002)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?WmNnbGw3b0MydUU2ZlFsaVBBdFdxSXNqcnNGa3NRYlpzOEd6SUs0MFZqdWww?= =?utf-8?B?VmwvaFNBc1laN1pPOE9uTWVvam9KUWVCY3YvaTU1N0VuTXo3SzBhUDIzdkdo?= =?utf-8?B?TGcydzRCNDRTaXNVSW1MS1plRUlhQmFFazhpM0xKU3A5Smx4R0kranFpT3lF?= =?utf-8?B?ekhKVEJ3OHdodWpLekxHLzRuZUVtbGZ1ZzRtR2s2OVk5YlZrZVlmdnhCTFZj?= =?utf-8?B?d2VLVGFYYjRUUXgrSG1KTitucGNuWmpXTlRwbkNwSGdvbklPVWtiVUFtelFj?= =?utf-8?B?V2p1NjhaKzBZam0vRXdVczdQbGRoVUkxUGFUaXl6aE05amtQRXFzVkR2QnJO?= =?utf-8?B?V216aHVzNUQvcDhrTUs3ZU4wTlJFVmVNa3gyM1VBdjU2UmpZQm1QL2JIVElR?= =?utf-8?B?aURwdnF5NndFcmQ3TFRwVytkMEZYOE1keUtSd1d2NG5HVVdFZWFJckJjcWRX?= =?utf-8?B?UGxIQlJnclJQRFZTMXdZNDZkSnhIUmFyc3RDeXpubU1ON2VFQ21qeVhUbFY2?= =?utf-8?B?SSsyYTR2SkVYbjhESEJPSXV4S0Q2N3JNdmRnL015c1htOG94UGdMYit0cmNv?= =?utf-8?B?cHEzcDNybExiNkhkaE5wZE5DNno2ZjFsa0wxWnJOTm5admVMeWJRT0NYVXhn?= =?utf-8?B?TncrSTdCSHR1bGlrSHJtYUpxMW01dnlINkhENWZqWTBHdms4MllPUnViT3Zu?= =?utf-8?B?Z1dzZm9ka0FGbjhWYzVlTDR2UzF5RDJIQVJSZDJkd2N0WjF3QmhrNDNvQWp2?= =?utf-8?B?anJjOUFGWkptdlVGSG1PSG54SVB0aE9IdHB5UEt6QWpyNnN4c0ZSaUE4UTFi?= =?utf-8?B?NnlTQ3l3RVMyY2wwOFdUSGh1KzF0TEZoUEprQStXTmdOVklVcm1WOXZHMGx5?= =?utf-8?B?SVo5VSt6dzd6Z3FBU29ZNUNab24xWEUvYlA0a1NTUExGUkozanJaaDNNNnVR?= =?utf-8?B?Y2FIS0Nhd0tOejhTNHRXeTdSaEo1aE0rcW5CMGdzcVd3SzZtZ09SeC9Eam9K?= =?utf-8?B?WndBNmxUdWNMSmZwZFR6NEhLUUYrMWlaVlo4WTN4Tmlib0NSNHBPMFp4eVht?= =?utf-8?B?WnlJV0J5L0N6ZDR2WXdpY3Y2dTMyTkl0NW1MYXpqQ1BRSFBvT1FqWm42WUt5?= =?utf-8?B?UjdqVElBWkRVZTNWRmU1UTZZbm9tYVZkUnlxbWVzYy93emd4S1J3U2lrWjlz?= =?utf-8?B?cU5ObkMvVXJSWlJlSjl1WE42UkQ0NWppUnpidmRmeDd2ekpIcjdkb1VsNHg4?= =?utf-8?B?K0phekxNd2pRYmtmNkIwUzZjT1lmVkJUZHNhczQvVlE2OVEzTjZpN2o4QjhU?= =?utf-8?B?MlRmWkdEWjFNTEVkaldOcGtZS1M1dVVSc0RZTFdNSDZ5ZnRSUVI3WC9ORjZ2?= =?utf-8?B?elU1MmNWTkgva0JBT2taZ3IzakVSS2FGT2dhQXJCWUo0L1JOZ1ZOK2VXd1c2?= =?utf-8?B?SVA5dE5lVFluMjZHUmlsUU1xR25CWTJ2TmNoZFJ5aWd5NjlRamdDUnZHemFF?= =?utf-8?B?RXgzL3hrL0ZCQzlMOVpSVDZ4Wlpid1JZOUxyUGM4WkJMNHpaNVBLRFBEZUE1?= =?utf-8?B?TUp3aDBQcGV4ZzNSdElHRndEdDR5bnV4OUNZVlV6R001MDJRNXJYcDhkRUY3?= =?utf-8?B?QnYwRzNTTVJIL3I4TGJqeENJZk5kNkg4WnNMcmlTZEkzMk56TEUwV0FpTUl1?= =?utf-8?B?VW1lNk1rUDBoeGxpVDd5SHlWYk90d0lHbE51UEFjcE9ubXo3RnVVUHdCNzVC?= =?utf-8?B?cnVyZmFrZDNXblNoWk0rOXVyMEJuNzZqRDdudko3SlZKYjkrY3J4WEVpSjRR?= =?utf-8?B?Q2pyVGpJNU9aTGJVcW1BTUw0STFaamh3dlhxR29WZUdWUEJHVDFsTmN1Y25U?= =?utf-8?Q?6OpIMzrPwdE8C?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa13e93b-284b-45ab-30dc-08d91b3e51d8 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2021 03:21:12.6879 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GMQkp8d3l5FASHD0+OJs2VPC2gZP6g8GfCTFZL8EuXpQXRi0mhCDcxFXxpeM28Ueh4MbeeQLlAbXPwsEWiZ1rA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4541 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Use it to call disply code dependent on device->drv_data before it's set to NULL on device unplug v5: Move HW finilization into this callback to prevent MMIO accesses post cpi remove. v7: Split kfd suspend from device exit to expdite HW related stuff to amdgpu_pci_remove v8: Squash previous KFD commit into this commit to avoid compile break. Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 59 +++++++++++++------ drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 +- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 +++- drivers/gpu/drm/amd/include/amd_shared.h | 2 + 6 files changed, 56 insertions(+), 24 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 5f6696a3c778..2b06dee9a0ce 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -170,7 +170,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) } } -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev) +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev) { if (adev->kfd.dev) { kgd2kfd_device_exit(adev->kfd.dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 5ffb07b02810..d8a537e8aea5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -127,7 +127,7 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev, const void *ih_ring_entry); void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev); +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, uint32_t vmid, uint64_t gpu_addr, uint32_t *ib_cmd, uint32_t ib_len); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 8bee95ad32d9..bc75e35dd8d8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2558,34 +2558,26 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) return 0; } -/** - * amdgpu_device_ip_fini - run fini for hardware IPs - * - * @adev: amdgpu_device pointer - * - * Main teardown pass for hardware IPs. The list of all the hardware - * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks - * are run. hw_fini tears down the hardware associated with each IP - * and sw_fini tears down any software state associated with each IP. - * Returns 0 on success, negative error code on failure. - */ -static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) { int i, r; - if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) - amdgpu_virt_release_ras_err_handler_data(adev); + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].version->funcs->early_fini) + continue; - amdgpu_ras_pre_fini(adev); + r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev); + if (r) { + DRM_DEBUG("early_fini of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + } + } - if (adev->gmc.xgmi.num_physical_nodes > 1) - amdgpu_xgmi_remove_device(adev); + amdgpu_amdkfd_suspend(adev, false); amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); - amdgpu_amdkfd_device_fini(adev); - /* need to disable SMC first */ for (i = 0; i < adev->num_ip_blocks; i++) { if (!adev->ip_blocks[i].status.hw) @@ -2616,6 +2608,33 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) adev->ip_blocks[i].status.hw = false; } + return 0; +} + +/** + * amdgpu_device_ip_fini - run fini for hardware IPs + * + * @adev: amdgpu_device pointer + * + * Main teardown pass for hardware IPs. The list of all the hardware + * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks + * are run. hw_fini tears down the hardware associated with each IP + * and sw_fini tears down any software state associated with each IP. + * Returns 0 on success, negative error code on failure. + */ +static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +{ + int i, r; + + if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) + amdgpu_virt_release_ras_err_handler_data(adev); + + amdgpu_ras_pre_fini(adev); + + if (adev->gmc.xgmi.num_physical_nodes > 1) + amdgpu_xgmi_remove_device(adev); + + amdgpu_amdkfd_device_fini_sw(adev); for (i = adev->num_ip_blocks - 1; i >= 0; i--) { if (!adev->ip_blocks[i].status.sw) @@ -3681,6 +3700,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); + + amdgpu_device_ip_fini_early(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 357b9bf62a1c..ab6d2a43c9a3 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -858,10 +858,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, return kfd->init_complete; } + + void kgd2kfd_device_exit(struct kfd_dev *kfd) { if (kfd->init_complete) { - kgd2kfd_suspend(kfd, false); device_queue_manager_uninit(kfd->dqm); kfd_interrupt_exit(kfd); kfd_topology_remove_device(kfd); diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 9ca517b65854..f7112865269a 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1251,6 +1251,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) return -EINVAL; } +static int amdgpu_dm_early_fini(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + amdgpu_dm_audio_fini(adev); + + return 0; +} + static void amdgpu_dm_fini(struct amdgpu_device *adev) { int i; @@ -1259,8 +1268,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev) drm_encoder_cleanup(&adev->dm.mst_encoders[i].base); } - amdgpu_dm_audio_fini(adev); - amdgpu_dm_destroy_drm_device(&adev->dm); #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) @@ -2298,6 +2305,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = { .late_init = dm_late_init, .sw_init = dm_sw_init, .sw_fini = dm_sw_fini, + .early_fini = amdgpu_dm_early_fini, .hw_init = dm_hw_init, .hw_fini = dm_hw_fini, .suspend = dm_suspend, diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 43ed6291b2b8..1ad56da486e4 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -240,6 +240,7 @@ enum amd_dpm_forced_level; * @late_init: sets up late driver/hw state (post hw_init) - Optional * @sw_init: sets up driver state, does not configure hw * @sw_fini: tears down driver state, does not configure hw + * @early_fini: tears down stuff before dev detached from driver * @hw_init: sets up the hw state * @hw_fini: tears down the hw state * @late_fini: final cleanup @@ -268,6 +269,7 @@ struct amd_ip_funcs { int (*late_init)(void *handle); int (*sw_init)(void *handle); int (*sw_fini)(void *handle); + int (*early_fini)(void *handle); int (*hw_init)(void *handle); int (*hw_fini)(void *handle); void (*late_fini)(void *handle); -- 2.25.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFEE3C433ED for ; Thu, 20 May 2021 03:21:19 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 78A1561184 for ; Thu, 20 May 2021 03:21:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78A1561184 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5EC966E323; Thu, 20 May 2021 03:21:16 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2072.outbound.protection.outlook.com [40.107.237.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id CCCB26E323; Thu, 20 May 2021 03:21:14 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YGs2ARB+FP8pdegzsalwbq5XDnEB+23yL425/0HxwHIIWQWt/Z8IS1FyhfO6+6E1X82bGoHOknYh1gh9Rv8+bXXtZZZeus7VTDUehE7XjDB2AylJGflVizPIGxIPennNTT/EnUcqDROfKwCQe0e57xWFPbkLt6MAhry4wDue4JF+c1eDDt3Nx7kLlR3ahhuNd3ZCZW0xKdQTUpz/GnV6f088StjU14ASVwQhEi77lVbiw1Vr6nEOovV9BrGALNbkdbcdu7i/ZWlHCDqQWItsATkjf5yxQvNrbLfNcgByWw7+A8rOL8l+reAb4x88SKHHUrsMiXNGPHHLa7Hrr0n84g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFuDU6XxOnvDBO9eG4wfkvbyFAjPnORcxBq1DCHGBuk=; b=dVjWPZqQxu1WAj/FBJ3FSSPPPiNGM7vGz39UdJYOLGpvu5ontfh1/xCpMI8Oon7TnzH9yCXmuLiowFoTpm1fnu3vYF/wlCtNURT5XJdv7us5QxHF7yuCroqZn9k6vfYLmEXLJA9tfkhdJsU4Sx1HMClH36iZzcBscRX1Nvk0PWee1dE9jxzci21mUn5G8uywCRj/j7c4m6zqEpkC7NnVG7fcYxzQNyOJfp3z7qvEZkm7t6US7kD2xIzL4tHEa/QxQroRl3ud7DxKK44tOZGoBmJ7Zz4INH2bQIo5Jar+e+sIIbESlJKf+vFvqgi9VakEOz81+18FmiTMPabxZPzorw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFuDU6XxOnvDBO9eG4wfkvbyFAjPnORcxBq1DCHGBuk=; b=wf4X/6de3V45nRMmW2V6agAgFz5UGkMhKxQncreaJzYbtOLrvXuA+7Nl67+CnUYOCpUbRLrGVsS5Xn+1jRx6hnhh5hsBOvGQGB1hkckoBTzosz+ypZx+PrNVkH0fmn0EyWKlQBn3xB6pfKoLRhmPe3tzMzjkAdyyFVWqX/iTnoM= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4541.namprd12.prod.outlook.com (2603:10b6:806:9e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.26; Thu, 20 May 2021 03:21:13 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.033; Thu, 20 May 2021 03:21:12 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com, Felix.Kuehling@amd.com Subject: [PATCH] drm/amdgpu: Add early fini callback Date: Wed, 19 May 2021 23:20:57 -0400 Message-Id: <20210520032057.497334-1-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Originating-IP: [2607:fea8:3edf:49b0:9c48:15d0:c2cd:c6e0] X-ClientProxiedBy: YT1PR01CA0043.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2e::12) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:9c48:15d0:c2cd:c6e0) by YT1PR01CA0043.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Thu, 20 May 2021 03:21:11 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aa13e93b-284b-45ab-30dc-08d91b3e51d8 X-MS-TrafficTypeDiagnostic: SA0PR12MB4541: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:551; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CXCaiveV9+oEa46rjkOIDAf8qy3dmAXme9BGaCqRiADDGd1sYH/QesXdPHxcWU8pjLxNIyV85GcUcaidrwEcZ24cr1d1CGlQ0f4PKkEkHqvslBuK1TJwQ6Fixdxnd84f0SPiYkb62T8WOWEhetjocpSx3ZJkk0TdeHeg3Ldcbx+3FcPRhOI9WS7oXdI4/UBQ0zTlhGdIBMg94vy1qc7k5SGimF4ytlQ3uDvGj2gzEvXJwos9gcJRl8r1+Ag+166zi3CBNILA/rt2p1Tun7gLPmlAvJYgCDSko3eXC1eN8R0jmuTrkdnLbNAsQvUChnWHV18iOQi1FbxFWt2jNlfJ8Z5oMp2eidNvfCeHUR9G8i4nMgbm2ozBQVd929eTsFO2DMjgVAj8qzeonLsZ7w79Cc26+K2NiL7TvNzJqvhzIodkpGOPAZDHvb3n35I3Txnz7fQmlmzmQoz/sx6tVqiajbIvJOHuX7PAOJnNWQBN5w0VmPf0N3+8tcqblBsPimSo756gXq678SO64JGHP2Yh8+SFg0o8ZbPTMM9UXZX5O+8ZAUXZPdrXEiUeyEvksjnfeaMYQairvyoeDoQQy+hvVg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(346002)(366004)(136003)(396003)(2906002)(316002)(16526019)(6666004)(186003)(2616005)(86362001)(8676002)(6636002)(36756003)(54906003)(66946007)(478600001)(1076003)(4326008)(66556008)(6512007)(66476007)(66574015)(6486002)(5660300002)(83380400001)(6506007)(44832011)(52116002)(8936002)(38100700002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?WmNnbGw3b0MydUU2ZlFsaVBBdFdxSXNqcnNGa3NRYlpzOEd6SUs0MFZqdWww?= =?utf-8?B?VmwvaFNBc1laN1pPOE9uTWVvam9KUWVCY3YvaTU1N0VuTXo3SzBhUDIzdkdo?= =?utf-8?B?TGcydzRCNDRTaXNVSW1MS1plRUlhQmFFazhpM0xKU3A5Smx4R0kranFpT3lF?= =?utf-8?B?ekhKVEJ3OHdodWpLekxHLzRuZUVtbGZ1ZzRtR2s2OVk5YlZrZVlmdnhCTFZj?= =?utf-8?B?d2VLVGFYYjRUUXgrSG1KTitucGNuWmpXTlRwbkNwSGdvbklPVWtiVUFtelFj?= =?utf-8?B?V2p1NjhaKzBZam0vRXdVczdQbGRoVUkxUGFUaXl6aE05amtQRXFzVkR2QnJO?= =?utf-8?B?V216aHVzNUQvcDhrTUs3ZU4wTlJFVmVNa3gyM1VBdjU2UmpZQm1QL2JIVElR?= =?utf-8?B?aURwdnF5NndFcmQ3TFRwVytkMEZYOE1keUtSd1d2NG5HVVdFZWFJckJjcWRX?= =?utf-8?B?UGxIQlJnclJQRFZTMXdZNDZkSnhIUmFyc3RDeXpubU1ON2VFQ21qeVhUbFY2?= =?utf-8?B?SSsyYTR2SkVYbjhESEJPSXV4S0Q2N3JNdmRnL015c1htOG94UGdMYit0cmNv?= =?utf-8?B?cHEzcDNybExiNkhkaE5wZE5DNno2ZjFsa0wxWnJOTm5admVMeWJRT0NYVXhn?= =?utf-8?B?TncrSTdCSHR1bGlrSHJtYUpxMW01dnlINkhENWZqWTBHdms4MllPUnViT3Zu?= =?utf-8?B?Z1dzZm9ka0FGbjhWYzVlTDR2UzF5RDJIQVJSZDJkd2N0WjF3QmhrNDNvQWp2?= =?utf-8?B?anJjOUFGWkptdlVGSG1PSG54SVB0aE9IdHB5UEt6QWpyNnN4c0ZSaUE4UTFi?= =?utf-8?B?NnlTQ3l3RVMyY2wwOFdUSGh1KzF0TEZoUEprQStXTmdOVklVcm1WOXZHMGx5?= =?utf-8?B?SVo5VSt6dzd6Z3FBU29ZNUNab24xWEUvYlA0a1NTUExGUkozanJaaDNNNnVR?= =?utf-8?B?Y2FIS0Nhd0tOejhTNHRXeTdSaEo1aE0rcW5CMGdzcVd3SzZtZ09SeC9Eam9K?= =?utf-8?B?WndBNmxUdWNMSmZwZFR6NEhLUUYrMWlaVlo4WTN4Tmlib0NSNHBPMFp4eVht?= =?utf-8?B?WnlJV0J5L0N6ZDR2WXdpY3Y2dTMyTkl0NW1MYXpqQ1BRSFBvT1FqWm42WUt5?= =?utf-8?B?UjdqVElBWkRVZTNWRmU1UTZZbm9tYVZkUnlxbWVzYy93emd4S1J3U2lrWjlz?= =?utf-8?B?cU5ObkMvVXJSWlJlSjl1WE42UkQ0NWppUnpidmRmeDd2ekpIcjdkb1VsNHg4?= =?utf-8?B?K0phekxNd2pRYmtmNkIwUzZjT1lmVkJUZHNhczQvVlE2OVEzTjZpN2o4QjhU?= =?utf-8?B?MlRmWkdEWjFNTEVkaldOcGtZS1M1dVVSc0RZTFdNSDZ5ZnRSUVI3WC9ORjZ2?= =?utf-8?B?elU1MmNWTkgva0JBT2taZ3IzakVSS2FGT2dhQXJCWUo0L1JOZ1ZOK2VXd1c2?= =?utf-8?B?SVA5dE5lVFluMjZHUmlsUU1xR25CWTJ2TmNoZFJ5aWd5NjlRamdDUnZHemFF?= =?utf-8?B?RXgzL3hrL0ZCQzlMOVpSVDZ4Wlpid1JZOUxyUGM4WkJMNHpaNVBLRFBEZUE1?= =?utf-8?B?TUp3aDBQcGV4ZzNSdElHRndEdDR5bnV4OUNZVlV6R001MDJRNXJYcDhkRUY3?= =?utf-8?B?QnYwRzNTTVJIL3I4TGJqeENJZk5kNkg4WnNMcmlTZEkzMk56TEUwV0FpTUl1?= =?utf-8?B?VW1lNk1rUDBoeGxpVDd5SHlWYk90d0lHbE51UEFjcE9ubXo3RnVVUHdCNzVC?= =?utf-8?B?cnVyZmFrZDNXblNoWk0rOXVyMEJuNzZqRDdudko3SlZKYjkrY3J4WEVpSjRR?= =?utf-8?B?Q2pyVGpJNU9aTGJVcW1BTUw0STFaamh3dlhxR29WZUdWUEJHVDFsTmN1Y25U?= =?utf-8?Q?6OpIMzrPwdE8C?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa13e93b-284b-45ab-30dc-08d91b3e51d8 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2021 03:21:12.6879 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GMQkp8d3l5FASHD0+OJs2VPC2gZP6g8GfCTFZL8EuXpQXRi0mhCDcxFXxpeM28Ueh4MbeeQLlAbXPwsEWiZ1rA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4541 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, helgaas@kernel.org, Alexander.Deucher@amd.com, =?UTF-8?q?Christian=20K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use it to call disply code dependent on device->drv_data before it's set to NULL on device unplug v5: Move HW finilization into this callback to prevent MMIO accesses post cpi remove. v7: Split kfd suspend from device exit to expdite HW related stuff to amdgpu_pci_remove v8: Squash previous KFD commit into this commit to avoid compile break. Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 59 +++++++++++++------ drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 +- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 +++- drivers/gpu/drm/amd/include/amd_shared.h | 2 + 6 files changed, 56 insertions(+), 24 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 5f6696a3c778..2b06dee9a0ce 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -170,7 +170,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) } } -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev) +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev) { if (adev->kfd.dev) { kgd2kfd_device_exit(adev->kfd.dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 5ffb07b02810..d8a537e8aea5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -127,7 +127,7 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev, const void *ih_ring_entry); void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev); +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, uint32_t vmid, uint64_t gpu_addr, uint32_t *ib_cmd, uint32_t ib_len); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 8bee95ad32d9..bc75e35dd8d8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2558,34 +2558,26 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) return 0; } -/** - * amdgpu_device_ip_fini - run fini for hardware IPs - * - * @adev: amdgpu_device pointer - * - * Main teardown pass for hardware IPs. The list of all the hardware - * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks - * are run. hw_fini tears down the hardware associated with each IP - * and sw_fini tears down any software state associated with each IP. - * Returns 0 on success, negative error code on failure. - */ -static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) { int i, r; - if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) - amdgpu_virt_release_ras_err_handler_data(adev); + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].version->funcs->early_fini) + continue; - amdgpu_ras_pre_fini(adev); + r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev); + if (r) { + DRM_DEBUG("early_fini of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + } + } - if (adev->gmc.xgmi.num_physical_nodes > 1) - amdgpu_xgmi_remove_device(adev); + amdgpu_amdkfd_suspend(adev, false); amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); - amdgpu_amdkfd_device_fini(adev); - /* need to disable SMC first */ for (i = 0; i < adev->num_ip_blocks; i++) { if (!adev->ip_blocks[i].status.hw) @@ -2616,6 +2608,33 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) adev->ip_blocks[i].status.hw = false; } + return 0; +} + +/** + * amdgpu_device_ip_fini - run fini for hardware IPs + * + * @adev: amdgpu_device pointer + * + * Main teardown pass for hardware IPs. The list of all the hardware + * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks + * are run. hw_fini tears down the hardware associated with each IP + * and sw_fini tears down any software state associated with each IP. + * Returns 0 on success, negative error code on failure. + */ +static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +{ + int i, r; + + if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) + amdgpu_virt_release_ras_err_handler_data(adev); + + amdgpu_ras_pre_fini(adev); + + if (adev->gmc.xgmi.num_physical_nodes > 1) + amdgpu_xgmi_remove_device(adev); + + amdgpu_amdkfd_device_fini_sw(adev); for (i = adev->num_ip_blocks - 1; i >= 0; i--) { if (!adev->ip_blocks[i].status.sw) @@ -3681,6 +3700,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); + + amdgpu_device_ip_fini_early(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 357b9bf62a1c..ab6d2a43c9a3 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -858,10 +858,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, return kfd->init_complete; } + + void kgd2kfd_device_exit(struct kfd_dev *kfd) { if (kfd->init_complete) { - kgd2kfd_suspend(kfd, false); device_queue_manager_uninit(kfd->dqm); kfd_interrupt_exit(kfd); kfd_topology_remove_device(kfd); diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 9ca517b65854..f7112865269a 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1251,6 +1251,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) return -EINVAL; } +static int amdgpu_dm_early_fini(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + amdgpu_dm_audio_fini(adev); + + return 0; +} + static void amdgpu_dm_fini(struct amdgpu_device *adev) { int i; @@ -1259,8 +1268,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev) drm_encoder_cleanup(&adev->dm.mst_encoders[i].base); } - amdgpu_dm_audio_fini(adev); - amdgpu_dm_destroy_drm_device(&adev->dm); #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) @@ -2298,6 +2305,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = { .late_init = dm_late_init, .sw_init = dm_sw_init, .sw_fini = dm_sw_fini, + .early_fini = amdgpu_dm_early_fini, .hw_init = dm_hw_init, .hw_fini = dm_hw_fini, .suspend = dm_suspend, diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 43ed6291b2b8..1ad56da486e4 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -240,6 +240,7 @@ enum amd_dpm_forced_level; * @late_init: sets up late driver/hw state (post hw_init) - Optional * @sw_init: sets up driver state, does not configure hw * @sw_fini: tears down driver state, does not configure hw + * @early_fini: tears down stuff before dev detached from driver * @hw_init: sets up the hw state * @hw_fini: tears down the hw state * @late_fini: final cleanup @@ -268,6 +269,7 @@ struct amd_ip_funcs { int (*late_init)(void *handle); int (*sw_init)(void *handle); int (*sw_fini)(void *handle); + int (*early_fini)(void *handle); int (*hw_init)(void *handle); int (*hw_fini)(void *handle); void (*late_fini)(void *handle); -- 2.25.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C902C43460 for ; Thu, 20 May 2021 03:21:17 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 60C8661163 for ; Thu, 20 May 2021 03:21:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60C8661163 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 74E736E825; Thu, 20 May 2021 03:21:16 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2072.outbound.protection.outlook.com [40.107.237.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id CCCB26E323; Thu, 20 May 2021 03:21:14 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YGs2ARB+FP8pdegzsalwbq5XDnEB+23yL425/0HxwHIIWQWt/Z8IS1FyhfO6+6E1X82bGoHOknYh1gh9Rv8+bXXtZZZeus7VTDUehE7XjDB2AylJGflVizPIGxIPennNTT/EnUcqDROfKwCQe0e57xWFPbkLt6MAhry4wDue4JF+c1eDDt3Nx7kLlR3ahhuNd3ZCZW0xKdQTUpz/GnV6f088StjU14ASVwQhEi77lVbiw1Vr6nEOovV9BrGALNbkdbcdu7i/ZWlHCDqQWItsATkjf5yxQvNrbLfNcgByWw7+A8rOL8l+reAb4x88SKHHUrsMiXNGPHHLa7Hrr0n84g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFuDU6XxOnvDBO9eG4wfkvbyFAjPnORcxBq1DCHGBuk=; b=dVjWPZqQxu1WAj/FBJ3FSSPPPiNGM7vGz39UdJYOLGpvu5ontfh1/xCpMI8Oon7TnzH9yCXmuLiowFoTpm1fnu3vYF/wlCtNURT5XJdv7us5QxHF7yuCroqZn9k6vfYLmEXLJA9tfkhdJsU4Sx1HMClH36iZzcBscRX1Nvk0PWee1dE9jxzci21mUn5G8uywCRj/j7c4m6zqEpkC7NnVG7fcYxzQNyOJfp3z7qvEZkm7t6US7kD2xIzL4tHEa/QxQroRl3ud7DxKK44tOZGoBmJ7Zz4INH2bQIo5Jar+e+sIIbESlJKf+vFvqgi9VakEOz81+18FmiTMPabxZPzorw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JFuDU6XxOnvDBO9eG4wfkvbyFAjPnORcxBq1DCHGBuk=; b=wf4X/6de3V45nRMmW2V6agAgFz5UGkMhKxQncreaJzYbtOLrvXuA+7Nl67+CnUYOCpUbRLrGVsS5Xn+1jRx6hnhh5hsBOvGQGB1hkckoBTzosz+ypZx+PrNVkH0fmn0EyWKlQBn3xB6pfKoLRhmPe3tzMzjkAdyyFVWqX/iTnoM= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4541.namprd12.prod.outlook.com (2603:10b6:806:9e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.26; Thu, 20 May 2021 03:21:13 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.033; Thu, 20 May 2021 03:21:12 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com, Felix.Kuehling@amd.com Subject: [PATCH] drm/amdgpu: Add early fini callback Date: Wed, 19 May 2021 23:20:57 -0400 Message-Id: <20210520032057.497334-1-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: X-Originating-IP: [2607:fea8:3edf:49b0:9c48:15d0:c2cd:c6e0] X-ClientProxiedBy: YT1PR01CA0043.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2e::12) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:9c48:15d0:c2cd:c6e0) by YT1PR01CA0043.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Thu, 20 May 2021 03:21:11 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aa13e93b-284b-45ab-30dc-08d91b3e51d8 X-MS-TrafficTypeDiagnostic: SA0PR12MB4541: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:551; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CXCaiveV9+oEa46rjkOIDAf8qy3dmAXme9BGaCqRiADDGd1sYH/QesXdPHxcWU8pjLxNIyV85GcUcaidrwEcZ24cr1d1CGlQ0f4PKkEkHqvslBuK1TJwQ6Fixdxnd84f0SPiYkb62T8WOWEhetjocpSx3ZJkk0TdeHeg3Ldcbx+3FcPRhOI9WS7oXdI4/UBQ0zTlhGdIBMg94vy1qc7k5SGimF4ytlQ3uDvGj2gzEvXJwos9gcJRl8r1+Ag+166zi3CBNILA/rt2p1Tun7gLPmlAvJYgCDSko3eXC1eN8R0jmuTrkdnLbNAsQvUChnWHV18iOQi1FbxFWt2jNlfJ8Z5oMp2eidNvfCeHUR9G8i4nMgbm2ozBQVd929eTsFO2DMjgVAj8qzeonLsZ7w79Cc26+K2NiL7TvNzJqvhzIodkpGOPAZDHvb3n35I3Txnz7fQmlmzmQoz/sx6tVqiajbIvJOHuX7PAOJnNWQBN5w0VmPf0N3+8tcqblBsPimSo756gXq678SO64JGHP2Yh8+SFg0o8ZbPTMM9UXZX5O+8ZAUXZPdrXEiUeyEvksjnfeaMYQairvyoeDoQQy+hvVg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(346002)(366004)(136003)(396003)(2906002)(316002)(16526019)(6666004)(186003)(2616005)(86362001)(8676002)(6636002)(36756003)(54906003)(66946007)(478600001)(1076003)(4326008)(66556008)(6512007)(66476007)(66574015)(6486002)(5660300002)(83380400001)(6506007)(44832011)(52116002)(8936002)(38100700002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?WmNnbGw3b0MydUU2ZlFsaVBBdFdxSXNqcnNGa3NRYlpzOEd6SUs0MFZqdWww?= =?utf-8?B?VmwvaFNBc1laN1pPOE9uTWVvam9KUWVCY3YvaTU1N0VuTXo3SzBhUDIzdkdo?= =?utf-8?B?TGcydzRCNDRTaXNVSW1MS1plRUlhQmFFazhpM0xKU3A5Smx4R0kranFpT3lF?= =?utf-8?B?ekhKVEJ3OHdodWpLekxHLzRuZUVtbGZ1ZzRtR2s2OVk5YlZrZVlmdnhCTFZj?= =?utf-8?B?d2VLVGFYYjRUUXgrSG1KTitucGNuWmpXTlRwbkNwSGdvbklPVWtiVUFtelFj?= =?utf-8?B?V2p1NjhaKzBZam0vRXdVczdQbGRoVUkxUGFUaXl6aE05amtQRXFzVkR2QnJO?= =?utf-8?B?V216aHVzNUQvcDhrTUs3ZU4wTlJFVmVNa3gyM1VBdjU2UmpZQm1QL2JIVElR?= =?utf-8?B?aURwdnF5NndFcmQ3TFRwVytkMEZYOE1keUtSd1d2NG5HVVdFZWFJckJjcWRX?= =?utf-8?B?UGxIQlJnclJQRFZTMXdZNDZkSnhIUmFyc3RDeXpubU1ON2VFQ21qeVhUbFY2?= =?utf-8?B?SSsyYTR2SkVYbjhESEJPSXV4S0Q2N3JNdmRnL015c1htOG94UGdMYit0cmNv?= =?utf-8?B?cHEzcDNybExiNkhkaE5wZE5DNno2ZjFsa0wxWnJOTm5admVMeWJRT0NYVXhn?= =?utf-8?B?TncrSTdCSHR1bGlrSHJtYUpxMW01dnlINkhENWZqWTBHdms4MllPUnViT3Zu?= =?utf-8?B?Z1dzZm9ka0FGbjhWYzVlTDR2UzF5RDJIQVJSZDJkd2N0WjF3QmhrNDNvQWp2?= =?utf-8?B?anJjOUFGWkptdlVGSG1PSG54SVB0aE9IdHB5UEt6QWpyNnN4c0ZSaUE4UTFi?= =?utf-8?B?NnlTQ3l3RVMyY2wwOFdUSGh1KzF0TEZoUEprQStXTmdOVklVcm1WOXZHMGx5?= =?utf-8?B?SVo5VSt6dzd6Z3FBU29ZNUNab24xWEUvYlA0a1NTUExGUkozanJaaDNNNnVR?= =?utf-8?B?Y2FIS0Nhd0tOejhTNHRXeTdSaEo1aE0rcW5CMGdzcVd3SzZtZ09SeC9Eam9K?= =?utf-8?B?WndBNmxUdWNMSmZwZFR6NEhLUUYrMWlaVlo4WTN4Tmlib0NSNHBPMFp4eVht?= =?utf-8?B?WnlJV0J5L0N6ZDR2WXdpY3Y2dTMyTkl0NW1MYXpqQ1BRSFBvT1FqWm42WUt5?= =?utf-8?B?UjdqVElBWkRVZTNWRmU1UTZZbm9tYVZkUnlxbWVzYy93emd4S1J3U2lrWjlz?= =?utf-8?B?cU5ObkMvVXJSWlJlSjl1WE42UkQ0NWppUnpidmRmeDd2ekpIcjdkb1VsNHg4?= =?utf-8?B?K0phekxNd2pRYmtmNkIwUzZjT1lmVkJUZHNhczQvVlE2OVEzTjZpN2o4QjhU?= =?utf-8?B?MlRmWkdEWjFNTEVkaldOcGtZS1M1dVVSc0RZTFdNSDZ5ZnRSUVI3WC9ORjZ2?= =?utf-8?B?elU1MmNWTkgva0JBT2taZ3IzakVSS2FGT2dhQXJCWUo0L1JOZ1ZOK2VXd1c2?= =?utf-8?B?SVA5dE5lVFluMjZHUmlsUU1xR25CWTJ2TmNoZFJ5aWd5NjlRamdDUnZHemFF?= =?utf-8?B?RXgzL3hrL0ZCQzlMOVpSVDZ4Wlpid1JZOUxyUGM4WkJMNHpaNVBLRFBEZUE1?= =?utf-8?B?TUp3aDBQcGV4ZzNSdElHRndEdDR5bnV4OUNZVlV6R001MDJRNXJYcDhkRUY3?= =?utf-8?B?QnYwRzNTTVJIL3I4TGJqeENJZk5kNkg4WnNMcmlTZEkzMk56TEUwV0FpTUl1?= =?utf-8?B?VW1lNk1rUDBoeGxpVDd5SHlWYk90d0lHbE51UEFjcE9ubXo3RnVVUHdCNzVC?= =?utf-8?B?cnVyZmFrZDNXblNoWk0rOXVyMEJuNzZqRDdudko3SlZKYjkrY3J4WEVpSjRR?= =?utf-8?B?Q2pyVGpJNU9aTGJVcW1BTUw0STFaamh3dlhxR29WZUdWUEJHVDFsTmN1Y25U?= =?utf-8?Q?6OpIMzrPwdE8C?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa13e93b-284b-45ab-30dc-08d91b3e51d8 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2021 03:21:12.6879 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GMQkp8d3l5FASHD0+OJs2VPC2gZP6g8GfCTFZL8EuXpQXRi0mhCDcxFXxpeM28Ueh4MbeeQLlAbXPwsEWiZ1rA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4541 X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrey Grodzovsky , gregkh@linuxfoundation.org, ppaalanen@gmail.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?UTF-8?q?Christian=20K=C3=B6nig?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" VXNlIGl0IHRvIGNhbGwgZGlzcGx5IGNvZGUgZGVwZW5kZW50IG9uIGRldmljZS0+ZHJ2X2RhdGEK YmVmb3JlIGl0J3Mgc2V0IHRvIE5VTEwgb24gZGV2aWNlIHVucGx1ZwoKdjU6Ck1vdmUgSFcgZmlu aWxpemF0aW9uIGludG8gdGhpcyBjYWxsYmFjayB0byBwcmV2ZW50IE1NSU8gYWNjZXNzZXMKcG9z dCBjcGkgcmVtb3ZlLgoKdjc6ClNwbGl0IGtmZCBzdXNwZW5kIGZyb20gZGV2aWNlIGV4aXQgdG8g ZXhwZGl0ZSBIVyByZWxhdGVkCnN0dWZmIHRvIGFtZGdwdV9wY2lfcmVtb3ZlCgp2ODoKU3F1YXNo IHByZXZpb3VzIEtGRCBjb21taXQgaW50byB0aGlzIGNvbW1pdCB0byBhdm9pZCBjb21waWxlIGJy ZWFrLgoKU2lnbmVkLW9mZi1ieTogQW5kcmV5IEdyb2R6b3Zza3kgPGFuZHJleS5ncm9kem92c2t5 QGFtZC5jb20+CkFja2VkLWJ5OiBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFt ZC5jb20+Ci0tLQogZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2FtZGtmZC5jICAg IHwgIDIgKy0KIGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9hbWRrZmQuaCAgICB8 ICAyICstCiBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMgICAgfCA1 OSArKysrKysrKysrKysrLS0tLS0tCiBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGtmZC9rZmRfZGV2 aWNlLmMgICAgICAgfCAgMyArLQogLi4uL2dwdS9kcm0vYW1kL2Rpc3BsYXkvYW1kZ3B1X2RtL2Ft ZGdwdV9kbS5jIHwgMTIgKysrLQogZHJpdmVycy9ncHUvZHJtL2FtZC9pbmNsdWRlL2FtZF9zaGFy ZWQuaCAgICAgIHwgIDIgKwogNiBmaWxlcyBjaGFuZ2VkLCA1NiBpbnNlcnRpb25zKCspLCAyNCBk ZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRn cHVfYW1ka2ZkLmMgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfYW1ka2ZkLmMK aW5kZXggNWY2Njk2YTNjNzc4Li4yYjA2ZGVlOWEwY2UgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1 L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9hbWRrZmQuYworKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1k L2FtZGdwdS9hbWRncHVfYW1ka2ZkLmMKQEAgLTE3MCw3ICsxNzAsNyBAQCB2b2lkIGFtZGdwdV9h bWRrZmRfZGV2aWNlX2luaXQoc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYpCiAJfQogfQogCi12 b2lkIGFtZGdwdV9hbWRrZmRfZGV2aWNlX2Zpbmkoc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYp Cit2b2lkIGFtZGdwdV9hbWRrZmRfZGV2aWNlX2Zpbmlfc3coc3RydWN0IGFtZGdwdV9kZXZpY2Ug KmFkZXYpCiB7CiAJaWYgKGFkZXYtPmtmZC5kZXYpIHsKIAkJa2dkMmtmZF9kZXZpY2VfZXhpdChh ZGV2LT5rZmQuZGV2KTsKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2Ft ZGdwdV9hbWRrZmQuaCBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9hbWRrZmQu aAppbmRleCA1ZmZiMDdiMDI4MTAuLmQ4YTUzN2U4YWVhNSAxMDA2NDQKLS0tIGEvZHJpdmVycy9n cHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2FtZGtmZC5oCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9h bWQvYW1kZ3B1L2FtZGdwdV9hbWRrZmQuaApAQCAtMTI3LDcgKzEyNyw3IEBAIHZvaWQgYW1kZ3B1 X2FtZGtmZF9pbnRlcnJ1cHQoc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsCiAJCQljb25zdCB2 b2lkICppaF9yaW5nX2VudHJ5KTsKIHZvaWQgYW1kZ3B1X2FtZGtmZF9kZXZpY2VfcHJvYmUoc3Ry dWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYpOwogdm9pZCBhbWRncHVfYW1ka2ZkX2RldmljZV9pbml0 KHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KTsKLXZvaWQgYW1kZ3B1X2FtZGtmZF9kZXZpY2Vf ZmluaShzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldik7Cit2b2lkIGFtZGdwdV9hbWRrZmRfZGV2 aWNlX2Zpbmlfc3coc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYpOwogaW50IGFtZGdwdV9hbWRr ZmRfc3VibWl0X2liKHN0cnVjdCBrZ2RfZGV2ICprZ2QsIGVudW0ga2dkX2VuZ2luZV90eXBlIGVu Z2luZSwKIAkJCQl1aW50MzJfdCB2bWlkLCB1aW50NjRfdCBncHVfYWRkciwKIAkJCQl1aW50MzJf dCAqaWJfY21kLCB1aW50MzJfdCBpYl9sZW4pOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJt L2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv YW1kZ3B1X2RldmljZS5jCmluZGV4IDhiZWU5NWFkMzJkOS4uYmM3NWUzNWRkOGQ4IDEwMDY0NAot LS0gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMKKysrIGIvZHJp dmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jCkBAIC0yNTU4LDM0ICsyNTU4 LDI2IEBAIHN0YXRpYyBpbnQgYW1kZ3B1X2RldmljZV9pcF9sYXRlX2luaXQoc3RydWN0IGFtZGdw dV9kZXZpY2UgKmFkZXYpCiAJcmV0dXJuIDA7CiB9CiAKLS8qKgotICogYW1kZ3B1X2RldmljZV9p cF9maW5pIC0gcnVuIGZpbmkgZm9yIGhhcmR3YXJlIElQcwotICoKLSAqIEBhZGV2OiBhbWRncHVf ZGV2aWNlIHBvaW50ZXIKLSAqCi0gKiBNYWluIHRlYXJkb3duIHBhc3MgZm9yIGhhcmR3YXJlIElQ cy4gIFRoZSBsaXN0IG9mIGFsbCB0aGUgaGFyZHdhcmUKLSAqIElQcyB0aGF0IG1ha2UgdXAgdGhl IGFzaWMgaXMgd2Fsa2VkIGFuZCB0aGUgaHdfZmluaSBhbmQgc3dfZmluaSBjYWxsYmFja3MKLSAq IGFyZSBydW4uICBod19maW5pIHRlYXJzIGRvd24gdGhlIGhhcmR3YXJlIGFzc29jaWF0ZWQgd2l0 aCBlYWNoIElQCi0gKiBhbmQgc3dfZmluaSB0ZWFycyBkb3duIGFueSBzb2Z0d2FyZSBzdGF0ZSBh c3NvY2lhdGVkIHdpdGggZWFjaCBJUC4KLSAqIFJldHVybnMgMCBvbiBzdWNjZXNzLCBuZWdhdGl2 ZSBlcnJvciBjb2RlIG9uIGZhaWx1cmUuCi0gKi8KLXN0YXRpYyBpbnQgYW1kZ3B1X2RldmljZV9p cF9maW5pKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KQorc3RhdGljIGludCBhbWRncHVfZGV2 aWNlX2lwX2ZpbmlfZWFybHkoc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYpCiB7CiAJaW50IGks IHI7CiAKLQlpZiAoYW1kZ3B1X3NyaW92X3ZmKGFkZXYpICYmIGFkZXYtPnZpcnQucmFzX2luaXRf ZG9uZSkKLQkJYW1kZ3B1X3ZpcnRfcmVsZWFzZV9yYXNfZXJyX2hhbmRsZXJfZGF0YShhZGV2KTsK Kwlmb3IgKGkgPSAwOyBpIDwgYWRldi0+bnVtX2lwX2Jsb2NrczsgaSsrKSB7CisJCWlmICghYWRl di0+aXBfYmxvY2tzW2ldLnZlcnNpb24tPmZ1bmNzLT5lYXJseV9maW5pKQorCQkJY29udGludWU7 CiAKLQlhbWRncHVfcmFzX3ByZV9maW5pKGFkZXYpOworCQlyID0gYWRldi0+aXBfYmxvY2tzW2ld LnZlcnNpb24tPmZ1bmNzLT5lYXJseV9maW5pKCh2b2lkICopYWRldik7CisJCWlmIChyKSB7CisJ CQlEUk1fREVCVUcoImVhcmx5X2Zpbmkgb2YgSVAgYmxvY2sgPCVzPiBmYWlsZWQgJWRcbiIsCisJ CQkJICBhZGV2LT5pcF9ibG9ja3NbaV0udmVyc2lvbi0+ZnVuY3MtPm5hbWUsIHIpOworCQl9CisJ fQogCi0JaWYgKGFkZXYtPmdtYy54Z21pLm51bV9waHlzaWNhbF9ub2RlcyA+IDEpCi0JCWFtZGdw dV94Z21pX3JlbW92ZV9kZXZpY2UoYWRldik7CisJYW1kZ3B1X2FtZGtmZF9zdXNwZW5kKGFkZXYs IGZhbHNlKTsKIAogCWFtZGdwdV9kZXZpY2Vfc2V0X3BnX3N0YXRlKGFkZXYsIEFNRF9QR19TVEFU RV9VTkdBVEUpOwogCWFtZGdwdV9kZXZpY2Vfc2V0X2NnX3N0YXRlKGFkZXYsIEFNRF9DR19TVEFU RV9VTkdBVEUpOwogCi0JYW1kZ3B1X2FtZGtmZF9kZXZpY2VfZmluaShhZGV2KTsKLQogCS8qIG5l ZWQgdG8gZGlzYWJsZSBTTUMgZmlyc3QgKi8KIAlmb3IgKGkgPSAwOyBpIDwgYWRldi0+bnVtX2lw X2Jsb2NrczsgaSsrKSB7CiAJCWlmICghYWRldi0+aXBfYmxvY2tzW2ldLnN0YXR1cy5odykKQEAg LTI2MTYsNiArMjYwOCwzMyBAQCBzdGF0aWMgaW50IGFtZGdwdV9kZXZpY2VfaXBfZmluaShzdHJ1 Y3QgYW1kZ3B1X2RldmljZSAqYWRldikKIAkJYWRldi0+aXBfYmxvY2tzW2ldLnN0YXR1cy5odyA9 IGZhbHNlOwogCX0KIAorCXJldHVybiAwOworfQorCisvKioKKyAqIGFtZGdwdV9kZXZpY2VfaXBf ZmluaSAtIHJ1biBmaW5pIGZvciBoYXJkd2FyZSBJUHMKKyAqCisgKiBAYWRldjogYW1kZ3B1X2Rl dmljZSBwb2ludGVyCisgKgorICogTWFpbiB0ZWFyZG93biBwYXNzIGZvciBoYXJkd2FyZSBJUHMu ICBUaGUgbGlzdCBvZiBhbGwgdGhlIGhhcmR3YXJlCisgKiBJUHMgdGhhdCBtYWtlIHVwIHRoZSBh c2ljIGlzIHdhbGtlZCBhbmQgdGhlIGh3X2ZpbmkgYW5kIHN3X2ZpbmkgY2FsbGJhY2tzCisgKiBh cmUgcnVuLiAgaHdfZmluaSB0ZWFycyBkb3duIHRoZSBoYXJkd2FyZSBhc3NvY2lhdGVkIHdpdGgg ZWFjaCBJUAorICogYW5kIHN3X2ZpbmkgdGVhcnMgZG93biBhbnkgc29mdHdhcmUgc3RhdGUgYXNz b2NpYXRlZCB3aXRoIGVhY2ggSVAuCisgKiBSZXR1cm5zIDAgb24gc3VjY2VzcywgbmVnYXRpdmUg ZXJyb3IgY29kZSBvbiBmYWlsdXJlLgorICovCitzdGF0aWMgaW50IGFtZGdwdV9kZXZpY2VfaXBf ZmluaShzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldikKK3sKKwlpbnQgaSwgcjsKKworCWlmIChh bWRncHVfc3Jpb3ZfdmYoYWRldikgJiYgYWRldi0+dmlydC5yYXNfaW5pdF9kb25lKQorCQlhbWRn cHVfdmlydF9yZWxlYXNlX3Jhc19lcnJfaGFuZGxlcl9kYXRhKGFkZXYpOworCisJYW1kZ3B1X3Jh c19wcmVfZmluaShhZGV2KTsKKworCWlmIChhZGV2LT5nbWMueGdtaS5udW1fcGh5c2ljYWxfbm9k ZXMgPiAxKQorCQlhbWRncHVfeGdtaV9yZW1vdmVfZGV2aWNlKGFkZXYpOworCisJYW1kZ3B1X2Ft ZGtmZF9kZXZpY2VfZmluaV9zdyhhZGV2KTsKIAogCWZvciAoaSA9IGFkZXYtPm51bV9pcF9ibG9j a3MgLSAxOyBpID49IDA7IGktLSkgewogCQlpZiAoIWFkZXYtPmlwX2Jsb2Nrc1tpXS5zdGF0dXMu c3cpCkBAIC0zNjgxLDYgKzM3MDAsOCBAQCB2b2lkIGFtZGdwdV9kZXZpY2VfZmluaV9odyhzdHJ1 Y3QgYW1kZ3B1X2RldmljZSAqYWRldikKIAlhbWRncHVfZmJkZXZfZmluaShhZGV2KTsKIAogCWFt ZGdwdV9pcnFfZmluaV9odyhhZGV2KTsKKworCWFtZGdwdV9kZXZpY2VfaXBfZmluaV9lYXJseShh ZGV2KTsKIH0KIAogdm9pZCBhbWRncHVfZGV2aWNlX2Zpbmlfc3coc3RydWN0IGFtZGdwdV9kZXZp Y2UgKmFkZXYpCmRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGtmZC9rZmRfZGV2 aWNlLmMgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGtmZC9rZmRfZGV2aWNlLmMKaW5kZXggMzU3 YjliZjYyYTFjLi5hYjZkMmE0M2M5YTMgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQv YW1ka2ZkL2tmZF9kZXZpY2UuYworKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGtmZC9rZmRf ZGV2aWNlLmMKQEAgLTg1OCwxMCArODU4LDExIEBAIGJvb2wga2dkMmtmZF9kZXZpY2VfaW5pdChz dHJ1Y3Qga2ZkX2RldiAqa2ZkLAogCXJldHVybiBrZmQtPmluaXRfY29tcGxldGU7CiB9CiAKKwor CiB2b2lkIGtnZDJrZmRfZGV2aWNlX2V4aXQoc3RydWN0IGtmZF9kZXYgKmtmZCkKIHsKIAlpZiAo a2ZkLT5pbml0X2NvbXBsZXRlKSB7Ci0JCWtnZDJrZmRfc3VzcGVuZChrZmQsIGZhbHNlKTsKIAkJ ZGV2aWNlX3F1ZXVlX21hbmFnZXJfdW5pbml0KGtmZC0+ZHFtKTsKIAkJa2ZkX2ludGVycnVwdF9l eGl0KGtmZCk7CiAJCWtmZF90b3BvbG9neV9yZW1vdmVfZGV2aWNlKGtmZCk7CmRpZmYgLS1naXQg YS9kcml2ZXJzL2dwdS9kcm0vYW1kL2Rpc3BsYXkvYW1kZ3B1X2RtL2FtZGdwdV9kbS5jIGIvZHJp dmVycy9ncHUvZHJtL2FtZC9kaXNwbGF5L2FtZGdwdV9kbS9hbWRncHVfZG0uYwppbmRleCA5Y2E1 MTdiNjU4NTQuLmY3MTEyODY1MjY5YSAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9k aXNwbGF5L2FtZGdwdV9kbS9hbWRncHVfZG0uYworKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2Rp c3BsYXkvYW1kZ3B1X2RtL2FtZGdwdV9kbS5jCkBAIC0xMjUxLDYgKzEyNTEsMTUgQEAgc3RhdGlj IGludCBhbWRncHVfZG1faW5pdChzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldikKIAlyZXR1cm4g LUVJTlZBTDsKIH0KIAorc3RhdGljIGludCBhbWRncHVfZG1fZWFybHlfZmluaSh2b2lkICpoYW5k bGUpCit7CisJc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYgPSAoc3RydWN0IGFtZGdwdV9kZXZp Y2UgKiloYW5kbGU7CisKKwlhbWRncHVfZG1fYXVkaW9fZmluaShhZGV2KTsKKworCXJldHVybiAw OworfQorCiBzdGF0aWMgdm9pZCBhbWRncHVfZG1fZmluaShzdHJ1Y3QgYW1kZ3B1X2RldmljZSAq YWRldikKIHsKIAlpbnQgaTsKQEAgLTEyNTksOCArMTI2OCw2IEBAIHN0YXRpYyB2b2lkIGFtZGdw dV9kbV9maW5pKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KQogCQlkcm1fZW5jb2Rlcl9jbGVh bnVwKCZhZGV2LT5kbS5tc3RfZW5jb2RlcnNbaV0uYmFzZSk7CiAJfQogCi0JYW1kZ3B1X2RtX2F1 ZGlvX2ZpbmkoYWRldik7Ci0KIAlhbWRncHVfZG1fZGVzdHJveV9kcm1fZGV2aWNlKCZhZGV2LT5k bSk7CiAKICNpZiBkZWZpbmVkKENPTkZJR19EUk1fQU1EX1NFQ1VSRV9ESVNQTEFZKQpAQCAtMjI5 OCw2ICsyMzA1LDcgQEAgc3RhdGljIGNvbnN0IHN0cnVjdCBhbWRfaXBfZnVuY3MgYW1kZ3B1X2Rt X2Z1bmNzID0gewogCS5sYXRlX2luaXQgPSBkbV9sYXRlX2luaXQsCiAJLnN3X2luaXQgPSBkbV9z d19pbml0LAogCS5zd19maW5pID0gZG1fc3dfZmluaSwKKwkuZWFybHlfZmluaSA9IGFtZGdwdV9k bV9lYXJseV9maW5pLAogCS5od19pbml0ID0gZG1faHdfaW5pdCwKIAkuaHdfZmluaSA9IGRtX2h3 X2ZpbmksCiAJLnN1c3BlbmQgPSBkbV9zdXNwZW5kLApkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUv ZHJtL2FtZC9pbmNsdWRlL2FtZF9zaGFyZWQuaCBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvaW5jbHVk ZS9hbWRfc2hhcmVkLmgKaW5kZXggNDNlZDYyOTFiMmI4Li4xYWQ1NmRhNDg2ZTQgMTAwNjQ0Ci0t LSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvaW5jbHVkZS9hbWRfc2hhcmVkLmgKKysrIGIvZHJpdmVy cy9ncHUvZHJtL2FtZC9pbmNsdWRlL2FtZF9zaGFyZWQuaApAQCAtMjQwLDYgKzI0MCw3IEBAIGVu dW0gYW1kX2RwbV9mb3JjZWRfbGV2ZWw7CiAgKiBAbGF0ZV9pbml0OiBzZXRzIHVwIGxhdGUgZHJp dmVyL2h3IHN0YXRlIChwb3N0IGh3X2luaXQpIC0gT3B0aW9uYWwKICAqIEBzd19pbml0OiBzZXRz IHVwIGRyaXZlciBzdGF0ZSwgZG9lcyBub3QgY29uZmlndXJlIGh3CiAgKiBAc3dfZmluaTogdGVh cnMgZG93biBkcml2ZXIgc3RhdGUsIGRvZXMgbm90IGNvbmZpZ3VyZSBodworICogQGVhcmx5X2Zp bmk6IHRlYXJzIGRvd24gc3R1ZmYgYmVmb3JlIGRldiBkZXRhY2hlZCBmcm9tIGRyaXZlcgogICog QGh3X2luaXQ6IHNldHMgdXAgdGhlIGh3IHN0YXRlCiAgKiBAaHdfZmluaTogdGVhcnMgZG93biB0 aGUgaHcgc3RhdGUKICAqIEBsYXRlX2Zpbmk6IGZpbmFsIGNsZWFudXAKQEAgLTI2OCw2ICsyNjks NyBAQCBzdHJ1Y3QgYW1kX2lwX2Z1bmNzIHsKIAlpbnQgKCpsYXRlX2luaXQpKHZvaWQgKmhhbmRs ZSk7CiAJaW50ICgqc3dfaW5pdCkodm9pZCAqaGFuZGxlKTsKIAlpbnQgKCpzd19maW5pKSh2b2lk ICpoYW5kbGUpOworCWludCAoKmVhcmx5X2ZpbmkpKHZvaWQgKmhhbmRsZSk7CiAJaW50ICgqaHdf aW5pdCkodm9pZCAqaGFuZGxlKTsKIAlpbnQgKCpod19maW5pKSh2b2lkICpoYW5kbGUpOwogCXZv aWQgKCpsYXRlX2ZpbmkpKHZvaWQgKmhhbmRsZSk7Ci0tIAoyLjI1LjEKCl9fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmFtZC1nZnggbWFpbGluZyBsaXN0CmFt ZC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3Jn L21haWxtYW4vbGlzdGluZm8vYW1kLWdmeAo=