From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83AAEC3F2D1 for ; Tue, 3 Mar 2020 16:07:09 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0DA3F206E2 for ; Tue, 3 Mar 2020 16:07:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="hcB24YFx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0DA3F206E2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BD5846E059; Tue, 3 Mar 2020 16:07:08 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2043.outbound.protection.outlook.com [40.107.94.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7195B6E059 for ; Tue, 3 Mar 2020 16:07:07 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jP6DxLlyRAnD+OYPgCKDmxOx6HTPvTXoO3As5pnzCEWX8WxaZ3mly7PjVu47pHp719mcmichHyaYuKPIWK2LjlAdOgLwQS80ZVR5BAgUz+0k51TzPddaev0O9RU2eE7GTFWdl0O+oLcjpGHWsoGgvZj3Ia8oEf2YrHSIPBaShndtJc5h94Y+Jvw/5Xu8CguS390volMhaMD3RYBRF6GVJGDDNMgK4K6181+S9MiWcqL5sHyDObLVvasx694jgAy2HUwBogUR7ssLefiarcBN4/qzWts2CKNdTceNWnwH+YRaGtWEroVxgP+RX0T6xhImgcuRZCC0w20VzuqHVRPU6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Qt+2tvt8toe2nApfKz+1dynfqexsv64RlPMwD3Ttx8U=; b=HUU7hkq2S7xWwNXfkaSS7Av51BTCy6CiQ6Q1cdPZ2HDnKPEZXigOo1MOpqZz0QpnaPfrgd/JXcAq/DkEh1bTx/AYkPiU2KjK0kWs5oeugErIiznKgGa17kTtmXC4WxrgeG649Zw6bnseB1SWYm1LJSnabV2qdQ6weI5T8fHcO4L6851cwspf9ArzeXDwoe8MXoJRwPPYf29YmTeZdXQ/RsLhM3zPd90HmIXt1JY31m0EU1+U2SoTSWuFq/O0U9ct5LjyCDJBfqBVW7sB/tt5mYcUzQo60GBnVTJFDOOnlJpNZsKuGnumXbddSGK4Yhf1xcaTH+IvOeZchLrf+CzOmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Qt+2tvt8toe2nApfKz+1dynfqexsv64RlPMwD3Ttx8U=; b=hcB24YFxz7YlAeowX9bKjae4JFSq7fjUy1Kw8byqMfxBtr9TZ49QgU0qL5txu+sCOaIFNa0V4DdXJa18/W/o14qa1aBSi01So6FRlIs/hMowg1mad+LxD5JIDKU7jQ39V1ejxVtrenDSUcOdlUDB46inmWuAYRwn7SLshBgpcjU= Received: from MN2PR12MB3376.namprd12.prod.outlook.com (2603:10b6:208:c2::20) by MN2PR12MB4271.namprd12.prod.outlook.com (2603:10b6:208:1d7::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2772.14; Tue, 3 Mar 2020 16:07:06 +0000 Received: from MN2PR12MB3376.namprd12.prod.outlook.com ([fe80::61bc:e52b:bdcf:4f9b]) by MN2PR12MB3376.namprd12.prod.outlook.com ([fe80::61bc:e52b:bdcf:4f9b%4]) with mapi id 15.20.2772.019; Tue, 3 Mar 2020 16:07:06 +0000 From: "He, Jacob" To: "Koenig, Christian" , "amd-gfx@lists.freedesktop.org" Subject: RE: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid when application reserves the vmid Thread-Topic: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid when application reserves the vmid Thread-Index: AQHV8FRyok5cRxATf0SYBiJ9i8QZh6g27AOAgAAERJiAAAjwgIAAAzfWgAAF1YCAAAWQtQ== Date: Tue, 3 Mar 2020 16:07:06 +0000 Message-ID: References: <20200302053529.5736-1-jacob.he@amd.com> <0bbac128-473d-b8d1-9b5a-ceb25357c81c@gmail.com> <06d9d31f-87c5-b1f1-da93-992531dd57ad@amd.com> , <21a8a363-1ebe-a4cb-150d-f84442f322fc@amd.com> In-Reply-To: <21a8a363-1ebe-a4cb-150d-f84442f322fc@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_76546daa-41b6-470c-bb85-f6f40f044d7f_Enabled=True; MSIP_Label_76546daa-41b6-470c-bb85-f6f40f044d7f_SiteId=3dd8961f-e488-4e60-8e11-a82d994e183d; MSIP_Label_76546daa-41b6-470c-bb85-f6f40f044d7f_SetDate=2020-03-03T15:56:03.3867436Z; MSIP_Label_76546daa-41b6-470c-bb85-f6f40f044d7f_ContentBits=0; MSIP_Label_76546daa-41b6-470c-bb85-f6f40f044d7f_Method=Privileged authentication-results: spf=none (sender IP is ) smtp.mailfrom=Jacob.He@amd.com; x-originating-ip: [180.167.199.185] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 0a809bfb-79ff-4ab0-6c49-08d7bf8ceb7c x-ms-traffictypediagnostic: MN2PR12MB4271:|MN2PR12MB4271: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-forefront-prvs: 03319F6FEF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(376002)(396003)(136003)(346002)(366004)(39860400002)(189003)(199004)(91956017)(478600001)(6506007)(110136005)(2906002)(86362001)(316002)(7696005)(71200400001)(52536014)(5660300002)(76116006)(8676002)(53546011)(81166006)(26005)(186003)(33656002)(15650500001)(81156014)(9686003)(55016002)(66446008)(66556008)(64756008)(66574012)(66946007)(8936002)(66476007); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR12MB4271; H:MN2PR12MB3376.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: amd.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ZQPw+dKWf7WXBwv5CmV1n34CcPgLjlvL6943yMu1UrG2AYsBsMot3gZsDkWgM9GN/RYLmtFGYWIJhg52D6YHfCmzsh4V4/KdHOdTVqD3cXG+i7tQSlQiJpJsImfhCwHDbUcHnUJIFwli0Y6Juer3dohlF4BdGjXzqWFzVI40jsDIwUr6b4HeuCMLhvqPxMcI2aO2xQ1jkIuzSbtOxHbJs/VMVt4Nl44UVx9yW9Shhuv1qSqKhNqBsKfvAERvSW3h2RjVOq2La0wcyLYOkfYheUE4XWzqJSrqmUT1IaaJA6mdH6WhYFaDwfc7n8OqV7ybcR9sPnMpm74rT3Jb0DYLAXtbLlDMMRKJZny/E3HdNRJn2rtaHA++sl3fJEbgGrzdEncNlimuabdnjZVlSHn2fIZkmRxAhAiCxbrY88/tsXu1gHT4iAVqYUdLQCJREBEx x-ms-exchange-antispam-messagedata: ZGlIyg74SJ3Qm/6kCMbtmSA5BeSVDHJZDrLOACcBA1xK63kUNEOixOTKVUzWAAw9m6AwYJLKjb7FHUKysRfELXU1lEiYnfMZFvLEYpRxKGYzNk2oxXH5JX+wzTiHZEX5c+WLmEVgQ+2BIYE9zjRCZQ== MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a809bfb-79ff-4ab0-6c49-08d7bf8ceb7c X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2020 16:07:06.2031 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: zV1InK+ItG5cnT0yZD0dG7kTyl66N+odgcG68cSwPKhoVZ96DcbdUEan1kBBA/p2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4271 X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============0391016793==" Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" --===============0391016793== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_MN2PR12MB33768819020B42D37DFDB6809BE40MN2PR12MB3376namp_" --_000_MN2PR12MB33768819020B42D37DFDB6809BE40MN2PR12MB3376namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable [AMD Official Use Only - Internal Distribution Only] Oh, you are right! If SPM_VMID is updated by other process while the SPM en= abled commands is executing, that will cause VM fault. Is the wait vm idle right before unreserve vmid still necessary if using as= ynchroneously setting SPM_VMID? Thanks Jacob From: Koenig, Christian Sent: Tuesday, March 3, 2020 11:36 PM To: He, Jacob; amd-gfx@lists.freedesktop.org Subject: Re: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid when a= pplication reserves the vmid See the SPM buffer address is set using CP commands as well, right? And tho= se execute asynchronously. When we now synchronously update the SPM VMID we risk that we switch from o= ne process to another while the new process is not ready yet with its setup= . That could have quite a bunch of unforeseen consequences, including acciden= tally writing SPM data into the new process address space at whatever buffe= r address was used before. This is something we at least should try to avoid. Regards, Christian. Am 03.03.20 um 16:28 schrieb He, Jacob: [AMD Official Use Only - Internal Distribution Only] Thanks! Could you please take an example of trouble =93This way we avoid = a bunch of trouble when one process drops the VMID reservation and another = one grabs it.=94? Thanks Jacob From: Koenig, Christian Sent: Tuesday, March 3, 2020 11:03 PM To: He, Jacob; amd-gfx@lists.freedesktop.org Subject: Re: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid when a= pplication reserves the vmid Am 03.03.20 um 15:34 schrieb He, Jacob: [AMD Official Use Only - Internal Distribution Only] It would be better if we could do that asynchronously with a register write on the ring. Sorry, I don=92t get your point. Could you please elaborate more? You pass the ring from amdgpu_vm_flush() to the *_update_spm_vmid() functio= ns. And then instead of using WREG32() you call amdgpu_ring_emit_wreg() to make= the write asynchronously on the ring buffer using a CP command. This way we avoid a bunch of trouble when one process drops the VMID reserv= ation and another one grabs it. Regards, Christian. Thanks Jacob From: Christian K=F6nig Sent: Tuesday, March 3, 2020 10:16 PM To: He, Jacob; amd-gfx@lists.freedesktop.org Subject: Re: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid when a= pplication reserves the vmid Am 02.03.20 um 06:35 schrieb Jacob He: > SPM access the video memory according to SPM_VMID. It should be updated > with the job's vmid right before the job is scheduled. SPM_VMID is a > global resource > > Change-Id: Id3881908960398f87e7c95026a54ff83ff826700 > Signed-off-by: Jacob He > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd= /amdgpu/amdgpu_vm.c > index c00696f3017e..c761d3a0b6e8 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c > @@ -1080,8 +1080,12 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, stru= ct amdgpu_job *job, > struct dma_fence *fence =3D NULL; > bool pasid_mapping_needed =3D false; > unsigned patch_offset =3D 0; > + bool update_spm_vmid_needed =3D (job->vm && (job->vm->reserved_vmid= [vmhub] !=3D NULL)); > int r; > > + if (update_spm_vmid_needed && adev->gfx.rlc.funcs->update_spm_vmid) > + adev->gfx.rlc.funcs->update_spm_vmid(adev, job->vmid); > + It would be better if we could do that asynchronously with a register write on the ring. The alternative is that we block for the VM to be idle in amdgpu_vm_ioctl() before unreserving the VMID. In other words lock the reservation object of the root PD and call amdgpu_vm_wait_idle() before calling amdgpu_vmid_free_reserved(). Regards, Christian. > if (amdgpu_vmid_had_gpu_reset(adev, id)) { > gds_switch_needed =3D true; > vm_flush_needed =3D true; --_000_MN2PR12MB33768819020B42D37DFDB6809BE40MN2PR12MB3376namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

[AMD Official Use Only - Internal Distribution Only]


Oh, you are right! If SPM_VMID is updated by other p= rocess while the SPM enabled commands is executing, that will cause VM faul= t.

 

Is the wait vm idle right before unreserve vmid stil= l necessary if using asynchroneously setting SPM_VMID?

 

Thanks

Jacob

 

From: Koenig, Christian
Sent: Tuesday, March 3, 2020 11:36 PM
To: He, Jacob; amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid= when application reserves the vmid

 

See the SPM buffer address is set using CP commands = as well, right? And those execute asynchronously.

When we now synchronously update the SPM VMID we risk that we switch from o= ne process to another while the new process is not ready yet with its setup= .

That could have quite a bunch of unforeseen consequences, including acciden= tally writing SPM data into the new process address space at whatever buffe= r address was used before.

This is something we at least should try to avoid.

Regards,
Christian.

Am 03.03.20 um 16:28 schrieb He, Jacob:

[AMD Official Use Only - Internal D= istribution Only]

 =

Thanks!  Could you please take an example of tr= ouble  =93This way we avoid a bunch of tro= uble when one process drops the VMID reservation and another one grabs it.<= /span>=94?

 

Thanks

Jacob

 

From: Koenig, Christian
Sent: Tuesday, March 3, 2020 11:03 PM
To: He, Jacob; amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid= when application reserves the vmid

 

Am 03.03.20 um 15:34 sch= rieb He, Jacob:

[AMD Official Use Only - Internal D= istribution Only]

 =

It would be better if we could do that asynchronously with a regi= ster
write on the ring.

Sorry, I don=92t get you= r point. Could you please elaborate more?


You pass the ring from amdgpu_vm_flush() to the *_update_spm_vmid() functio= ns.

And then instead of using WREG32() you call amdgpu_ring_emit_wreg() to make= the write asynchronously on the ring buffer using a CP command.

This way we avoid a bunch of trouble when one process drops the VMID reserv= ation and another one grabs it.

Regards,
Christian.



 =

Thanks

Jacob

 =

From: Christian K=F6nig
Sent: Tuesday, March 3, 2020 10:16 PM
To: He, Jacob; amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH] drm/amdgpu: Update SPM_VMID with the job's vmid= when application reserves the vmid

 =

Am 02.03.20 um 06:35 schrieb Jacob He:
> SPM access the video memory according to SPM_VMID. It should be update= d
> with the job's vmid right before the job is scheduled. SPM_VMID is a > global resource
>
> Change-Id: Id3881908960398f87e7c95026a54ff83ff826700
> Signed-off-by: Jacob He <jacob.= he@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 +++= +
>   1 file changed, 4 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/= amd/amdgpu/amdgpu_vm.c
> index c00696f3017e..c761d3a0b6e8 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1080,8 +1080,12 @@ int amdgpu_vm_flush(struct amdgpu_ring *rin= g, struct amdgpu_job *job,
>        struct dma_fence *fence =3D = NULL;
>        bool pasid_mapping_needed = =3D false;
>        unsigned patch_offset =3D 0;=
> +     bool update_spm_vmid_needed =3D (job->= ;vm && (job->vm->reserved_vmid[vmhub] !=3D NULL));
>        int r;
>  
> +     if (update_spm_vmid_needed && ad= ev->gfx.rlc.funcs->update_spm_vmid)
> +           = ;  adev->gfx.rlc.funcs->update_spm_vmid(adev, job->vmid);
> +

It would be better if we could do that asynchronously with a register
write on the ring.

The alternative is that we block for the VM to be idle in
amdgpu_vm_ioctl() before unreserving the VMID.

In other words lock the reservation object of the root PD and call
amdgpu_vm_wait_idle() before calling amdgpu_vmid_free_reserved().

Regards,
Christian.

>        if (amdgpu_vmid_had_gpu_rese= t(adev, id)) {
>            = ;    gds_switch_needed =3D true;
>            = ;    vm_flush_needed =3D true;

 =

 =

 =

 =

 

--_000_MN2PR12MB33768819020B42D37DFDB6809BE40MN2PR12MB3376namp_-- --===============0391016793== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx --===============0391016793==--