From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: ** X-Spam-Status: No, score=2.5 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2E10C10F29 for ; Tue, 17 Mar 2020 17:13:38 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 85A202073E for ; Tue, 17 Mar 2020 17:13:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dhIjs+Oa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85A202073E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 675746E5C6; Tue, 17 Mar 2020 17:13:37 +0000 (UTC) Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4D7836E5C1; Tue, 17 Mar 2020 17:13:36 +0000 (UTC) Received: by mail-pj1-x102b.google.com with SMTP id q16so145628pje.1; Tue, 17 Mar 2020 10:13:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1qMDA0z5nbIODUb7Pwz016ch/4OWaMpBoMI+4oFO3Aw=; b=dhIjs+Oa8sELelQCMp30Zq11NosW5tfRvfBvKIe+MWGbNdE1DNrTMAGX7h0yrcZLe+ Juqe5RCw+fNKtnqQH1Vlg52Q+OobKrAjBoyqoAVJi8HVTaijNgHM2WE5qAh9yFl73CyI kWfbipCzug5Rh81Wb2UXLZBpveDnZVxNVgXiCF5thcaNLBFFs+JJLm7KT+Pyyps+BYeU 8s/XHzj1lYa1b7T2tcOaSupXkQ6B5+zCH8h6hAdKyDzzM0CE8rBPXIu8cYtuord+jwcs akmTe6Ytjc7h1H+GxMzwEyuEnvlzYOUtQpq+9ic0FS3bmgk+X0mMeCfKUM7ufXipYYQy R2Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1qMDA0z5nbIODUb7Pwz016ch/4OWaMpBoMI+4oFO3Aw=; b=PwgFo75iuhfvmu5+EgyKbgZoia9e4wydDgbxj3MKA4rOxoIGWtzLcIz9EB4C7R0t3P YM/+7VVPd9cLPvuul/zXlE5zQEklyJKL9q47aqihFLeiIazhONGAJrwHytUktR182Atn yDkDI6HmMpmRMZaqQpVdABbx9rNsUy0pIAsgME76Kkm07n4cH6Bbj6bvUsS7Cq5YG/LS VcFON6Qp5QqIeUY0gVXd1bPZLm5CxcVmt//gkwCXo8udEP6mu0I8H2aqq3LjUZs9ZoJA U1dc6PuxEucHZm5t+2G68p8ElxKnrRonS/f4SCpn338p2hZNXJPZ8sVAUO5ek+tq7xmG HQfQ== X-Gm-Message-State: ANhLgQ01W3oxWmTH5s6tQewhyjDwH1Z5IKCZ17NtTi7U6bTwdOGBUWB6 i8ybI5yNJrNZDxOSu4B0VPp0Vw1EqXhHFMlfyy0= X-Google-Smtp-Source: ADFU+vsclagzX/qtexJGNM4YjehC885UyWiX0M+GY3Um7qRXrDCvQTMm1+BqBbVnRypekxH1gTVWHlLnpsYjxUnDmXA= X-Received: by 2002:a17:902:d202:: with SMTP id t2mr5447676ply.5.1584465215702; Tue, 17 Mar 2020 10:13:35 -0700 (PDT) MIME-Version: 1.0 References: <170e13edbb0.27ad.c6988b7ea6112e3e892765a0d4287e0c@jlekstrand.net> In-Reply-To: From: =?UTF-8?B?TWFyZWsgT2zFocOhaw==?= Date: Tue, 17 Mar 2020 13:13:22 -0400 Message-ID: Subject: Re: Plumbing explicit synchronization through the Linux ecosystem To: =?UTF-8?Q?Michel_D=C3=A4nzer?= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , xorg-devel , Maling list - DRI developers , "wayland-devel @ lists . freedesktop . org" , Discussion of the development of and with GStreamer , Jason Ekstrand , ML mesa-dev , linux-media@vger.kernel.org Content-Type: multipart/mixed; boundary="===============0621417444==" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" --===============0621417444== Content-Type: multipart/alternative; boundary="000000000000574cae05a1100e6c" --000000000000574cae05a1100e6c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue., Mar. 17, 2020, 06:02 Michel D=C3=A4nzer, wrot= e: > On 2020-03-16 7:33 p.m., Marek Ol=C5=A1=C3=A1k wrote: > > On Mon, Mar 16, 2020 at 5:57 AM Michel D=C3=A4nzer > wrote: > >> On 2020-03-16 4:50 a.m., Marek Ol=C5=A1=C3=A1k wrote: > >>> The synchronization works because the Mesa driver waits for idle > (drains > >>> the GFX pipeline) at the end of command buffers and there is only 1 > >>> graphics queue, so everything is ordered. > >>> > >>> The GFX pipeline runs asynchronously to the command buffer, meaning t= he > >>> command buffer only starts draws and doesn't wait for completion. If > the > >>> Mesa driver didn't wait at the end of the command buffer, the command > >>> buffer would finish and a different process could start execution of > its > >>> own command buffer while shaders of the previous process are still > >> running. > >>> > >>> If the Mesa driver submits a command buffer internally (because it's > >> full), > >>> it doesn't wait, so the GFX pipeline doesn't notice that a command > buffer > >>> ended and a new one started. > >>> > >>> The waiting at the end of command buffers happens only when the flush > is > >>> external (Swap buffers, glFlush). > >>> > >>> It's a performance problem, because the GFX queue is blocked until th= e > >> GFX > >>> pipeline is drained at the end of every frame at least. > >>> > >>> So explicit fences for SwapBuffers would help. > >> > >> Not sure what difference it would make, since the same thing needs to = be > >> done for explicit fences as well, doesn't it? > > > > No. Explicit fences don't require userspace to wait for idle in the > command > > buffer. Fences are signalled when the last draw is complete and caches > are > > flushed. Before that happens, any command buffer that is not dependent = on > > the fence can start execution. There is never a need for the GPU to be > idle > > if there is enough independent work to do. > > I don't think explicit fences in the context of this discussion imply > using that different fence signalling mechanism though. My understanding > is that the API proposed by Jason allows implicit fences to be used as > explicit ones and vice versa, so presumably they have to use the same > signalling mechanism. > > > Anyway, maybe the different fence signalling mechanism you describe > could be used by the amdgpu kernel driver in general, then Mesa could > drop the waits for idle and get the benefits with implicit sync as well? > Yes. If there is any waiting, or should be done in the GPU scheduler, not in the command buffer, so that independent command buffers can use the GFX queue. Marek > > -- > Earthling Michel D=C3=A4nzer | https://redhat= .com > Libre software enthusiast | Mesa and X developer > --000000000000574cae05a1100e6c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue., Mar. 17, 2020, 06:02 Michel D=C3=A4nzer, <= michel@daenzer.net> wrote:
=
On 2020-03-16 7:33 p.m., Marek Ol=C5= =A1=C3=A1k wrote:
> On Mon, Mar 16, 2020 at 5:57 AM Michel D=C3=A4nzer <michel@daenzer.= net> wrote:
>> On 2020-03-16 4:50 a.m., Marek Ol=C5=A1=C3=A1k wrote:
>>> The synchronization works because the Mesa driver waits for id= le (drains
>>> the GFX pipeline) at the end of command buffers and there is o= nly 1
>>> graphics queue, so everything is ordered.
>>>
>>> The GFX pipeline runs asynchronously to the command buffer, me= aning the
>>> command buffer only starts draws and doesn't wait for comp= letion. If the
>>> Mesa driver didn't wait at the end of the command buffer, = the command
>>> buffer would finish and a different process could start execut= ion of its
>>> own command buffer while shaders of the previous process are s= till
>> running.
>>>
>>> If the Mesa driver submits a command buffer internally (becaus= e it's
>> full),
>>> it doesn't wait, so the GFX pipeline doesn't notice th= at a command buffer
>>> ended and a new one started.
>>>
>>> The waiting at the end of command buffers happens only when th= e flush is
>>> external (Swap buffers, glFlush).
>>>
>>> It's a performance problem, because the GFX queue is block= ed until the
>> GFX
>>> pipeline is drained at the end of every frame at least.
>>>
>>> So explicit fences for SwapBuffers would help.
>>
>> Not sure what difference it would make, since the same thing needs= to be
>> done for explicit fences as well, doesn't it?
>
> No. Explicit fences don't require userspace to wait for idle in th= e command
> buffer. Fences are signalled when the last draw is complete and caches= are
> flushed. Before that happens, any command buffer that is not dependent= on
> the fence can start execution. There is never a need for the GPU to be= idle
> if there is enough independent work to do.

I don't think explicit fences in the context of this discussion imply using that different fence signalling mechanism though. My understanding is that the API proposed by Jason allows implicit fences to be used as
explicit ones and vice versa, so presumably they have to use the same
signalling mechanism.


Anyway, maybe the different fence signalling mechanism you describe
could be used by the amdgpu kernel driver in general, then Mesa could
drop the waits for idle and get the benefits with implicit sync as well?

Yes= . If there is any waiting, or should be done in the GPU scheduler, not in t= he command buffer, so that independent command buffers can use the GFX queu= e.

Marek



--
Earthling Michel D=C3=A4nzer=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0|=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0htt= ps://redhat.com
Libre software enthusiast=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Mesa and X developer
--000000000000574cae05a1100e6c-- --===============0621417444== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel --===============0621417444==--