From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: ** X-Spam-Status: No, score=2.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B0F6C433ED for ; Wed, 28 Apr 2021 05:20:19 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BD3BC61417 for ; Wed, 28 Apr 2021 05:20:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD3BC61417 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 287AA6EA99; Wed, 28 Apr 2021 05:20:17 +0000 (UTC) Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by gabe.freedesktop.org (Postfix) with ESMTPS id 12A7D6EA99; Wed, 28 Apr 2021 05:20:06 +0000 (UTC) Received: by mail-pl1-x634.google.com with SMTP id t21so1055948plo.2; Tue, 27 Apr 2021 22:20:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0nJdSUOT2ZLHt/Ci8K1eVf9vho0jKv1jKO+dhCK1E9E=; b=mPLWdHvCyYsbQHkn5yiRL0XODthOi3hM/g10uHRf7gmYDUnAzuf8lj5HHRB5axLjgw 9zZpwt85420BGFjOh4bQaC4ah1JADL759ohMShakr9+zSO6KHyP9GN9EGiG0KtMuBZzp kHXEwcrLSN8O3BBP0dNEPt9q084GNmJ1Im3W22/9ICMTGk1apEknFIQoNMGX/uT9ekNO CXf52PeU8WE4tJtKuNTwhEIZih6C6ZAHj7SluWbfLLAUxgCJ08gNzcQ61in4AkGItku4 ydv4LknWSGtIJyMJmAQTnFvBoRE5lXUXS0FHQa5AlNvRhqtpiP8O+Lq/sbgVbBg7Vugs ccEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0nJdSUOT2ZLHt/Ci8K1eVf9vho0jKv1jKO+dhCK1E9E=; b=fxfIhWcFU1OBWOLI39QcolIWarC7uK6xHYKTWz8HdKPUXmvFxCgKBmlEsAY4T+O5R/ FlQRsg81wpxPoO91VhtL45eAObM0WoT4lWI8THYFcLX29JdiNIB+0NU/l8bfkOaDjmkD bKoWXH+2jnAE/eDxWoD8zO74566h60Bqqx21RUnLQe8kM+v4IAmySsh5BBM6bGtYrBaz Ht1czoNs7haww4B4zGG1Njgdn7AHNyKEEtJ6BABfviTJhkJJDufEO4uKPC55RJf4rMnD 3fv/u5armbcsGA9U7DVTxiSQDXdUdJzhrFFxMdwJf+yYFefJH1yRach+gk59/4WIRmwy kjPQ== X-Gm-Message-State: AOAM531wLsD4VidcEt3CULf1AX9hVcVI8h5Kq65Oog8m2rc/YgaIAY3K UD2722uBQUQAhuwr8HNUScjZXuwX3468JRAc254DbyNR2pO1ww== X-Google-Smtp-Source: ABdhPJwFnozyf5ZYCeLVE346Q1sQAmOmPO5eNJuvFkWiqC7bjcUiFOhxiVx8JbxjuU/tf/SlB6ZqzOSF3krGxKD+VxI= X-Received: by 2002:a17:90b:3796:: with SMTP id mz22mr1994510pjb.80.1619587205567; Tue, 27 Apr 2021 22:20:05 -0700 (PDT) MIME-Version: 1.0 References: <735e0d2e-f2c9-c546-ea6c-b5bbb0fe03a6@gmail.com> <23ea06c825279c7a9f7678b335c7f89437d387ed.camel@pengutronix.de> In-Reply-To: From: =?UTF-8?B?TWFyZWsgT2zFocOhaw==?= Date: Wed, 28 Apr 2021 01:19:16 -0400 Message-ID: Subject: Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal To: Jason Ekstrand X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?Q?Christian_K=C3=B6nig?= , ML Mesa-dev , dri-devel Content-Type: multipart/mixed; boundary="===============1695728040==" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" --===============1695728040== Content-Type: multipart/alternative; boundary="0000000000001217be05c1018801" --0000000000001217be05c1018801 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed., Apr. 28, 2021, 00:01 Jason Ekstrand, wrote: > On Tue, Apr 27, 2021 at 4:59 PM Marek Ol=C5=A1=C3=A1k = wrote: > > > > Jason, both memory-based signalling as well as interrupt-based > signalling to the CPU would be supported by amdgpu. External devices don'= t > need to support memory-based sync objects. The only limitation is that th= ey > can't convert amdgpu sync objects to dma_fence. > > Sure. I'm not worried about the mechanism. We just need a word that > means "the new fence thing" and I've been throwing "memory fence" > around for that. Other mechanisms may work as well. > > > The sad thing is that "external -> amdgpu" dependencies are really > "external <-> amdgpu" dependencies due to mutually-exclusive access > required by non-explicitly-sync'd buffers, so amdgpu-amdgpu interop is th= e > only interop that would initially work with those buffers. Explicitly > sync'd buffers also won't work if other drivers convert explicit fences t= o > dma_fence. Thus, both implicit sync and explicit sync might not work with > other drivers at all. The only interop that would initially work is > explicit fences with memory-based waiting and signalling on the external > device to keep the kernel out of the picture. > > Yup. This is where things get hard. That said, I'm not quite ready > to give up on memory/interrupt fences just yet. > > One thought that came to mind which might help would be if we added an > extremely strict concept of memory ownership. The idea would be that > any given BO would be in one of two states at any given time: > > 1. legacy: dma_fences and implicit sync works as normal but it cannot > be resident in any "modern" (direct submission, ULLS, whatever you > want to call it) context > > 2. modern: In this mode they should not be used by any legacy > context. We can't strictly prevent this, unfortunately, but maybe we > can say reading produces garbage and writes may be discarded. In this > mode, they can be bound to modern contexts. > > In theory, when in "modern" mode, you could bind the same buffer in > multiple modern contexts at a time. However, when that's the case, it > makes ownership really tricky to track. Therefore, we might want some > sort of dma-buf create flag for "always modern" vs. "switchable" and > only allow binding to one modern context at a time when it's > switchable. > > If we did this, we may be able to move any dma_fence shenanigans to > the ownership transition points. We'd still need some sort of "wait > for fence and transition" which has a timeout. However, then we'd be > fairly well guaranteed that the application (not just Mesa!) has > really and truly decided it's done with the buffer and we wouldn't (I > hope!) end up with the accidental edges in the dependency graph. > > Of course, I've not yet proven any of this correct so feel free to > tell me why it won't work. :-) It was just one of those "about to go > to bed and had a thunk" type thoughts. > We'd like to keep userspace outside of Mesa drivers intact and working except for interop where we don't have much choice. At the same time, future hw may remove support for kernel queues, so we might not have much choice there either, depending on what the hw interface will look like. The idea is to have an ioctl for querying a timeline semaphore buffer associated with a shared BO, and an ioctl for querying the next wait and signal number (e.g. n and n+1) for that semaphore. Waiting for n would be like mutex lock and signaling would be like mutex unlock. The next process would use the same ioctl and get n+1 and n+2, etc. There is a deadlock condition because one process can do lock A, lock B, and another can do lock B, lock A, which can be prevented such that the ioctl that returns the numbers would return them for multiple buffers at once. This solution needs no changes to userspace outside of Mesa drivers, and we'll also keep the BO wait ioctl for GPU-CPU sync. Marek > --Jason > > P.S. Daniel was 100% right when he said this discussion needs a glossary= . > > > > Marek > > > > > > On Tue, Apr 27, 2021 at 3:41 PM Jason Ekstrand > wrote: > >> > >> Trying to figure out which e-mail in this mess is the right one to > reply to.... > >> > >> On Tue, Apr 27, 2021 at 12:31 PM Lucas Stach > wrote: > >> > > >> > Hi, > >> > > >> > Am Dienstag, dem 27.04.2021 um 09:26 -0400 schrieb Marek Ol=C5=A1=C3= =A1k: > >> > > Ok. So that would only make the following use cases broken for now= : > >> > > - amd render -> external gpu > >> > >> Assuming said external GPU doesn't support memory fences. If we do > >> amdgpu and i915 at the same time, that covers basically most of the > >> external GPU use-cases. Of course, we'd want to convert nouveau as > >> well for the rest. > >> > >> > > - amd video encode -> network device > >> > > >> > FWIW, "only" breaking amd render -> external gpu will make us pretty > >> > unhappy, as we have some cases where we are combining an AMD APU wit= h > a > >> > FPGA based graphics card. I can't go into the specifics of this use- > >> > case too much but basically the AMD graphics is rendering content th= at > >> > gets composited on top of a live video pipeline running through the > >> > FPGA. > >> > >> I think it's worth taking a step back and asking what's being here > >> before we freak out too much. If we do go this route, it doesn't mean > >> that your FPGA use-case can't work, it just means it won't work > >> out-of-the box anymore. You'll have to separate execution and memory > >> dependencies inside your FPGA driver. That's still not great but it's > >> not as bad as you maybe made it sound. > >> > >> > > What about the case when we get a buffer from an external device a= nd > >> > > we're supposed to make it "busy" when we are using it, and the > >> > > external device wants to wait until we stop using it? Is it > something > >> > > that can happen, thus turning "external -> amd" into "external <-> > >> > > amd"? > >> > > >> > Zero-copy texture sampling from a video input certainly appreciates > >> > this very much. Trying to pass the render fence through the various > >> > layers of userspace to be able to tell when the video input can reus= e > a > >> > buffer is a great experience in yak shaving. Allowing the video inpu= t > >> > to reuse the buffer as soon as the read dma_fence from the GPU is > >> > signaled is much more straight forward. > >> > >> Oh, it's definitely worse than that. Every window system interaction > >> is bi-directional. The X server has to wait on the client before > >> compositing from it and the client has to wait on X before re-using > >> that back-buffer. Of course, we can break that later dependency by > >> doing a full CPU wait but that's going to mean either more latency or > >> reserving more back buffers. There's no good clean way to claim that > >> any of this is one-directional. > >> > >> --Jason > --0000000000001217be05c1018801 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Wed., Apr. 28, 2021, 00:01 Jason Ekstrand, <jason@jlek= strand.net> wrote:
On Tue, = Apr 27, 2021 at 4:59 PM Marek Ol=C5=A1=C3=A1k <maraeo@gmail.com= > wrote:
>
> Jason, both memory-based signalling as well as interrupt-based signall= ing to the CPU would be supported by amdgpu. External devices don't nee= d to support memory-based sync objects. The only limitation is that they ca= n't convert amdgpu sync objects to dma_fence.

Sure.=C2=A0 I'm not worried about the mechanism.=C2=A0 We just need a w= ord that
means "the new fence thing" and I've been throwing "memo= ry fence"
around for that.=C2=A0 Other mechanisms may work as well.

> The sad thing is that "external -> amdgpu" dependencies a= re really "external <-> amdgpu" dependencies due to mutuall= y-exclusive access required by non-explicitly-sync'd buffers, so amdgpu= -amdgpu interop is the only interop that would initially work with those bu= ffers. Explicitly sync'd buffers also won't work if other drivers c= onvert explicit fences to dma_fence. Thus, both implicit sync and explicit = sync might not work with other drivers at all. The only interop that would = initially work is explicit fences with memory-based waiting and signalling = on the external device to keep the kernel out of the picture.

Yup.=C2=A0 This is where things get hard.=C2=A0 That said, I'm not quit= e ready
to give up on memory/interrupt fences just yet.

One thought that came to mind which might help would be if we added an
extremely strict concept of memory ownership.=C2=A0 The idea would be that<= br> any given BO would be in one of two states at any given time:

=C2=A01. legacy: dma_fences and implicit sync works as normal but it cannot=
be resident in any "modern" (direct submission, ULLS, whatever yo= u
want to call it) context

=C2=A02. modern: In this mode they should not be used by any legacy
context.=C2=A0 We can't strictly prevent this, unfortunately, but maybe= we
can say reading produces garbage and writes may be discarded.=C2=A0 In this=
mode, they can be bound to modern contexts.

In theory, when in "modern" mode, you could bind the same buffer = in
multiple modern contexts at a time.=C2=A0 However, when that's the case= , it
makes ownership really tricky to track.=C2=A0 Therefore, we might want some=
sort of dma-buf create flag for "always modern" vs. "switcha= ble" and
only allow binding to one modern context at a time when it's
switchable.

If we did this, we may be able to move any dma_fence shenanigans to
the ownership transition points.=C2=A0 We'd still need some sort of &qu= ot;wait
for fence and transition" which has a timeout.=C2=A0 However, then we&= #39;d be
fairly well guaranteed that the application (not just Mesa!) has
really and truly decided it's done with the buffer and we wouldn't = (I
hope!) end up with the accidental edges in the dependency graph.

Of course, I've not yet proven any of this correct so feel free to
tell me why it won't work. :-)=C2=A0 It was just one of those "abo= ut to go
to bed and had a thunk" type thoughts.

We'd like to keep userspace = outside of Mesa drivers intact and working except for interop where we don&= #39;t have much choice. At the same time, future hw may remove support for = kernel queues, so we might not have much choice there either, depending on = what the hw interface will look like.

The idea is to have an ioctl for querying a timeline semaphor= e buffer associated with a shared BO, and an ioctl for querying the next wa= it and signal number (e.g. n and n+1) for that semaphore. Waiting for n wou= ld be like mutex lock and signaling would be like mutex unlock. The next pr= ocess would use the same ioctl and get n+1 and n+2, etc. There is a deadloc= k condition because one process can do lock A, lock B, and another can do l= ock B, lock A, which can be prevented such that the ioctl that returns the = numbers would return them for multiple buffers at once. This solution needs= no changes to userspace outside of Mesa drivers, and we'll also keep t= he BO wait ioctl for GPU-CPU sync.

Marek


--Jason

P.S.=C2=A0 Daniel was 100% right when he said this discussion needs a gloss= ary.


> Marek
>
>
> On Tue, Apr 27, 2021 at 3:41 PM Jason Ekstrand <jason@= jlekstrand.net> wrote:
>>
>> Trying to figure out which e-mail in this mess is the right one to= reply to....
>>
>> On Tue, Apr 27, 2021 at 12:31 PM Lucas Stach <l.= stach@pengutronix.de> wrote:
>> >
>> > Hi,
>> >
>> > Am Dienstag, dem 27.04.2021 um 09:26 -0400 schrieb Marek Ol= =C5=A1=C3=A1k:
>> > > Ok. So that would only make the following use cases brok= en for now:
>> > > - amd render -> external gpu
>>
>> Assuming said external GPU doesn't support memory fences.=C2= =A0 If we do
>> amdgpu and i915 at the same time, that covers basically most of th= e
>> external GPU use-cases.=C2=A0 Of course, we'd want to convert = nouveau as
>> well for the rest.
>>
>> > > - amd video encode -> network device
>> >
>> > FWIW, "only" breaking amd render -> external gpu= will make us pretty
>> > unhappy, as we have some cases where we are combining an AMD = APU with a
>> > FPGA based graphics card. I can't go into the specifics o= f this use-
>> > case too much but basically the AMD graphics is rendering con= tent that
>> > gets composited on top of a live video pipeline running throu= gh the
>> > FPGA.
>>
>> I think it's worth taking a step back and asking what's be= ing here
>> before we freak out too much.=C2=A0 If we do go this route, it doe= sn't mean
>> that your FPGA use-case can't work, it just means it won't= work
>> out-of-the box anymore.=C2=A0 You'll have to separate executio= n and memory
>> dependencies inside your FPGA driver.=C2=A0 That's still not g= reat but it's
>> not as bad as you maybe made it sound.
>>
>> > > What about the case when we get a buffer from an externa= l device and
>> > > we're supposed to make it "busy" when we a= re using it, and the
>> > > external device wants to wait until we stop using it? Is= it something
>> > > that can happen, thus turning "external -> amd&q= uot; into "external <->
>> > > amd"?
>> >
>> > Zero-copy texture sampling from a video input certainly appre= ciates
>> > this very much. Trying to pass the render fence through the v= arious
>> > layers of userspace to be able to tell when the video input c= an reuse a
>> > buffer is a great experience in yak shaving. Allowing the vid= eo input
>> > to reuse the buffer as soon as the read dma_fence from the GP= U is
>> > signaled is much more straight forward.
>>
>> Oh, it's definitely worse than that.=C2=A0 Every window system= interaction
>> is bi-directional.=C2=A0 The X server has to wait on the client be= fore
>> compositing from it and the client has to wait on X before re-usin= g
>> that back-buffer.=C2=A0 Of course, we can break that later depende= ncy by
>> doing a full CPU wait but that's going to mean either more lat= ency or
>> reserving more back buffers.=C2=A0 There's no good clean way t= o claim that
>> any of this is one-directional.
>>
>> --Jason
--0000000000001217be05c1018801-- --===============1695728040== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel --===============1695728040==--