From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, HTML_MESSAGE,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EB97C433FE for ; Fri, 17 Sep 2021 01:07:03 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A9450611EE for ; Fri, 17 Sep 2021 01:07:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A9450611EE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 07E8D6EA54; Fri, 17 Sep 2021 01:07:02 +0000 (UTC) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) by gabe.freedesktop.org (Postfix) with ESMTPS id 788B86EA54 for ; Fri, 17 Sep 2021 01:07:00 +0000 (UTC) Received: by mail-lf1-x12a.google.com with SMTP id x27so25909263lfu.5 for ; Thu, 16 Sep 2021 18:07:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Jsh4uOg+deq2eeZ/agcGKShwP5MDbGUASvB4Cfvw8Ac=; b=lHD3VY83Ir/JQFQCJh5FsIXwjozMWi5qS/ugRn4Ua0RQv1685HB9smWyjpo+QXPbW/ tu1VXDgKqWxfbxmRIvyXmTwKC2tWSAW87IJfeEddmsj6751mYlkaAqLlzvzg8lUD169R YM0yHomeKZmvmuTtmewi/lvfjIqft3VzYyYRE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Jsh4uOg+deq2eeZ/agcGKShwP5MDbGUASvB4Cfvw8Ac=; b=1lDNIYlqXvBxMurpZ9qWxiHzSusV8zWm1iEKYfeGT+r5/ZQfOaspBCcfGdvbeO91QO pT0HPzOqegBupG/CblBYqSAXGjqSoUuwcOdVrHVqud1LUsTI8lZxUaJo3Juyl5fKntKU 6DwZ1dRrYH0pspyjqWvbuytZXU1dcT8fm0WRPxfR9iMScQCA+Lt6H27xvzwXiwXf8pRK VOAZUa4ltLL/2uPB5dagXhCz8HQkOwKzbC2l2oDjlb8Pkpx2gM2cWrDZw/wuM4LGa6+t H0FuwveZWhW8pXF45RmXaOkIsMS+QWTiEFEa1IRBeTuoccP40RZXSJ19cT5j+zr9jSl9 JvMA== X-Gm-Message-State: AOAM531EoOq2glfRY1T1R4iFWzWBLd+Cq2DXJXucDr9FgF5L9HUaREmO dHFBqnV5Eo1dFVGhllZx52tmds7cF1fUeg== X-Google-Smtp-Source: ABdhPJybY8VBpY5k3CYmKnTR0CnWwA1F8/qc9BRgzYFuDKk/cGvai476m3L66mqUGMleRkxM9Z7EBA== X-Received: by 2002:ac2:4e05:: with SMTP id e5mr5908868lfr.632.1631840818174; Thu, 16 Sep 2021 18:06:58 -0700 (PDT) Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com. [209.85.167.50]) by smtp.gmail.com with ESMTPSA id s2sm518985lji.1.2021.09.16.18.06.57 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 16 Sep 2021 18:06:57 -0700 (PDT) Received: by mail-lf1-f50.google.com with SMTP id y28so25844732lfb.0 for ; Thu, 16 Sep 2021 18:06:57 -0700 (PDT) X-Received: by 2002:a2e:bb93:: with SMTP id y19mr7064519lje.79.1631840816595; Thu, 16 Sep 2021 18:06:56 -0700 (PDT) MIME-Version: 1.0 References: <20210909013717.861-1-gurchetansingh@chromium.org> <20210909013717.861-10-gurchetansingh@chromium.org> In-Reply-To: From: Gurchetan Singh Date: Thu, 16 Sep 2021 18:06:45 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [virtio-dev] [PATCH v1 09/12] drm/virtio: implement context init: allocate an array of fence contexts To: Chia-I Wu Cc: ML dri-devel , virtio-dev@lists.oasis-open.org, Gerd Hoffmann Content-Type: multipart/alternative; boundary="00000000000034a7b305cc268cab" X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" --00000000000034a7b305cc268cab Content-Type: text/plain; charset="UTF-8" On Wed, Sep 15, 2021 at 5:11 PM Chia-I Wu wrote: > i > > On Tue, Sep 14, 2021 at 6:26 PM Gurchetan Singh > wrote: > > > > > > > > On Tue, Sep 14, 2021 at 10:53 AM Chia-I Wu wrote: > >> > >> ,On Mon, Sep 13, 2021 at 6:57 PM Gurchetan Singh > >> wrote: > >> > > >> > > >> > > >> > > >> > On Mon, Sep 13, 2021 at 11:52 AM Chia-I Wu wrote: > >> >> > >> >> . > >> >> > >> >> On Mon, Sep 13, 2021 at 10:48 AM Gurchetan Singh > >> >> wrote: > >> >> > > >> >> > > >> >> > > >> >> > On Fri, Sep 10, 2021 at 12:33 PM Chia-I Wu > wrote: > >> >> >> > >> >> >> On Wed, Sep 8, 2021 at 6:37 PM Gurchetan Singh > >> >> >> wrote: > >> >> >> > > >> >> >> > We don't want fences from different 3D contexts (virgl, > gfxstream, > >> >> >> > venus) to be on the same timeline. With explicit context > creation, > >> >> >> > we can specify the number of ring each context wants. > >> >> >> > > >> >> >> > Execbuffer can specify which ring to use. > >> >> >> > > >> >> >> > Signed-off-by: Gurchetan Singh > >> >> >> > Acked-by: Lingfeng Yang > >> >> >> > --- > >> >> >> > drivers/gpu/drm/virtio/virtgpu_drv.h | 3 +++ > >> >> >> > drivers/gpu/drm/virtio/virtgpu_ioctl.c | 34 > ++++++++++++++++++++++++-- > >> >> >> > 2 files changed, 35 insertions(+), 2 deletions(-) > >> >> >> > > >> >> >> > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h > b/drivers/gpu/drm/virtio/virtgpu_drv.h > >> >> >> > index a5142d60c2fa..cca9ab505deb 100644 > >> >> >> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h > >> >> >> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h > >> >> >> > @@ -56,6 +56,7 @@ > >> >> >> > #define STATE_ERR 2 > >> >> >> > > >> >> >> > #define MAX_CAPSET_ID 63 > >> >> >> > +#define MAX_RINGS 64 > >> >> >> > > >> >> >> > struct virtio_gpu_object_params { > >> >> >> > unsigned long size; > >> >> >> > @@ -263,6 +264,8 @@ struct virtio_gpu_fpriv { > >> >> >> > uint32_t ctx_id; > >> >> >> > uint32_t context_init; > >> >> >> > bool context_created; > >> >> >> > + uint32_t num_rings; > >> >> >> > + uint64_t base_fence_ctx; > >> >> >> > struct mutex context_lock; > >> >> >> > }; > >> >> >> > > >> >> >> > diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c > b/drivers/gpu/drm/virtio/virtgpu_ioctl.c > >> >> >> > index f51f3393a194..262f79210283 100644 > >> >> >> > --- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c > >> >> >> > +++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c > >> >> >> > @@ -99,6 +99,11 @@ static int > virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data, > >> >> >> > int in_fence_fd = exbuf->fence_fd; > >> >> >> > int out_fence_fd = -1; > >> >> >> > void *buf; > >> >> >> > + uint64_t fence_ctx; > >> >> >> > + uint32_t ring_idx; > >> >> >> > + > >> >> >> > + fence_ctx = vgdev->fence_drv.context; > >> >> >> > + ring_idx = 0; > >> >> >> > > >> >> >> > if (vgdev->has_virgl_3d == false) > >> >> >> > return -ENOSYS; > >> >> >> > @@ -106,6 +111,17 @@ static int > virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data, > >> >> >> > if ((exbuf->flags & ~VIRTGPU_EXECBUF_FLAGS)) > >> >> >> > return -EINVAL; > >> >> >> > > >> >> >> > + if ((exbuf->flags & VIRTGPU_EXECBUF_RING_IDX)) { > >> >> >> > + if (exbuf->ring_idx >= vfpriv->num_rings) > >> >> >> > + return -EINVAL; > >> >> >> > + > >> >> >> > + if (!vfpriv->base_fence_ctx) > >> >> >> > + return -EINVAL; > >> >> >> > + > >> >> >> > + fence_ctx = vfpriv->base_fence_ctx; > >> >> >> > + ring_idx = exbuf->ring_idx; > >> >> >> > + } > >> >> >> > + > >> >> >> > exbuf->fence_fd = -1; > >> >> >> > > >> >> >> > virtio_gpu_create_context(dev, file); > >> >> >> > @@ -173,7 +189,7 @@ static int > virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data, > >> >> >> > goto out_memdup; > >> >> >> > } > >> >> >> > > >> >> >> > - out_fence = virtio_gpu_fence_alloc(vgdev, > vgdev->fence_drv.context, 0); > >> >> >> > + out_fence = virtio_gpu_fence_alloc(vgdev, fence_ctx, > ring_idx); > >> >> >> > if(!out_fence) { > >> >> >> > ret = -ENOMEM; > >> >> >> > goto out_unresv; > >> >> >> > @@ -691,7 +707,7 @@ static int > virtio_gpu_context_init_ioctl(struct drm_device *dev, > >> >> >> > return -EINVAL; > >> >> >> > > >> >> >> > /* Number of unique parameters supported at this time. > */ > >> >> >> > - if (num_params > 1) > >> >> >> > + if (num_params > 2) > >> >> >> > return -EINVAL; > >> >> >> > > >> >> >> > ctx_set_params = > memdup_user(u64_to_user_ptr(args->ctx_set_params), > >> >> >> > @@ -731,6 +747,20 @@ static int > virtio_gpu_context_init_ioctl(struct drm_device *dev, > >> >> >> > > >> >> >> > vfpriv->context_init |= value; > >> >> >> > break; > >> >> >> > + case VIRTGPU_CONTEXT_PARAM_NUM_RINGS: > >> >> >> > + if (vfpriv->base_fence_ctx) { > >> >> >> > + ret = -EINVAL; > >> >> >> > + goto out_unlock; > >> >> >> > + } > >> >> >> > + > >> >> >> > + if (value > MAX_RINGS) { > >> >> >> > + ret = -EINVAL; > >> >> >> > + goto out_unlock; > >> >> >> > + } > >> >> >> > + > >> >> >> > + vfpriv->base_fence_ctx = > dma_fence_context_alloc(value); > >> >> >> With multiple fence contexts, we should do something about > implicit fencing. > >> >> >> > >> >> >> The classic example is Mesa and X server. When both use virgl > and the > >> >> >> global fence context, no dma_fence_wait is fine. But when Mesa > uses > >> >> >> venus and the ring fence context, dma_fence_wait should be > inserted. > >> >> > > >> >> > > >> >> > If I read your comment correctly, the use case is: > >> >> > > >> >> > context A (venus) > >> >> > > >> >> > sharing a render target with > >> >> > > >> >> > context B (Xserver backed virgl) > >> >> > > >> >> > ? > >> >> > > >> >> > Which function do you envisage dma_fence_wait(...) to be > inserted? Doesn't implicit synchronization mean there's no fence to share > between contexts (only buffer objects)? > >> >> > >> >> Fences can be implicitly shared via reservation objects associated > >> >> with buffer objects. > >> >> > >> >> > It may be possible to wait on the reservation object associated > with a buffer object from a different context (userspace can also do > DRM_IOCTL_VIRTGPU_WAIT), but not sure if that's what you're looking for. > >> >> > >> >> Right, that's what I am looking for. Userspace expects implicit > >> >> fencing to work. While there are works to move the userspace to do > >> >> explicit fencing, it is not there yet in general and we can't require > >> >> the userspace to do explicit fencing or DRM_IOCTL_VIRTGPU_WAIT. > >> > > >> > > >> > Another option would be to use the upcoming > DMA_BUF_IOCTL_EXPORT_SYNC_FILE + VIRTGPU_EXECBUF_FENCE_FD_IN (which checks > the dma_fence context). > >> That requires the X server / compositors to be modified. For example, > >> venus works under Android (where there is explicit fencing) or under a > >> modified compositor (which does DMA_BUF_IOCTL_EXPORT_SYNC_FILE or > >> DRM_IOCTL_VIRTGPU_WAIT). But it does not work too well with an > >> unmodified X server. > > > > > > Some semi-recent virgl modifications will be needed regardless for > interop, such as VIRGL_CAP_V2_UNTYPED_RESOURCE (?). > > > > Not sure aren't too many virgl users (most developers) > > > > Does Xserver just pick up the latest Mesa release (including > virgl/venus)? Suppose context types land in 5.16, the userspace changes > land (both Venus/Virgl) in 21.2 stable releases. > > > > https://docs.mesa3d.org/release-calendar.html > > > >> > >> > > >> > Generally, if it only requires virgl changes, userspace changes are > fine since OpenGL drivers implement implicit sync in many ways. Waiting on > the reservation object in the kernel is fine too though. > >> I don't think we want to assume virgl to be the only consumer of > >> dma-bufs, despite that it is the most common use case. > >> > >> > >> > > >> > Though venus doesn't use the NUM_RINGS param yet. Getting all > permutations of context type + display integration working would take some > time (patchset mostly tested with wayland + gfxstream/Android [no implicit > sync]). > >> > > >> > WDYT of someone figuring out virgl/venus interop later, independently > of this patchset? > >> > >> I think we should understand the implications of multiple fence > >> contexts better, even if some changes are not included in this > >> patchset. > >> > >> From my view, we don't need implicit fencing in most cases and > >> implicit fencing should be considered a legacy path. But X server / > >> compositors today happen to require it. Other drivers seem to use a > >> flag to control whether implicit fences are set up or waited (e.g., > >> AMDGPU_GEM_CREATE_EXPLICIT_SYNC, MSM_SUBMIT_NO_IMPLICIT, or > >> EXEC_OBJECT_WRITE). It seems to be the least surprising thing to do. > > > > > > IMO, the easiest way is just to limit the change to userspace if > possible since implicit sync is legacy/something we want to deprecate over > time. > > > > Another option is to add something like VIRTGPU_EXECBUF_EXPLICIT_SYNC > (similar to MSM_SUBMIT_NO_IMPLICIT), where the reservation objects are > waited on / added to without that flag. Since explicit sync will need new > hypercalls/params and is a major, that feature is expected to be > independent of context types. > > > > With that option, waiting on the reservation object would just be > another bug fix + addition to 5.16 (perhaps by you) so we can proceed in > parallel faster. VIRTGPU_EXECBUF_EXPLICIT_SYNC (or an equivalent) would be > added later. > > It is fine to add it later, but VIRTGPU_EXECBUF_EXPLICIT_SYNC sounds > better to me than a userspace solution. I don't think we need a new > hypercall as the wait can be a guest-side wait, similar to how > VIRTGPU_EXECBUF_FENCE_FD_IN is a guest-side wait. The flag can also > suppress VIRTIO_GPU_FLAG_FENCE and make the SUBMIT_3D hypercall > cheaper. > Okay, I will add a note regarding the plan in patch 9 of v2 for context types that need implicit sync. > > Even before this patchset, unless I miss something, it seems the fact > that we have a global fence context and assume all host GL contexts > are on the same timeline is not exactly correct. When glFlush is > called on two host GL contexts, the flush order is not exactly the > same as the execution order. But that is a different issue that can > be solved in virglrenderer. > > > > > >> > >> > >> > > >> >> > >> >> > >> >> > >> >> > >> >> > > >> >> >> > >> >> >> > >> >> >> > + vfpriv->num_rings = value; > >> >> >> > + break; > >> >> >> > default: > >> >> >> > ret = -EINVAL; > >> >> >> > goto out_unlock; > >> >> >> > -- > >> >> >> > 2.33.0.153.gba50c8fa24-goog > >> >> >> > > >> >> >> > > >> >> >> > > --------------------------------------------------------------------- > >> >> >> > To unsubscribe, e-mail: > virtio-dev-unsubscribe@lists.oasis-open.org > >> >> >> > For additional commands, e-mail: > virtio-dev-help@lists.oasis-open.org > >> >> >> > > --00000000000034a7b305cc268cab Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Wed, Sep 15, 2021 at 5:11 PM Chia-= I Wu <olvaffe@gmail.com> wro= te:
=C2=A0i

On Tue, Sep 14, 2021 at 6:26 PM Gurchetan Singh
<gurche= tansingh@chromium.org> wrote:
>
>
>
> On Tue, Sep 14, 2021 at 10:53 AM Chia-I Wu <olvaffe@gmail.com> wrote:
>>
>> ,On Mon, Sep 13, 2021 at 6:57 PM Gurchetan Singh
>> <gurchetansingh@chromium.org> wrote:
>> >
>> >
>> >
>> >
>> > On Mon, Sep 13, 2021 at 11:52 AM Chia-I Wu <olvaffe@gmail.com> wrote: >> >>
>> >> .
>> >>
>> >> On Mon, Sep 13, 2021 at 10:48 AM Gurchetan Singh
>> >> <gurchetansingh@chromium.org> wrote:
>> >> >
>> >> >
>> >> >
>> >> > On Fri, Sep 10, 2021 at 12:33 PM Chia-I Wu <olvaffe@gmail.com>= wrote:
>> >> >>
>> >> >> On Wed, Sep 8, 2021 at 6:37 PM Gurchetan Singh >> >> >> <gurchetansingh@chromium.org> wrote:
>> >> >> >
>> >> >> > We don't want fences from different 3D = contexts (virgl, gfxstream,
>> >> >> > venus) to be on the same timeline.=C2=A0 Wi= th explicit context creation,
>> >> >> > we can specify the number of ring each cont= ext wants.
>> >> >> >
>> >> >> > Execbuffer can specify which ring to use. >> >> >> >
>> >> >> > Signed-off-by: Gurchetan Singh <gurchetansingh@ch= romium.org>
>> >> >> > Acked-by: Lingfeng Yang <lfy@google.com>
>> >> >> > ---
>> >> >> >=C2=A0 drivers/gpu/drm/virtio/virtgpu_drv.h= =C2=A0 =C2=A0|=C2=A0 3 +++
>> >> >> >=C2=A0 drivers/gpu/drm/virtio/virtgpu_ioctl.= c | 34 ++++++++++++++++++++++++--
>> >> >> >=C2=A0 2 files changed, 35 insertions(+), 2 = deletions(-)
>> >> >> >
>> >> >> > diff --git a/drivers/gpu/drm/virtio/virtgpu= _drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> >> >> > index a5142d60c2fa..cca9ab505deb 100644
>> >> >> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h<= br> >> >> >> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h<= br> >> >> >> > @@ -56,6 +56,7 @@
>> >> >> >=C2=A0 #define STATE_ERR 2
>> >> >> >
>> >> >> >=C2=A0 #define MAX_CAPSET_ID 63
>> >> >> > +#define MAX_RINGS 64
>> >> >> >
>> >> >> >=C2=A0 struct virtio_gpu_object_params {
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0unsigned l= ong size;
>> >> >> > @@ -263,6 +264,8 @@ struct virtio_gpu_fpriv= {
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t c= tx_id;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t c= ontext_init;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bool conte= xt_created;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t num_ri= ngs;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0uint64_t base_f= ence_ctx;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0struct mut= ex context_lock;
>> >> >> >=C2=A0 };
>> >> >> >
>> >> >> > diff --git a/drivers/gpu/drm/virtio/virtgpu= _ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
>> >> >> > index f51f3393a194..262f79210283 100644
>> >> >> > --- a/drivers/gpu/drm/virtio/virtgpu_ioctl.= c
>> >> >> > +++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.= c
>> >> >> > @@ -99,6 +99,11 @@ static int virtio_gpu_ex= ecbuffer_ioctl(struct drm_device *dev, void *data,
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0int in_fen= ce_fd =3D exbuf->fence_fd;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0int out_fe= nce_fd =3D -1;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0void *buf;=
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0uint64_t fence_= ctx;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t ring_i= dx;
>> >> >> > +
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0fence_ctx =3D v= gdev->fence_drv.context;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0ring_idx =3D 0;=
>> >> >> >
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (vgdev-= >has_virgl_3d =3D=3D false)
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0return -ENOSYS;
>> >> >> > @@ -106,6 +111,17 @@ static int virtio_gpu_= execbuffer_ioctl(struct drm_device *dev, void *data,
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if ((exbuf= ->flags & ~VIRTGPU_EXECBUF_FLAGS))
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0return -EINVAL;
>> >> >> >
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0if ((exbuf->= flags & VIRTGPU_EXECBUF_RING_IDX)) {
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0if (exbuf->ring_idx >=3D vfpriv->num_rings)
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
>> >> >> > +
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0if (!vfpriv->base_fence_ctx)
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
>> >> >> > +
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0fence_ctx =3D vfpriv->base_fence_ctx;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0ring_idx =3D exbuf->ring_idx;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0}
>> >> >> > +
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0exbuf->= fence_fd =3D -1;
>> >> >> >
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0virtio_gpu= _create_context(dev, file);
>> >> >> > @@ -173,7 +189,7 @@ static int virtio_gpu_e= xecbuffer_ioctl(struct drm_device *dev, void *data,
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto out_memdup;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
>> >> >> >
>> >> >> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0out_fence =3D v= irtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0out_fence =3D v= irtio_gpu_fence_alloc(vgdev, fence_ctx, ring_idx);
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if(!out_fe= nce) {
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0ret =3D -ENOMEM;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0goto out_unresv;
>> >> >> > @@ -691,7 +707,7 @@ static int virtio_gpu_c= ontext_init_ioctl(struct drm_device *dev,
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0return -EINVAL;
>> >> >> >
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* Number = of unique parameters supported at this time. */
>> >> >> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0if (num_params = > 1)
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0if (num_params = > 2)
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0return -EINVAL;
>> >> >> >
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ctx_set_pa= rams =3D memdup_user(u64_to_user_ptr(args->ctx_set_params),
>> >> >> > @@ -731,6 +747,20 @@ static int virtio_gpu_= context_init_ioctl(struct drm_device *dev,
>> >> >> >
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vfpriv->context_init |= =3D value;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0break;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0case VIRTGPU_CONTEXT_PARAM_NUM_RINGS:
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (vfpriv->base_fence_ctx) { >> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ret = =3D -EINVAL;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto o= ut_unlock;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
>> >> >> > +
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (value > MAX_RINGS) {
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ret = =3D -EINVAL;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto o= ut_unlock;
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
>> >> >> > +
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vfpriv->base_fence_ctx =3D dma_= fence_context_alloc(value);
>> >> >> With multiple fence contexts, we should do somet= hing about implicit fencing.
>> >> >>
>> >> >> The classic example is Mesa and X server.=C2=A0 = When both use virgl and the
>> >> >> global fence context, no dma_fence_wait is fine.= =C2=A0 But when Mesa uses
>> >> >> venus and the ring fence context, dma_fence_wait= should be inserted.
>> >> >
>> >> >
>> >> >=C2=A0 If I read your comment correctly, the use case= is:
>> >> >
>> >> > context A (venus)
>> >> >
>> >> > sharing a render target with
>> >> >
>> >> > context B (Xserver backed virgl)
>> >> >
>> >> > ?
>> >> >
>> >> > Which function do you envisage dma_fence_wait(...) t= o be inserted?=C2=A0 Doesn't implicit synchronization mean there's = no fence to share between contexts (only buffer objects)?
>> >>
>> >> Fences can be implicitly shared via reservation objects a= ssociated
>> >> with buffer objects.
>> >>
>> >> > It may be possible to wait on the reservation object= associated with a buffer object from a different context (userspace can al= so do DRM_IOCTL_VIRTGPU_WAIT), but not sure if that's what you're l= ooking for.
>> >>
>> >> Right, that's what I am looking for.=C2=A0 Userspace = expects implicit
>> >> fencing to work.=C2=A0 While there are works to move the = userspace to do
>> >> explicit fencing, it is not there yet in general and we c= an't require
>> >> the userspace to do explicit fencing or DRM_IOCTL_VIRTGPU= _WAIT.
>> >
>> >
>> > Another option would be to use the upcoming DMA_BUF_IOCTL_EXP= ORT_SYNC_FILE + VIRTGPU_EXECBUF_FENCE_FD_IN (which checks the dma_fence con= text).
>> That requires the X server / compositors to be modified.=C2=A0 For= example,
>> venus works under Android (where there is explicit fencing) or und= er a
>> modified compositor (which does DMA_BUF_IOCTL_EXPORT_SYNC_FILE or<= br> >> DRM_IOCTL_VIRTGPU_WAIT).=C2=A0 But it does not work too well with = an
>> unmodified X server.
>
>
> Some semi-recent virgl modifications will be needed regardless for int= erop, such as VIRGL_CAP_V2_UNTYPED_RESOURCE (?).
>
> Not sure aren't too many virgl users (most developers)
>
> Does Xserver just pick up the latest Mesa release (including virgl/ven= us)?=C2=A0 Suppose context types land in 5.16, the userspace changes land (= both Venus/Virgl) in 21.2 stable releases.
>
> https://docs.mesa3d.org/release-calendar.html<= br> >
>>
>> >
>> > Generally, if it only requires virgl changes, userspace chang= es are fine since OpenGL drivers implement implicit sync in many ways.=C2= =A0 Waiting on the reservation object in the kernel is fine too though.
>> I don't think we want to assume virgl to be the only consumer = of
>> dma-bufs, despite that it is the most common use case.
>>
>>
>> >
>> > Though venus doesn't use the NUM_RINGS param yet.=C2=A0 G= etting all permutations of context type + display integration working would= take some time (patchset mostly tested with wayland + gfxstream/Android [n= o implicit sync]).
>> >
>> > WDYT of someone figuring out virgl/venus interop later, indep= endently of this patchset?
>>
>> I think we should understand the implications of multiple fence >> contexts better, even if some changes are not included in this
>> patchset.
>>
>> From my view, we don't need implicit fencing in most cases and=
>> implicit fencing should be considered a legacy path.=C2=A0 But X s= erver /
>> compositors today happen to require it.=C2=A0 Other drivers seem t= o use a
>> flag to control whether implicit fences are set up or waited (e.g.= ,
>> AMDGPU_GEM_CREATE_EXPLICIT_SYNC, MSM_SUBMIT_NO_IMPLICIT, or
>> EXEC_OBJECT_WRITE).=C2=A0 It seems to be the least surprising thin= g to do.
>
>
> IMO, the easiest way is just to limit the change to userspace if possi= ble since implicit sync is legacy/something we want to deprecate over time.=
>
> Another option is to add something like VIRTGPU_EXECBUF_EXPLICIT_SYNC = (similar to MSM_SUBMIT_NO_IMPLICIT), where the reservation objects are wait= ed on / added to without that flag.=C2=A0 Since explicit sync will need new= hypercalls/params and is a major, that feature is expected to be independe= nt of context types.
>
> With that option, waiting on the reservation object would just be anot= her bug fix + addition to 5.16 (perhaps by you) so we can proceed in parall= el faster.=C2=A0 VIRTGPU_EXECBUF_EXPLICIT_SYNC (or an equivalent) would be = added later.

It is fine to add it later, but VIRTGPU_EXECBUF_EXPLICIT_SYNC sounds
better to me than a userspace solution.=C2=A0 I don't think we need a n= ew
hypercall as the wait can be a guest-side wait, similar to how
VIRTGPU_EXECBUF_FENCE_FD_IN is a guest-side wait.=C2=A0 The flag can also suppress VIRTIO_GPU_FLAG_FENCE and make the SUBMIT_3D hypercall
cheaper.

Okay, I will add a note regard= ing the plan in patch 9 of v2 for context types that need implicit sync.
=C2=A0

Even before this patchset, unless I miss something, it seems the fact
that we have a global fence context and assume all host GL contexts
are on the same timeline is not exactly correct.=C2=A0 When glFlush is
called on two host GL contexts, the flush order is not exactly the
same as the execution order.=C2=A0 But that is a different issue that can be solved in virglrenderer.


>
>>
>>
>> >
>> >>
>> >>
>> >>
>> >>
>> >> >
>> >> >>
>> >> >>
>> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vfpriv->num_rings =3D value; >> >> >> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0break;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0default:
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ret =3D -EINVAL;
>> >> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto out_unlock;
>> >> >> > --
>> >> >> > 2.33.0.153.gba50c8fa24-goog
>> >> >> >
>> >> >> >
>> >> >> > -------------------------------------------= --------------------------
>> >> >> > To unsubscribe, e-mail: virtio-dev-un= subscribe@lists.oasis-open.org
>> >> >> > For additional commands, e-mail: virtio-dev-= help@lists.oasis-open.org
>> >> >> >
--00000000000034a7b305cc268cab--