From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13C74C2D0D1 for ; Mon, 6 Jan 2020 10:30:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5EF3020731 for ; Mon, 6 Jan 2020 10:30:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="iy3acYcX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726080AbgAFKah (ORCPT ); Mon, 6 Jan 2020 05:30:37 -0500 Received: from mail-lj1-f193.google.com ([209.85.208.193]:39496 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725821AbgAFKag (ORCPT ); Mon, 6 Jan 2020 05:30:36 -0500 Received: by mail-lj1-f193.google.com with SMTP id l2so50407915lja.6 for ; Mon, 06 Jan 2020 02:30:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xRwHkpnYCU80CqiFriSq26/Ueft4nTQ1j9puNzPJVzc=; b=iy3acYcXY84jtCWQ6YvJtd8ju46EL9DaFUwbhkqPEHuujsbB7k9MHqjzfKoi4T53A8 ZZiDDDrbwo5HUj+cmTQhawaZmfZzneIyimcMbQ4jrOSNpcc6FLiOskevof/BmaNtd5Tp a6T3s/sXM0iwgIBIElx6vfRGgbKFMPxs/BUu4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xRwHkpnYCU80CqiFriSq26/Ueft4nTQ1j9puNzPJVzc=; b=iHIARM+TCBjokwAWMpYLTzngtUkun/Fww7Jm+YdJGQwm3fKu2invlUZCI70pLLhCKG 63c+/+HlJlUd8usu4/ezQmos5PgZ6RRhiM2vk1ZplZ5CNjE8KJgFYO5kjakyMafmD25F A9o46RVoC0LFEYPwCzvQBNMHfgLfFB4UKhKi5KvR899l2GLjkQR9udZdAd8aEVVwsO8u GQwoGaHPkF8N/xFCasfu+r2cOuFkdRUJehBzQELpoLr8Ah8RrHX5B+/m59FZcpdXUNzX fZm9i6oqlGco9rHxRoyGm4YUY/7JV44pl7pXV3N0zjOpL2cJzAevp094ERBnH3R+g8oB Jpvw== X-Gm-Message-State: APjAAAXSUnS7VEb5enP0kYac1HwMvQBo/kIERNa+0c7uYtLhapny4k4E Jh1vRf+wXxu08gThSvYby9HIpt01eAkjYF0poA83bQ== X-Google-Smtp-Source: APXvYqym4TugE7eYBCSm5iKyuwnULB5X20dwN7X0c9gpX6iyBuwSVp8/2FKQmr7WrPdc1N7hZW9TPPycokTodv56BEc= X-Received: by 2002:a2e:9cd8:: with SMTP id g24mr59248816ljj.243.1578306633543; Mon, 06 Jan 2020 02:30:33 -0800 (PST) MIME-Version: 1.0 References: <20191218130214.170703-1-keiichiw@chromium.org> <3016670.ToaXtcqt80@os-lin-dmo> In-Reply-To: <3016670.ToaXtcqt80@os-lin-dmo> From: Keiichi Watanabe Date: Mon, 6 Jan 2020 19:30:22 +0900 Message-ID: Subject: Re: [PATCH v2 0/1] VirtIO video device specification To: Dmitry Sepp Cc: Tomasz Figa , virtio-dev@lists.oasis-open.org, Linux Media Mailing List , Alexandre Courbot , Alex Lau , Daniel Vetter , Dylan Reid , Enrico Granata , Frediano Ziglio , Hans Verkuil , Gerd Hoffmann , =?UTF-8?Q?St=C3=A9phane_Marchesin?= , Pawel Osciak , spice-devel@lists.freedesktop.org, David Stevens , uril@redhat.com Content-Type: text/plain; charset="UTF-8" Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Hi Dmitry, Tomasz, On Fri, Jan 3, 2020 at 10:05 PM Dmitry Sepp wrote: > > Hi Tomasz, Keiichi, > > On Samstag, 21. Dezember 2019 07:19:23 CET Tomasz Figa wrote: > > On Sat, Dec 21, 2019 at 3:18 PM Tomasz Figa wrote: > > > On Sat, Dec 21, 2019 at 1:36 PM Keiichi Watanabe > wrote: > > > > Hi Dmitry, > > > > > > > > On Sat, Dec 21, 2019 at 12:59 AM Dmitry Sepp > > > > > > > > wrote: > > > > > Hi Keiichi, > > > > > > > > > > On Mittwoch, 18. Dezember 2019 14:02:13 CET Keiichi Watanabe wrote: > > > > > > Hi, > > > > > > This is the 2nd version of virtio-video patch. The PDF is available > > > > > > in [1]. > > > > > > The first version was sent at [2]. > > > > > > > > > > > > Any feedback would be appreciated. Thank you. > > > > > > > > > > > > Best, > > > > > > Keiichi > > > > > > > > > > > > [1]: > > > > > > https://drive.google.com/drive/folders/1eT5fEckBoor2iHZR4f4GLxYzFMVa > > > > > > pOFx?us > > > > > > p=sharing [2]: https://markmail.org/message/gc6h25acct22niut > > > > > > > > > > > > Change log: > > > > > > > > > > > > v2: > > > > > > * Removed functionalities except encoding and decoding. > > > > > > * Splited encoder and decoder into different devices that use the > > > > > > same > > > > > > protocol. * Replaced GET_FUNCS with GET_CAPABILITY. > > > > > > * Updated structs for capabilities. > > > > > > > > > > > > - Defined new structs and enums such as image formats, profiles, > > > > > > range > > > > > > > > > > > > (min, max, step), etc > > > > > > > > > > > > * For virtio_video_pixel_format, chose a naming convention that > > > > > > is used > > > > > > > > > > > > in DRM. We removed XBGR, NV21 and I422, as they are not used > > > > > > in the > > > > > > current draft implementation. https://lwn.net/Articles/806416/ > > > > > > > > > > > > - Removed virtio_video_control, whose usage was not documented yet > > > > > > and > > > > > > > > > > > > which is not necessary for the simplest decoding scenario. > > > > > > > > > > > > - Removed virtio_video_desc, as it is no longer needed. > > > > > > > > > > > > * Updated struct virtio_video_config for changes around > > > > > > capabilities. > > > > > > * Added a way to represent supported combinations of formats. > > > > > > > > > > > > - A field "mask" in virtio_video_format_desc plays this role. > > > > > > > > > > > > * Removed VIRTIO_VIDEO_T_STREAM_{START,STOP} because they don't play > > > > > > any > > > > > > meaningful roles. * Removed VIRTIO_VIDEO_T_STREAM_{ATTACH, > > > > > > DETACH}_BACKING > > > > > > and merged them into RESOURCE_{CREATE, DESTROY}. * Added a way to > > > > > > notify/specify resource creation method. > > > > > > > > > > > > - Added a feature flag. > > > > > > - Defined enum virtio_video_mem_type. > > > > > > - Added new fields in video_stream_create. > > > > > > > > > > > > * Modified fields in virtio_video_params. > > > > > > > > > > > > - Added crop information. > > > > > > > > > > > > * Removed enum virtio_video_channel_type because we can get this > > > > > > information by image format. > > > > > > > > > > Could you please explain this? How do you get the information? > > > > > > > > It means that if image formats are well-defined, channel information > > > > (e.g. the order of channels) is uniquely determined. > > > > > > > > > Suppose you have some piece of HW on the host side that wants I420 as > > > > > one > > > > > contig buffer w/ some offsets. But on the driver side, say, gralloc > > > > > gives you three separate buffers, one per channel. How do we pass > > > > > those to the device then? > > > > > > > > You're talking about CrOS use case where buffers are allocated by > > > > virtio-gpu, right? > > > > In this case, virtio-gpu allocates one contiguous host-side buffer and > > > > the client regards a pair of (buffer FD, offset) as one channel. > > > > And, we can register this pair to the device when the buffer is > > > > imported. > > > > In the virtio-vdec spec draft, this pair corresponds to struct > > > > virtio_vdec_plane in struct virtio_vdec_plane. > > > > > > > > So, I suppose we will need similar structs when we add a control to > > > > import buffers. However, I don't think it's necessary when guest pages > > > > are used. > > > > > > I think we need some way for the guest to know whether it can allocate > > > the planes in separate buffers, even when guest pages are used. This > > > would be equivalent to V4L2 M and non-M formats, but mixing this into > > > FourCC in V4L2 is an acknowledged mistake, so we should add a query or > > > something. > > > > > Yes, this is what I mean. In fact, we already do face the situation when the > device side is not happy with the sgt and wants contig. I think we'll add a > module parameter for now. Okay. So, I suppose we'll be able to update structs: * Add a flag in virtio_video_format_desc that indicates whether planes can be in separate buffers, and * Add a flag in virtio_video_format_desc that indicates that the device requires contiguous buffers for this format. Does it make sense? Best regards, Keiichi > > Regards, > Dmitry. > > > > For future V4L2 development we came up with the idea of a format flag > > > which could mean that the hardware allows putting planes in separate > > > buffers. We could have a similar per-format flag in the capabilities, > > > as we already have a list of all the supported formats there. > > > > Sorry, forgot to paste the link from future V4L2 work notes from this year > > ELCE: https://www.spinics.net/lists/linux-media/msg159789.html > > > > > Best regards, > > > Tomasz > > > > > > > Best regards, > > > > Keiichi > > > > > > > > > Best regards, > > > > > Dmitry. > > > > > > > > > > > * Renamed virtio_video_pin to virtio_video_buf_type. > > > > > > > > > > > > - It's similar to V4L2_BUF_TYPE_VIDEO_{OUTPUT, CAPTURE}. > > > > > > > > > > > > * Added an error event. > > > > > > * Reordered some subsections. > > > > > > * Changed styles to make it consistent with other devices. > > > > > > > > > > > > Dmitry Sepp (1): > > > > > > virtio-video: Add virtio video device specification > > > > > > > > > > > > content.tex | 1 + > > > > > > virtio-video.tex | 579 > > > > > > +++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > 2 files changed, 580 insertions(+) > > > > > > create mode 100644 virtio-video.tex > > > > > > > > > > > > -- > > > > > > 2.24.1.735.g03f4e72817-goog > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: virtio-dev-return-6600-cohuck=redhat.com@lists.oasis-open.org Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [10.110.1.242]) by lists.oasis-open.org (Postfix) with ESMTP id 418C9984400 for ; Mon, 6 Jan 2020 10:30:35 +0000 (UTC) MIME-Version: 1.0 References: <20191218130214.170703-1-keiichiw@chromium.org> <3016670.ToaXtcqt80@os-lin-dmo> In-Reply-To: <3016670.ToaXtcqt80@os-lin-dmo> From: Keiichi Watanabe Date: Mon, 6 Jan 2020 19:30:22 +0900 Message-ID: Subject: [virtio-dev] Re: [PATCH v2 0/1] VirtIO video device specification Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable To: Dmitry Sepp Cc: Tomasz Figa , virtio-dev@lists.oasis-open.org, Linux Media Mailing List , Alexandre Courbot , Alex Lau , Daniel Vetter , Dylan Reid , Enrico Granata , Frediano Ziglio , Hans Verkuil , Gerd Hoffmann , =?UTF-8?Q?St=C3=A9phane_Marchesin?= , Pawel Osciak , spice-devel@lists.freedesktop.org, David Stevens , uril@redhat.com List-ID: Hi Dmitry, Tomasz, On Fri, Jan 3, 2020 at 10:05 PM Dmitry Sepp w= rote: > > Hi Tomasz, Keiichi, > > On Samstag, 21. Dezember 2019 07:19:23 CET Tomasz Figa wrote: > > On Sat, Dec 21, 2019 at 3:18 PM Tomasz Figa wrote: > > > On Sat, Dec 21, 2019 at 1:36 PM Keiichi Watanabe > wrote: > > > > Hi Dmitry, > > > > > > > > On Sat, Dec 21, 2019 at 12:59 AM Dmitry Sepp > > > > > > > > wrote: > > > > > Hi Keiichi, > > > > > > > > > > On Mittwoch, 18. Dezember 2019 14:02:13 CET Keiichi Watanabe wrot= e: > > > > > > Hi, > > > > > > This is the 2nd version of virtio-video patch. The PDF is avail= able > > > > > > in [1]. > > > > > > The first version was sent at [2]. > > > > > > > > > > > > Any feedback would be appreciated. Thank you. > > > > > > > > > > > > Best, > > > > > > Keiichi > > > > > > > > > > > > [1]: > > > > > > https://drive.google.com/drive/folders/1eT5fEckBoor2iHZR4f4GLxY= zFMVa > > > > > > pOFx?us > > > > > > p=3Dsharing [2]: https://markmail.org/message/gc6h25acct22niut > > > > > > > > > > > > Change log: > > > > > > > > > > > > v2: > > > > > > * Removed functionalities except encoding and decoding. > > > > > > * Splited encoder and decoder into different devices that use t= he > > > > > > same > > > > > > protocol. * Replaced GET_FUNCS with GET_CAPABILITY. > > > > > > * Updated structs for capabilities. > > > > > > > > > > > > - Defined new structs and enums such as image formats, profil= es, > > > > > > range > > > > > > > > > > > > (min, max, step), etc > > > > > > > > > > > > * For virtio_video_pixel_format, chose a naming convention = that > > > > > > is used > > > > > > > > > > > > in DRM. We removed XBGR, NV21 and I422, as they are not u= sed > > > > > > in the > > > > > > current draft implementation. https://lwn.net/Articles/80= 6416/ > > > > > > > > > > > > - Removed virtio_video_control, whose usage was not documente= d yet > > > > > > and > > > > > > > > > > > > which is not necessary for the simplest decoding scenario. > > > > > > > > > > > > - Removed virtio_video_desc, as it is no longer needed. > > > > > > > > > > > > * Updated struct virtio_video_config for changes around > > > > > > capabilities. > > > > > > * Added a way to represent supported combinations of formats. > > > > > > > > > > > > - A field "mask" in virtio_video_format_desc plays this role. > > > > > > > > > > > > * Removed VIRTIO_VIDEO_T_STREAM_{START,STOP} because they don't= play > > > > > > any > > > > > > meaningful roles. * Removed VIRTIO_VIDEO_T_STREAM_{ATTACH, > > > > > > DETACH}_BACKING > > > > > > and merged them into RESOURCE_{CREATE, DESTROY}. * Added a way = to > > > > > > notify/specify resource creation method. > > > > > > > > > > > > - Added a feature flag. > > > > > > - Defined enum virtio_video_mem_type. > > > > > > - Added new fields in video_stream_create. > > > > > > > > > > > > * Modified fields in virtio_video_params. > > > > > > > > > > > > - Added crop information. > > > > > > > > > > > > * Removed enum virtio_video_channel_type because we can get thi= s > > > > > > information by image format. > > > > > > > > > > Could you please explain this? How do you get the information? > > > > > > > > It means that if image formats are well-defined, channel informatio= n > > > > (e.g. the order of channels) is uniquely determined. > > > > > > > > > Suppose you have some piece of HW on the host side that wants I42= 0 as > > > > > one > > > > > contig buffer w/ some offsets. But on the driver side, say, grall= oc > > > > > gives you three separate buffers, one per channel. How do we pass > > > > > those to the device then? > > > > > > > > You're talking about CrOS use case where buffers are allocated by > > > > virtio-gpu, right? > > > > In this case, virtio-gpu allocates one contiguous host-side buffer = and > > > > the client regards a pair of (buffer FD, offset) as one channel. > > > > And, we can register this pair to the device when the buffer is > > > > imported. > > > > In the virtio-vdec spec draft, this pair corresponds to struct > > > > virtio_vdec_plane in struct virtio_vdec_plane. > > > > > > > > So, I suppose we will need similar structs when we add a control to > > > > import buffers. However, I don't think it's necessary when guest pa= ges > > > > are used. > > > > > > I think we need some way for the guest to know whether it can allocat= e > > > the planes in separate buffers, even when guest pages are used. This > > > would be equivalent to V4L2 M and non-M formats, but mixing this into > > > FourCC in V4L2 is an acknowledged mistake, so we should add a query o= r > > > something. > > > > > Yes, this is what I mean. In fact, we already do face the situation when = the > device side is not happy with the sgt and wants contig. I think we'll add= a > module parameter for now. Okay. So, I suppose we'll be able to update structs: * Add a flag in virtio_video_format_desc that indicates whether planes can be in separate buffers, and * Add a flag in virtio_video_format_desc that indicates that the device requires contiguous buffers for this format. Does it make sense? Best regards, Keiichi > > Regards, > Dmitry. > > > > For future V4L2 development we came up with the idea of a format flag > > > which could mean that the hardware allows putting planes in separate > > > buffers. We could have a similar per-format flag in the capabilities, > > > as we already have a list of all the supported formats there. > > > > Sorry, forgot to paste the link from future V4L2 work notes from this y= ear > > ELCE: https://www.spinics.net/lists/linux-media/msg159789.html > > > > > Best regards, > > > Tomasz > > > > > > > Best regards, > > > > Keiichi > > > > > > > > > Best regards, > > > > > Dmitry. > > > > > > > > > > > * Renamed virtio_video_pin to virtio_video_buf_type. > > > > > > > > > > > > - It's similar to V4L2_BUF_TYPE_VIDEO_{OUTPUT, CAPTURE}. > > > > > > > > > > > > * Added an error event. > > > > > > * Reordered some subsections. > > > > > > * Changed styles to make it consistent with other devices. > > > > > > > > > > > > Dmitry Sepp (1): > > > > > > virtio-video: Add virtio video device specification > > > > > > > > > > > > content.tex | 1 + > > > > > > virtio-video.tex | 579 > > > > > > +++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > 2 files changed, 580 insertions(+) > > > > > > create mode 100644 virtio-video.tex > > > > > > > > > > > > -- > > > > > > 2.24.1.735.g03f4e72817-goog > > --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org