From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from lb2-smtp-cloud7.xs4all.net ([194.109.24.28]:47673 "EHLO lb2-smtp-cloud7.xs4all.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727727AbeIJRUF (ORCPT ); Mon, 10 Sep 2018 13:20:05 -0400 Subject: Re: [Xen-devel][PATCH 1/1] cameraif: add ABI for para-virtual camera To: Oleksandr Andrushchenko , "Oleksandr_Andrushchenko@epam.com" , xen-devel@lists.xenproject.org, konrad.wilk@oracle.com, jgross@suse.com, boris.ostrovsky@oracle.com, mchehab@kernel.org, linux-media@vger.kernel.org, sakari.ailus@linux.intel.com, koji.matsuoka.xm@renesas.com References: <20180731093142.3828-1-andr2000@gmail.com> <20180731093142.3828-2-andr2000@gmail.com> <73b69e31-d36d-d89f-20d6-d59dbefe395e@xs4all.nl> <7134b3ad-9fcf-0139-41b3-67a3dbc8224d@xs4all.nl> <51f97715-454a-0242-b381-29944d77d5b5@gmail.com> <3c6bb5c8-eeb4-fd09-407a-5a77b29b56c3@xs4all.nl> <2a39c994-118f-a17e-c40a-f5fbbad1cb03@epam.com> <30d7c91a-4515-157b-fc29-90c2e6f0008b@xs4all.nl> From: Hans Verkuil Message-ID: Date: Mon, 10 Sep 2018 14:26:09 +0200 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-media-owner@vger.kernel.org List-ID: On 09/10/2018 01:49 PM, Oleksandr Andrushchenko wrote: > On 09/10/2018 02:09 PM, Hans Verkuil wrote: >> On 09/10/2018 11:52 AM, Oleksandr Andrushchenko wrote: >>> On 09/10/2018 12:04 PM, Hans Verkuil wrote: >>>> On 09/10/2018 10:24 AM, Oleksandr Andrushchenko wrote: >>>>> On 09/10/2018 10:53 AM, Hans Verkuil wrote: >>>>>> Hi Oleksandr, >>>>>> >>>>>> On 09/10/2018 09:16 AM, Oleksandr Andrushchenko wrote: >>>> >>>> >>>>>>>> I suspect that you likely will want to support such sources eventually, so >>>>>>>> it pays to design this with that in mind. >>>>>>> Again, I think that this is the backend to hide these >>>>>>> use-cases from the frontend. >>>>>> I'm not sure you can: say you are playing a bluray connected to the system >>>>>> with HDMI, then if there is a resolution change, what do you do? You can tear >>>>>> everything down and build it up again, or you can just tell frontends that >>>>>> something changed and that they have to look at the new vcamera configuration. >>>>>> >>>>>> The latter seems to be more sensible to me. It is really not much that you >>>>>> need to do: all you really need is an event signalling that something changed. >>>>>> In V4L2 that's the V4L2_EVENT_SOURCE_CHANGE. >>>>> well, this complicates things a lot as I'll have to >>>>> re-allocate buffers - right? >>>> Right. Different resolutions means different sized buffers and usually lots of >>>> changes throughout the whole video pipeline, which in this case can even >>>> go into multiple VMs. >>>> >>>> One additional thing to keep in mind for the future: V4L2_EVENT_SOURCE_CHANGE >>>> has a flags field that tells userspace what changed. Right now that is just the >>>> resolution, but in the future you can expect flags for cases where just the >>>> colorspace information changes, but not the resolution. >>>> >>>> Which reminds me of two important missing pieces of information in your protocol: >>>> >>>> 1) You need to communicate the colorspace data: >>>> >>>> - colorspace >>>> - xfer_func >>>> - ycbcr_enc/hsv_enc (unlikely you ever want to support HSV pixelformats, so I >>>> think you can ignore hsv_enc) >>>> - quantization >>>> >>>> See https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-v4l2.html#c.v4l2_pix_format >>>> and the links to the colorspace sections in the V4L2 spec for details). >>>> >>>> This information is part of the format, it is reported by the driver. >>> I'll take a look and think what can be put and how into the protocol, >>> do you think I'll have to implement all the above for >>> this stage? >> Yes. Without it VMs will have no way of knowing how to reproduce the right colors. >> They don't *have* to use this information, but it should be there. For cameras >> this isn't all that important, for SDTV/HDTV sources this becomes more relevant >> (esp. the quantization and ycbcr_enc information) and for sources with BT.2020/HDR >> formats this is critical. > ok, then I'll add the following to the set_config request/response: > > uint32_t colorspace; > uint32_t xfer_func; > uint32_t ycbcr_enc; > uint32_t quantization; > > With this respect, I will need to put some OS agnostic constants > into the protocol, so if backend and frontend are not Linux/V4L2 > based they can still talk to each other. > I see that V4L2 already defines constants for the above: [1], [2], [3], [4]. > > Do you think I can define the same replacing V4L2_ prefix > with XENCAMERA_, e.g. V4L2_XFER_FUNC_SRGB -> XENCAMERA_XFER_FUNC_SRGB? Yes. > > Do I need to define all those or there can be some subset of the > above for my simpler use-case? Most of these defines directly map to standards. I would skip the following defines: V4L2_COLORSPACE_DEFAULT (not applicable) V4L2_COLORSPACE_470_SYSTEM_* (rarely used, if received by the HW the Xen backend should map this to V4L2_COLORSPACE_SMPTE170M) V4L2_COLORSPACE_JPEG (historical V4L2 artifact, see here how to map: https://hverkuil.home.xs4all.nl/spec/uapi/v4l/colorspaces-details.html#col-jpeg) V4L2_COLORSPACE_SMPTE240M (rarely used, map to V4L2_COLORSPACE_SMPTE170M if seen in backend) V4L2_XFER_FUNC_SMPTE240M (rarely used, map to V4L2_XFER_FUNC_709) V4L2_YCBCR_ENC_SMPTE240M (rarely used, map to V4L2_YCBCR_ENC_709) While V4L2 allows 0 (DEFAULT) values for xfer_func, ycbcr_enc and quantization, and provides macros to map default values to the actual values (for legacy reasons), the Xen backend should always fill this in explicitly, using those same mapping macros (see e.g. V4L2_MAP_XFER_FUNC_DEFAULT). The V4L2 spec has extensive information on colorspaces (sections 2.14-2.17). > >> The vivid driver can actually reproduce all combinations, so that's a good driver >> to test this with. > You mean I can use it on backend side instead of real HW camera and > test all the configurations possible/those of interest? Right. Regards, Hans From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hans Verkuil Subject: Re: [PATCH 1/1] cameraif: add ABI for para-virtual camera Date: Mon, 10 Sep 2018 14:26:09 +0200 Message-ID: References: <20180731093142.3828-1-andr2000@gmail.com> <20180731093142.3828-2-andr2000@gmail.com> <73b69e31-d36d-d89f-20d6-d59dbefe395e@xs4all.nl> <7134b3ad-9fcf-0139-41b3-67a3dbc8224d@xs4all.nl> <51f97715-454a-0242-b381-29944d77d5b5@gmail.com> <3c6bb5c8-eeb4-fd09-407a-5a77b29b56c3@xs4all.nl> <2a39c994-118f-a17e-c40a-f5fbbad1cb03@epam.com> <30d7c91a-4515-157b-fc29-90c2e6f0008b@xs4all.nl> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1fzLGi-0000P0-70 for xen-devel@lists.xenproject.org; Mon, 10 Sep 2018 12:26:16 +0000 In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" To: Oleksandr Andrushchenko , "Oleksandr_Andrushchenko@epam.com" , xen-devel@lists.xenproject.org, konrad.wilk@oracle.com, jgross@suse.com, boris.ostrovsky@oracle.com, mchehab@kernel.org, linux-media@vger.kernel.org, sakari.ailus@linux.intel.com, koji.matsuoka.xm@renesas.com List-Id: xen-devel@lists.xenproject.org T24gMDkvMTAvMjAxOCAwMTo0OSBQTSwgT2xla3NhbmRyIEFuZHJ1c2hjaGVua28gd3JvdGU6Cj4g T24gMDkvMTAvMjAxOCAwMjowOSBQTSwgSGFucyBWZXJrdWlsIHdyb3RlOgo+PiBPbiAwOS8xMC8y MDE4IDExOjUyIEFNLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToKPj4+IE9uIDA5LzEw LzIwMTggMTI6MDQgUE0sIEhhbnMgVmVya3VpbCB3cm90ZToKPj4+PiBPbiAwOS8xMC8yMDE4IDEw OjI0IEFNLCBPbGVrc2FuZHIgQW5kcnVzaGNoZW5rbyB3cm90ZToKPj4+Pj4gT24gMDkvMTAvMjAx OCAxMDo1MyBBTSwgSGFucyBWZXJrdWlsIHdyb3RlOgo+Pj4+Pj4gSGkgT2xla3NhbmRyLAo+Pj4+ Pj4KPj4+Pj4+IE9uIDA5LzEwLzIwMTggMDk6MTYgQU0sIE9sZWtzYW5kciBBbmRydXNoY2hlbmtv IHdyb3RlOgo+Pj4+IDxzbmlwPgo+Pj4+Cj4+Pj4+Pj4+IEkgc3VzcGVjdCB0aGF0IHlvdSBsaWtl bHkgd2lsbCB3YW50IHRvIHN1cHBvcnQgc3VjaCBzb3VyY2VzIGV2ZW50dWFsbHksIHNvCj4+Pj4+ Pj4+IGl0IHBheXMgdG8gZGVzaWduIHRoaXMgd2l0aCB0aGF0IGluIG1pbmQuCj4+Pj4+Pj4gQWdh aW4sIEkgdGhpbmsgdGhhdCB0aGlzIGlzIHRoZSBiYWNrZW5kIHRvIGhpZGUgdGhlc2UKPj4+Pj4+ PiB1c2UtY2FzZXMgZnJvbSB0aGUgZnJvbnRlbmQuCj4+Pj4+PiBJJ20gbm90IHN1cmUgeW91IGNh bjogc2F5IHlvdSBhcmUgcGxheWluZyBhIGJsdXJheSBjb25uZWN0ZWQgdG8gdGhlIHN5c3RlbQo+ Pj4+Pj4gd2l0aCBIRE1JLCB0aGVuIGlmIHRoZXJlIGlzIGEgcmVzb2x1dGlvbiBjaGFuZ2UsIHdo YXQgZG8geW91IGRvPyBZb3UgY2FuIHRlYXIKPj4+Pj4+IGV2ZXJ5dGhpbmcgZG93biBhbmQgYnVp bGQgaXQgdXAgYWdhaW4sIG9yIHlvdSBjYW4ganVzdCB0ZWxsIGZyb250ZW5kcyB0aGF0Cj4+Pj4+ PiBzb21ldGhpbmcgY2hhbmdlZCBhbmQgdGhhdCB0aGV5IGhhdmUgdG8gbG9vayBhdCB0aGUgbmV3 IHZjYW1lcmEgY29uZmlndXJhdGlvbi4KPj4+Pj4+Cj4+Pj4+PiBUaGUgbGF0dGVyIHNlZW1zIHRv IGJlIG1vcmUgc2Vuc2libGUgdG8gbWUuIEl0IGlzIHJlYWxseSBub3QgbXVjaCB0aGF0IHlvdQo+ Pj4+Pj4gbmVlZCB0byBkbzogYWxsIHlvdSByZWFsbHkgbmVlZCBpcyBhbiBldmVudCBzaWduYWxs aW5nIHRoYXQgc29tZXRoaW5nIGNoYW5nZWQuCj4+Pj4+PiBJbiBWNEwyIHRoYXQncyB0aGUgVjRM Ml9FVkVOVF9TT1VSQ0VfQ0hBTkdFLgo+Pj4+PiB3ZWxsLCB0aGlzIGNvbXBsaWNhdGVzIHRoaW5n cyBhIGxvdCBhcyBJJ2xsIGhhdmUgdG8KPj4+Pj4gcmUtYWxsb2NhdGUgYnVmZmVycyAtIHJpZ2h0 Pwo+Pj4+IFJpZ2h0LiBEaWZmZXJlbnQgcmVzb2x1dGlvbnMgbWVhbnMgZGlmZmVyZW50IHNpemVk IGJ1ZmZlcnMgYW5kIHVzdWFsbHkgbG90cyBvZgo+Pj4+IGNoYW5nZXMgdGhyb3VnaG91dCB0aGUg d2hvbGUgdmlkZW8gcGlwZWxpbmUsIHdoaWNoIGluIHRoaXMgY2FzZSBjYW4gZXZlbgo+Pj4+IGdv IGludG8gbXVsdGlwbGUgVk1zLgo+Pj4+Cj4+Pj4gT25lIGFkZGl0aW9uYWwgdGhpbmcgdG8ga2Vl cCBpbiBtaW5kIGZvciB0aGUgZnV0dXJlOiBWNEwyX0VWRU5UX1NPVVJDRV9DSEFOR0UKPj4+PiBo YXMgYSBmbGFncyBmaWVsZCB0aGF0IHRlbGxzIHVzZXJzcGFjZSB3aGF0IGNoYW5nZWQuIFJpZ2h0 IG5vdyB0aGF0IGlzIGp1c3QgdGhlCj4+Pj4gcmVzb2x1dGlvbiwgYnV0IGluIHRoZSBmdXR1cmUg eW91IGNhbiBleHBlY3QgZmxhZ3MgZm9yIGNhc2VzIHdoZXJlIGp1c3QgdGhlCj4+Pj4gY29sb3Jz cGFjZSBpbmZvcm1hdGlvbiBjaGFuZ2VzLCBidXQgbm90IHRoZSByZXNvbHV0aW9uLgo+Pj4+Cj4+ Pj4gV2hpY2ggcmVtaW5kcyBtZSBvZiB0d28gaW1wb3J0YW50IG1pc3NpbmcgcGllY2VzIG9mIGlu Zm9ybWF0aW9uIGluIHlvdXIgcHJvdG9jb2w6Cj4+Pj4KPj4+PiAxKSBZb3UgbmVlZCB0byBjb21t dW5pY2F0ZSB0aGUgY29sb3JzcGFjZSBkYXRhOgo+Pj4+Cj4+Pj4gLSBjb2xvcnNwYWNlCj4+Pj4g LSB4ZmVyX2Z1bmMKPj4+PiAtIHljYmNyX2VuYy9oc3ZfZW5jICh1bmxpa2VseSB5b3UgZXZlciB3 YW50IHRvIHN1cHBvcnQgSFNWIHBpeGVsZm9ybWF0cywgc28gSQo+Pj4+ICAgICB0aGluayB5b3Ug Y2FuIGlnbm9yZSBoc3ZfZW5jKQo+Pj4+IC0gcXVhbnRpemF0aW9uCj4+Pj4KPj4+PiBTZWUgaHR0 cHM6Ly9odmVya3VpbC5ob21lLnhzNGFsbC5ubC9zcGVjL3VhcGkvdjRsL3BpeGZtdC12NGwyLmh0 bWwjYy52NGwyX3BpeF9mb3JtYXQKPj4+PiBhbmQgdGhlIGxpbmtzIHRvIHRoZSBjb2xvcnNwYWNl IHNlY3Rpb25zIGluIHRoZSBWNEwyIHNwZWMgZm9yIGRldGFpbHMpLgo+Pj4+Cj4+Pj4gVGhpcyBp bmZvcm1hdGlvbiBpcyBwYXJ0IG9mIHRoZSBmb3JtYXQsIGl0IGlzIHJlcG9ydGVkIGJ5IHRoZSBk cml2ZXIuCj4+PiBJJ2xsIHRha2UgYSBsb29rIGFuZCB0aGluayB3aGF0IGNhbiBiZSBwdXQgYW5k IGhvdyBpbnRvIHRoZSBwcm90b2NvbCwKPj4+IGRvIHlvdSB0aGluayBJJ2xsIGhhdmUgdG8gaW1w bGVtZW50IGFsbCB0aGUgYWJvdmUgZm9yCj4+PiB0aGlzIHN0YWdlPwo+PiBZZXMuIFdpdGhvdXQg aXQgVk1zIHdpbGwgaGF2ZSBubyB3YXkgb2Yga25vd2luZyBob3cgdG8gcmVwcm9kdWNlIHRoZSBy aWdodCBjb2xvcnMuCj4+IFRoZXkgZG9uJ3QgKmhhdmUqIHRvIHVzZSB0aGlzIGluZm9ybWF0aW9u LCBidXQgaXQgc2hvdWxkIGJlIHRoZXJlLiBGb3IgY2FtZXJhcwo+PiB0aGlzIGlzbid0IGFsbCB0 aGF0IGltcG9ydGFudCwgZm9yIFNEVFYvSERUViBzb3VyY2VzIHRoaXMgYmVjb21lcyBtb3JlIHJl bGV2YW50Cj4+IChlc3AuIHRoZSBxdWFudGl6YXRpb24gYW5kIHljYmNyX2VuYyBpbmZvcm1hdGlv bikgYW5kIGZvciBzb3VyY2VzIHdpdGggQlQuMjAyMC9IRFIKPj4gZm9ybWF0cyB0aGlzIGlzIGNy aXRpY2FsLgo+IG9rLCB0aGVuIEknbGwgYWRkIHRoZSBmb2xsb3dpbmcgdG8gdGhlIHNldF9jb25m aWcgcmVxdWVzdC9yZXNwb25zZToKPiAKPiAgICAgIHVpbnQzMl90IGNvbG9yc3BhY2U7Cj4gICAg ICB1aW50MzJfdCB4ZmVyX2Z1bmM7Cj4gICAgICB1aW50MzJfdCB5Y2Jjcl9lbmM7Cj4gICAgICB1 aW50MzJfdCBxdWFudGl6YXRpb247Cj4gCj4gV2l0aCB0aGlzIHJlc3BlY3QsIEkgd2lsbCBuZWVk IHRvIHB1dCBzb21lIE9TIGFnbm9zdGljIGNvbnN0YW50cwo+IGludG8gdGhlIHByb3RvY29sLCBz byBpZiBiYWNrZW5kIGFuZCBmcm9udGVuZCBhcmUgbm90IExpbnV4L1Y0TDIKPiBiYXNlZCB0aGV5 IGNhbiBzdGlsbCB0YWxrIHRvIGVhY2ggb3RoZXIuCj4gSSBzZWUgdGhhdCBWNEwyIGFscmVhZHkg ZGVmaW5lcyBjb25zdGFudHMgZm9yIHRoZSBhYm92ZTogWzFdLCBbMl0sIFszXSwgWzRdLgo+IAo+ IERvIHlvdSB0aGluayBJIGNhbiBkZWZpbmUgdGhlIHNhbWUgcmVwbGFjaW5nIFY0TDJfIHByZWZp eAo+IHdpdGggWEVOQ0FNRVJBXywgZS5nLiBWNEwyX1hGRVJfRlVOQ19TUkdCIC0+IFhFTkNBTUVS QV9YRkVSX0ZVTkNfU1JHQj8KClllcy4KCj4gCj4gRG8gSSBuZWVkIHRvIGRlZmluZSBhbGwgdGhv c2Ugb3IgdGhlcmUgY2FuIGJlIHNvbWUgc3Vic2V0IG9mIHRoZQo+IGFib3ZlIGZvciBteSBzaW1w bGVyIHVzZS1jYXNlPwoKTW9zdCBvZiB0aGVzZSBkZWZpbmVzIGRpcmVjdGx5IG1hcCB0byBzdGFu ZGFyZHMuIEkgd291bGQgc2tpcCB0aGUgZm9sbG93aW5nCmRlZmluZXM6CgpWNEwyX0NPTE9SU1BB Q0VfREVGQVVMVCAobm90IGFwcGxpY2FibGUpClY0TDJfQ09MT1JTUEFDRV80NzBfU1lTVEVNXyog IChyYXJlbHkgdXNlZCwgaWYgcmVjZWl2ZWQgYnkgdGhlIEhXIHRoZSBYZW4gYmFja2VuZAoJCQlz aG91bGQgbWFwIHRoaXMgdG8gVjRMMl9DT0xPUlNQQUNFX1NNUFRFMTcwTSkKVjRMMl9DT0xPUlNQ QUNFX0pQRUcgKGhpc3RvcmljYWwgVjRMMiBhcnRpZmFjdCwgc2VlIGhlcmUgaG93IHRvIG1hcDoK CSBodHRwczovL2h2ZXJrdWlsLmhvbWUueHM0YWxsLm5sL3NwZWMvdWFwaS92NGwvY29sb3JzcGFj ZXMtZGV0YWlscy5odG1sI2NvbC1qcGVnKQoKVjRMMl9DT0xPUlNQQUNFX1NNUFRFMjQwTSAocmFy ZWx5IHVzZWQsIG1hcCB0byBWNEwyX0NPTE9SU1BBQ0VfU01QVEUxNzBNIGlmIHNlZW4gaW4gYmFj a2VuZCkKClY0TDJfWEZFUl9GVU5DX1NNUFRFMjQwTSAocmFyZWx5IHVzZWQsIG1hcCB0byBWNEwy X1hGRVJfRlVOQ183MDkpCgpWNEwyX1lDQkNSX0VOQ19TTVBURTI0ME0gKHJhcmVseSB1c2VkLCBt YXAgdG8gVjRMMl9ZQ0JDUl9FTkNfNzA5KQoKV2hpbGUgVjRMMiBhbGxvd3MgMCAoREVGQVVMVCkg dmFsdWVzIGZvciB4ZmVyX2Z1bmMsIHljYmNyX2VuYyBhbmQgcXVhbnRpemF0aW9uLCBhbmQKcHJv dmlkZXMgbWFjcm9zIHRvIG1hcCBkZWZhdWx0IHZhbHVlcyB0byB0aGUgYWN0dWFsIHZhbHVlcyAo Zm9yIGxlZ2FjeSByZWFzb25zKSwKdGhlIFhlbiBiYWNrZW5kIHNob3VsZCBhbHdheXMgZmlsbCB0 aGlzIGluIGV4cGxpY2l0bHksIHVzaW5nIHRob3NlIHNhbWUgbWFwcGluZwptYWNyb3MgKHNlZSBl LmcuIFY0TDJfTUFQX1hGRVJfRlVOQ19ERUZBVUxUKS4KClRoZSBWNEwyIHNwZWMgaGFzIGV4dGVu c2l2ZSBpbmZvcm1hdGlvbiBvbiBjb2xvcnNwYWNlcyAoc2VjdGlvbnMgMi4xNC0yLjE3KS4KCj4g Cj4+IFRoZSB2aXZpZCBkcml2ZXIgY2FuIGFjdHVhbGx5IHJlcHJvZHVjZSBhbGwgY29tYmluYXRp b25zLCBzbyB0aGF0J3MgYSBnb29kIGRyaXZlcgo+PiB0byB0ZXN0IHRoaXMgd2l0aC4KPiBZb3Ug bWVhbiBJIGNhbiB1c2UgaXQgb24gYmFja2VuZCBzaWRlIGluc3RlYWQgb2YgcmVhbCBIVyBjYW1l cmEgYW5kCj4gdGVzdCBhbGwgdGhlIGNvbmZpZ3VyYXRpb25zIHBvc3NpYmxlL3Rob3NlIG9mIGlu dGVyZXN0PwoKUmlnaHQuCgpSZWdhcmRzLAoKCUhhbnMKCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fClhlbi1kZXZlbCBtYWlsaW5nIGxpc3QKWGVuLWRldmVs QGxpc3RzLnhlbnByb2plY3Qub3JnCmh0dHBzOi8vbGlzdHMueGVucHJvamVjdC5vcmcvbWFpbG1h bi9saXN0aW5mby94ZW4tZGV2ZWw=