From: Pekka Paalanen <ppaalanen@gmail.com> To: Esaki Tomohito <etom@igel.co.jp> Cc: "Enrico Weigelt, metux IT consult" <lkml@metux.net>, devicetree@vger.kernel.org, Takanari Hayama <taki@igel.co.jp>, Thomas Zimmermann <tzimmermann@suse.de>, linux-doc@vger.kernel.org, David Airlie <airlied@linux.ie>, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-renesas-soc@vger.kernel.org, Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>, Laurent Pinchart <laurent.pinchart@ideasonboard.com>, Damian Hobson-Garcia <dhobsong@igel.co.jp> Subject: Re: [PATH 0/4] [RFC] Support virtual DRM Date: Tue, 22 Jun 2021 11:12:39 +0300 [thread overview] Message-ID: <20210622111239.73aa87aa@eldfell> (raw) In-Reply-To: <1cfab5f9-f275-aa53-00de-5da3fcea71c5@igel.co.jp> [-- Attachment #1: Type: text/plain, Size: 3567 bytes --] On Tue, 22 Jun 2021 13:03:39 +0900 Esaki Tomohito <etom@igel.co.jp> wrote: > Hi, Enrico Weigelt > Thank you for reply. > > On 2021/06/22 1:05, Enrico Weigelt, metux IT consult wrote: > > On 21.06.21 08:27, Tomohito Esaki wrote: > > > > Hi, > > > >> Virtual DRM splits the overlay planes of a display controller into multiple > >> virtual devices to allow each plane to be accessed by each process. > >> > >> This makes it possible to overlay images output from multiple processes on a > >> display. For example, one process displays the camera image without compositor > >> while another process overlays the UI. > > > > Are you attempting to create an simple in-kernel compositor ? > > I think the basic idea is the same as DRMlease. Hi, indeed. Why not use DRM leases instead? > We want to separate the resources from the master in units of planes, > so we proposed virtual DRM. > I think the advantage of vDRM is that you can use general DRM APIs > in userland. You do that with DRM leases too. > > I don't think that's not the way to go, at least not by touching each > > single display driver, and not hardcoding the planes in DT. > > Thank you for comment. I will reconsider about DT. > > > What's the actual use case you're doing that for ? Why not using some > > userland compositor ? > > I think when latency is important (e.g., AR, VR, for displaying camera > images in IVI systems), there may be use cases where the compositor > cannot be used. > Normally, when the image is passed through the compositor, it is > displayed after 2 VSYNC at most, because the compositor combines the > image with VSYNC synchronization. On the other hand, if we use vDRM, the > image will be displayed at the next VSYNC, so it will be displayed after > 1 VSYNC at most. As I said in my other email, this is false in the general sense. > Also, since the compositor is a single point of failure, we may not want > to make it dependent on it. This... I'm not quite sure I buy it. If any of all the programs using virtual KMS crashes, you still lose some crucial components from your display. Maybe that program, while crashing, uploads such a bad state to its very own KMS plane, that it causes other KMS planes to malfunction. Then you need to detect this situation and still restart everything, not just the crashed program. I would think a userspace compositor approach is actually more reliable. You write the compositor to be extremely robust. Exactly because the compositor is in control of the complete display device and not just little pieces of it, it can see what is happening and it can mitigate problems. If you have more unreliable components needing access to display, make those clients to the compositor, so they can crash and malfunction on their own without potentially killing the whole display device. If you are as concerned about latency as XR people are, then use DRM leases. Also, what if your virtual KMS driver has a bug? Restarting the kernel is much harder that restarting a userspace compositor that hands out DRM leases. The userspace compositor could even be such that it does nothing more than handing out DRM leases. However, DRM leases have the problem that there is no single entity responsible for keeping the display device working, but that responsibility is split between several processes and none of them sees the whole picture. Btw. VKMS is an existing DRM driver, so your name choice is conflicting. Thanks, pq [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: Pekka Paalanen <ppaalanen@gmail.com> To: Esaki Tomohito <etom@igel.co.jp> Cc: devicetree@vger.kernel.org, Takanari Hayama <taki@igel.co.jp>, linux-doc@vger.kernel.org, David Airlie <airlied@linux.ie>, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-renesas-soc@vger.kernel.org, Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>, "Enrico Weigelt, metux IT consult" <lkml@metux.net>, Laurent Pinchart <laurent.pinchart@ideasonboard.com>, Thomas Zimmermann <tzimmermann@suse.de>, Damian Hobson-Garcia <dhobsong@igel.co.jp> Subject: Re: [PATH 0/4] [RFC] Support virtual DRM Date: Tue, 22 Jun 2021 11:12:39 +0300 [thread overview] Message-ID: <20210622111239.73aa87aa@eldfell> (raw) In-Reply-To: <1cfab5f9-f275-aa53-00de-5da3fcea71c5@igel.co.jp> [-- Attachment #1: Type: text/plain, Size: 3567 bytes --] On Tue, 22 Jun 2021 13:03:39 +0900 Esaki Tomohito <etom@igel.co.jp> wrote: > Hi, Enrico Weigelt > Thank you for reply. > > On 2021/06/22 1:05, Enrico Weigelt, metux IT consult wrote: > > On 21.06.21 08:27, Tomohito Esaki wrote: > > > > Hi, > > > >> Virtual DRM splits the overlay planes of a display controller into multiple > >> virtual devices to allow each plane to be accessed by each process. > >> > >> This makes it possible to overlay images output from multiple processes on a > >> display. For example, one process displays the camera image without compositor > >> while another process overlays the UI. > > > > Are you attempting to create an simple in-kernel compositor ? > > I think the basic idea is the same as DRMlease. Hi, indeed. Why not use DRM leases instead? > We want to separate the resources from the master in units of planes, > so we proposed virtual DRM. > I think the advantage of vDRM is that you can use general DRM APIs > in userland. You do that with DRM leases too. > > I don't think that's not the way to go, at least not by touching each > > single display driver, and not hardcoding the planes in DT. > > Thank you for comment. I will reconsider about DT. > > > What's the actual use case you're doing that for ? Why not using some > > userland compositor ? > > I think when latency is important (e.g., AR, VR, for displaying camera > images in IVI systems), there may be use cases where the compositor > cannot be used. > Normally, when the image is passed through the compositor, it is > displayed after 2 VSYNC at most, because the compositor combines the > image with VSYNC synchronization. On the other hand, if we use vDRM, the > image will be displayed at the next VSYNC, so it will be displayed after > 1 VSYNC at most. As I said in my other email, this is false in the general sense. > Also, since the compositor is a single point of failure, we may not want > to make it dependent on it. This... I'm not quite sure I buy it. If any of all the programs using virtual KMS crashes, you still lose some crucial components from your display. Maybe that program, while crashing, uploads such a bad state to its very own KMS plane, that it causes other KMS planes to malfunction. Then you need to detect this situation and still restart everything, not just the crashed program. I would think a userspace compositor approach is actually more reliable. You write the compositor to be extremely robust. Exactly because the compositor is in control of the complete display device and not just little pieces of it, it can see what is happening and it can mitigate problems. If you have more unreliable components needing access to display, make those clients to the compositor, so they can crash and malfunction on their own without potentially killing the whole display device. If you are as concerned about latency as XR people are, then use DRM leases. Also, what if your virtual KMS driver has a bug? Restarting the kernel is much harder that restarting a userspace compositor that hands out DRM leases. The userspace compositor could even be such that it does nothing more than handing out DRM leases. However, DRM leases have the problem that there is no single entity responsible for keeping the display device working, but that responsibility is split between several processes and none of them sees the whole picture. Btw. VKMS is an existing DRM driver, so your name choice is conflicting. Thanks, pq [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2021-06-22 8:12 UTC|newest] Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-06-21 6:27 [PATH 0/4] [RFC] Support virtual DRM Tomohito Esaki 2021-06-21 6:27 ` [PATH 1/4] drm: Add Virtual DRM device driver Tomohito Esaki 2021-06-21 6:27 ` [PATH 2/4] rcar-du: Add support virtual DRM device Tomohito Esaki 2021-06-21 6:27 ` [PATH 3/4] dt-bindings: display: Add virtual DRM Tomohito Esaki 2021-06-21 6:27 ` [PATH 4/4] doc-rst: Add virtual DRM documentation Tomohito Esaki 2021-06-21 7:10 ` [PATH 0/4] [RFC] Support virtual DRM Thomas Zimmermann 2021-06-21 9:24 ` Maxime Ripard 2021-06-22 4:36 ` Esaki Tomohito 2021-06-22 4:36 ` Esaki Tomohito 2021-06-23 14:39 ` Maxime Ripard 2021-06-23 14:39 ` Maxime Ripard 2021-06-22 4:02 ` Esaki Tomohito 2021-06-22 4:02 ` Esaki Tomohito 2021-06-22 7:57 ` Pekka Paalanen 2021-06-22 7:57 ` Pekka Paalanen 2021-06-23 8:04 ` Michel Dänzer 2021-06-23 8:21 ` Esaki Tomohito 2021-06-22 9:12 ` Thomas Zimmermann 2021-06-22 9:12 ` Thomas Zimmermann 2021-06-21 16:05 ` Enrico Weigelt, metux IT consult 2021-06-22 4:03 ` Esaki Tomohito 2021-06-22 4:03 ` Esaki Tomohito 2021-06-22 8:12 ` Pekka Paalanen [this message] 2021-06-22 8:12 ` Pekka Paalanen 2021-06-22 19:12 ` Daniel Vetter 2021-06-22 19:12 ` Daniel Vetter 2021-06-23 6:56 ` Esaki Tomohito 2021-06-23 6:56 ` Esaki Tomohito 2021-06-23 8:39 ` Pekka Paalanen 2021-06-23 8:39 ` Pekka Paalanen 2021-06-23 9:22 ` Esaki Tomohito 2021-06-23 9:22 ` Esaki Tomohito 2021-06-23 11:41 ` Pekka Paalanen 2021-06-23 11:41 ` Pekka Paalanen 2021-06-25 1:55 ` Esaki Tomohito 2021-06-25 1:55 ` Esaki Tomohito 2021-06-21 6:43 Tomohito Esaki 2021-06-21 6:43 ` Tomohito Esaki 2021-06-22 8:04 ` Simon Ser 2021-06-22 8:04 ` Simon Ser
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210622111239.73aa87aa@eldfell \ --to=ppaalanen@gmail.com \ --cc=airlied@linux.ie \ --cc=devicetree@vger.kernel.org \ --cc=dhobsong@igel.co.jp \ --cc=dri-devel@lists.freedesktop.org \ --cc=etom@igel.co.jp \ --cc=kieran.bingham+renesas@ideasonboard.com \ --cc=laurent.pinchart@ideasonboard.com \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-renesas-soc@vger.kernel.org \ --cc=lkml@metux.net \ --cc=taki@igel.co.jp \ --cc=tzimmermann@suse.de \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.