From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C981C49EA3 for ; Mon, 21 Jun 2021 06:44:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 26DDB61040 for ; Mon, 21 Jun 2021 06:44:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229949AbhFUGqu (ORCPT ); Mon, 21 Jun 2021 02:46:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229618AbhFUGqr (ORCPT ); Mon, 21 Jun 2021 02:46:47 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07DB8C061756 for ; Sun, 20 Jun 2021 23:44:33 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id t13so1549711pgu.11 for ; Sun, 20 Jun 2021 23:44:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igel-co-jp.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qYWJuyK1bUpXUNYlMG8y+I7VziTPyHJu5fz70XXNt7o=; b=zSsBl46XOGZe6gvr8o3kSN6cTIPzym+tkv/0ToxwVp80014S9cjzeYVaJTe7IchkT3 KzO0Rrnnnb0txcx1hIR4xdteZ4vsg1vsE5Y2rnAl/G3FEoMMUe2KYfexSXNGEnx8OtaS ECt8k6XlbFEv1yvP34tPW4y704OmqqlCktx5y0nZXNr5gSEpSpphC9BAjnL5HgbeUDxo TAUXuI8SUtGC7pJsajJykD2gc7WtGj2LQTUZ5G9Rv5/EQwUDJDHyBKw9zMX/1Iq/8Y1Y i7XI+UqHlL1TCvgn8DVn8B21N4EHMGd/uaasHWQaYYI8Ky00kLsBtw7bGJMey2M24OBS 7H9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qYWJuyK1bUpXUNYlMG8y+I7VziTPyHJu5fz70XXNt7o=; b=gE4u2zJswd39cL/PIJTTXpDj1kqbxmHNeCxrbWGbPtLDJ9AZIbZ1s8tYVeKoFR+R8g tNNEG2OA+taPUpIf2X62QW/Zl6fuaHRkwd/9xdo91mLmYHHTZa3PqydWemtk1c8ae27t Ank1QsCwpPofDalGhBiH2KSwdcdlOWgm5MWq5Z+RdZuvpfBMDCmcjUuriPj651MN+9Yw U7VFKyw42CcL9Z4w0l8m8qiqyRa9HP1lb1XXyhNhqzo8/BO4bpjzW/ZjDCGmkU1AWSwe t8Ls7EURBqf0EQZ27NXQzIZfgctutnNM4pIdWYJ/LNk6pg1OhrmAAXTx/vwI8phfAJ8F DcaA== X-Gm-Message-State: AOAM533T8u1JDBN2fdkpmCzuUtdeKU2d4Tw7Fkxq/oVYbzCjYxB0Jnqf iUrEX0r0k8pFZSL7zSnggSjQcw== X-Google-Smtp-Source: ABdhPJyU0p37To1lNBWRzUCiNdn9mBwEeKIBU3xGQdaMaX1k0DNbUC1ZknIAhuYlvctiMOE5qL9jKg== X-Received: by 2002:a62:ee16:0:b029:2fe:ffcf:775a with SMTP id e22-20020a62ee160000b02902feffcf775amr17476747pfi.59.1624257872488; Sun, 20 Jun 2021 23:44:32 -0700 (PDT) Received: from localhost.localdomain ([240b:10:c9a0:ca00:61b8:26ec:dc30:8823]) by smtp.gmail.com with ESMTPSA id o34sm12744450pgm.6.2021.06.20.23.44.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:44:32 -0700 (PDT) From: Tomohito Esaki To: Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter , Laurent Pinchart , Kieran Bingham Cc: dri-devel@lists.freedesktop.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-doc@vger.kernel.org, Tomohito Esaki Subject: [PATH 0/4] [RFC] Support virtual DRM Date: Mon, 21 Jun 2021 15:43:59 +0900 Message-Id: <20210621064403.26663-1-etom@igel.co.jp> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello Sorry, there was a typo in the dri-devel mail address, so I've resend it. --- Virtual DRM splits the overlay planes of a display controller into multiple virtual devices to allow each plane to be accessed by each process. This makes it possible to overlay images output from multiple processes on a display. For example, one process displays the camera image without compositor while another process overlays the UI. Virtual DRM driver doesn’t directly control the display hardware and has no access to the physical bus. Instead, the virtual DRM driver issues requests to the standard DRM device driver (parent) when the hardware needs to be controlled. The parent is modified to notify the virtual DRM driver of interruptevents from the display hardware. Therefore, in order to use virtual DRM, each DRM device driver needs to add code to support virutal DRM. The only driver supported in this patch series is rcar-du. This patch series is divided into multiple. The first patch adds vDRM feature to DRM, and the second patch support vDRM for the rcar-du driver. The other patches add documentation. In particular, I would appreciate your advice on the following points: * virtual DRM generalization I've only tested with rcar-du, is there anything I should consider to make virtual DRM work with other drivers? * Integration to upstream I think it is a good idea to add virtual DRM to the DRM core functionality, but I would appreciate any suggestions on what needs to be improved for integration to upstream. * dumb_create and fb_create callback I think that the dumb_create and fb_create callbacks need to be done by the parent, and it is preferable to use the parent's callbacks as they are. However, since the dumb buffer needs to be registered in the parent and the fb handle needs to be registered in the drm_file of the vDRM, the dumb_create callbacks from the parent driver cannot be used as is. Therefore, the current implementation of the dumb_create callback is workarround. What do you think is the best way to deal with this issue? Tomohito Esaki (4): Add Virtual DRM device driver rcar-du: Add support virtual DRM device dt-bindings: display: Add virtual DRM doc-rst: Add virtual DRM documentation .../devicetree/bindings/display/vdrm.yaml | 67 ++ Documentation/gpu/drivers.rst | 1 + Documentation/gpu/vdrm.rst | 51 ++ drivers/gpu/drm/Kconfig | 7 + drivers/gpu/drm/Makefile | 1 + drivers/gpu/drm/rcar-du/Kconfig | 4 + drivers/gpu/drm/rcar-du/Makefile | 1 + drivers/gpu/drm/rcar-du/rcar_du_crtc.c | 42 + drivers/gpu/drm/rcar-du/rcar_du_crtc.h | 13 + drivers/gpu/drm/rcar-du/rcar_du_drv.c | 13 + drivers/gpu/drm/rcar-du/rcar_du_drv.h | 3 + drivers/gpu/drm/rcar-du/rcar_du_vdrm.c | 191 ++++ drivers/gpu/drm/rcar-du/rcar_du_vdrm.h | 67 ++ drivers/gpu/drm/rcar-du/rcar_du_vsp.c | 22 + drivers/gpu/drm/rcar-du/rcar_du_vsp.h | 1 + drivers/gpu/drm/vdrm/vdrm_api.h | 68 ++ drivers/gpu/drm/vdrm/vdrm_drv.c | 859 ++++++++++++++++++ drivers/gpu/drm/vdrm/vdrm_drv.h | 80 ++ 18 files changed, 1491 insertions(+) create mode 100644 Documentation/devicetree/bindings/display/vdrm.yaml create mode 100644 Documentation/gpu/vdrm.rst create mode 100644 drivers/gpu/drm/rcar-du/rcar_du_vdrm.c create mode 100644 drivers/gpu/drm/rcar-du/rcar_du_vdrm.h create mode 100644 drivers/gpu/drm/vdrm/vdrm_api.h create mode 100644 drivers/gpu/drm/vdrm/vdrm_drv.c create mode 100644 drivers/gpu/drm/vdrm/vdrm_drv.h -- 2.25.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01A9BC49361 for ; Mon, 21 Jun 2021 07:08:05 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C3DD0611CC for ; Mon, 21 Jun 2021 07:08:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C3DD0611CC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igel.co.jp Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1185C89D77; Mon, 21 Jun 2021 07:07:59 +0000 (UTC) Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by gabe.freedesktop.org (Postfix) with ESMTPS id CDC6F89A35 for ; Mon, 21 Jun 2021 06:44:32 +0000 (UTC) Received: by mail-pf1-x433.google.com with SMTP id h26so1762507pfo.5 for ; Sun, 20 Jun 2021 23:44:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igel-co-jp.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qYWJuyK1bUpXUNYlMG8y+I7VziTPyHJu5fz70XXNt7o=; b=zSsBl46XOGZe6gvr8o3kSN6cTIPzym+tkv/0ToxwVp80014S9cjzeYVaJTe7IchkT3 KzO0Rrnnnb0txcx1hIR4xdteZ4vsg1vsE5Y2rnAl/G3FEoMMUe2KYfexSXNGEnx8OtaS ECt8k6XlbFEv1yvP34tPW4y704OmqqlCktx5y0nZXNr5gSEpSpphC9BAjnL5HgbeUDxo TAUXuI8SUtGC7pJsajJykD2gc7WtGj2LQTUZ5G9Rv5/EQwUDJDHyBKw9zMX/1Iq/8Y1Y i7XI+UqHlL1TCvgn8DVn8B21N4EHMGd/uaasHWQaYYI8Ky00kLsBtw7bGJMey2M24OBS 7H9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=qYWJuyK1bUpXUNYlMG8y+I7VziTPyHJu5fz70XXNt7o=; b=MRIRvgTvBtbYU1fXVwzj5KshputhwZdPho3XmYmzahIPgNH9y3iuq2au18YcKCATBj i6pZDyhF8njvUuE1EOILk0F54BDBofWtiQ8LND2GeQUSw7sUbTuQVFfIxJdIqMcndzp3 VhVv1BUw0R7Vfgd/tXM0z1i8JwZnt9ZSUO26DLKUV0IwQmbRRHnnqJTgj/QxtKl/qGGS D8RJ5vjaDIRkW8h1dlWrLWGm+VigksIpeXFA5eBNwuhoamvqJc21Ef/SRRyTUSPIidxT fYk3CWNsYCKw1oHuzD0ddteiMQ/0XTx3w2NAXB5E/vpEa1dOOF5Ewzi4FnrOqox36zC+ LLDA== X-Gm-Message-State: AOAM533zCm9v6KWZC76ZGwfoa8ILYOaE5fDi+FSuXFW3adbguIkRgDlJ mAY/vqdwpV6m2fTktZr+QP8D6Fzux1JeqqPh X-Google-Smtp-Source: ABdhPJyU0p37To1lNBWRzUCiNdn9mBwEeKIBU3xGQdaMaX1k0DNbUC1ZknIAhuYlvctiMOE5qL9jKg== X-Received: by 2002:a62:ee16:0:b029:2fe:ffcf:775a with SMTP id e22-20020a62ee160000b02902feffcf775amr17476747pfi.59.1624257872488; Sun, 20 Jun 2021 23:44:32 -0700 (PDT) Received: from localhost.localdomain ([240b:10:c9a0:ca00:61b8:26ec:dc30:8823]) by smtp.gmail.com with ESMTPSA id o34sm12744450pgm.6.2021.06.20.23.44.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:44:32 -0700 (PDT) From: Tomohito Esaki To: Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter , Laurent Pinchart , Kieran Bingham Subject: [PATH 0/4] [RFC] Support virtual DRM Date: Mon, 21 Jun 2021 15:43:59 +0900 Message-Id: <20210621064403.26663-1-etom@igel.co.jp> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Mon, 21 Jun 2021 07:07:50 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: devicetree@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-renesas-soc@vger.kernel.org, Tomohito Esaki Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Hello Sorry, there was a typo in the dri-devel mail address, so I've resend it. --- Virtual DRM splits the overlay planes of a display controller into multiple virtual devices to allow each plane to be accessed by each process. This makes it possible to overlay images output from multiple processes on a display. For example, one process displays the camera image without compositor while another process overlays the UI. Virtual DRM driver doesn’t directly control the display hardware and has no access to the physical bus. Instead, the virtual DRM driver issues requests to the standard DRM device driver (parent) when the hardware needs to be controlled. The parent is modified to notify the virtual DRM driver of interruptevents from the display hardware. Therefore, in order to use virtual DRM, each DRM device driver needs to add code to support virutal DRM. The only driver supported in this patch series is rcar-du. This patch series is divided into multiple. The first patch adds vDRM feature to DRM, and the second patch support vDRM for the rcar-du driver. The other patches add documentation. In particular, I would appreciate your advice on the following points: * virtual DRM generalization I've only tested with rcar-du, is there anything I should consider to make virtual DRM work with other drivers? * Integration to upstream I think it is a good idea to add virtual DRM to the DRM core functionality, but I would appreciate any suggestions on what needs to be improved for integration to upstream. * dumb_create and fb_create callback I think that the dumb_create and fb_create callbacks need to be done by the parent, and it is preferable to use the parent's callbacks as they are. However, since the dumb buffer needs to be registered in the parent and the fb handle needs to be registered in the drm_file of the vDRM, the dumb_create callbacks from the parent driver cannot be used as is. Therefore, the current implementation of the dumb_create callback is workarround. What do you think is the best way to deal with this issue? Tomohito Esaki (4): Add Virtual DRM device driver rcar-du: Add support virtual DRM device dt-bindings: display: Add virtual DRM doc-rst: Add virtual DRM documentation .../devicetree/bindings/display/vdrm.yaml | 67 ++ Documentation/gpu/drivers.rst | 1 + Documentation/gpu/vdrm.rst | 51 ++ drivers/gpu/drm/Kconfig | 7 + drivers/gpu/drm/Makefile | 1 + drivers/gpu/drm/rcar-du/Kconfig | 4 + drivers/gpu/drm/rcar-du/Makefile | 1 + drivers/gpu/drm/rcar-du/rcar_du_crtc.c | 42 + drivers/gpu/drm/rcar-du/rcar_du_crtc.h | 13 + drivers/gpu/drm/rcar-du/rcar_du_drv.c | 13 + drivers/gpu/drm/rcar-du/rcar_du_drv.h | 3 + drivers/gpu/drm/rcar-du/rcar_du_vdrm.c | 191 ++++ drivers/gpu/drm/rcar-du/rcar_du_vdrm.h | 67 ++ drivers/gpu/drm/rcar-du/rcar_du_vsp.c | 22 + drivers/gpu/drm/rcar-du/rcar_du_vsp.h | 1 + drivers/gpu/drm/vdrm/vdrm_api.h | 68 ++ drivers/gpu/drm/vdrm/vdrm_drv.c | 859 ++++++++++++++++++ drivers/gpu/drm/vdrm/vdrm_drv.h | 80 ++ 18 files changed, 1491 insertions(+) create mode 100644 Documentation/devicetree/bindings/display/vdrm.yaml create mode 100644 Documentation/gpu/vdrm.rst create mode 100644 drivers/gpu/drm/rcar-du/rcar_du_vdrm.c create mode 100644 drivers/gpu/drm/rcar-du/rcar_du_vdrm.h create mode 100644 drivers/gpu/drm/vdrm/vdrm_api.h create mode 100644 drivers/gpu/drm/vdrm/vdrm_drv.c create mode 100644 drivers/gpu/drm/vdrm/vdrm_drv.h -- 2.25.1