From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD295C4360C for ; Fri, 15 Jan 2021 20:03:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2A0023A58 for ; Fri, 15 Jan 2021 20:03:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388712AbhAOUDP (ORCPT ); Fri, 15 Jan 2021 15:03:15 -0500 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:33653 "EHLO relay5-d.mail.gandi.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388572AbhAOUCz (ORCPT ); Fri, 15 Jan 2021 15:02:55 -0500 X-Originating-IP: 93.29.109.196 Received: from localhost.localdomain (196.109.29.93.rev.sfr.net [93.29.109.196]) (Authenticated sender: paul.kocialkowski@bootlin.com) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 96C0A1C000C; Fri, 15 Jan 2021 20:02:09 +0000 (UTC) From: Paul Kocialkowski To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, devel@driverdev.osuosl.org, linux-sunxi@googlegroups.com Cc: Yong Deng , Mauro Carvalho Chehab , Rob Herring , Maxime Ripard , Chen-Yu Tsai , Jernej Skrabec , Paul Kocialkowski , Jonathan Corbet , Kishon Vijay Abraham I , Vinod Koul , Helen Koike , Dafna Hirschfeld , Greg Kroah-Hartman , Philipp Zabel , Sakari Ailus , Hans Verkuil , Thomas Petazzoni , kevin.lhopital@hotmail.com, Ezequiel Garcia Subject: [PATCH v5 08/16] media: sun6i-csi: Add support for MIPI CSI-2 bridge input Date: Fri, 15 Jan 2021 21:01:33 +0100 Message-Id: <20210115200141.1397785-9-paul.kocialkowski@bootlin.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210115200141.1397785-1-paul.kocialkowski@bootlin.com> References: <20210115200141.1397785-1-paul.kocialkowski@bootlin.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The A31 CSI controller supports a MIPI CSI-2 bridge input, which has its own dedicated port in the fwnode graph. Support for this input is added with this change: - two pads are defined for the media entity instead of one and only one needs to be connected at a time; - the pads currently match the fwnode graph representation; - links are created between our pads and the subdevs for each interface and are no longer immutable so that userspace can select which interface to use in case both are bound to a subdev; - fwnode endpoints are parsed and stored for each interface; - the active subdev (and fwnode endpoint) is retrieved when validating the media link at stream on time and cleared at stream off; - an error is raised if both links are active at the same time; - the MIPI interface bit is set if the MIPI CSI-2 bridge endpoint is active. In the future, the media entity representation might evolve to: - distinguish the internal parallel bridge and data formatter; - represent each of the 4 internal channels that can exist between the parallel bridge (for BT656 time-multiplex) and MIPI CSI-2 (internal channels can be mapped to virtual channels); - connect the controller's output to the ISP instead of its DMA engine. Finally note that the MIPI CSI-2 bridges should not be linked in the fwnode graph unless they have a sensor subdev attached. Signed-off-by: Paul Kocialkowski Acked-by: Maxime Ripard --- .../platform/sunxi/sun6i-csi/sun6i_csi.c | 111 +++++++++++++++--- .../platform/sunxi/sun6i-csi/sun6i_csi.h | 9 +- .../platform/sunxi/sun6i-csi/sun6i_video.c | 57 +++++---- .../platform/sunxi/sun6i-csi/sun6i_video.h | 7 +- 4 files changed, 137 insertions(+), 47 deletions(-) diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c index d06028f42534..55cac91f8282 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -49,6 +49,7 @@ static inline struct sun6i_csi_dev *sun6i_csi_to_dev(struct sun6i_csi *csi) /* TODO add 10&12 bit YUV, RGB support */ bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, + struct v4l2_fwnode_endpoint *endpoint, u32 pixformat, u32 mbus_code) { struct sun6i_csi_dev *sdev = sun6i_csi_to_dev(csi); @@ -58,9 +59,9 @@ bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, * 8bit and 16bit bus width. * Identify the media bus format from device tree. */ - if ((sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_PARALLEL - || sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_BT656) - && sdev->csi.v4l2_ep.bus.parallel.bus_width == 16) { + if ((endpoint->bus_type == V4L2_MBUS_PARALLEL || + endpoint->bus_type == V4L2_MBUS_BT656) && + endpoint->bus.parallel.bus_width == 16) { switch (pixformat) { case V4L2_PIX_FMT_HM12: case V4L2_PIX_FMT_NV12: @@ -373,7 +374,7 @@ static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_dev *sdev, static void sun6i_csi_setup_bus(struct sun6i_csi_dev *sdev) { - struct v4l2_fwnode_endpoint *endpoint = &sdev->csi.v4l2_ep; + struct v4l2_fwnode_endpoint *endpoint = sdev->csi.video.source_endpoint; struct sun6i_csi *csi = &sdev->csi; unsigned char bus_width; u32 flags; @@ -459,6 +460,9 @@ static void sun6i_csi_setup_bus(struct sun6i_csi_dev *sdev) if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING) cfg |= CSI_IF_CFG_CLK_POL_FALLING_EDGE; break; + case V4L2_MBUS_CSI2_DPHY: + cfg |= CSI_IF_CFG_MIPI_IF_MIPI; + break; default: dev_warn(sdev->dev, "Unsupported bus type: %d\n", endpoint->bus_type); @@ -636,11 +640,11 @@ void sun6i_csi_set_stream(struct sun6i_csi *csi, bool enable) * Media Controller and V4L2 */ static int sun6i_csi_link_entity(struct sun6i_csi *csi, + struct media_pad *sink_pad, struct media_entity *entity, - struct fwnode_handle *fwnode) + struct fwnode_handle *fwnode, bool enabled) { struct media_entity *sink; - struct media_pad *sink_pad; int src_pad_index; int ret; @@ -654,14 +658,12 @@ static int sun6i_csi_link_entity(struct sun6i_csi *csi, src_pad_index = ret; sink = &csi->video.vdev.entity; - sink_pad = &csi->video.pad; dev_dbg(csi->dev, "creating %s:%u -> %s:%u link\n", entity->name, src_pad_index, sink->name, sink_pad->index); ret = media_create_pad_link(entity, src_pad_index, sink, sink_pad->index, - MEDIA_LNK_FL_ENABLED | - MEDIA_LNK_FL_IMMUTABLE); + enabled ? MEDIA_LNK_FL_ENABLED : 0); if (ret < 0) { dev_err(csi->dev, "failed to create %s:%u -> %s:%u link\n", entity->name, src_pad_index, @@ -676,19 +678,67 @@ static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier) { struct sun6i_csi *csi = container_of(notifier, struct sun6i_csi, notifier); + struct sun6i_video *video = &csi->video; struct v4l2_device *v4l2_dev = &csi->v4l2_dev; - struct v4l2_subdev *sd; + struct v4l2_subdev *parallel_sd = NULL; + struct v4l2_subdev *mipi_csi2_bridge_sd = NULL; + struct fwnode_handle *handle = NULL; int ret; dev_dbg(csi->dev, "notify complete, all subdevs registered\n"); - sd = list_first_entry(&v4l2_dev->subdevs, struct v4l2_subdev, list); - if (!sd) - return -EINVAL; + /* Find the subdevs that match our fwnode ports. */ + while (1) { + struct v4l2_fwnode_link link; + struct v4l2_subdev *sd; - ret = sun6i_csi_link_entity(csi, &sd->entity, sd->fwnode); - if (ret < 0) - return ret; + handle = fwnode_graph_get_next_endpoint(dev_fwnode(csi->dev), + handle); + if (!handle) + break; + + ret = v4l2_fwnode_parse_link(handle, &link); + if (ret) + break; + + list_for_each_entry(sd, &v4l2_dev->subdevs, list) { + if (!sd->fwnode || link.remote_node != sd->fwnode) + continue; + + switch (link.local_port) { + case 0: + parallel_sd = sd; + break; + case 1: + mipi_csi2_bridge_sd = sd; + break; + } + } + + v4l2_fwnode_put_link(&link); + } + + if (parallel_sd) { + dev_dbg(csi->dev, "linking parallel interface subdev\n"); + + ret = sun6i_csi_link_entity(csi, &video->pads[0], + ¶llel_sd->entity, + parallel_sd->fwnode, true); + if (ret < 0) + return ret; + } + + if (mipi_csi2_bridge_sd) { + dev_dbg(csi->dev, "linking MIPI CSI-2 bridge subdev\n"); + + /* Mark the link as disabled if a parallel subdev is there. */ + ret = sun6i_csi_link_entity(csi, &video->pads[1], + &mipi_csi2_bridge_sd->entity, + mipi_csi2_bridge_sd->fwnode, + parallel_sd ? false : true); + if (ret < 0) + return ret; + } ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev); if (ret < 0) @@ -714,17 +764,18 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi *csi) static int sun6i_csi_v4l2_fwnode_init(struct sun6i_csi *csi) { + struct sun6i_video *video = &csi->video; struct v4l2_fwnode_endpoint *endpoint = NULL; struct fwnode_handle *handle = NULL; int ret; /* Parallel */ - endpoint = &csi->v4l2_ep; + endpoint = &video->parallel_endpoint; handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(csi->dev), 0, 0, FWNODE_GRAPH_ENDPOINT_NEXT); if (!handle) - return 0; + goto mipi_csi2_bridge; ret = v4l2_fwnode_endpoint_parse(handle, endpoint); if (ret) @@ -745,6 +796,30 @@ static int sun6i_csi_v4l2_fwnode_init(struct sun6i_csi *csi) fwnode_handle_put(handle); + /* MIPI CSI-2 bridge */ + +mipi_csi2_bridge: + endpoint = &video->mipi_csi2_bridge_endpoint; + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(csi->dev), 1, 0, + FWNODE_GRAPH_ENDPOINT_NEXT); + if (!handle) + return 0; + + ret = v4l2_fwnode_endpoint_parse(handle, endpoint); + if (ret) + goto error; + + ret = v4l2_async_notifier_add_fwnode_remote_subdev(&csi->notifier, + handle, + &csi->subdev); + if (ret) + goto error; + + fwnode_handle_put(handle); + + /* Set this for our local convenience. */ + endpoint->bus_type = V4L2_MBUS_CSI2_DPHY; + return 0; error: diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h index 7cd23cd74685..0fd2d12e0322 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h @@ -41,9 +41,6 @@ struct sun6i_csi { struct v4l2_async_subdev subdev; struct v4l2_async_notifier notifier; - /* video port settings */ - struct v4l2_fwnode_endpoint v4l2_ep; - struct sun6i_csi_config config; struct sun6i_video video; @@ -52,11 +49,13 @@ struct sun6i_csi { /** * sun6i_csi_is_format_supported() - check if the format supported by csi * @csi: pointer to the csi + * @endpoint: pointer to the endpoint to check * @pixformat: v4l2 pixel format (V4L2_PIX_FMT_*) * @mbus_code: media bus format code (MEDIA_BUS_FMT_*) */ -bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, u32 pixformat, - u32 mbus_code); +bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, + struct v4l2_fwnode_endpoint *endpoint, + u32 pixformat, u32 mbus_code); /** * sun6i_csi_set_power() - power on/off the csi diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c index b55de9ab64d8..e968c0009711 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c @@ -72,22 +72,6 @@ static bool is_pixformat_valid(unsigned int pixformat) return false; } -static struct v4l2_subdev * -sun6i_video_remote_subdev(struct sun6i_video *video, u32 *pad) -{ - struct media_pad *remote; - - remote = media_entity_remote_pad(&video->pad); - - if (!remote || !is_media_entity_v4l2_subdev(remote->entity)) - return NULL; - - if (pad) - *pad = remote->index; - - return media_entity_to_v4l2_subdev(remote->entity); -} - static int sun6i_video_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers, unsigned int *nplanes, @@ -150,7 +134,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count) goto stop_media_pipeline; } - subdev = sun6i_video_remote_subdev(video, NULL); + subdev = video->source_subdev; if (!subdev) goto stop_media_pipeline; @@ -206,6 +190,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count) sun6i_csi_set_stream(video->csi, false); stop_media_pipeline: media_pipeline_stop(&video->vdev.entity); + video->source_subdev = NULL; clear_dma_queue: spin_lock_irqsave(&video->dma_queue_lock, flags); list_for_each_entry(buf, &video->dma_queue, list) @@ -223,13 +208,16 @@ static void sun6i_video_stop_streaming(struct vb2_queue *vq) unsigned long flags; struct sun6i_csi_buffer *buf; - subdev = sun6i_video_remote_subdev(video, NULL); + subdev = video->source_subdev; if (subdev) v4l2_subdev_call(subdev, video, s_stream, 0); sun6i_csi_set_stream(video->csi, false); - media_pipeline_stop(&video->vdev.entity); + if (subdev) + media_pipeline_stop(&video->vdev.entity); + + video->source_subdev = NULL; /* Release all active buffers */ spin_lock_irqsave(&video->dma_queue_lock, flags); @@ -549,22 +537,42 @@ static int sun6i_video_link_validate(struct media_link *link) struct video_device *vdev = container_of(link->sink->entity, struct video_device, entity); struct sun6i_video *video = video_get_drvdata(vdev); + struct v4l2_fwnode_endpoint *endpoint; struct v4l2_subdev_format source_fmt; + struct v4l2_subdev *subdev; int ret; video->mbus_code = 0; - if (!media_entity_remote_pad(link->sink->entity->pads)) { + if (!link->source) { dev_info(video->csi->dev, "video node %s pad not connected\n", vdev->name); return -ENOLINK; } + if (!is_media_entity_v4l2_subdev(link->source->entity)) + return -ENODEV; + + subdev = media_entity_to_v4l2_subdev(link->source->entity); + + if (video->source_subdev) { + dev_err(video->csi->dev, + "unable to connect both parallel and MIPI CSI-2 bridge interfaces\n"); + return -ENOLINK; + } + + if (link->sink == &video->pads[0]) + endpoint = &video->parallel_endpoint; + else if (link->sink == &video->pads[1]) + endpoint = &video->mipi_csi2_bridge_endpoint; + else + return -EINVAL; + ret = sun6i_video_link_validate_get_format(link->source, &source_fmt); if (ret < 0) return ret; - if (!sun6i_csi_is_format_supported(video->csi, + if (!sun6i_csi_is_format_supported(video->csi, endpoint, video->fmt.fmt.pix.pixelformat, source_fmt.format.code)) { dev_err(video->csi->dev, @@ -584,6 +592,8 @@ static int sun6i_video_link_validate(struct media_link *link) } video->mbus_code = source_fmt.format.code; + video->source_endpoint = endpoint; + video->source_subdev = subdev; return 0; } @@ -603,9 +613,10 @@ int sun6i_video_init(struct sun6i_video *video, struct sun6i_csi *csi, video->csi = csi; /* Initialize the media entity... */ - video->pad.flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT; + video->pads[0].flags = MEDIA_PAD_FL_SINK; + video->pads[1].flags = MEDIA_PAD_FL_SINK; vdev->entity.ops = &sun6i_video_media_ops; - ret = media_entity_pads_init(&vdev->entity, 1, &video->pad); + ret = media_entity_pads_init(&vdev->entity, 2, video->pads); if (ret < 0) return ret; diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h index b9cd919c24ac..30d56d98d5e9 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h @@ -15,11 +15,16 @@ struct sun6i_csi; struct sun6i_video { struct video_device vdev; - struct media_pad pad; + struct media_pad pads[2]; struct sun6i_csi *csi; struct mutex lock; + struct v4l2_fwnode_endpoint parallel_endpoint; + struct v4l2_fwnode_endpoint mipi_csi2_bridge_endpoint; + struct v4l2_fwnode_endpoint *source_endpoint; + struct v4l2_subdev *source_subdev; + struct vb2_queue vb2_vidq; spinlock_t dma_queue_lock; struct list_head dma_queue; -- 2.30.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 297F9C433E0 for ; Fri, 15 Jan 2021 20:02:26 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D63A023AC4 for ; Fri, 15 Jan 2021 20:02:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D63A023AC4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=bootlin.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=driverdev-devel-bounces@linuxdriverproject.org Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 553082E118; Fri, 15 Jan 2021 20:02:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nSD+F7CI99WU; Fri, 15 Jan 2021 20:02:17 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id B138727484; Fri, 15 Jan 2021 20:02:16 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id BA5B11BF423 for ; Fri, 15 Jan 2021 20:02:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id B6DFC87492 for ; Fri, 15 Jan 2021 20:02:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6XPgIH9avrUv for ; Fri, 15 Jan 2021 20:02:13 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3917587322 for ; Fri, 15 Jan 2021 20:02:13 +0000 (UTC) X-Originating-IP: 93.29.109.196 Received: from localhost.localdomain (196.109.29.93.rev.sfr.net [93.29.109.196]) (Authenticated sender: paul.kocialkowski@bootlin.com) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 96C0A1C000C; Fri, 15 Jan 2021 20:02:09 +0000 (UTC) From: Paul Kocialkowski To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, devel@driverdev.osuosl.org, linux-sunxi@googlegroups.com Subject: [PATCH v5 08/16] media: sun6i-csi: Add support for MIPI CSI-2 bridge input Date: Fri, 15 Jan 2021 21:01:33 +0100 Message-Id: <20210115200141.1397785-9-paul.kocialkowski@bootlin.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210115200141.1397785-1-paul.kocialkowski@bootlin.com> References: <20210115200141.1397785-1-paul.kocialkowski@bootlin.com> MIME-Version: 1.0 X-BeenThere: driverdev-devel@linuxdriverproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux Driver Project Developer List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jernej Skrabec , Dafna Hirschfeld , Philipp Zabel , Jonathan Corbet , Thomas Petazzoni , Greg Kroah-Hartman , Helen Koike , Vinod Koul , Maxime Ripard , Kishon Vijay Abraham I , Paul Kocialkowski , Chen-Yu Tsai , Rob Herring , Hans Verkuil , Ezequiel Garcia , Yong Deng , Sakari Ailus , Mauro Carvalho Chehab , kevin.lhopital@hotmail.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: driverdev-devel-bounces@linuxdriverproject.org Sender: "devel" The A31 CSI controller supports a MIPI CSI-2 bridge input, which has its own dedicated port in the fwnode graph. Support for this input is added with this change: - two pads are defined for the media entity instead of one and only one needs to be connected at a time; - the pads currently match the fwnode graph representation; - links are created between our pads and the subdevs for each interface and are no longer immutable so that userspace can select which interface to use in case both are bound to a subdev; - fwnode endpoints are parsed and stored for each interface; - the active subdev (and fwnode endpoint) is retrieved when validating the media link at stream on time and cleared at stream off; - an error is raised if both links are active at the same time; - the MIPI interface bit is set if the MIPI CSI-2 bridge endpoint is active. In the future, the media entity representation might evolve to: - distinguish the internal parallel bridge and data formatter; - represent each of the 4 internal channels that can exist between the parallel bridge (for BT656 time-multiplex) and MIPI CSI-2 (internal channels can be mapped to virtual channels); - connect the controller's output to the ISP instead of its DMA engine. Finally note that the MIPI CSI-2 bridges should not be linked in the fwnode graph unless they have a sensor subdev attached. Signed-off-by: Paul Kocialkowski Acked-by: Maxime Ripard --- .../platform/sunxi/sun6i-csi/sun6i_csi.c | 111 +++++++++++++++--- .../platform/sunxi/sun6i-csi/sun6i_csi.h | 9 +- .../platform/sunxi/sun6i-csi/sun6i_video.c | 57 +++++---- .../platform/sunxi/sun6i-csi/sun6i_video.h | 7 +- 4 files changed, 137 insertions(+), 47 deletions(-) diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c index d06028f42534..55cac91f8282 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -49,6 +49,7 @@ static inline struct sun6i_csi_dev *sun6i_csi_to_dev(struct sun6i_csi *csi) /* TODO add 10&12 bit YUV, RGB support */ bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, + struct v4l2_fwnode_endpoint *endpoint, u32 pixformat, u32 mbus_code) { struct sun6i_csi_dev *sdev = sun6i_csi_to_dev(csi); @@ -58,9 +59,9 @@ bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, * 8bit and 16bit bus width. * Identify the media bus format from device tree. */ - if ((sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_PARALLEL - || sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_BT656) - && sdev->csi.v4l2_ep.bus.parallel.bus_width == 16) { + if ((endpoint->bus_type == V4L2_MBUS_PARALLEL || + endpoint->bus_type == V4L2_MBUS_BT656) && + endpoint->bus.parallel.bus_width == 16) { switch (pixformat) { case V4L2_PIX_FMT_HM12: case V4L2_PIX_FMT_NV12: @@ -373,7 +374,7 @@ static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_dev *sdev, static void sun6i_csi_setup_bus(struct sun6i_csi_dev *sdev) { - struct v4l2_fwnode_endpoint *endpoint = &sdev->csi.v4l2_ep; + struct v4l2_fwnode_endpoint *endpoint = sdev->csi.video.source_endpoint; struct sun6i_csi *csi = &sdev->csi; unsigned char bus_width; u32 flags; @@ -459,6 +460,9 @@ static void sun6i_csi_setup_bus(struct sun6i_csi_dev *sdev) if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING) cfg |= CSI_IF_CFG_CLK_POL_FALLING_EDGE; break; + case V4L2_MBUS_CSI2_DPHY: + cfg |= CSI_IF_CFG_MIPI_IF_MIPI; + break; default: dev_warn(sdev->dev, "Unsupported bus type: %d\n", endpoint->bus_type); @@ -636,11 +640,11 @@ void sun6i_csi_set_stream(struct sun6i_csi *csi, bool enable) * Media Controller and V4L2 */ static int sun6i_csi_link_entity(struct sun6i_csi *csi, + struct media_pad *sink_pad, struct media_entity *entity, - struct fwnode_handle *fwnode) + struct fwnode_handle *fwnode, bool enabled) { struct media_entity *sink; - struct media_pad *sink_pad; int src_pad_index; int ret; @@ -654,14 +658,12 @@ static int sun6i_csi_link_entity(struct sun6i_csi *csi, src_pad_index = ret; sink = &csi->video.vdev.entity; - sink_pad = &csi->video.pad; dev_dbg(csi->dev, "creating %s:%u -> %s:%u link\n", entity->name, src_pad_index, sink->name, sink_pad->index); ret = media_create_pad_link(entity, src_pad_index, sink, sink_pad->index, - MEDIA_LNK_FL_ENABLED | - MEDIA_LNK_FL_IMMUTABLE); + enabled ? MEDIA_LNK_FL_ENABLED : 0); if (ret < 0) { dev_err(csi->dev, "failed to create %s:%u -> %s:%u link\n", entity->name, src_pad_index, @@ -676,19 +678,67 @@ static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier) { struct sun6i_csi *csi = container_of(notifier, struct sun6i_csi, notifier); + struct sun6i_video *video = &csi->video; struct v4l2_device *v4l2_dev = &csi->v4l2_dev; - struct v4l2_subdev *sd; + struct v4l2_subdev *parallel_sd = NULL; + struct v4l2_subdev *mipi_csi2_bridge_sd = NULL; + struct fwnode_handle *handle = NULL; int ret; dev_dbg(csi->dev, "notify complete, all subdevs registered\n"); - sd = list_first_entry(&v4l2_dev->subdevs, struct v4l2_subdev, list); - if (!sd) - return -EINVAL; + /* Find the subdevs that match our fwnode ports. */ + while (1) { + struct v4l2_fwnode_link link; + struct v4l2_subdev *sd; - ret = sun6i_csi_link_entity(csi, &sd->entity, sd->fwnode); - if (ret < 0) - return ret; + handle = fwnode_graph_get_next_endpoint(dev_fwnode(csi->dev), + handle); + if (!handle) + break; + + ret = v4l2_fwnode_parse_link(handle, &link); + if (ret) + break; + + list_for_each_entry(sd, &v4l2_dev->subdevs, list) { + if (!sd->fwnode || link.remote_node != sd->fwnode) + continue; + + switch (link.local_port) { + case 0: + parallel_sd = sd; + break; + case 1: + mipi_csi2_bridge_sd = sd; + break; + } + } + + v4l2_fwnode_put_link(&link); + } + + if (parallel_sd) { + dev_dbg(csi->dev, "linking parallel interface subdev\n"); + + ret = sun6i_csi_link_entity(csi, &video->pads[0], + ¶llel_sd->entity, + parallel_sd->fwnode, true); + if (ret < 0) + return ret; + } + + if (mipi_csi2_bridge_sd) { + dev_dbg(csi->dev, "linking MIPI CSI-2 bridge subdev\n"); + + /* Mark the link as disabled if a parallel subdev is there. */ + ret = sun6i_csi_link_entity(csi, &video->pads[1], + &mipi_csi2_bridge_sd->entity, + mipi_csi2_bridge_sd->fwnode, + parallel_sd ? false : true); + if (ret < 0) + return ret; + } ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev); if (ret < 0) @@ -714,17 +764,18 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi *csi) static int sun6i_csi_v4l2_fwnode_init(struct sun6i_csi *csi) { + struct sun6i_video *video = &csi->video; struct v4l2_fwnode_endpoint *endpoint = NULL; struct fwnode_handle *handle = NULL; int ret; /* Parallel */ - endpoint = &csi->v4l2_ep; + endpoint = &video->parallel_endpoint; handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(csi->dev), 0, 0, FWNODE_GRAPH_ENDPOINT_NEXT); if (!handle) - return 0; + goto mipi_csi2_bridge; ret = v4l2_fwnode_endpoint_parse(handle, endpoint); if (ret) @@ -745,6 +796,30 @@ static int sun6i_csi_v4l2_fwnode_init(struct sun6i_csi *csi) fwnode_handle_put(handle); + /* MIPI CSI-2 bridge */ + +mipi_csi2_bridge: + endpoint = &video->mipi_csi2_bridge_endpoint; + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(csi->dev), 1, 0, + FWNODE_GRAPH_ENDPOINT_NEXT); + if (!handle) + return 0; + + ret = v4l2_fwnode_endpoint_parse(handle, endpoint); + if (ret) + goto error; + + ret = v4l2_async_notifier_add_fwnode_remote_subdev(&csi->notifier, + handle, + &csi->subdev); + if (ret) + goto error; + + fwnode_handle_put(handle); + + /* Set this for our local convenience. */ + endpoint->bus_type = V4L2_MBUS_CSI2_DPHY; + return 0; error: diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h index 7cd23cd74685..0fd2d12e0322 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h @@ -41,9 +41,6 @@ struct sun6i_csi { struct v4l2_async_subdev subdev; struct v4l2_async_notifier notifier; - /* video port settings */ - struct v4l2_fwnode_endpoint v4l2_ep; - struct sun6i_csi_config config; struct sun6i_video video; @@ -52,11 +49,13 @@ struct sun6i_csi { /** * sun6i_csi_is_format_supported() - check if the format supported by csi * @csi: pointer to the csi + * @endpoint: pointer to the endpoint to check * @pixformat: v4l2 pixel format (V4L2_PIX_FMT_*) * @mbus_code: media bus format code (MEDIA_BUS_FMT_*) */ -bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, u32 pixformat, - u32 mbus_code); +bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, + struct v4l2_fwnode_endpoint *endpoint, + u32 pixformat, u32 mbus_code); /** * sun6i_csi_set_power() - power on/off the csi diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c index b55de9ab64d8..e968c0009711 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c @@ -72,22 +72,6 @@ static bool is_pixformat_valid(unsigned int pixformat) return false; } -static struct v4l2_subdev * -sun6i_video_remote_subdev(struct sun6i_video *video, u32 *pad) -{ - struct media_pad *remote; - - remote = media_entity_remote_pad(&video->pad); - - if (!remote || !is_media_entity_v4l2_subdev(remote->entity)) - return NULL; - - if (pad) - *pad = remote->index; - - return media_entity_to_v4l2_subdev(remote->entity); -} - static int sun6i_video_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers, unsigned int *nplanes, @@ -150,7 +134,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count) goto stop_media_pipeline; } - subdev = sun6i_video_remote_subdev(video, NULL); + subdev = video->source_subdev; if (!subdev) goto stop_media_pipeline; @@ -206,6 +190,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count) sun6i_csi_set_stream(video->csi, false); stop_media_pipeline: media_pipeline_stop(&video->vdev.entity); + video->source_subdev = NULL; clear_dma_queue: spin_lock_irqsave(&video->dma_queue_lock, flags); list_for_each_entry(buf, &video->dma_queue, list) @@ -223,13 +208,16 @@ static void sun6i_video_stop_streaming(struct vb2_queue *vq) unsigned long flags; struct sun6i_csi_buffer *buf; - subdev = sun6i_video_remote_subdev(video, NULL); + subdev = video->source_subdev; if (subdev) v4l2_subdev_call(subdev, video, s_stream, 0); sun6i_csi_set_stream(video->csi, false); - media_pipeline_stop(&video->vdev.entity); + if (subdev) + media_pipeline_stop(&video->vdev.entity); + + video->source_subdev = NULL; /* Release all active buffers */ spin_lock_irqsave(&video->dma_queue_lock, flags); @@ -549,22 +537,42 @@ static int sun6i_video_link_validate(struct media_link *link) struct video_device *vdev = container_of(link->sink->entity, struct video_device, entity); struct sun6i_video *video = video_get_drvdata(vdev); + struct v4l2_fwnode_endpoint *endpoint; struct v4l2_subdev_format source_fmt; + struct v4l2_subdev *subdev; int ret; video->mbus_code = 0; - if (!media_entity_remote_pad(link->sink->entity->pads)) { + if (!link->source) { dev_info(video->csi->dev, "video node %s pad not connected\n", vdev->name); return -ENOLINK; } + if (!is_media_entity_v4l2_subdev(link->source->entity)) + return -ENODEV; + + subdev = media_entity_to_v4l2_subdev(link->source->entity); + + if (video->source_subdev) { + dev_err(video->csi->dev, + "unable to connect both parallel and MIPI CSI-2 bridge interfaces\n"); + return -ENOLINK; + } + + if (link->sink == &video->pads[0]) + endpoint = &video->parallel_endpoint; + else if (link->sink == &video->pads[1]) + endpoint = &video->mipi_csi2_bridge_endpoint; + else + return -EINVAL; + ret = sun6i_video_link_validate_get_format(link->source, &source_fmt); if (ret < 0) return ret; - if (!sun6i_csi_is_format_supported(video->csi, + if (!sun6i_csi_is_format_supported(video->csi, endpoint, video->fmt.fmt.pix.pixelformat, source_fmt.format.code)) { dev_err(video->csi->dev, @@ -584,6 +592,8 @@ static int sun6i_video_link_validate(struct media_link *link) } video->mbus_code = source_fmt.format.code; + video->source_endpoint = endpoint; + video->source_subdev = subdev; return 0; } @@ -603,9 +613,10 @@ int sun6i_video_init(struct sun6i_video *video, struct sun6i_csi *csi, video->csi = csi; /* Initialize the media entity... */ - video->pad.flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT; + video->pads[0].flags = MEDIA_PAD_FL_SINK; + video->pads[1].flags = MEDIA_PAD_FL_SINK; vdev->entity.ops = &sun6i_video_media_ops; - ret = media_entity_pads_init(&vdev->entity, 1, &video->pad); + ret = media_entity_pads_init(&vdev->entity, 2, video->pads); if (ret < 0) return ret; diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h index b9cd919c24ac..30d56d98d5e9 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h @@ -15,11 +15,16 @@ struct sun6i_csi; struct sun6i_video { struct video_device vdev; - struct media_pad pad; + struct media_pad pads[2]; struct sun6i_csi *csi; struct mutex lock; + struct v4l2_fwnode_endpoint parallel_endpoint; + struct v4l2_fwnode_endpoint mipi_csi2_bridge_endpoint; + struct v4l2_fwnode_endpoint *source_endpoint; + struct v4l2_subdev *source_subdev; + struct vb2_queue vb2_vidq; spinlock_t dma_queue_lock; struct list_head dma_queue; -- 2.30.0 _______________________________________________ devel mailing list devel@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44419C433E0 for ; Fri, 15 Jan 2021 20:05:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AF92B21D7A for ; Fri, 15 Jan 2021 20:05:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF92B21D7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=bootlin.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=4DQserd2u8g02iSYilvVOB6DBx0bu75KuzFOtwDSTag=; b=PVCiLQHnKuOi2KV/7xswvFfH1 VbQ9NKL4IYpUmADfd4TNoM7s9oNc64Xm68oHhq+CKIvIXDezp/MRtnOHtJ78/ufwu5SegeGd5Gmrd 7jRfF+kJxZ4ZSSJjZHUwRQcytHiRN8gbUe1/6mP3+dtE8PRY4mTAG+4nCp3sOPq6hqT1C0Apt7QQZ XCu4KeaE/sFYTuZWTcEcBlqv2D4/Ga3eFP91antPtwTYShvhUQLQm+7vPKSM/M4TmFMKUtTaoOk7O NiBCl7wJff7ULIN17crlX3szatFsKSx//6+qNRRnZdEneAy8ifw3OCxKHNiGPMq78LcWFth4DM4VL TAW/ZHioQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l0VJU-0002s5-IZ; Fri, 15 Jan 2021 20:03:17 +0000 Received: from relay5-d.mail.gandi.net ([217.70.183.197]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l0VIS-0002XZ-Pn for linux-arm-kernel@lists.infradead.org; Fri, 15 Jan 2021 20:02:26 +0000 X-Originating-IP: 93.29.109.196 Received: from localhost.localdomain (196.109.29.93.rev.sfr.net [93.29.109.196]) (Authenticated sender: paul.kocialkowski@bootlin.com) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 96C0A1C000C; Fri, 15 Jan 2021 20:02:09 +0000 (UTC) From: Paul Kocialkowski To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, devel@driverdev.osuosl.org, linux-sunxi@googlegroups.com Subject: [PATCH v5 08/16] media: sun6i-csi: Add support for MIPI CSI-2 bridge input Date: Fri, 15 Jan 2021 21:01:33 +0100 Message-Id: <20210115200141.1397785-9-paul.kocialkowski@bootlin.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210115200141.1397785-1-paul.kocialkowski@bootlin.com> References: <20210115200141.1397785-1-paul.kocialkowski@bootlin.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210115_150213_170288_4B8CDC05 X-CRM114-Status: GOOD ( 29.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jernej Skrabec , Dafna Hirschfeld , Philipp Zabel , Jonathan Corbet , Thomas Petazzoni , Greg Kroah-Hartman , Helen Koike , Vinod Koul , Maxime Ripard , Kishon Vijay Abraham I , Paul Kocialkowski , Chen-Yu Tsai , Rob Herring , Hans Verkuil , Ezequiel Garcia , Yong Deng , Sakari Ailus , Mauro Carvalho Chehab , kevin.lhopital@hotmail.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The A31 CSI controller supports a MIPI CSI-2 bridge input, which has its own dedicated port in the fwnode graph. Support for this input is added with this change: - two pads are defined for the media entity instead of one and only one needs to be connected at a time; - the pads currently match the fwnode graph representation; - links are created between our pads and the subdevs for each interface and are no longer immutable so that userspace can select which interface to use in case both are bound to a subdev; - fwnode endpoints are parsed and stored for each interface; - the active subdev (and fwnode endpoint) is retrieved when validating the media link at stream on time and cleared at stream off; - an error is raised if both links are active at the same time; - the MIPI interface bit is set if the MIPI CSI-2 bridge endpoint is active. In the future, the media entity representation might evolve to: - distinguish the internal parallel bridge and data formatter; - represent each of the 4 internal channels that can exist between the parallel bridge (for BT656 time-multiplex) and MIPI CSI-2 (internal channels can be mapped to virtual channels); - connect the controller's output to the ISP instead of its DMA engine. Finally note that the MIPI CSI-2 bridges should not be linked in the fwnode graph unless they have a sensor subdev attached. Signed-off-by: Paul Kocialkowski Acked-by: Maxime Ripard --- .../platform/sunxi/sun6i-csi/sun6i_csi.c | 111 +++++++++++++++--- .../platform/sunxi/sun6i-csi/sun6i_csi.h | 9 +- .../platform/sunxi/sun6i-csi/sun6i_video.c | 57 +++++---- .../platform/sunxi/sun6i-csi/sun6i_video.h | 7 +- 4 files changed, 137 insertions(+), 47 deletions(-) diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c index d06028f42534..55cac91f8282 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -49,6 +49,7 @@ static inline struct sun6i_csi_dev *sun6i_csi_to_dev(struct sun6i_csi *csi) /* TODO add 10&12 bit YUV, RGB support */ bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, + struct v4l2_fwnode_endpoint *endpoint, u32 pixformat, u32 mbus_code) { struct sun6i_csi_dev *sdev = sun6i_csi_to_dev(csi); @@ -58,9 +59,9 @@ bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, * 8bit and 16bit bus width. * Identify the media bus format from device tree. */ - if ((sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_PARALLEL - || sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_BT656) - && sdev->csi.v4l2_ep.bus.parallel.bus_width == 16) { + if ((endpoint->bus_type == V4L2_MBUS_PARALLEL || + endpoint->bus_type == V4L2_MBUS_BT656) && + endpoint->bus.parallel.bus_width == 16) { switch (pixformat) { case V4L2_PIX_FMT_HM12: case V4L2_PIX_FMT_NV12: @@ -373,7 +374,7 @@ static enum csi_input_seq get_csi_input_seq(struct sun6i_csi_dev *sdev, static void sun6i_csi_setup_bus(struct sun6i_csi_dev *sdev) { - struct v4l2_fwnode_endpoint *endpoint = &sdev->csi.v4l2_ep; + struct v4l2_fwnode_endpoint *endpoint = sdev->csi.video.source_endpoint; struct sun6i_csi *csi = &sdev->csi; unsigned char bus_width; u32 flags; @@ -459,6 +460,9 @@ static void sun6i_csi_setup_bus(struct sun6i_csi_dev *sdev) if (flags & V4L2_MBUS_PCLK_SAMPLE_FALLING) cfg |= CSI_IF_CFG_CLK_POL_FALLING_EDGE; break; + case V4L2_MBUS_CSI2_DPHY: + cfg |= CSI_IF_CFG_MIPI_IF_MIPI; + break; default: dev_warn(sdev->dev, "Unsupported bus type: %d\n", endpoint->bus_type); @@ -636,11 +640,11 @@ void sun6i_csi_set_stream(struct sun6i_csi *csi, bool enable) * Media Controller and V4L2 */ static int sun6i_csi_link_entity(struct sun6i_csi *csi, + struct media_pad *sink_pad, struct media_entity *entity, - struct fwnode_handle *fwnode) + struct fwnode_handle *fwnode, bool enabled) { struct media_entity *sink; - struct media_pad *sink_pad; int src_pad_index; int ret; @@ -654,14 +658,12 @@ static int sun6i_csi_link_entity(struct sun6i_csi *csi, src_pad_index = ret; sink = &csi->video.vdev.entity; - sink_pad = &csi->video.pad; dev_dbg(csi->dev, "creating %s:%u -> %s:%u link\n", entity->name, src_pad_index, sink->name, sink_pad->index); ret = media_create_pad_link(entity, src_pad_index, sink, sink_pad->index, - MEDIA_LNK_FL_ENABLED | - MEDIA_LNK_FL_IMMUTABLE); + enabled ? MEDIA_LNK_FL_ENABLED : 0); if (ret < 0) { dev_err(csi->dev, "failed to create %s:%u -> %s:%u link\n", entity->name, src_pad_index, @@ -676,19 +678,67 @@ static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier) { struct sun6i_csi *csi = container_of(notifier, struct sun6i_csi, notifier); + struct sun6i_video *video = &csi->video; struct v4l2_device *v4l2_dev = &csi->v4l2_dev; - struct v4l2_subdev *sd; + struct v4l2_subdev *parallel_sd = NULL; + struct v4l2_subdev *mipi_csi2_bridge_sd = NULL; + struct fwnode_handle *handle = NULL; int ret; dev_dbg(csi->dev, "notify complete, all subdevs registered\n"); - sd = list_first_entry(&v4l2_dev->subdevs, struct v4l2_subdev, list); - if (!sd) - return -EINVAL; + /* Find the subdevs that match our fwnode ports. */ + while (1) { + struct v4l2_fwnode_link link; + struct v4l2_subdev *sd; - ret = sun6i_csi_link_entity(csi, &sd->entity, sd->fwnode); - if (ret < 0) - return ret; + handle = fwnode_graph_get_next_endpoint(dev_fwnode(csi->dev), + handle); + if (!handle) + break; + + ret = v4l2_fwnode_parse_link(handle, &link); + if (ret) + break; + + list_for_each_entry(sd, &v4l2_dev->subdevs, list) { + if (!sd->fwnode || link.remote_node != sd->fwnode) + continue; + + switch (link.local_port) { + case 0: + parallel_sd = sd; + break; + case 1: + mipi_csi2_bridge_sd = sd; + break; + } + } + + v4l2_fwnode_put_link(&link); + } + + if (parallel_sd) { + dev_dbg(csi->dev, "linking parallel interface subdev\n"); + + ret = sun6i_csi_link_entity(csi, &video->pads[0], + ¶llel_sd->entity, + parallel_sd->fwnode, true); + if (ret < 0) + return ret; + } + + if (mipi_csi2_bridge_sd) { + dev_dbg(csi->dev, "linking MIPI CSI-2 bridge subdev\n"); + + /* Mark the link as disabled if a parallel subdev is there. */ + ret = sun6i_csi_link_entity(csi, &video->pads[1], + &mipi_csi2_bridge_sd->entity, + mipi_csi2_bridge_sd->fwnode, + parallel_sd ? false : true); + if (ret < 0) + return ret; + } ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev); if (ret < 0) @@ -714,17 +764,18 @@ static void sun6i_csi_v4l2_cleanup(struct sun6i_csi *csi) static int sun6i_csi_v4l2_fwnode_init(struct sun6i_csi *csi) { + struct sun6i_video *video = &csi->video; struct v4l2_fwnode_endpoint *endpoint = NULL; struct fwnode_handle *handle = NULL; int ret; /* Parallel */ - endpoint = &csi->v4l2_ep; + endpoint = &video->parallel_endpoint; handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(csi->dev), 0, 0, FWNODE_GRAPH_ENDPOINT_NEXT); if (!handle) - return 0; + goto mipi_csi2_bridge; ret = v4l2_fwnode_endpoint_parse(handle, endpoint); if (ret) @@ -745,6 +796,30 @@ static int sun6i_csi_v4l2_fwnode_init(struct sun6i_csi *csi) fwnode_handle_put(handle); + /* MIPI CSI-2 bridge */ + +mipi_csi2_bridge: + endpoint = &video->mipi_csi2_bridge_endpoint; + handle = fwnode_graph_get_endpoint_by_id(dev_fwnode(csi->dev), 1, 0, + FWNODE_GRAPH_ENDPOINT_NEXT); + if (!handle) + return 0; + + ret = v4l2_fwnode_endpoint_parse(handle, endpoint); + if (ret) + goto error; + + ret = v4l2_async_notifier_add_fwnode_remote_subdev(&csi->notifier, + handle, + &csi->subdev); + if (ret) + goto error; + + fwnode_handle_put(handle); + + /* Set this for our local convenience. */ + endpoint->bus_type = V4L2_MBUS_CSI2_DPHY; + return 0; error: diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h index 7cd23cd74685..0fd2d12e0322 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h @@ -41,9 +41,6 @@ struct sun6i_csi { struct v4l2_async_subdev subdev; struct v4l2_async_notifier notifier; - /* video port settings */ - struct v4l2_fwnode_endpoint v4l2_ep; - struct sun6i_csi_config config; struct sun6i_video video; @@ -52,11 +49,13 @@ struct sun6i_csi { /** * sun6i_csi_is_format_supported() - check if the format supported by csi * @csi: pointer to the csi + * @endpoint: pointer to the endpoint to check * @pixformat: v4l2 pixel format (V4L2_PIX_FMT_*) * @mbus_code: media bus format code (MEDIA_BUS_FMT_*) */ -bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, u32 pixformat, - u32 mbus_code); +bool sun6i_csi_is_format_supported(struct sun6i_csi *csi, + struct v4l2_fwnode_endpoint *endpoint, + u32 pixformat, u32 mbus_code); /** * sun6i_csi_set_power() - power on/off the csi diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c index b55de9ab64d8..e968c0009711 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c @@ -72,22 +72,6 @@ static bool is_pixformat_valid(unsigned int pixformat) return false; } -static struct v4l2_subdev * -sun6i_video_remote_subdev(struct sun6i_video *video, u32 *pad) -{ - struct media_pad *remote; - - remote = media_entity_remote_pad(&video->pad); - - if (!remote || !is_media_entity_v4l2_subdev(remote->entity)) - return NULL; - - if (pad) - *pad = remote->index; - - return media_entity_to_v4l2_subdev(remote->entity); -} - static int sun6i_video_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers, unsigned int *nplanes, @@ -150,7 +134,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count) goto stop_media_pipeline; } - subdev = sun6i_video_remote_subdev(video, NULL); + subdev = video->source_subdev; if (!subdev) goto stop_media_pipeline; @@ -206,6 +190,7 @@ static int sun6i_video_start_streaming(struct vb2_queue *vq, unsigned int count) sun6i_csi_set_stream(video->csi, false); stop_media_pipeline: media_pipeline_stop(&video->vdev.entity); + video->source_subdev = NULL; clear_dma_queue: spin_lock_irqsave(&video->dma_queue_lock, flags); list_for_each_entry(buf, &video->dma_queue, list) @@ -223,13 +208,16 @@ static void sun6i_video_stop_streaming(struct vb2_queue *vq) unsigned long flags; struct sun6i_csi_buffer *buf; - subdev = sun6i_video_remote_subdev(video, NULL); + subdev = video->source_subdev; if (subdev) v4l2_subdev_call(subdev, video, s_stream, 0); sun6i_csi_set_stream(video->csi, false); - media_pipeline_stop(&video->vdev.entity); + if (subdev) + media_pipeline_stop(&video->vdev.entity); + + video->source_subdev = NULL; /* Release all active buffers */ spin_lock_irqsave(&video->dma_queue_lock, flags); @@ -549,22 +537,42 @@ static int sun6i_video_link_validate(struct media_link *link) struct video_device *vdev = container_of(link->sink->entity, struct video_device, entity); struct sun6i_video *video = video_get_drvdata(vdev); + struct v4l2_fwnode_endpoint *endpoint; struct v4l2_subdev_format source_fmt; + struct v4l2_subdev *subdev; int ret; video->mbus_code = 0; - if (!media_entity_remote_pad(link->sink->entity->pads)) { + if (!link->source) { dev_info(video->csi->dev, "video node %s pad not connected\n", vdev->name); return -ENOLINK; } + if (!is_media_entity_v4l2_subdev(link->source->entity)) + return -ENODEV; + + subdev = media_entity_to_v4l2_subdev(link->source->entity); + + if (video->source_subdev) { + dev_err(video->csi->dev, + "unable to connect both parallel and MIPI CSI-2 bridge interfaces\n"); + return -ENOLINK; + } + + if (link->sink == &video->pads[0]) + endpoint = &video->parallel_endpoint; + else if (link->sink == &video->pads[1]) + endpoint = &video->mipi_csi2_bridge_endpoint; + else + return -EINVAL; + ret = sun6i_video_link_validate_get_format(link->source, &source_fmt); if (ret < 0) return ret; - if (!sun6i_csi_is_format_supported(video->csi, + if (!sun6i_csi_is_format_supported(video->csi, endpoint, video->fmt.fmt.pix.pixelformat, source_fmt.format.code)) { dev_err(video->csi->dev, @@ -584,6 +592,8 @@ static int sun6i_video_link_validate(struct media_link *link) } video->mbus_code = source_fmt.format.code; + video->source_endpoint = endpoint; + video->source_subdev = subdev; return 0; } @@ -603,9 +613,10 @@ int sun6i_video_init(struct sun6i_video *video, struct sun6i_csi *csi, video->csi = csi; /* Initialize the media entity... */ - video->pad.flags = MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_MUST_CONNECT; + video->pads[0].flags = MEDIA_PAD_FL_SINK; + video->pads[1].flags = MEDIA_PAD_FL_SINK; vdev->entity.ops = &sun6i_video_media_ops; - ret = media_entity_pads_init(&vdev->entity, 1, &video->pad); + ret = media_entity_pads_init(&vdev->entity, 2, video->pads); if (ret < 0) return ret; diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h index b9cd919c24ac..30d56d98d5e9 100644 --- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h @@ -15,11 +15,16 @@ struct sun6i_csi; struct sun6i_video { struct video_device vdev; - struct media_pad pad; + struct media_pad pads[2]; struct sun6i_csi *csi; struct mutex lock; + struct v4l2_fwnode_endpoint parallel_endpoint; + struct v4l2_fwnode_endpoint mipi_csi2_bridge_endpoint; + struct v4l2_fwnode_endpoint *source_endpoint; + struct v4l2_subdev *source_subdev; + struct vb2_queue vb2_vidq; spinlock_t dma_queue_lock; struct list_head dma_queue; -- 2.30.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel