From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E63BB2C83 for ; Sat, 20 Nov 2021 15:36:27 +0000 (UTC) Received: by mail-ed1-f43.google.com with SMTP id z5so55938732edd.3 for ; Sat, 20 Nov 2021 07:36:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lFBgdagvfj9hhxv3IEdDgaDBd52bTGyq0WBPlUMIXIA=; b=EuFwYQiqpqkm9mUUFv3u+9eUrbqyoTO+gtiK5tuX+U3rA55RCyUxDKrCn1OrMEdNid K9lYt6jhzqExdcRUMRMR2pL3/YdTz2eA5SrnBgq+QGPf8IMAnB1S3vE03t0yWP4oL77R mGLLrPRnO4bpQShcMq/Ew8avvbdbHCbJ263lHt0ev865YxhHiDTLKTc9U9wdczePtR84 t6d1CdFKOlK0YfXUJ11otswsXe6sFkd6Ja6G4tR86oXT50O94ZhV81D5H9WpRc/OHP6i f6NNZcYG5QhCH+vFcetJnMilNcjIpss2CNJndtTJbI2GUr9zLHR3sncvx/JpH6QkgeXj cLgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lFBgdagvfj9hhxv3IEdDgaDBd52bTGyq0WBPlUMIXIA=; b=B1jaUfodsOhErMl+2OxN7BjEs5LFTJZvECVb0hgsoS2JFPVo2UWClofWM6HoUqzuiG DZo6IV6Pknj3ZG0ENS1kU8HXDqVxP25NjWHsYNw3EtcZlqfvvqC+f9Q7QaAAcW6H0Hpu Ndn0AWFFAaCqVBlCk8H/dS3Ps2Iycbneu14SjoU9vLE2h8S3YYIJ4msrYFAILfyrV0Nb GTJFRv/UWc9LQBTqZzaZsI+hxMM9fatuzBeGUNO7pVcspqoq1j0+fm7OC07/2NmroEkq IKSOtxZFRUdppZvOBXJ1AKdLdr3YO6E9Pb0pOMoCuzfYCaVdPyvfzXG26+f1PIH3vqWq gxXg== X-Gm-Message-State: AOAM533s2C5CfJ75iAx3mpKDJeDTIZ7RLarmEdxNE9xtpkb3DIIvRo+Q 2tJSrVSDoKtim8rJ7qFfBqFOhRyXAWNJAa+p+1o= X-Google-Smtp-Source: ABdhPJwf1oM/7o8Qza8mLwNjoE4V0eaLDBF+tVzyW98wajqYlqH36VC/WpDIrPHJprCtda3CK101ib4ne75HgIGnieg= X-Received: by 2002:a17:907:94c2:: with SMTP id dn2mr7972165ejc.325.1637422585888; Sat, 20 Nov 2021 07:36:25 -0800 (PST) Precedence: bulk X-Mailing-List: linux-staging@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20211106183802.893285-1-aford173@gmail.com> <718f7f6d6cd564d031c1963f1590c62d549ae725.camel@ndufresne.ca> <8db00a4b6faa99c940d9bc86e17161eb0db5efe3.camel@ndufresne.ca> <7f94eaacfddb8c5434c17f1e069ea87a17657ce9.camel@ndufresne.ca> In-Reply-To: From: Adam Ford Date: Sat, 20 Nov 2021 09:36:14 -0600 Message-ID: Subject: Re: [RFC 0/5] arm64: imx8mm: Enable Hantro VPUs To: Nicolas Dufresne Cc: Tim Harvey , linux-media , Schrempf Frieder , Marek Vasut , Jagan Teki , Adam Ford-BE , cstevens@beaconembedded.com, Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Rob Herring , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Greg Kroah-Hartman , Heiko Stuebner , Lucas Stach , Joakim Zhang , Alice Guo , Peng Fan , "open list:HANTRO VPU CODEC DRIVER" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , open list , "open list:STAGING SUBSYSTEM" Content-Type: text/plain; charset="UTF-8" On Fri, Nov 19, 2021 at 5:37 PM Adam Ford wrote: > > On Fri, Nov 19, 2021 at 10:29 AM Nicolas Dufresne wrote: > > > > Hi Adam, Tim, > > > > [...] > > > > > > Nicolas and Adam, > > > > > > > > > > > > For the H1 patches in this series: I've been able to test the IMX8MM > > > > > > H1 JPEG encode using GStreamer 1.18.5: > > > > > > $ gst-inspect-1.0 | grep -e "v4l2.*enc" > > > > > > video4linux2: v4l2jpegenc: V4L2 JPEG Encoder > > > > > > $ gst-launch-1.0 videotestsrc ! jpegenc ! rtpjpegpay ! udpsink > > > > > ^ v4l2jpegenc > > > > > > > > > > This is just a transcript error ? > > > > > > > > Nicolas, > > > > > > > > No! Thanks for catching my mistake. I was testing with software encode... ooops! > > > > > > > > 'gst-launch-1.0 videotestsrc ! v4l2jpegenc ! fakesink' actually hangs > > > > the board so likely a power-domain issue there? > > > > > > The v4l2-compliance tests fail on the h1 decoder with a hang, but I > > > think we're writing to registers which are not documented in the Mini > > > TRM. The Mini TRM doesn't explicitly show the JPEG encoding as a > > > feature, but some of the registers state JPEG, but because some of the > > > registers written for the H1 are not documented in the TRM. If those > > > registers are restricted or not in this SoC, I am concerned that it > > > might be related. I'll try to run some more tests this weekend to > > > check on the status of the power-domain stuff. > > > > To verify if the HW support JPEG encoding you can read SWREG63 bit 25. This is > > in the TRM, just not labelled properly. To mimic the decoding side, would be "HW > > synthesis config register X" with the bit labelled SW_ENC_JPEG_PROF (but > > PROF/profile is on or off). If your board hang while reading this, you likely > > didn't get the power bit right. > > > > IMX8 has an undocumented control block thing that we have been fighting with in > > imx8q, perhaps that's your issue. Few driver was proposed, we are still pending > > on NXP solution to be submitted (they asked us to wait, still waiting =)). > > Nicolas, > > Thanks for the suggestion to read offset FC. There was an attempt > made by Lucas Stach to develop a VPU blk-ctrl driver to coordinate the > power-domains with the GPC driver. Unfortunately, it does appear to > hang, so it might not be operating correctly. > > Lucas, > > Do you have any idea of stuff I can try to see if the power domain is > coming online correctly? > > [ 10.434727] imx-pgc imx-pgc-domain.6: request the vpumix domain to power up > [ 10.463647] imx-pgc imx-pgc-domain.6: request the vpumix ADB400 to power up > [ 10.517155] imx-pgc imx-pgc-domain.6: genpd vpumix success > [ 10.728927] vpu: set fuse bits to enable > [ 10.825500] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g1 GPC domain > [ 10.878986] imx-pgc imx-pgc-domain.7: request the vpu-g1 domain to power up > [ 10.932429] imx-pgc imx-pgc-domain.7: genpd vpu-g1 success > [ 10.971988] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g1 success > [ 11.004726] hantro-vpu 38300000.video-codec: registered > nxp,imx8mm-vpu-dec as /dev/video0 > [ 11.040760] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g2 GPC domain > [ 11.066181] imx-pgc imx-pgc-domain.8: request the vpu-g2 domain to power up > [ 11.087887] imx-pgc imx-pgc-domain.8: genpd vpu-g2 success > [ 11.113808] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g2 success > [ 11.139634] hantro-vpu 38310000.video-codec: registered > nxp,imx8mm-vpu-g2-dec as /dev/video1 > [ 11.156463] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-h1 GPC domain > [ 11.170817] imx-pgc imx-pgc-domain.9: request the vpu-h1 domain to power up > [ 11.232990] imx-pgc imx-pgc-domain.9: genpd vpu-h1 success > [ 11.252546] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-h1 success > [ 11.266152] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc > > > > adam > Nicolas, Tim, and Lucas, I think I have the hanging resolved in the power domains, and I'll be pushing the fix to the GPCv2. For the H1 Encoder, I added some debugging code to read the offset 0xfc and print some data based on the findings of that VPU-h1 offset. I basically check the various bits per the TRM to see if they are set and print some splat to indicate whether or not the function is supported. [ 8.861865] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc [ 8.870594] hantro-vpu 38320000.video-codec: Stabilization supported by HW [ 8.889341] hantro-vpu 38320000.video-codec: VP8 encoding supported by HW [ 8.899386] hantro-vpu 38320000.video-codec: H.264 encoding supported by HW [ 8.918171] hantro-vpu 38320000.video-codec: RGB to YUV conversion supported by HW [ 8.934067] hantro-vpu 38320000.video-codec: registered nxp,imx8mm-vpu-h1-enc as /dev/video2 Unfortunately, JPEG is not listed as supported. :-( However, the hanging stops occurring, so I'll be posting a patch to update the GPCv2 code. I can reduce sone device tree duplication, and the G2 throws some splat, but that will be a separate discussion. I can also run v4l2-compliance on the H1 node, and it responds without hanging. root@beacon-imx8mm-kit:~# v4l2-compliance -d2 v4l2-compliance SHA: not available , 64 bits, 64-bit time_t Compliance test for hantro-vpu device /dev/video2: Driver Info: Driver name : hantro-vpu Card type : nxp,imx8mm-vpu-h1-enc Bus info : platform: hantro-vpu Driver version : 5.16.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar < snip> Total for hantro-vpu device /dev/video2: 46, Succeeded: 46, Failed: 0, Warnings: 0 I'll do an RFCv2 on the Hantro G1 and G2 with the H1 removed based on the updated GPCv2 code I'll be pushing shortly, but at least the system doesn't hang, so I'm fairly confident the power domains are working better now even if we cannot support the JPEG. adam > > > > > > > > > > > > > > > > > host=192.168.1.146 port=5000 > > > > > > viewed on client@192.168.1.146 via: > > > > > > $ gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,payload=96 ! > > > > > > rtpjpegdepay ! jpegdec ! autovideosink > > > > > > > > > > > > For the G1/G2 patches in the series I don't see any Gstreamer > > > > > > 'v4l2.*dec' elements. Perhaps I need a newer version of Gstreamer. > > > > > > > > > > Most likely yes, I suggest building gstreamer/ branch "main", GStreamer has now > > > > > a single repository. We are very close to 1.20, which will include stable API > > > > > support of H264, MPEG2 and VP8 decoding. > > > > > > > > > > > > > Ok, let me see if I can navigate through the build process and I'll > > > > get back to you. > > > > > > > > Thanks, > > > > > > > > Tim > > > > > > > > > > > > > > > > I have CSI capture and DSI display currently working on > > > > > > imx8mm-venice-gw73xx-0x that I can play with. The CSI sensor only > > > > > > supports RAW8/RAW10 (and gstreamer currently only supports RAW8) and I > > > > > > can't efficiently convert to something the JPEG encoder likes without > > > > > > bayer2rgbneon (a libneon version). > > > > > > > > > > > > I see from the IMX8MMRM that the 2D GPU supports scaling etc with a > > > > > > wide range of data formats but I'm not sure how to tap into this as > > > > > > that hardware is managed by the vivante driver. On the IMX6QDL there > > > > > > is a separate IPU block that Philipp Zabel wrote a nice mem2mem > > > > > > csc/scaler driver for but I don't see any equivalent currently for > > > > > > IMX8MM. > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Tim > > > > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53846C433F5 for ; Sat, 20 Nov 2021 15:36:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fFWusPPIu6lQ1pZnGbrv9HVOYYbJjTzWTSe7t0XCXDw=; b=bgoMt2fAtc4Y1t mzU0y4oBEDmtVvxPzup5ibFIDL8q64kU9QoAzG2bb6rtocVemYEyRYoWRXlDD8xM83DpBBY4FD5Ki zicjdMVgjjHw5mNpqjBQV2IsTQ6hq6crz9xsL6SErCwYCLwGZfSj0jmc3DJghLVYG6YgAvq83Kn/f 3k9YGTOMojbxUaEakIJ3HUxGch1CxHmxnuuSh3yVbce12TJDB6JOGqS6DkWLrZWumdyKlKb6Jt7yS cG8GzuasWMDyRRQHXyqGwuAZDvzsefi8InMC1eN1EV+cBU8ZraELN820cHbHcuTsAJCdCMronx11h y3hqGcuKovaGD9DmZQRA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1moSPp-00Cev8-MS; Sat, 20 Nov 2021 15:36:33 +0000 Received: from mail-ed1-x52b.google.com ([2a00:1450:4864:20::52b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1moSPm-00Ceu1-9v; Sat, 20 Nov 2021 15:36:32 +0000 Received: by mail-ed1-x52b.google.com with SMTP id o20so11407992eds.10; Sat, 20 Nov 2021 07:36:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lFBgdagvfj9hhxv3IEdDgaDBd52bTGyq0WBPlUMIXIA=; b=EuFwYQiqpqkm9mUUFv3u+9eUrbqyoTO+gtiK5tuX+U3rA55RCyUxDKrCn1OrMEdNid K9lYt6jhzqExdcRUMRMR2pL3/YdTz2eA5SrnBgq+QGPf8IMAnB1S3vE03t0yWP4oL77R mGLLrPRnO4bpQShcMq/Ew8avvbdbHCbJ263lHt0ev865YxhHiDTLKTc9U9wdczePtR84 t6d1CdFKOlK0YfXUJ11otswsXe6sFkd6Ja6G4tR86oXT50O94ZhV81D5H9WpRc/OHP6i f6NNZcYG5QhCH+vFcetJnMilNcjIpss2CNJndtTJbI2GUr9zLHR3sncvx/JpH6QkgeXj cLgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lFBgdagvfj9hhxv3IEdDgaDBd52bTGyq0WBPlUMIXIA=; b=0tSlIXJ2wDIt5pQevVxONU9fMTBu/12J6z0gnbIefNmrDz/UR+UuDOYtBItCqcdz6b t/4XuVZHb8gQ2Ov9E8KBAPmote7mbpWd+CamUwd99EHZmqgSj72Hgad18gx6z9VzAm6Z mRQHnHIEA6BFMuM+TiMUPiYKIlkEDo7Lu7oIlBSOpOWjKi/z7E4o9bfjkRze/ddTpLro HgDoRhBNLWeJVZj5fNJc3zxUSsoh0zA0zwluKFFddOjqY2JgrbRdDZ39Ulu8q61zQLm/ HfaOR4wGabiFbofRUao1STpz165tIPaW3vuXmdeDWKkPXmISkSnmXbi8PSKMugvZguPU 9lrA== X-Gm-Message-State: AOAM532PmD5UUli0AJjEMoKJF6YDHlqrAl2XckAUu1UU/bawgvmGxk0r eWD0DdS4bMDBcGYbmb6fNwVbkO7mlIAXa3ldgKU= X-Google-Smtp-Source: ABdhPJwf1oM/7o8Qza8mLwNjoE4V0eaLDBF+tVzyW98wajqYlqH36VC/WpDIrPHJprCtda3CK101ib4ne75HgIGnieg= X-Received: by 2002:a17:907:94c2:: with SMTP id dn2mr7972165ejc.325.1637422585888; Sat, 20 Nov 2021 07:36:25 -0800 (PST) MIME-Version: 1.0 References: <20211106183802.893285-1-aford173@gmail.com> <718f7f6d6cd564d031c1963f1590c62d549ae725.camel@ndufresne.ca> <8db00a4b6faa99c940d9bc86e17161eb0db5efe3.camel@ndufresne.ca> <7f94eaacfddb8c5434c17f1e069ea87a17657ce9.camel@ndufresne.ca> In-Reply-To: From: Adam Ford Date: Sat, 20 Nov 2021 09:36:14 -0600 Message-ID: Subject: Re: [RFC 0/5] arm64: imx8mm: Enable Hantro VPUs To: Nicolas Dufresne Cc: Tim Harvey , linux-media , Schrempf Frieder , Marek Vasut , Jagan Teki , Adam Ford-BE , cstevens@beaconembedded.com, Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Rob Herring , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Greg Kroah-Hartman , Heiko Stuebner , Lucas Stach , Joakim Zhang , Alice Guo , Peng Fan , "open list:HANTRO VPU CODEC DRIVER" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , open list , "open list:STAGING SUBSYSTEM" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211120_073630_357747_D78D0FF2 X-CRM114-Status: GOOD ( 53.28 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org On Fri, Nov 19, 2021 at 5:37 PM Adam Ford wrote: > > On Fri, Nov 19, 2021 at 10:29 AM Nicolas Dufresne wrote: > > > > Hi Adam, Tim, > > > > [...] > > > > > > Nicolas and Adam, > > > > > > > > > > > > For the H1 patches in this series: I've been able to test the IMX8MM > > > > > > H1 JPEG encode using GStreamer 1.18.5: > > > > > > $ gst-inspect-1.0 | grep -e "v4l2.*enc" > > > > > > video4linux2: v4l2jpegenc: V4L2 JPEG Encoder > > > > > > $ gst-launch-1.0 videotestsrc ! jpegenc ! rtpjpegpay ! udpsink > > > > > ^ v4l2jpegenc > > > > > > > > > > This is just a transcript error ? > > > > > > > > Nicolas, > > > > > > > > No! Thanks for catching my mistake. I was testing with software encode... ooops! > > > > > > > > 'gst-launch-1.0 videotestsrc ! v4l2jpegenc ! fakesink' actually hangs > > > > the board so likely a power-domain issue there? > > > > > > The v4l2-compliance tests fail on the h1 decoder with a hang, but I > > > think we're writing to registers which are not documented in the Mini > > > TRM. The Mini TRM doesn't explicitly show the JPEG encoding as a > > > feature, but some of the registers state JPEG, but because some of the > > > registers written for the H1 are not documented in the TRM. If those > > > registers are restricted or not in this SoC, I am concerned that it > > > might be related. I'll try to run some more tests this weekend to > > > check on the status of the power-domain stuff. > > > > To verify if the HW support JPEG encoding you can read SWREG63 bit 25. This is > > in the TRM, just not labelled properly. To mimic the decoding side, would be "HW > > synthesis config register X" with the bit labelled SW_ENC_JPEG_PROF (but > > PROF/profile is on or off). If your board hang while reading this, you likely > > didn't get the power bit right. > > > > IMX8 has an undocumented control block thing that we have been fighting with in > > imx8q, perhaps that's your issue. Few driver was proposed, we are still pending > > on NXP solution to be submitted (they asked us to wait, still waiting =)). > > Nicolas, > > Thanks for the suggestion to read offset FC. There was an attempt > made by Lucas Stach to develop a VPU blk-ctrl driver to coordinate the > power-domains with the GPC driver. Unfortunately, it does appear to > hang, so it might not be operating correctly. > > Lucas, > > Do you have any idea of stuff I can try to see if the power domain is > coming online correctly? > > [ 10.434727] imx-pgc imx-pgc-domain.6: request the vpumix domain to power up > [ 10.463647] imx-pgc imx-pgc-domain.6: request the vpumix ADB400 to power up > [ 10.517155] imx-pgc imx-pgc-domain.6: genpd vpumix success > [ 10.728927] vpu: set fuse bits to enable > [ 10.825500] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g1 GPC domain > [ 10.878986] imx-pgc imx-pgc-domain.7: request the vpu-g1 domain to power up > [ 10.932429] imx-pgc imx-pgc-domain.7: genpd vpu-g1 success > [ 10.971988] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g1 success > [ 11.004726] hantro-vpu 38300000.video-codec: registered > nxp,imx8mm-vpu-dec as /dev/video0 > [ 11.040760] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g2 GPC domain > [ 11.066181] imx-pgc imx-pgc-domain.8: request the vpu-g2 domain to power up > [ 11.087887] imx-pgc imx-pgc-domain.8: genpd vpu-g2 success > [ 11.113808] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g2 success > [ 11.139634] hantro-vpu 38310000.video-codec: registered > nxp,imx8mm-vpu-g2-dec as /dev/video1 > [ 11.156463] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-h1 GPC domain > [ 11.170817] imx-pgc imx-pgc-domain.9: request the vpu-h1 domain to power up > [ 11.232990] imx-pgc imx-pgc-domain.9: genpd vpu-h1 success > [ 11.252546] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-h1 success > [ 11.266152] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc > > > > adam > Nicolas, Tim, and Lucas, I think I have the hanging resolved in the power domains, and I'll be pushing the fix to the GPCv2. For the H1 Encoder, I added some debugging code to read the offset 0xfc and print some data based on the findings of that VPU-h1 offset. I basically check the various bits per the TRM to see if they are set and print some splat to indicate whether or not the function is supported. [ 8.861865] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc [ 8.870594] hantro-vpu 38320000.video-codec: Stabilization supported by HW [ 8.889341] hantro-vpu 38320000.video-codec: VP8 encoding supported by HW [ 8.899386] hantro-vpu 38320000.video-codec: H.264 encoding supported by HW [ 8.918171] hantro-vpu 38320000.video-codec: RGB to YUV conversion supported by HW [ 8.934067] hantro-vpu 38320000.video-codec: registered nxp,imx8mm-vpu-h1-enc as /dev/video2 Unfortunately, JPEG is not listed as supported. :-( However, the hanging stops occurring, so I'll be posting a patch to update the GPCv2 code. I can reduce sone device tree duplication, and the G2 throws some splat, but that will be a separate discussion. I can also run v4l2-compliance on the H1 node, and it responds without hanging. root@beacon-imx8mm-kit:~# v4l2-compliance -d2 v4l2-compliance SHA: not available , 64 bits, 64-bit time_t Compliance test for hantro-vpu device /dev/video2: Driver Info: Driver name : hantro-vpu Card type : nxp,imx8mm-vpu-h1-enc Bus info : platform: hantro-vpu Driver version : 5.16.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar < snip> Total for hantro-vpu device /dev/video2: 46, Succeeded: 46, Failed: 0, Warnings: 0 I'll do an RFCv2 on the Hantro G1 and G2 with the H1 removed based on the updated GPCv2 code I'll be pushing shortly, but at least the system doesn't hang, so I'm fairly confident the power domains are working better now even if we cannot support the JPEG. adam > > > > > > > > > > > > > > > > > host=192.168.1.146 port=5000 > > > > > > viewed on client@192.168.1.146 via: > > > > > > $ gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,payload=96 ! > > > > > > rtpjpegdepay ! jpegdec ! autovideosink > > > > > > > > > > > > For the G1/G2 patches in the series I don't see any Gstreamer > > > > > > 'v4l2.*dec' elements. Perhaps I need a newer version of Gstreamer. > > > > > > > > > > Most likely yes, I suggest building gstreamer/ branch "main", GStreamer has now > > > > > a single repository. We are very close to 1.20, which will include stable API > > > > > support of H264, MPEG2 and VP8 decoding. > > > > > > > > > > > > > Ok, let me see if I can navigate through the build process and I'll > > > > get back to you. > > > > > > > > Thanks, > > > > > > > > Tim > > > > > > > > > > > > > > > > I have CSI capture and DSI display currently working on > > > > > > imx8mm-venice-gw73xx-0x that I can play with. The CSI sensor only > > > > > > supports RAW8/RAW10 (and gstreamer currently only supports RAW8) and I > > > > > > can't efficiently convert to something the JPEG encoder likes without > > > > > > bayer2rgbneon (a libneon version). > > > > > > > > > > > > I see from the IMX8MMRM that the 2D GPU supports scaling etc with a > > > > > > wide range of data formats but I'm not sure how to tap into this as > > > > > > that hardware is managed by the vivante driver. On the IMX6QDL there > > > > > > is a separate IPU block that Philipp Zabel wrote a nice mem2mem > > > > > > csc/scaler driver for but I don't see any equivalent currently for > > > > > > IMX8MM. > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Tim > > > > > > > _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8F2BBC433F5 for ; Sat, 20 Nov 2021 15:38:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AfH9ZCrKHSKiJFyGOlDUrvXpLqp1i0zZUOeq9lZzThM=; b=3SICAJCXdbY1Q0 7Dem9wVNs0AimK9vOobK1aMHSyawFw1o5l1J3H2AQv18cbgWjBdGcR+19xNDn2iIOl91t7XrFk37l VbwcytXnQDa1X6EXUrV0kUmPZcyc+zk5kk/viIi8A2ZeTulInwITEqahqywj/Xb9VsrbPQNP2Bs3V 6LlZ7MgSjZlDApdyWGL4Eu5FtdV4uIgnKVDKsvLfmCcmEROq3r5xJ3K8Sc/z7SRQgnBFxVcIggwvv WFrxKe2ItDIZakH+s/4HNxOY6KbYoFY8ln37VM37WvNJvRJKn4O8KDklwu045o03QN+VxA+DBL9td DGFxHl/QBvXpO6tOCbDw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1moSPr-00CevE-6O; Sat, 20 Nov 2021 15:36:35 +0000 Received: from mail-ed1-x52b.google.com ([2a00:1450:4864:20::52b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1moSPm-00Ceu1-9v; Sat, 20 Nov 2021 15:36:32 +0000 Received: by mail-ed1-x52b.google.com with SMTP id o20so11407992eds.10; Sat, 20 Nov 2021 07:36:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lFBgdagvfj9hhxv3IEdDgaDBd52bTGyq0WBPlUMIXIA=; b=EuFwYQiqpqkm9mUUFv3u+9eUrbqyoTO+gtiK5tuX+U3rA55RCyUxDKrCn1OrMEdNid K9lYt6jhzqExdcRUMRMR2pL3/YdTz2eA5SrnBgq+QGPf8IMAnB1S3vE03t0yWP4oL77R mGLLrPRnO4bpQShcMq/Ew8avvbdbHCbJ263lHt0ev865YxhHiDTLKTc9U9wdczePtR84 t6d1CdFKOlK0YfXUJ11otswsXe6sFkd6Ja6G4tR86oXT50O94ZhV81D5H9WpRc/OHP6i f6NNZcYG5QhCH+vFcetJnMilNcjIpss2CNJndtTJbI2GUr9zLHR3sncvx/JpH6QkgeXj cLgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lFBgdagvfj9hhxv3IEdDgaDBd52bTGyq0WBPlUMIXIA=; b=0tSlIXJ2wDIt5pQevVxONU9fMTBu/12J6z0gnbIefNmrDz/UR+UuDOYtBItCqcdz6b t/4XuVZHb8gQ2Ov9E8KBAPmote7mbpWd+CamUwd99EHZmqgSj72Hgad18gx6z9VzAm6Z mRQHnHIEA6BFMuM+TiMUPiYKIlkEDo7Lu7oIlBSOpOWjKi/z7E4o9bfjkRze/ddTpLro HgDoRhBNLWeJVZj5fNJc3zxUSsoh0zA0zwluKFFddOjqY2JgrbRdDZ39Ulu8q61zQLm/ HfaOR4wGabiFbofRUao1STpz165tIPaW3vuXmdeDWKkPXmISkSnmXbi8PSKMugvZguPU 9lrA== X-Gm-Message-State: AOAM532PmD5UUli0AJjEMoKJF6YDHlqrAl2XckAUu1UU/bawgvmGxk0r eWD0DdS4bMDBcGYbmb6fNwVbkO7mlIAXa3ldgKU= X-Google-Smtp-Source: ABdhPJwf1oM/7o8Qza8mLwNjoE4V0eaLDBF+tVzyW98wajqYlqH36VC/WpDIrPHJprCtda3CK101ib4ne75HgIGnieg= X-Received: by 2002:a17:907:94c2:: with SMTP id dn2mr7972165ejc.325.1637422585888; Sat, 20 Nov 2021 07:36:25 -0800 (PST) MIME-Version: 1.0 References: <20211106183802.893285-1-aford173@gmail.com> <718f7f6d6cd564d031c1963f1590c62d549ae725.camel@ndufresne.ca> <8db00a4b6faa99c940d9bc86e17161eb0db5efe3.camel@ndufresne.ca> <7f94eaacfddb8c5434c17f1e069ea87a17657ce9.camel@ndufresne.ca> In-Reply-To: From: Adam Ford Date: Sat, 20 Nov 2021 09:36:14 -0600 Message-ID: Subject: Re: [RFC 0/5] arm64: imx8mm: Enable Hantro VPUs To: Nicolas Dufresne Cc: Tim Harvey , linux-media , Schrempf Frieder , Marek Vasut , Jagan Teki , Adam Ford-BE , cstevens@beaconembedded.com, Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Rob Herring , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Greg Kroah-Hartman , Heiko Stuebner , Lucas Stach , Joakim Zhang , Alice Guo , Peng Fan , "open list:HANTRO VPU CODEC DRIVER" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , open list , "open list:STAGING SUBSYSTEM" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211120_073630_357747_D78D0FF2 X-CRM114-Status: GOOD ( 53.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Nov 19, 2021 at 5:37 PM Adam Ford wrote: > > On Fri, Nov 19, 2021 at 10:29 AM Nicolas Dufresne wrote: > > > > Hi Adam, Tim, > > > > [...] > > > > > > Nicolas and Adam, > > > > > > > > > > > > For the H1 patches in this series: I've been able to test the IMX8MM > > > > > > H1 JPEG encode using GStreamer 1.18.5: > > > > > > $ gst-inspect-1.0 | grep -e "v4l2.*enc" > > > > > > video4linux2: v4l2jpegenc: V4L2 JPEG Encoder > > > > > > $ gst-launch-1.0 videotestsrc ! jpegenc ! rtpjpegpay ! udpsink > > > > > ^ v4l2jpegenc > > > > > > > > > > This is just a transcript error ? > > > > > > > > Nicolas, > > > > > > > > No! Thanks for catching my mistake. I was testing with software encode... ooops! > > > > > > > > 'gst-launch-1.0 videotestsrc ! v4l2jpegenc ! fakesink' actually hangs > > > > the board so likely a power-domain issue there? > > > > > > The v4l2-compliance tests fail on the h1 decoder with a hang, but I > > > think we're writing to registers which are not documented in the Mini > > > TRM. The Mini TRM doesn't explicitly show the JPEG encoding as a > > > feature, but some of the registers state JPEG, but because some of the > > > registers written for the H1 are not documented in the TRM. If those > > > registers are restricted or not in this SoC, I am concerned that it > > > might be related. I'll try to run some more tests this weekend to > > > check on the status of the power-domain stuff. > > > > To verify if the HW support JPEG encoding you can read SWREG63 bit 25. This is > > in the TRM, just not labelled properly. To mimic the decoding side, would be "HW > > synthesis config register X" with the bit labelled SW_ENC_JPEG_PROF (but > > PROF/profile is on or off). If your board hang while reading this, you likely > > didn't get the power bit right. > > > > IMX8 has an undocumented control block thing that we have been fighting with in > > imx8q, perhaps that's your issue. Few driver was proposed, we are still pending > > on NXP solution to be submitted (they asked us to wait, still waiting =)). > > Nicolas, > > Thanks for the suggestion to read offset FC. There was an attempt > made by Lucas Stach to develop a VPU blk-ctrl driver to coordinate the > power-domains with the GPC driver. Unfortunately, it does appear to > hang, so it might not be operating correctly. > > Lucas, > > Do you have any idea of stuff I can try to see if the power domain is > coming online correctly? > > [ 10.434727] imx-pgc imx-pgc-domain.6: request the vpumix domain to power up > [ 10.463647] imx-pgc imx-pgc-domain.6: request the vpumix ADB400 to power up > [ 10.517155] imx-pgc imx-pgc-domain.6: genpd vpumix success > [ 10.728927] vpu: set fuse bits to enable > [ 10.825500] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g1 GPC domain > [ 10.878986] imx-pgc imx-pgc-domain.7: request the vpu-g1 domain to power up > [ 10.932429] imx-pgc imx-pgc-domain.7: genpd vpu-g1 success > [ 10.971988] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g1 success > [ 11.004726] hantro-vpu 38300000.video-codec: registered > nxp,imx8mm-vpu-dec as /dev/video0 > [ 11.040760] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g2 GPC domain > [ 11.066181] imx-pgc imx-pgc-domain.8: request the vpu-g2 domain to power up > [ 11.087887] imx-pgc imx-pgc-domain.8: genpd vpu-g2 success > [ 11.113808] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g2 success > [ 11.139634] hantro-vpu 38310000.video-codec: registered > nxp,imx8mm-vpu-g2-dec as /dev/video1 > [ 11.156463] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-h1 GPC domain > [ 11.170817] imx-pgc imx-pgc-domain.9: request the vpu-h1 domain to power up > [ 11.232990] imx-pgc imx-pgc-domain.9: genpd vpu-h1 success > [ 11.252546] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-h1 success > [ 11.266152] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc > > > > adam > Nicolas, Tim, and Lucas, I think I have the hanging resolved in the power domains, and I'll be pushing the fix to the GPCv2. For the H1 Encoder, I added some debugging code to read the offset 0xfc and print some data based on the findings of that VPU-h1 offset. I basically check the various bits per the TRM to see if they are set and print some splat to indicate whether or not the function is supported. [ 8.861865] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc [ 8.870594] hantro-vpu 38320000.video-codec: Stabilization supported by HW [ 8.889341] hantro-vpu 38320000.video-codec: VP8 encoding supported by HW [ 8.899386] hantro-vpu 38320000.video-codec: H.264 encoding supported by HW [ 8.918171] hantro-vpu 38320000.video-codec: RGB to YUV conversion supported by HW [ 8.934067] hantro-vpu 38320000.video-codec: registered nxp,imx8mm-vpu-h1-enc as /dev/video2 Unfortunately, JPEG is not listed as supported. :-( However, the hanging stops occurring, so I'll be posting a patch to update the GPCv2 code. I can reduce sone device tree duplication, and the G2 throws some splat, but that will be a separate discussion. I can also run v4l2-compliance on the H1 node, and it responds without hanging. root@beacon-imx8mm-kit:~# v4l2-compliance -d2 v4l2-compliance SHA: not available , 64 bits, 64-bit time_t Compliance test for hantro-vpu device /dev/video2: Driver Info: Driver name : hantro-vpu Card type : nxp,imx8mm-vpu-h1-enc Bus info : platform: hantro-vpu Driver version : 5.16.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar < snip> Total for hantro-vpu device /dev/video2: 46, Succeeded: 46, Failed: 0, Warnings: 0 I'll do an RFCv2 on the Hantro G1 and G2 with the H1 removed based on the updated GPCv2 code I'll be pushing shortly, but at least the system doesn't hang, so I'm fairly confident the power domains are working better now even if we cannot support the JPEG. adam > > > > > > > > > > > > > > > > > host=192.168.1.146 port=5000 > > > > > > viewed on client@192.168.1.146 via: > > > > > > $ gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,payload=96 ! > > > > > > rtpjpegdepay ! jpegdec ! autovideosink > > > > > > > > > > > > For the G1/G2 patches in the series I don't see any Gstreamer > > > > > > 'v4l2.*dec' elements. Perhaps I need a newer version of Gstreamer. > > > > > > > > > > Most likely yes, I suggest building gstreamer/ branch "main", GStreamer has now > > > > > a single repository. We are very close to 1.20, which will include stable API > > > > > support of H264, MPEG2 and VP8 decoding. > > > > > > > > > > > > > Ok, let me see if I can navigate through the build process and I'll > > > > get back to you. > > > > > > > > Thanks, > > > > > > > > Tim > > > > > > > > > > > > > > > > I have CSI capture and DSI display currently working on > > > > > > imx8mm-venice-gw73xx-0x that I can play with. The CSI sensor only > > > > > > supports RAW8/RAW10 (and gstreamer currently only supports RAW8) and I > > > > > > can't efficiently convert to something the JPEG encoder likes without > > > > > > bayer2rgbneon (a libneon version). > > > > > > > > > > > > I see from the IMX8MMRM that the 2D GPU supports scaling etc with a > > > > > > wide range of data formats but I'm not sure how to tap into this as > > > > > > that hardware is managed by the vivante driver. On the IMX6QDL there > > > > > > is a separate IPU block that Philipp Zabel wrote a nice mem2mem > > > > > > csc/scaler driver for but I don't see any equivalent currently for > > > > > > IMX8MM. > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Tim > > > > > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel