From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-il1-f180.google.com (mail-il1-f180.google.com [209.85.166.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19B4D68 for ; Fri, 19 Nov 2021 16:29:52 +0000 (UTC) Received: by mail-il1-f180.google.com with SMTP id w15so10780579ill.2 for ; Fri, 19 Nov 2021 08:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ndufresne-ca.20210112.gappssmtp.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=NA+2l6X3VrlJxsJamfaAblN1iQr9G5GeyeDjrzqWldE=; b=pWYGr8brYDg2I9xcuGuhurThossAPr1q87fBQPqCiM2BAja4C7uFnX65Fix38gLbMR at4/FYFXGd1o9zn1i0YRa4Hl3P62bx1s2GsXPKOGSzrVsAT1Xw8HlYNWbORdzoDIOeqe 0qslCNUJ66VzbM5OECQeydfyhNNqUWwGP6afEEwmpoOMoNIZnNaZfchouPoszwNYzV1v y1bimltU6qqxfdJSKgW6NjHgo6foggqrexAHaTR5BhNExaTs0kreVzSvKeiuvCJBGMM5 4hhlHH8MqKp2ss3LA9dpdsxQOKHOGkUr880PDTOgcILg0r5nps7M4Bb//54zUt1tJBXH UXjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=NA+2l6X3VrlJxsJamfaAblN1iQr9G5GeyeDjrzqWldE=; b=7As5YMpkTV8va2H9rpHBy8b9HSBrxYMwmH3a5RZ4SgVRf2fAmyF3vdegbHl7Crc7TV 9eeIBKJmBw4yns98+V8p86yInPpTsguc0mb3uq2NyjQZdaNfG1p35EeedbzK2ttkIPSn Sp+6E1oAe5tTMrumcQ4l/H05sOPSeQNl1TXDSdzQ2oK+kLl8vwZC5GERFUGT6SZMzbW/ Y8i62slY/i+v+OvKrYxWG2zr77rwLWx3uh0Mdmf82FNhhknzMRN9BvHAeao4ADkP+9nV mX7+IP67q6W0apD3PQbOLHpHNR9PPcpTgWC6zBbqoLHkwhg5//TgctpvoeDpHyKtHhjU BSCQ== X-Gm-Message-State: AOAM532Ut4aMTVlGcDX2Iue4yJNrTiF2deQu/iswSgf44vwOjWVKR3lD /zmisca7ngAd1CnSkq1Agi7Neg== X-Google-Smtp-Source: ABdhPJy9+PK69iHyuVZcAK9TkCljtUM0G3YiAMHQdYjwnsW/n1HgnSVsWmM0mEP/36cqcfaj9/5ecQ== X-Received: by 2002:a92:d343:: with SMTP id a3mr5688828ilh.136.1637339392190; Fri, 19 Nov 2021 08:29:52 -0800 (PST) Received: from nicolas-tpx395.localdomain (mtl.collabora.ca. [66.171.169.34]) by smtp.gmail.com with ESMTPSA id c7sm108143iob.28.2021.11.19.08.29.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Nov 2021 08:29:51 -0800 (PST) Message-ID: <7f94eaacfddb8c5434c17f1e069ea87a17657ce9.camel@ndufresne.ca> Subject: Re: [RFC 0/5] arm64: imx8mm: Enable Hantro VPUs From: Nicolas Dufresne To: Adam Ford , Tim Harvey Cc: linux-media , Schrempf Frieder , Marek Vasut , Jagan Teki , Adam Ford-BE , cstevens@beaconembedded.com, Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Rob Herring , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Greg Kroah-Hartman , Heiko Stuebner , Lucas Stach , Joakim Zhang , Alice Guo , Peng Fan , "open list:HANTRO VPU CODEC DRIVER" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , open list , "open list:STAGING SUBSYSTEM" Date: Fri, 19 Nov 2021 11:29:48 -0500 In-Reply-To: References: <20211106183802.893285-1-aford173@gmail.com> <718f7f6d6cd564d031c1963f1590c62d549ae725.camel@ndufresne.ca> <8db00a4b6faa99c940d9bc86e17161eb0db5efe3.camel@ndufresne.ca> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 (3.42.1-1.fc35) Precedence: bulk X-Mailing-List: linux-staging@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Hi Adam, Tim, [...] > > > > Nicolas and Adam, > > > > > > > > For the H1 patches in this series: I've been able to test the IMX8MM > > > > H1 JPEG encode using GStreamer 1.18.5: > > > > $ gst-inspect-1.0 | grep -e "v4l2.*enc" > > > > video4linux2: v4l2jpegenc: V4L2 JPEG Encoder > > > > $ gst-launch-1.0 videotestsrc ! jpegenc ! rtpjpegpay ! udpsink > > > ^ v4l2jpegenc > > > > > > This is just a transcript error ? > > > > Nicolas, > > > > No! Thanks for catching my mistake. I was testing with software encode... ooops! > > > > 'gst-launch-1.0 videotestsrc ! v4l2jpegenc ! fakesink' actually hangs > > the board so likely a power-domain issue there? > > The v4l2-compliance tests fail on the h1 decoder with a hang, but I > think we're writing to registers which are not documented in the Mini > TRM. The Mini TRM doesn't explicitly show the JPEG encoding as a > feature, but some of the registers state JPEG, but because some of the > registers written for the H1 are not documented in the TRM. If those > registers are restricted or not in this SoC, I am concerned that it > might be related. I'll try to run some more tests this weekend to > check on the status of the power-domain stuff. To verify if the HW support JPEG encoding you can read SWREG63 bit 25. This is in the TRM, just not labelled properly. To mimic the decoding side, would be "HW synthesis config register X" with the bit labelled SW_ENC_JPEG_PROF (but PROF/profile is on or off). If your board hang while reading this, you likely didn't get the power bit right. IMX8 has an undocumented control block thing that we have been fighting with in imx8q, perhaps that's your issue. Few driver was proposed, we are still pending on NXP solution to be submitted (they asked us to wait, still waiting =)). > > > > > > > > > host=192.168.1.146 port=5000 > > > > viewed on client@192.168.1.146 via: > > > > $ gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,payload=96 ! > > > > rtpjpegdepay ! jpegdec ! autovideosink > > > > > > > > For the G1/G2 patches in the series I don't see any Gstreamer > > > > 'v4l2.*dec' elements. Perhaps I need a newer version of Gstreamer. > > > > > > Most likely yes, I suggest building gstreamer/ branch "main", GStreamer has now > > > a single repository. We are very close to 1.20, which will include stable API > > > support of H264, MPEG2 and VP8 decoding. > > > > > > > Ok, let me see if I can navigate through the build process and I'll > > get back to you. > > > > Thanks, > > > > Tim > > > > > > > > > > I have CSI capture and DSI display currently working on > > > > imx8mm-venice-gw73xx-0x that I can play with. The CSI sensor only > > > > supports RAW8/RAW10 (and gstreamer currently only supports RAW8) and I > > > > can't efficiently convert to something the JPEG encoder likes without > > > > bayer2rgbneon (a libneon version). > > > > > > > > I see from the IMX8MMRM that the 2D GPU supports scaling etc with a > > > > wide range of data formats but I'm not sure how to tap into this as > > > > that hardware is managed by the vivante driver. On the IMX6QDL there > > > > is a separate IPU block that Philipp Zabel wrote a nice mem2mem > > > > csc/scaler driver for but I don't see any equivalent currently for > > > > IMX8MM. > > > > > > > > Best regards, > > > > > > > > Tim > > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B35B8C433F5 for ; Fri, 19 Nov 2021 16:29:59 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 774A761B1E for ; Fri, 19 Nov 2021 16:29:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 774A761B1E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ndufresne.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:Cc:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=D8XzijA2Mb+YJw3O+K9c0eil4MbWLTSct/cYN6z3ac8=; b=u1hiSDWW1WpC+S g92ZdeNwLq10l49Lh1StVpTSmOby93JOhaJ50m4vC2Tekg5wq/VycPLXJSv327KZQz9aRK9meWx95 jKJBIJO9X1cSIBvPa//hbLcI7TALzk0qjg/h2wrhihjDY4wA2l5NJfz8wihjbBasJvCDxr523XWQs OWUmYjmq7dIr2Ys9lAqQpjIpGpIjj6DhDTQyhnzDEha+64lvUUniDG6m9Y70RruOJwhv779NvbzwL BghcBnM3wBqsSWbCDNhQWgrND2n56pcf0OGl3r6CkQ9Mk4Pu45KvnoLjh/N4w3nSCaJDJnDS/sPBS N9PTmufm/BTe4wzXIKyQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mo6lv-00B1h0-Md; Fri, 19 Nov 2021 16:29:55 +0000 Received: from mail-il1-x129.google.com ([2607:f8b0:4864:20::129]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mo6lt-00B1f2-00 for linux-rockchip@lists.infradead.org; Fri, 19 Nov 2021 16:29:54 +0000 Received: by mail-il1-x129.google.com with SMTP id e8so10756314ilu.9 for ; Fri, 19 Nov 2021 08:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ndufresne-ca.20210112.gappssmtp.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=NA+2l6X3VrlJxsJamfaAblN1iQr9G5GeyeDjrzqWldE=; b=pWYGr8brYDg2I9xcuGuhurThossAPr1q87fBQPqCiM2BAja4C7uFnX65Fix38gLbMR at4/FYFXGd1o9zn1i0YRa4Hl3P62bx1s2GsXPKOGSzrVsAT1Xw8HlYNWbORdzoDIOeqe 0qslCNUJ66VzbM5OECQeydfyhNNqUWwGP6afEEwmpoOMoNIZnNaZfchouPoszwNYzV1v y1bimltU6qqxfdJSKgW6NjHgo6foggqrexAHaTR5BhNExaTs0kreVzSvKeiuvCJBGMM5 4hhlHH8MqKp2ss3LA9dpdsxQOKHOGkUr880PDTOgcILg0r5nps7M4Bb//54zUt1tJBXH UXjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=NA+2l6X3VrlJxsJamfaAblN1iQr9G5GeyeDjrzqWldE=; b=t8AIF/ol0+LAdmOWEmio1Zh6vOpewHV2Q3Km2C7yOY7eJcVCEjffX2qJ6dC7IFjsvW /2fH8jVVAzykKV/ZJYIwYfiJiECR6n/kuqdZGQijGS2IJDo9UG9F1ayezIg1SbDm/LJN Ij1/EM1GKwC+MoFkWuWPcOt347/jheY6esQ52hDNRK6Ad0ZIfY0COBwK7G73IkkRRWKB W1NgZkA4aUEvtd8VVSbBSinsuDbk2q9Ol7bUr+3Nb9u97b3/35cW7x30e9LdGkfjefda LUUVhmpHD4mAzHmQ6QfKjOekaoC8cVec2EvBl1IF8up2FUuRDq6/XGai9DybiVdXG0yE Fpaw== X-Gm-Message-State: AOAM532qtc7qov1vuD6WvkpgJaW94Hljaqlgk805Ex1O+bd2eorhZe/5 eiTDFIFOU/TdSxE+8IltHM8eTg== X-Google-Smtp-Source: ABdhPJy9+PK69iHyuVZcAK9TkCljtUM0G3YiAMHQdYjwnsW/n1HgnSVsWmM0mEP/36cqcfaj9/5ecQ== X-Received: by 2002:a92:d343:: with SMTP id a3mr5688828ilh.136.1637339392190; Fri, 19 Nov 2021 08:29:52 -0800 (PST) Received: from nicolas-tpx395.localdomain (mtl.collabora.ca. [66.171.169.34]) by smtp.gmail.com with ESMTPSA id c7sm108143iob.28.2021.11.19.08.29.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Nov 2021 08:29:51 -0800 (PST) Message-ID: <7f94eaacfddb8c5434c17f1e069ea87a17657ce9.camel@ndufresne.ca> Subject: Re: [RFC 0/5] arm64: imx8mm: Enable Hantro VPUs From: Nicolas Dufresne To: Adam Ford , Tim Harvey Cc: linux-media , Schrempf Frieder , Marek Vasut , Jagan Teki , Adam Ford-BE , cstevens@beaconembedded.com, Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Rob Herring , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Greg Kroah-Hartman , Heiko Stuebner , Lucas Stach , Joakim Zhang , Alice Guo , Peng Fan , "open list:HANTRO VPU CODEC DRIVER" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , open list , "open list:STAGING SUBSYSTEM" Date: Fri, 19 Nov 2021 11:29:48 -0500 In-Reply-To: References: <20211106183802.893285-1-aford173@gmail.com> <718f7f6d6cd564d031c1963f1590c62d549ae725.camel@ndufresne.ca> <8db00a4b6faa99c940d9bc86e17161eb0db5efe3.camel@ndufresne.ca> User-Agent: Evolution 3.42.1 (3.42.1-1.fc35) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211119_082953_071017_7D170FF5 X-CRM114-Status: GOOD ( 34.20 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org Hi Adam, Tim, [...] > > > > Nicolas and Adam, > > > > > > > > For the H1 patches in this series: I've been able to test the IMX8MM > > > > H1 JPEG encode using GStreamer 1.18.5: > > > > $ gst-inspect-1.0 | grep -e "v4l2.*enc" > > > > video4linux2: v4l2jpegenc: V4L2 JPEG Encoder > > > > $ gst-launch-1.0 videotestsrc ! jpegenc ! rtpjpegpay ! udpsink > > > ^ v4l2jpegenc > > > > > > This is just a transcript error ? > > > > Nicolas, > > > > No! Thanks for catching my mistake. I was testing with software encode... ooops! > > > > 'gst-launch-1.0 videotestsrc ! v4l2jpegenc ! fakesink' actually hangs > > the board so likely a power-domain issue there? > > The v4l2-compliance tests fail on the h1 decoder with a hang, but I > think we're writing to registers which are not documented in the Mini > TRM. The Mini TRM doesn't explicitly show the JPEG encoding as a > feature, but some of the registers state JPEG, but because some of the > registers written for the H1 are not documented in the TRM. If those > registers are restricted or not in this SoC, I am concerned that it > might be related. I'll try to run some more tests this weekend to > check on the status of the power-domain stuff. To verify if the HW support JPEG encoding you can read SWREG63 bit 25. This is in the TRM, just not labelled properly. To mimic the decoding side, would be "HW synthesis config register X" with the bit labelled SW_ENC_JPEG_PROF (but PROF/profile is on or off). If your board hang while reading this, you likely didn't get the power bit right. IMX8 has an undocumented control block thing that we have been fighting with in imx8q, perhaps that's your issue. Few driver was proposed, we are still pending on NXP solution to be submitted (they asked us to wait, still waiting =)). > > > > > > > > > host=192.168.1.146 port=5000 > > > > viewed on client@192.168.1.146 via: > > > > $ gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,payload=96 ! > > > > rtpjpegdepay ! jpegdec ! autovideosink > > > > > > > > For the G1/G2 patches in the series I don't see any Gstreamer > > > > 'v4l2.*dec' elements. Perhaps I need a newer version of Gstreamer. > > > > > > Most likely yes, I suggest building gstreamer/ branch "main", GStreamer has now > > > a single repository. We are very close to 1.20, which will include stable API > > > support of H264, MPEG2 and VP8 decoding. > > > > > > > Ok, let me see if I can navigate through the build process and I'll > > get back to you. > > > > Thanks, > > > > Tim > > > > > > > > > > I have CSI capture and DSI display currently working on > > > > imx8mm-venice-gw73xx-0x that I can play with. The CSI sensor only > > > > supports RAW8/RAW10 (and gstreamer currently only supports RAW8) and I > > > > can't efficiently convert to something the JPEG encoder likes without > > > > bayer2rgbneon (a libneon version). > > > > > > > > I see from the IMX8MMRM that the 2D GPU supports scaling etc with a > > > > wide range of data formats but I'm not sure how to tap into this as > > > > that hardware is managed by the vivante driver. On the IMX6QDL there > > > > is a separate IPU block that Philipp Zabel wrote a nice mem2mem > > > > csc/scaler driver for but I don't see any equivalent currently for > > > > IMX8MM. > > > > > > > > Best regards, > > > > > > > > Tim > > > _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 634E8C433F5 for ; Fri, 19 Nov 2021 16:31:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 34CB361B27 for ; Fri, 19 Nov 2021 16:31:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 34CB361B27 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ndufresne.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:Cc:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=to06M4oAlPBSVuzI53XG6XkDcVtkgfT168+CysvrifY=; b=RLjiIk5G00XYsP PaQo+etM2ovnjI1LxQZuQR5Ie2ZmPj8qAXqBarWfDjNdITfmQwjz8HYQQ89Ku9Wjj1U31sQrmpSPW 5tNFYGwo1BSGwTStEveyJqb7+vcondnQO22nM+Gs4FC/aW0YuZV5by3MhCFGcSRbz2C0SVz85uP3h TdLJZxC7hrIRSc0SWjPZSpylgd8d0gijSPD99CmVSwMgMPc6ApbQT+JQhtfjDy5OblHg8bShzUs2t 2xmktrr3QD15B8l8vq2Bodcv8T1eJXTKEmFSbyptH2jLGhA4f5l4IXV/4ju2rMAsrv+zQ2dQmDnEq G9AlAiQQE3S2xpLKUmhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mo6ly-00B1hJ-Nf; Fri, 19 Nov 2021 16:29:58 +0000 Received: from mail-il1-x12a.google.com ([2607:f8b0:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mo6ls-00B1f3-Ux for linux-arm-kernel@lists.infradead.org; Fri, 19 Nov 2021 16:29:54 +0000 Received: by mail-il1-x12a.google.com with SMTP id e8so10756315ilu.9 for ; Fri, 19 Nov 2021 08:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ndufresne-ca.20210112.gappssmtp.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=NA+2l6X3VrlJxsJamfaAblN1iQr9G5GeyeDjrzqWldE=; b=pWYGr8brYDg2I9xcuGuhurThossAPr1q87fBQPqCiM2BAja4C7uFnX65Fix38gLbMR at4/FYFXGd1o9zn1i0YRa4Hl3P62bx1s2GsXPKOGSzrVsAT1Xw8HlYNWbORdzoDIOeqe 0qslCNUJ66VzbM5OECQeydfyhNNqUWwGP6afEEwmpoOMoNIZnNaZfchouPoszwNYzV1v y1bimltU6qqxfdJSKgW6NjHgo6foggqrexAHaTR5BhNExaTs0kreVzSvKeiuvCJBGMM5 4hhlHH8MqKp2ss3LA9dpdsxQOKHOGkUr880PDTOgcILg0r5nps7M4Bb//54zUt1tJBXH UXjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=NA+2l6X3VrlJxsJamfaAblN1iQr9G5GeyeDjrzqWldE=; b=5yczuD3nQ+/x+wqwZKSbNlVc60FdL0K069NWBn2irpcm0bJZeTqLu+5jixEdkkM5A2 Xhs7m04SUGWKh47xXz7vFs3BUsBj4TmSkUpoix0MkSd7Us3aVmfitLZ9oerrdNAmPaGh L+K/kcp/1Y+j3z4ICLXDufyQlwk+d5DjndAcMBZnyzrvM1JRt5n/q5xgxY70DZW3U2xX 18NRL7ACO8SNPJbnqGXxPkc7bYIKAO6e430GNG4l9rJXy/JWmE1sHVuwvf7pUrYq+qQI l3hDcC5Au2NLJb8xgS0fo9kv5/xHdJ+p3uaQm19sK+gZe5Hbw+UpezJswRhfgM+ElhgX bpyw== X-Gm-Message-State: AOAM5329KCgzMh3avWDCRtcytPgEWsKqat+ujL5MKNX3XMDU0m984Hqj p2RBIdTHZnmefu8818aHp7AmXQ== X-Google-Smtp-Source: ABdhPJy9+PK69iHyuVZcAK9TkCljtUM0G3YiAMHQdYjwnsW/n1HgnSVsWmM0mEP/36cqcfaj9/5ecQ== X-Received: by 2002:a92:d343:: with SMTP id a3mr5688828ilh.136.1637339392190; Fri, 19 Nov 2021 08:29:52 -0800 (PST) Received: from nicolas-tpx395.localdomain (mtl.collabora.ca. [66.171.169.34]) by smtp.gmail.com with ESMTPSA id c7sm108143iob.28.2021.11.19.08.29.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Nov 2021 08:29:51 -0800 (PST) Message-ID: <7f94eaacfddb8c5434c17f1e069ea87a17657ce9.camel@ndufresne.ca> Subject: Re: [RFC 0/5] arm64: imx8mm: Enable Hantro VPUs From: Nicolas Dufresne To: Adam Ford , Tim Harvey Cc: linux-media , Schrempf Frieder , Marek Vasut , Jagan Teki , Adam Ford-BE , cstevens@beaconembedded.com, Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Rob Herring , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Greg Kroah-Hartman , Heiko Stuebner , Lucas Stach , Joakim Zhang , Alice Guo , Peng Fan , "open list:HANTRO VPU CODEC DRIVER" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" , open list , "open list:STAGING SUBSYSTEM" Date: Fri, 19 Nov 2021 11:29:48 -0500 In-Reply-To: References: <20211106183802.893285-1-aford173@gmail.com> <718f7f6d6cd564d031c1963f1590c62d549ae725.camel@ndufresne.ca> <8db00a4b6faa99c940d9bc86e17161eb0db5efe3.camel@ndufresne.ca> User-Agent: Evolution 3.42.1 (3.42.1-1.fc35) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211119_082953_063449_2216E0CD X-CRM114-Status: GOOD ( 35.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Adam, Tim, [...] > > > > Nicolas and Adam, > > > > > > > > For the H1 patches in this series: I've been able to test the IMX8MM > > > > H1 JPEG encode using GStreamer 1.18.5: > > > > $ gst-inspect-1.0 | grep -e "v4l2.*enc" > > > > video4linux2: v4l2jpegenc: V4L2 JPEG Encoder > > > > $ gst-launch-1.0 videotestsrc ! jpegenc ! rtpjpegpay ! udpsink > > > ^ v4l2jpegenc > > > > > > This is just a transcript error ? > > > > Nicolas, > > > > No! Thanks for catching my mistake. I was testing with software encode... ooops! > > > > 'gst-launch-1.0 videotestsrc ! v4l2jpegenc ! fakesink' actually hangs > > the board so likely a power-domain issue there? > > The v4l2-compliance tests fail on the h1 decoder with a hang, but I > think we're writing to registers which are not documented in the Mini > TRM. The Mini TRM doesn't explicitly show the JPEG encoding as a > feature, but some of the registers state JPEG, but because some of the > registers written for the H1 are not documented in the TRM. If those > registers are restricted or not in this SoC, I am concerned that it > might be related. I'll try to run some more tests this weekend to > check on the status of the power-domain stuff. To verify if the HW support JPEG encoding you can read SWREG63 bit 25. This is in the TRM, just not labelled properly. To mimic the decoding side, would be "HW synthesis config register X" with the bit labelled SW_ENC_JPEG_PROF (but PROF/profile is on or off). If your board hang while reading this, you likely didn't get the power bit right. IMX8 has an undocumented control block thing that we have been fighting with in imx8q, perhaps that's your issue. Few driver was proposed, we are still pending on NXP solution to be submitted (they asked us to wait, still waiting =)). > > > > > > > > > host=192.168.1.146 port=5000 > > > > viewed on client@192.168.1.146 via: > > > > $ gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,payload=96 ! > > > > rtpjpegdepay ! jpegdec ! autovideosink > > > > > > > > For the G1/G2 patches in the series I don't see any Gstreamer > > > > 'v4l2.*dec' elements. Perhaps I need a newer version of Gstreamer. > > > > > > Most likely yes, I suggest building gstreamer/ branch "main", GStreamer has now > > > a single repository. We are very close to 1.20, which will include stable API > > > support of H264, MPEG2 and VP8 decoding. > > > > > > > Ok, let me see if I can navigate through the build process and I'll > > get back to you. > > > > Thanks, > > > > Tim > > > > > > > > > > I have CSI capture and DSI display currently working on > > > > imx8mm-venice-gw73xx-0x that I can play with. The CSI sensor only > > > > supports RAW8/RAW10 (and gstreamer currently only supports RAW8) and I > > > > can't efficiently convert to something the JPEG encoder likes without > > > > bayer2rgbneon (a libneon version). > > > > > > > > I see from the IMX8MMRM that the 2D GPU supports scaling etc with a > > > > wide range of data formats but I'm not sure how to tap into this as > > > > that hardware is managed by the vivante driver. On the IMX6QDL there > > > > is a separate IPU block that Philipp Zabel wrote a nice mem2mem > > > > csc/scaler driver for but I don't see any equivalent currently for > > > > IMX8MM. > > > > > > > > Best regards, > > > > > > > > Tim > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel