From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B04BCC433EF for ; Tue, 14 Jun 2022 14:17:55 +0000 (UTC) Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by mx.groups.io with SMTP id smtpd.web12.6558.1655216275075713099 for ; Tue, 14 Jun 2022 07:17:55 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20210112 header.b=QS5dCGgC; spf=pass (domain: gmail.com, ip: 209.85.221.54, mailfrom: martin.jansa@gmail.com) Received: by mail-wr1-f54.google.com with SMTP id x17so11498681wrg.6 for ; Tue, 14 Jun 2022 07:17:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1JDBnBZPjpBjSAWpOXVXVLLGjTx4TwJv+lCoh2uo0U4=; b=QS5dCGgCW53PXSjtfE1Or8s9TMuNdzWAajV5GvRwh6WdQFqNCO7fqO0BSNAconwkrp 7MB9xqOeUG/OTGA+WAI4iW2l/VL3RQ9BCemxaU1EanbX9DcUgqXSnDNIPj8+MdcjULjC XCKyOMjGOlPyyxSf0/iAeeKVAJWMv1XKLlEmbOk7Xyb+CBXbjeBtR/fXR4XDMCwWa+b1 iQfv5Au88WiLEmJrXm7n84yF7W21MRcqlGnYAAJx/XJ5f4llIhpibcbWbGzfy8QY/GgU ag7QyBsodazQaDoGpVOh4SbgXLzM1vjrV6gwe5FcLKX5uhol/Y2SPfsxh2WDJqX702Nk CKxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1JDBnBZPjpBjSAWpOXVXVLLGjTx4TwJv+lCoh2uo0U4=; b=e9RAMbA0Y4W0Vmi2hcbKJnwN2TzpAKI8B3GFjJQHdD5Dd7v0931LdL2AbEyVLzQkUy uCY1UfeM5+48Pj2n4hC4NiuR+J1DHuNuXpr9tHMUu7PGaQgx/c91UDxr/xsSTv5aU7RS dkhFkEkUxc3PZUh9HdRhVc0cU54Hb/sBvjdNNdKEOf4sEp/nep/ixTb2gMYYXZ5qwxYY LIUMIJx5ecGrORd/BiJqu/QdTr67mbjRgWHa0Wa1+aav1ylnUALkO9k1LAD9q7odVndY bwMeIUZNUNYBijxGmr2lqKgny1ZGpxrUfLnLUY4ehpazbbyI2EqOtZNl8y0plKd3jg+8 oatw== X-Gm-Message-State: AJIora8nTOWnZ4ctQnD/4/v8W2cZmrHaWciO6xAobDSEvEVhjAMcCE5L w7Pd/+2D9T+fsJ6rH9caq56WgyW/QWLA9RaeDxg= X-Google-Smtp-Source: AGRyM1ve1hQWpZDJX1spoXwgj1gIULFH5sN4mF49+V/sZT34wreVZ99fNrx2X3mmUQD61RmplFgxFjxdGI5N2cJNeyA= X-Received: by 2002:adf:b358:0:b0:216:508c:e0bf with SMTP id k24-20020adfb358000000b00216508ce0bfmr5175249wrd.204.1655216273485; Tue, 14 Jun 2022 07:17:53 -0700 (PDT) MIME-Version: 1.0 References: <20220606155807.1414519-1-ptsneves@gmail.com> <20220614131652.566471-1-ptsneves@gmail.com> <20220614131652.566471-6-ptsneves@gmail.com> <0b06d351-7d98-42e4-0872-d8b0e4fe9ce7@gmail.com> In-Reply-To: <0b06d351-7d98-42e4-0872-d8b0e4fe9ce7@gmail.com> From: Martin Jansa Date: Tue, 14 Jun 2022 16:17:37 +0200 Message-ID: Subject: Re: [OE-core] [PATCH v4 6/7] utils: Add cmdline_shebang_wrapper util. To: Paulo Neves Cc: openembedded-core@lists.openembedded.org Content-Type: multipart/alternative; boundary="00000000000001f23505e16912e5" List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 14 Jun 2022 14:17:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/166893 --00000000000001f23505e16912e5 Content-Type: text/plain; charset="UTF-8" You can use bitbake-getvar to see the default FILESPATH, but "files" next to recipe is included by default (I prefer to use BPN directory for slightly faster lookup - unless the files are shared between different recipes). FILESEXTRAPATHS is usually only needed from bbappends which add new files to SRC_URI. For LICENSE you can use MIT and LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" like many "pure-metadata" recipes in oe-core. On Tue, Jun 14, 2022 at 3:46 PM Paulo Neves wrote: > Regarding the FILESEXTRAPATHS:prepend i was convinced that > ${THISDIR}/files was not automatically added and was needed for the > test.awk file. I guess I have been cargo culting for some time on this. > Will remove it. > > This is a recipe for a test and if i add a license different than CLOSED > than i need to add a license file, which seems a bit overkill for a one > line. Am I wrong? What license should it even be? Dont Panic is a common > string used in awk docs. > > Paulo Neves > > On 6/14/22 15:39, Martin Jansa wrote: > > FILESEXTRAPATHS:prepend doesn't seem to be needed and LICENSE is really > CLOSED? > > On Tue, Jun 14, 2022 at 3:17 PM Paulo Neves wrote: > >> Useful to work around shebang relocation issues, where >> shebangs are too long or have arguments in them, thus preventing them >> from using the /usr/bin/env shebang. >> --- >> .../wrapper/cmdline-shebang-wrapper-test.bb | 21 ++++++++++++ >> .../recipes-test/wrapper/files/test.awk | 2 ++ >> meta/classes/utils.bbclass | 34 +++++++++++++++++++ >> meta/lib/oeqa/selftest/cases/wrapper.py | 11 ++++++ >> 4 files changed, 68 insertions(+) >> create mode 100644 meta-selftest/recipes-test/wrapper/ >> cmdline-shebang-wrapper-test.bb >> create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk >> create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py >> >> diff --git a/meta-selftest/recipes-test/wrapper/ >> cmdline-shebang-wrapper-test.bb b/meta-selftest/recipes-test/wrapper/ >> cmdline-shebang-wrapper-test.bb >> new file mode 100644 >> index 0000000000..302eea8901 >> --- /dev/null >> +++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb >> @@ -0,0 +1,21 @@ >> +FILESEXTRAPATHS:prepend := "${THISDIR}/files:" >> +SUMMARY = "Check that create_cmdline_shebang works" >> +LICENSE = "CLOSED" >> +INHIBIT_DEFAULT_DEPS = "1" >> + >> +SRC_URI += "file://test.awk" >> + >> +EXCLUDE_FROM_WORLD = "1" >> +do_install() { >> + install -d ${D}${bindir} >> + install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test >> + sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test >> + create_cmdline_shebang_wrapper ${D}${bindir}/test >> + if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then >> + bbfatal "Wrapper is broken" >> + else >> + bbnote "Wrapper is good" >> + fi >> +} >> + >> +BBCLASSEXTEND = "native" >> diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk >> b/meta-selftest/recipes-test/wrapper/files/test.awk >> new file mode 100644 >> index 0000000000..91429197b1 >> --- /dev/null >> +++ b/meta-selftest/recipes-test/wrapper/files/test.awk >> @@ -0,0 +1,2 @@ >> +#! @AWK_BIN@ -f >> +BEGIN { print "Don't Panic!" } >> diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass >> index b4eb3d38ab..b617632d9f 100644 >> --- a/meta/classes/utils.bbclass >> +++ b/meta/classes/utils.bbclass >> @@ -184,6 +184,40 @@ END >> chmod +x $cmd >> } >> >> +create_cmdline_shebang_wrapper () { >> + # Create a wrapper script where commandline options are needed >> + # >> + # These are useful to work around shebang relocation issues, >> where shebangs are too >> + # long or have arguments in them, thus preventing them from using the >> /usr/bin/env >> + # shebang >> + # >> + # Usage: create_cmdline_wrapper FILENAME >> + >> + cmd=$1 >> + shift >> + >> + echo "Generating wrapper script for $cmd" >> + >> + # Strip #! and get remaining interpreter + arg >> + argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")" >> + # strip the shebang from the real script as we do not want it to be >> usable anyway >> + tail -n +2 $cmd > $cmd.real >> + cmdname=$(basename $cmd) >> + dirname=$(dirname $cmd) >> + cmdoptions=$@ >> + if [ "${base_prefix}" != "" ]; then >> + relpath=`python3 -c "import os; >> print(os.path.relpath('${D}${base_prefix}', '$dirname'))"` >> + cmdoptions=`echo $@ | sed -e >> "s:${base_prefix}:\\$realdir/$relpath:g"` >> + fi >> + cat <$cmd >> +#!/usr/bin/env bash >> +realpath=\`readlink -fn \$0\` >> +realdir=\`dirname \$realpath\` >> +exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions >> "\$@" >> +END >> + chmod +x $cmd >> +} >> + >> create_wrapper () { >> # Create a wrapper script where extra environment variables are >> needed >> # >> diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py >> b/meta/lib/oeqa/selftest/cases/wrapper.py >> new file mode 100644 >> index 0000000000..6de63310c0 >> --- /dev/null >> +++ b/meta/lib/oeqa/selftest/cases/wrapper.py >> @@ -0,0 +1,11 @@ >> +from oeqa.selftest.case import OESelftestTestCase >> +from oeqa.utils.commands import bitbake >> + >> +class WrapperTests(OESelftestTestCase): >> + def test_shebang_wrapper(self): >> + """ >> + Summary: Build a recipe which will fail if the >> cmdline_shebang_wrapper function is defective. >> + Expected: Exit status to be 0. >> + Author: Paulo Neves >> + """ >> + res = bitbake("cmdline-shebang-wrapper-test -c install", >> ignore_status=False) >> -- >> 2.25.1 >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#166887): >> https://lists.openembedded.org/g/openembedded-core/message/166887 >> Mute This Topic: https://lists.openembedded.org/mt/91748692/3617156 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ >> Martin.Jansa@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> >> > --00000000000001f23505e16912e5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
You can use bitbake-getvar to see the default FILESPATH, b= ut "files" next to recipe is included by default (I prefer to use= BPN directory for slightly faster lookup - unless the files are shared bet= ween different recipes). FILESEXTRAPATHS is usually only needed from bbappe= nds which add new files to SRC_URI.

For LICENSE you can = use MIT and
LIC_FILES_CHKSUM =3D "file://${COREBASE}/meta/CO= PYING.MIT;md5=3D3da9cfbcb788c80a0384361b4de20420"
like m= any "pure-metadata" recipes in oe-core.

On Tue, Jun 14, 2022= at 3:46 PM Paulo Neves <ptsneves@= gmail.com> wrote:
=20 =20 =20
Regarding the FILESEXTRAPATHS:prepend=C2=A0 i was convinced that ${THISDIR}/files was not automatically added and was needed for the test.awk file. I guess I have been cargo culting for some time on this. Will remove it.

This is a recipe for a test and if i add a license different than CLOSED than i need to add a license file, which seems a bit overkill for a one line. Am I wrong? What license should it even be? Dont Panic is a common string used in awk docs.

Paulo Neves

On 6/14/22 15:39, Martin Jansa wrote:
=20
FILESEXTRAPATHS:prepend doesn't seem to be neede= d and LICENSE is really CLOSED?

On Tue, Jun 14, 2022 at 3:17 PM Paulo Neves <ptsneves@gmail.com> wrote:
Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
---
=C2=A0.../wrapper/cmdline-shebang-wrapper-test.bb=C2=A0 =C2=A0| 21 ++++++++++++
=C2=A0.../recipes-test/wrapper/files/test.awk=C2=A0 =C2=A0 =C2=A0= =C2=A0|=C2=A0 2 ++
=C2=A0meta/classes/utils.bbclass=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | 34 +++++++++++++++++++
=C2=A0meta/lib/oeqa/selftest/cases/wrapper.py=C2=A0 =C2=A0 =C2=A0= =C2=A0| 11 ++++++
=C2=A04 files changed, 68 insertions(+)
=C2=A0create mode 100644 meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
=C2=A0create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk
=C2=A0create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py<= br>
diff --git a/meta-selftest/recipes-test/wrapper/cmdl= ine-shebang-wrapper-test.bb b/meta-selftest/recipes-test/wrapper/cmdline-shebang= -wrapper-test.bb
new file mode 100644
index 0000000000..302eea8901
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/cmdline-she= bang-wrapper-test.bb
@@ -0,0 +1,21 @@
+FILESEXTRAPATHS:prepend :=3D "${THISDIR}/files:"
+SUMMARY =3D "Check that create_cmdline_shebang works"<= br> +LICENSE =3D "CLOSED"
+INHIBIT_DEFAULT_DEPS =3D "1"
+
+SRC_URI +=3D "file://test.awk"
+
+EXCLUDE_FROM_WORLD =3D "1"
+do_install() {
+=C2=A0 =C2=A0 install -d ${D}${bindir}
+=C2=A0 =C2=A0 install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/= test
+=C2=A0 =C2=A0 sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+=C2=A0 =C2=A0 create_cmdline_shebang_wrapper ${D}${bindir}/test<= br> +=C2=A0 =C2=A0 if [ $(${D}${bindir}/test) !=3D "Don't Pa= nic!" ]; then
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 bbfatal "Wrapper is broken"= ;
+=C2=A0 =C2=A0 else
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 bbnote "Wrapper is good" +=C2=A0 =C2=A0 fi
+}
+
+BBCLASSEXTEND =3D "native"
diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 0000000000..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b4eb3d38ab..b617632d9f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
=C2=A0 =C2=A0 =C2=A0 =C2=A0 chmod +x $cmd
=C2=A0}

+create_cmdline_shebang_wrapper () {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0# Create a wrapper script where comma= ndline options are needed
+=C2=A0 =C2=A0 =C2=A0 =C2=A0#
+=C2=A0 =C2=A0 =C2=A0 =C2=A0# These are useful to work around she= bang relocation issues, where shebangs are too
+=C2=A0 # long or have arguments in them, thus preventing them fr= om using the /usr/bin/env
+=C2=A0 =C2=A0 =C2=A0 =C2=A0# shebang
+=C2=A0 =C2=A0 =C2=A0 =C2=A0#
+=C2=A0 =C2=A0 =C2=A0 =C2=A0# Usage: create_cmdline_wrapper FILEN= AME <extra-options>
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0cmd=3D$1
+=C2=A0 =C2=A0 =C2=A0 =C2=A0shift
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0echo "Generating wrapper script = for $cmd"
+
+=C2=A0 # Strip #! and get remaining interpreter + arg
+=C2=A0 argument=3D"$(basename "$(head -n1 $cmd | sed -= e 's|#![ ]*||g' )")"
+=C2=A0 # strip the shebang from the real script as we do not wan= t it to be usable anyway
+=C2=A0 tail -n +2 $cmd > $cmd.real
+=C2=A0 =C2=A0 =C2=A0 =C2=A0cmdname=3D$(basename $cmd)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0dirname=3D$(dirname $cmd)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0cmdoptions=3D$@
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if [ "${base_prefix}" !=3D = "" ]; then
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0relpath= =3D`python3 -c "import os; print(os.path.relpath('${D}${base_prefix}', '$dirname= '))"`
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmdoption= s=3D`echo $@ | sed -e "s:${base_prefix}:\\$realdir/$relpath:g"`
+=C2=A0 =C2=A0 =C2=A0 =C2=A0fi
+=C2=A0 =C2=A0 =C2=A0 =C2=A0cat <<END >$cmd
+#!/usr/bin/env bash
+realpath=3D\`readlink -fn \$0\`
+realdir=3D\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+=C2=A0 =C2=A0 =C2=A0 =C2=A0chmod +x $cmd
+}
+
=C2=A0create_wrapper () {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # Create a wrapper script where extra= environment variables are needed
=C2=A0 =C2=A0 =C2=A0 =C2=A0 #
diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py b/meta/lib/oeqa/selftest/cases/wrapper.py
new file mode 100644
index 0000000000..6de63310c0
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/wrapper.py
@@ -0,0 +1,11 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake
+
+class WrapperTests(OESelftestTestCase):
+=C2=A0 =C2=A0 def test_shebang_wrapper(self):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Summary:=C2=A0 =C2=A0Build a recipe = which will fail if the cmdline_shebang_wrapper function is defective.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Expected:=C2=A0 Exit status to be 0.=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Author:=C2=A0 =C2=A0 Paulo Neves <= ;ptsneves@gmail.com= >
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 res =3D bitbake("cmdline-sheban= g-wrapper-test -c install", ignore_status=3DFalse)
--
2.25.1


-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-
Links: You receive all messages sent to this group.
View/Reply Online (#166887): https://lists.openembedded.org/g/openembedded-core/message/166887 Mute This Topic: https://lists.openembed= ded.org/mt/91748692/3617156
Group Owner: openembedded-core+owner@lists.openembedde= d.org
Unsubscribe: https://lists.openemb= edded.org/g/openembedded-core/unsub [Mart= in.Jansa@gmail.com]
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-


--00000000000001f23505e16912e5--