dwarves.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* pahole: Problems with lib/include/bpf symlink in a tarball with --transform
@ 2021-08-12 19:00 Arnaldo Carvalho de Melo
  2021-08-12 19:06 ` Arnaldo Carvalho de Melo
  2021-08-13 18:15 ` Frank Heckenbach
  0 siblings, 2 replies; 5+ messages in thread
From: Arnaldo Carvalho de Melo @ 2021-08-12 19:00 UTC (permalink / raw)
  To: Luca Boccassi; +Cc: Frank Heckenbach, dwarves

Hi Luca,

	When I generate a tarball I'm not being able to build, as it
can't find bpf/btf.h, for instance, see this cset where I tried to fix
it:

commit 00cb89fb9af0355b8b4b64e26d3afa2348efb80d (HEAD -> next, five/next)
Author: Arnaldo Carvalho de Melo <acme@redhat.com>
Date:   Thu Aug 12 15:29:28 2021 -0300

    MANIFEST: Add lib/include/bpf

    The tarball wasn't working without this file:

      [ 32%] Building C object CMakeFiles/dwarves.dir/btf_encoder.c.o
      /tmp/dwarves-1.22/btf_encoder.c:18:10: fatal error: bpf/btf.h: No such file or directory
       #include <bpf/btf.h>
                ^~~~~~~~~~~
      compilation terminated.

    Cc: Luca Boccassi <bluca@debian.org>
    Fixes: 82756ea3eaf1f6e7 ("libbpf: Fixup patch to allow to use packaged version")
    Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

diff --git a/MANIFEST b/MANIFEST
index 67f382e6f8f9..88f6193d5bc0 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -68,3 +68,4 @@ libctf.c
 libctf.h
 regtest
 lib/bpf/
+lib/include/bpf
[acme@seventh pahole]$

But this doesn't help as well, as to prepare the tarball I use
tar --transform option (see README.tarball), and with lib/include/bpf
(the symlink pointing to ../bpf/src) we end up with:

[acme@seventh pahole]$ tar xvf ~/rpmbuild/SOURCES/dwarves-1.22.tar.xz | tail
dwarves-1.22/lib/bpf/include/linux/compiler.h
dwarves-1.22/lib/bpf/include/linux/list.h
dwarves-1.22/lib/bpf/include/linux/kernel.h
dwarves-1.22/lib/bpf/include/linux/err.h
dwarves-1.22/lib/bpf/CHECKPOINT-COMMIT
dwarves-1.22/lib/bpf/LICENSE
dwarves-1.22/lib/bpf/LICENSE.BSD-2-Clause
dwarves-1.22/lib/bpf/README.md
dwarves-1.22/lib/bpf/BPF-CHECKPOINT-COMMIT
dwarves-1.22/lib/include/bpf
[acme@seventh pahole]$ ls -la dwarves-1.22/lib/include/
total 8
drwxr-xr-x. 2 acme acme 4096 Aug 12 15:54 .
drwxr-xr-x. 4 acme acme 4096 Aug 12 15:54 ..
lrwxrwxrwx. 1 acme acme   23 Jun 18 17:56 bpf -> dwarves-1.22/../bpf/src
[acme@seventh pahole]$

Without the 'tar --transform' that is, on a checked out git repo:

⬢[acme@toolbox pahole]$ ls -la lib/include/bpf
lrwxrwxrwx. 1 acme acme 10 Aug 11 08:55 lib/include/bpf -> ../bpf/src
⬢[acme@toolbox pahole]$ ls -la lib/bpf/src
total 892
drwxr-xr-x. 1 acme acme    692 Aug  5 16:12 .
drwxr-xr-x. 1 acme acme    280 Jun 14 15:51 ..
<SNIP>
-rw-r--r--. 1 acme acme 123802 Jun 14 15:51 btf.c
-rw-r--r--. 1 acme acme  40953 Jun 14 15:51 btf_dump.c
-rw-r--r--. 1 acme acme  11796 Jun 14 15:51 btf.h
<SNIP>
⬢[acme@toolbox pahole]$

The problem is described in this old Debian ticket:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775247

I'm now trying to figure out what Frank meant when referring to that
"S" flag.

This is all on the 'next' branch, but I guess it affects master as well:

https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next

libbpf-devel has it on /usr/include/bpf/btf.h so its all ok there, but
when building it from the tarball I'm stuck with that symlink trick not
working ;-\

I have to figure out what is this 'S' flag (local symlinks, etc) or stop
using this --transform mechanism to have the dwarves-${version} top
level directory added when generating the tarball :-\

- Arnaldo

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pahole: Problems with lib/include/bpf symlink in a tarball with --transform
  2021-08-12 19:00 pahole: Problems with lib/include/bpf symlink in a tarball with --transform Arnaldo Carvalho de Melo
@ 2021-08-12 19:06 ` Arnaldo Carvalho de Melo
  2021-08-12 21:50   ` Luca Boccassi
  2021-08-13 18:15 ` Frank Heckenbach
  1 sibling, 1 reply; 5+ messages in thread
From: Arnaldo Carvalho de Melo @ 2021-08-12 19:06 UTC (permalink / raw)
  To: Luca Boccassi; +Cc: Frank Heckenbach, dwarves

Em Thu, Aug 12, 2021 at 04:00:38PM -0300, Arnaldo Carvalho de Melo escreveu:
> [acme@seventh pahole]$ ls -la dwarves-1.22/lib/include/
> total 8
> drwxr-xr-x. 2 acme acme 4096 Aug 12 15:54 .
> drwxr-xr-x. 4 acme acme 4096 Aug 12 15:54 ..
> lrwxrwxrwx. 1 acme acme   23 Jun 18 17:56 bpf -> dwarves-1.22/../bpf/src
> [acme@seventh pahole]$

Ok, including the parent directory and then replacing it with the
desired package_name-package_version did the trick ;-)

⬢[acme@toolbox pahole]$ v=1.$(($(git tag | sort -V | tail -1 | cut -d. -f2) + 1)) ; tar cvfJ ~/rpmbuild/SOURCES/dwarves-${v}.tar.xz --transform "s,^pahole/,dwarves-${v}/," `sed s%^%../pahole/%g MANIFEST`
⬢[acme@toolbox pahole]$ tar tvf ~/rpmbuild/SOURCES/dwarves-1.22.tar.xz  | tail
-rw-r--r-- acme/acme       252 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/helpers.sh
-rwxr-xr-x acme/acme      3084 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/mkrootfs.sh
-rwxr-xr-x acme/acme        86 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests-4.9.0.sh
-rwxr-xr-x acme/acme        86 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests-5.5.0.sh
-rwxr-xr-x acme/acme       365 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests.sh
-rwxr-xr-x acme/acme     10841 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run.sh
-rwxr-xr-x acme/acme      1245 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run_selftests.sh
-rwxr-xr-x acme/acme      1132 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run_vmtest.sh
-rw-r--r-- acme/acme   2904227 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/vmlinux.h
lrwxrwxrwx acme/acme         0 2021-08-11 08:55 dwarves-1.22/lib/include/bpf -> ../bpf/src
⬢[acme@toolbox pahole]$

yeah, it now builds from the generated tarball, updating README.tarball 8-)

- Arnaldo

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pahole: Problems with lib/include/bpf symlink in a tarball with --transform
  2021-08-12 19:06 ` Arnaldo Carvalho de Melo
@ 2021-08-12 21:50   ` Luca Boccassi
  2021-08-13  0:05     ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 5+ messages in thread
From: Luca Boccassi @ 2021-08-12 21:50 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo; +Cc: Frank Heckenbach, dwarves

On Thu, 12 Aug 2021 at 20:06, Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
>
> Em Thu, Aug 12, 2021 at 04:00:38PM -0300, Arnaldo Carvalho de Melo escreveu:
> > [acme@seventh pahole]$ ls -la dwarves-1.22/lib/include/
> > total 8
> > drwxr-xr-x. 2 acme acme 4096 Aug 12 15:54 .
> > drwxr-xr-x. 4 acme acme 4096 Aug 12 15:54 ..
> > lrwxrwxrwx. 1 acme acme   23 Jun 18 17:56 bpf -> dwarves-1.22/../bpf/src
> > [acme@seventh pahole]$
>
> Ok, including the parent directory and then replacing it with the
> desired package_name-package_version did the trick ;-)
>
> ⬢[acme@toolbox pahole]$ v=1.$(($(git tag | sort -V | tail -1 | cut -d. -f2) + 1)) ; tar cvfJ ~/rpmbuild/SOURCES/dwarves-${v}.tar.xz --transform "s,^pahole/,dwarves-${v}/," `sed s%^%../pahole/%g MANIFEST`
> ⬢[acme@toolbox pahole]$ tar tvf ~/rpmbuild/SOURCES/dwarves-1.22.tar.xz  | tail
> -rw-r--r-- acme/acme       252 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/helpers.sh
> -rwxr-xr-x acme/acme      3084 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/mkrootfs.sh
> -rwxr-xr-x acme/acme        86 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests-4.9.0.sh
> -rwxr-xr-x acme/acme        86 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests-5.5.0.sh
> -rwxr-xr-x acme/acme       365 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests.sh
> -rwxr-xr-x acme/acme     10841 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run.sh
> -rwxr-xr-x acme/acme      1245 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run_selftests.sh
> -rwxr-xr-x acme/acme      1132 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run_vmtest.sh
> -rw-r--r-- acme/acme   2904227 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/vmlinux.h
> lrwxrwxrwx acme/acme         0 2021-08-11 08:55 dwarves-1.22/lib/include/bpf -> ../bpf/src
> ⬢[acme@toolbox pahole]$
>
> yeah, it now builds from the generated tarball, updating README.tarball 8-)
>
> - Arnaldo

Glad to hear you found a fix - I would have had no idea :-)

Kind regards,
Luca Boccassi

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pahole: Problems with lib/include/bpf symlink in a tarball with --transform
  2021-08-12 21:50   ` Luca Boccassi
@ 2021-08-13  0:05     ` Arnaldo Carvalho de Melo
  0 siblings, 0 replies; 5+ messages in thread
From: Arnaldo Carvalho de Melo @ 2021-08-13  0:05 UTC (permalink / raw)
  To: Luca Boccassi; +Cc: Frank Heckenbach, dwarves

Em Thu, Aug 12, 2021 at 10:50:00PM +0100, Luca Boccassi escreveu:
> On Thu, 12 Aug 2021 at 20:06, Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > Em Thu, Aug 12, 2021 at 04:00:38PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > [acme@seventh pahole]$ ls -la dwarves-1.22/lib/include/
> > > total 8
> > > drwxr-xr-x. 2 acme acme 4096 Aug 12 15:54 .
> > > drwxr-xr-x. 4 acme acme 4096 Aug 12 15:54 ..
> > > lrwxrwxrwx. 1 acme acme   23 Jun 18 17:56 bpf -> dwarves-1.22/../bpf/src
> > > [acme@seventh pahole]$

> > Ok, including the parent directory and then replacing it with the
> > desired package_name-package_version did the trick ;-)

> > ⬢[acme@toolbox pahole]$ v=1.$(($(git tag | sort -V | tail -1 | cut -d. -f2) + 1)) ; tar cvfJ ~/rpmbuild/SOURCES/dwarves-${v}.tar.xz --transform "s,^pahole/,dwarves-${v}/," `sed s%^%../pahole/%g MANIFEST`
> > ⬢[acme@toolbox pahole]$ tar tvf ~/rpmbuild/SOURCES/dwarves-1.22.tar.xz  | tail
> > -rw-r--r-- acme/acme       252 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/helpers.sh
> > -rwxr-xr-x acme/acme      3084 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/mkrootfs.sh
> > -rwxr-xr-x acme/acme        86 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests-4.9.0.sh
> > -rwxr-xr-x acme/acme        86 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests-5.5.0.sh
> > -rwxr-xr-x acme/acme       365 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/prepare_selftests.sh
> > -rwxr-xr-x acme/acme     10841 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run.sh
> > -rwxr-xr-x acme/acme      1245 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run_selftests.sh
> > -rwxr-xr-x acme/acme      1132 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run_vmtest.sh
> > -rw-r--r-- acme/acme   2904227 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/vmlinux.h
> > lrwxrwxrwx acme/acme         0 2021-08-11 08:55 dwarves-1.22/lib/include/bpf -> ../bpf/src
> > ⬢[acme@toolbox pahole]$

> > yeah, it now builds from the generated tarball, updating README.tarball 8-)
 
> Glad to hear you found a fix - I would have had no idea :-)

Now you have, but hopefully will forget about it soon. At least we have
this in place:

https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=next&id=0954ebc2f957cf2e90765479f64a4e8784f76a75

Its like that Debian bug from Frank does: documents what that
--transform is doing so that we don't scratch our heads that much :-)

- Arnaldo

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pahole: Problems with lib/include/bpf symlink in a tarball with --transform
  2021-08-12 19:00 pahole: Problems with lib/include/bpf symlink in a tarball with --transform Arnaldo Carvalho de Melo
  2021-08-12 19:06 ` Arnaldo Carvalho de Melo
@ 2021-08-13 18:15 ` Frank Heckenbach
  1 sibling, 0 replies; 5+ messages in thread
From: Frank Heckenbach @ 2021-08-13 18:15 UTC (permalink / raw)
  To: bluca, acme; +Cc: dwarves

Hi,

> The problem is described in this old Debian ticket:
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775247
> 
> I'm now trying to figure out what Frank meant when referring to that
> "S" flag.
> 
> I have to figure out what is this 'S' flag (local symlinks, etc) or stop
> using this --transform mechanism to have the dwarves-${version} top
> level directory added when generating the tarball :-\

FWIW, the "S" flag means:

  Do not apply transformation to symbolic link targets.

(see https://www.gnu.org/software/tar/manual/html_node/transform.html
-- it's a bit hard to find in the docs ...)

In my use case, this allows me to build a dist archive from my main
source directory, adding a directory level in the tar, while leaving
symlinks unchanged:

  --transform='s|^|$(DIST_NAME)/|S'

This is not a fully general solution to the problem as I described,
and it may or may not help for your use case.

Viele Grüße,
Frank

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-08-13 18:34 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-12 19:00 pahole: Problems with lib/include/bpf symlink in a tarball with --transform Arnaldo Carvalho de Melo
2021-08-12 19:06 ` Arnaldo Carvalho de Melo
2021-08-12 21:50   ` Luca Boccassi
2021-08-13  0:05     ` Arnaldo Carvalho de Melo
2021-08-13 18:15 ` Frank Heckenbach

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).