From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10A3FC35257 for ; Mon, 5 Oct 2020 03:48:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 856B6207FF for ; Mon, 5 Oct 2020 03:48:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aDUyffpk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725852AbgJEDsm (ORCPT ); Sun, 4 Oct 2020 23:48:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725845AbgJEDsm (ORCPT ); Sun, 4 Oct 2020 23:48:42 -0400 Received: from mail-il1-x142.google.com (mail-il1-x142.google.com [IPv6:2607:f8b0:4864:20::142]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F056EC0613CE; Sun, 4 Oct 2020 20:48:41 -0700 (PDT) Received: by mail-il1-x142.google.com with SMTP id z5so6554509ilq.5; Sun, 04 Oct 2020 20:48:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=dBhgz32sZMbx5kb9JCuOzPqh+A3JaVp4ByVa9zVTLVE=; b=aDUyffpkhi+f6OMwsWXBa+8miiLp17CTkihEpZxCX6ES5oriL4vATRtorKlEFv/F8L 4oCeHpl6gE7sNUNLBjJG4ZV7+6hjWxWqfIfkHqmGhtr6e8y6Mk2S09DuKz+00PPNbQ86 rHh3e1A9EsDYD585CqdU+/mQjw42/nYjNBvMf5igpW7p+kzQFEz+VAhMUmKt+jpCauTk +TwT66QfhlMl3S9F/mwGJmnXmuGfvf2lJa9uATVulPONeOmUglqX/syy6mEdngLnmGyd UMHxO8eaNXW5LLSkvOZ2Q1pqcJTuQ2Cq1A5Sss/ZztvSZxdp5+8nypxz70Z3I7CUoGAw v7Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=dBhgz32sZMbx5kb9JCuOzPqh+A3JaVp4ByVa9zVTLVE=; b=KQZOdOsXJ56bttDjiQuK60HRK1Y6C1mqzfxsId37CliT9zGKzkMPODP2vZ8NIpz8qg 1E71qogt/4rlJOISHxR7X9CR7lbFrcchXoppQ7wTg7VL7FF9TQNaGN8APVYqpbhwdO+b dOtEi7jZUeOxhMrrr+ika75veRl1zPn6WnGWXTS/Tl2zQLyDtBS42TYS737f3IKzxooL LSz4CJdii/qt5W1Bae12yaHXjnCxyd41cvDDqE2kjB4JD6WGMNHMjbqabefHuq1esDu8 Dpv7ZYQQWRJMw4Ae/dp2N5OunIqWNx79jCDoK3M86V//cLX3GR5Mfr9m26wpu85iHek3 IfEw== X-Gm-Message-State: AOAM532jTCuqezHUHFg/2BarJMJFnE2xeLdHB9qBa3BybXQVKD5N3XXl +OaBt8AiSORTS4iTkLG6hFJk8M97zpNtIkUuyJU= X-Google-Smtp-Source: ABdhPJxB1Cy3OzJH699TGu4tVstMbpXMq5mOwJ5CiXtqXwRfalD0JAYcckBiuLUj6J7Fr0EidDJEYEx+3t4Dv2mygNw= X-Received: by 2002:a92:cb8c:: with SMTP id z12mr9209889ilo.123.1601869721004; Sun, 04 Oct 2020 20:48:41 -0700 (PDT) MIME-Version: 1.0 From: Tony Ambardar Date: Sun, 4 Oct 2020 20:48:32 -0700 Message-ID: Subject: Re: [PATCH dwarves 00/11] Switch BTF loading and encoding to libbpf APIs To: andriin@fb.com Cc: bpf@vger.kernel.org, dwarves@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: dwarves@vger.kernel.org On Tue, Sep 29, 2020 at 09:27:31PM -0700, Andrii Nakryiko wrote: > This patch set switches pahole to use libbpf-provided BTF loading and enc= oding > APIs. This reduces pahole's own BTF encoding code, speeds up the process, > reduces amount of RAM needed for DWARF-to-BTF conversion. Also, pahole fi= nally > gets support to generating BTF for cross-compiled ELF binaries with diffe= rent > endianness (patch #11). > Hello Andrii, After a small hiccup (see below) I managed to build a modified 'pahole' and= test cross-compiling from x86_64 to mips 64/32-bit and big/little-endian targets. Using "bpftool btf dump file /sys/kernel/btf/vmlinux format c" succeeded on all targets, whereas prior to your changes running on big-endian targets would raise an error. (Note that the 'bpftool' used a 'libbpf' without any of your changes.) Thanks so much for tackling these BTF endianness problems; it's been a grea= t help for working with embedded systems. > Additionally, patch #6 fixes previously missed problem with invalid array > index type generation. > > Patches #7-10 are speeding up DWARF-to-BTF convertion/dedup pretty > significantly, saving overall about 9 seconds out of current 27 or so. > > Patch #8 revamps how per-CPU BTF variables are emitted, eliminating repea= ted > and expensive looping over ELF symbols table. > > Patch #10 admittedly has some hacky parts to satisfy CTF use case, but it= s > speed ups are greatest. So I'll understand if it gets dropped, but it wou= ld be > a pity. > Possibly a case of operator error, but I had to skip this patch to cleanly build 'pahole', and didn't have much chance to look into the compile error: [ 1%] Building C object CMakeFiles/bpf.dir/lib/bpf/src/ringbuf.c.o In file included from /home/kodidev/pahole/strings.h:9, from /usr/include/string.h:432, from /home/kodidev/pahole/lib/bpf/src/libbpf_common.h:12= , from /home/kodidev/pahole/lib/bpf/src/libbpf.h:20, from /home/kodidev/pahole/lib/bpf/src/ringbuf.c:20: /home/kodidev/pahole/lib/bpf/src/btf.h:33:11: error: expected =E2=80=98;= =E2=80=99 before =E2=80=98void=E2=80=99 33 | LIBBPF_API void btf__free(struct btf *btf); | ^~~~~ | ; Kind regards, Tony > More details could be found in respective patches. > > Andrii Nakryiko (11): > libbpf: update to latest libbpf version > btf_encoder: detect BTF encoding errors and exit > dwarves: expose and maintain active debug info loader operations > btf_loader: use libbpf to load BTF > btf_encoder: use libbpf APIs to encode BTF type info > btf_encoder: fix emitting __ARRAY_SIZE_TYPE__ as index range type > btf_encoder: discard CUs after BTF encoding > btf_encoder: revamp how per-CPU variables are encoded > dwarf_loader: increase the size of lookup hash map > strings: use BTF's string APIs for strings management > btf_encoder: support cross-compiled ELF binaries with different > endianness >