From: Yonghong Song <yhs@fb.com>
To: <sedat.dilek@gmail.com>
Cc: "Mark Wieelard" <mark@klomp.org>,
"Masahiro Yamada" <masahiroy@kernel.org>,
"Arnaldo Carvalho de Melo" <acme@kernel.org>,
"Arnaldo Carvalho de Melo" <arnaldo.melo@gmail.com>,
dwarves@vger.kernel.org,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
bpf@vger.kernel.org, "Jiri Olsa" <jolsa@kernel.org>,
"Jan Engelhardt" <jengelh@inai.de>,
"Domenico Andreoli" <cavok@debian.org>,
"Matthias Schwarzott" <zzam@gentoo.org>,
"Andrii Nakryiko" <andriin@fb.com>,
"Paul Moore" <paul@paul-moore.com>,
"Ondrej Mosnacek" <omosnace@redhat.com>,
"Daniel P. Berrangé" <berrange@redhat.com>,
"Tom Stellard" <tstellar@redhat.com>
Subject: Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type
Date: Sat, 6 Feb 2021 11:17:07 -0800 [thread overview]
Message-ID: <64206fbc-656a-5ffd-6e9d-739c8c6f7410@fb.com> (raw)
In-Reply-To: <CA+icZUVfTH=yONintyJ+T8kvTrR4Q0gumJYNUCs6Ybraff5Kpg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4115 bytes --]
On 2/6/21 10:10 AM, Sedat Dilek wrote:
> On Sat, Feb 6, 2021 at 6:53 PM Yonghong Song <yhs@fb.com> wrote:
>>
>>
>>
>> On 2/6/21 8:24 AM, Mark Wieelard wrote:
>>> Hi,
>>>
>>> On Sat, Feb 06, 2021 at 12:26:44AM -0800, Yonghong Song wrote:
>>>> With the above vmlinux, the issue appears to be handling
>>>> DW_ATE_signed_1, DW_ATE_unsigned_{1,24,40}.
>>>>
>>>> The following patch should fix the issue:
>>>
>>> That doesn't really make sense to me. Why is the compiler emitting a
>>> DW_TAG_base_type that needs to be interpreted according to the
>>> DW_AT_name attribute?
>>>
>>> If the issue is that the size of the base type cannot be expressed in
>>> bytes then the DWARF spec provides the following option:
>>>
>>> If the value of an object of the given type does not fully occupy
>>> the storage described by a byte size attribute, the base type
>>> entry may also have a DW_AT_bit_size and a DW_AT_data_bit_offset
>>> attribute, both of whose values are integer constant values (see
>>> Section 2.19 on page 55). The bit size attribute describes the
>>> actual size in bits used to represent values of the given
>>> type. The data bit offset attribute is the offset in bits from the
>>> beginning of the containing storage to the beginning of the
>>> value. Bits that are part of the offset are padding. If this
>>> attribute is omitted a default data bit offset of zero is assumed.
>>>
>>> Would it be possible to use that encoding of those special types? If
>>
>> I agree with you. I do not like comparing me as well. Unfortunately,
>> there is no enough information in dwarf to find out actual information.
>> The following is the dwarf dump with vmlinux (Sedat provided) for
>> DW_ATE_unsigned_1.
>>
>> 0x000e97e9: DW_TAG_base_type
>> DW_AT_name ("DW_ATE_unsigned_1")
>> DW_AT_encoding (DW_ATE_unsigned)
>> DW_AT_byte_size (0x00)
>>
>> There is no DW_AT_bit_size and DW_AT_bit_offset for base type.
>> AFAIK, these two attributes typically appear in struct/union members
>> together with DW_AT_byte_size.
>>
>> Maybe compilers (clang in this case) can emit DW_AT_bit_size = 1
>> and DW_AT_bit_offset = 0/7 (depending on big/little endian) and
>> this case, we just test and get DW_AT_bit_size and it should work.
>>
>> But I think BTF does not need this (DW_ATE_unsigned_1) for now.
>> I checked dwarf dump and it is mostly used for some arith operation
>> encoded in dump (in this case, e.g., shift by 1 bit)
>>
>> 0x000015cf: DW_TAG_base_type
>> DW_AT_name ("DW_ATE_unsigned_1")
>> DW_AT_encoding (DW_ATE_unsigned)
>> DW_AT_byte_size (0x00)
>>
>> 0x00010ed9: DW_TAG_formal_parameter
>> DW_AT_location (DW_OP_lit0, DW_OP_not,
>> DW_OP_convert (0x000015cf) "DW_ATE_unsigned_1", DW_OP_convert
>> (0x000015d4) "DW_ATE_unsigned_8", DW_OP_stack_value)
>> DW_AT_abstract_origin (0x00013984 "branch")
>>
>> Look at clang frontend, only the following types are encoded with
>> unsigned dwarf type.
>>
>> case BuiltinType::UShort:
>> case BuiltinType::UInt:
>> case BuiltinType::UInt128:
>> case BuiltinType::ULong:
>> case BuiltinType::WChar_U:
>> case BuiltinType::ULongLong:
>> Encoding = llvm::dwarf::DW_ATE_unsigned;
>> break;
>>
>>
>>> not, can we try to come up with some extension that doesn't require
>>> consumers to match magic names?
>>>
>
> You want me to upload mlx5_core.ko?
I just sent out a patch. You are cc'ed. I also attached in this email.
Yes, it would be great if you can upload mlx5_core.ko so I can
double check with this DW_ATE_unsigned_160 which is really usual.
>
> When looking with llvm-dwarf for DW_ATE_unsigned_160:
>
> 0x00d65616: DW_TAG_base_type
> DW_AT_name ("DW_ATE_unsigned_160")
> DW_AT_encoding (DW_ATE_unsigned)
> DW_AT_byte_size (0x14)
>
> If you need further information, please let me know.
>
> Thanks.
>
> - Sedat -
>
[-- Attachment #2: 0001-btf_encoder-sanitize-non-regular-int-base-type.patch --]
[-- Type: text/plain, Size: 5139 bytes --]
From 239c797090abbdc5253d0ff1e9e657c5006fbbee Mon Sep 17 00:00:00 2001
From: Yonghong Song <yhs@fb.com>
Date: Sat, 6 Feb 2021 10:21:45 -0800
Subject: [PATCH dwarves] btf_encoder: sanitize non-regular int base type
clang with dwarf5 may generate non-regular int base type,
i.e., not a signed/unsigned char/short/int/longlong/__int128.
Such base types are often used to describe
how an actual parameter or variable is generated. For example,
0x000015cf: DW_TAG_base_type
DW_AT_name ("DW_ATE_unsigned_1")
DW_AT_encoding (DW_ATE_unsigned)
DW_AT_byte_size (0x00)
0x00010ed9: DW_TAG_formal_parameter
DW_AT_location (DW_OP_lit0,
DW_OP_not,
DW_OP_convert (0x000015cf) "DW_ATE_unsigned_1",
DW_OP_convert (0x000015d4) "DW_ATE_unsigned_8",
DW_OP_stack_value)
DW_AT_abstract_origin (0x00013984 "branch")
What it does is with a literal "0", did a "not" operation, and the converted to
one-bit unsigned int and then 8-bit unsigned int.
Another example,
0x000e97e4: DW_TAG_base_type
DW_AT_name ("DW_ATE_unsigned_24")
DW_AT_encoding (DW_ATE_unsigned)
DW_AT_byte_size (0x03)
0x000f88f8: DW_TAG_variable
DW_AT_location (indexed (0x3c) loclist = 0x00008fb0:
[0xffffffff82808812, 0xffffffff82808817):
DW_OP_breg0 RAX+0,
DW_OP_convert (0x000e97d5) "DW_ATE_unsigned_64",
DW_OP_convert (0x000e97df) "DW_ATE_unsigned_8",
DW_OP_stack_value,
DW_OP_piece 0x1,
DW_OP_breg0 RAX+0,
DW_OP_convert (0x000e97d5) "DW_ATE_unsigned_64",
DW_OP_convert (0x000e97da) "DW_ATE_unsigned_32",
DW_OP_lit8,
DW_OP_shr,
DW_OP_convert (0x000e97da) "DW_ATE_unsigned_32",
DW_OP_convert (0x000e97e4) "DW_ATE_unsigned_24",
DW_OP_stack_value,
DW_OP_piece 0x3
......
At one point, a right shift by 8 happens and the result is converted to
32-bit unsigned int and then to 24-bit unsigned int.
BTF does not need any of these DW_OP_* information and such non-regular int
types will cause libbpf to emit errors.
Let us sanitize them to generate BTF acceptable to libbpf and kernel.
Cc: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
---
libbtf.c | 39 ++++++++++++++++++++++++++++++++++++++-
1 file changed, 38 insertions(+), 1 deletion(-)
diff --git a/libbtf.c b/libbtf.c
index 9f76283..93fe185 100644
--- a/libbtf.c
+++ b/libbtf.c
@@ -373,6 +373,7 @@ int32_t btf_elf__add_base_type(struct btf_elf *btfe, const struct base_type *bt,
struct btf *btf = btfe->btf;
const struct btf_type *t;
uint8_t encoding = 0;
+ uint16_t byte_sz;
int32_t id;
if (bt->is_signed) {
@@ -384,7 +385,43 @@ int32_t btf_elf__add_base_type(struct btf_elf *btfe, const struct base_type *bt,
return -1;
}
- id = btf__add_int(btf, name, BITS_ROUNDUP_BYTES(bt->bit_size), encoding);
+ /* dwarf5 may emit DW_ATE_[un]signed_{num} base types where
+ * {num} is not power of 2 and may exceed 128. Such attributes
+ * are mostly used to record operation for an actual parameter
+ * or variable.
+ * For example,
+ * DW_AT_location (indexed (0x3c) loclist = 0x00008fb0:
+ * [0xffffffff82808812, 0xffffffff82808817):
+ * DW_OP_breg0 RAX+0,
+ * DW_OP_convert (0x000e97d5) "DW_ATE_unsigned_64",
+ * DW_OP_convert (0x000e97df) "DW_ATE_unsigned_8",
+ * DW_OP_stack_value,
+ * DW_OP_piece 0x1,
+ * DW_OP_breg0 RAX+0,
+ * DW_OP_convert (0x000e97d5) "DW_ATE_unsigned_64",
+ * DW_OP_convert (0x000e97da) "DW_ATE_unsigned_32",
+ * DW_OP_lit8,
+ * DW_OP_shr,
+ * DW_OP_convert (0x000e97da) "DW_ATE_unsigned_32",
+ * DW_OP_convert (0x000e97e4) "DW_ATE_unsigned_24",
+ * DW_OP_stack_value, DW_OP_piece 0x3
+ * DW_AT_name ("ebx")
+ * DW_AT_decl_file ("/linux/arch/x86/events/intel/core.c")
+ *
+ * In the above example, at some point, one unsigned_32 value
+ * is right shifted by 8 and the result is converted to unsigned_32
+ * and then unsigned_24.
+ *
+ * BTF does not need such DW_OP_* information so let us sanitize
+ * these non-regular int types to avoid libbpf/kernel complaints.
+ */
+ byte_sz = BITS_ROUNDUP_BYTES(bt->bit_size);
+ if (!byte_sz || (byte_sz & (byte_sz - 1))) {
+ name = "sanitized_int";
+ byte_sz = 4;
+ }
+
+ id = btf__add_int(btf, name, byte_sz, encoding);
if (id < 0) {
btf_elf__log_err(btfe, BTF_KIND_INT, name, true, "Error emitting BTF type");
} else {
--
2.24.1
next prev parent reply other threads:[~2021-02-06 19:18 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-04 22:07 ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF) Arnaldo Carvalho de Melo
2021-02-04 22:10 ` Sedat Dilek
2021-02-05 0:01 ` Andrii Nakryiko
2021-02-05 4:33 ` Arnaldo Carvalho de Melo
2021-02-05 7:39 ` Andrii Nakryiko
2021-02-05 9:33 ` Arnaldo Carvalho de Melo
2021-02-05 16:25 ` Arnaldo Carvalho de Melo
2021-02-05 22:11 ` Andrii Nakryiko
2021-02-05 23:55 ` Arnaldo Carvalho de Melo
2021-02-07 6:40 ` Andrii Nakryiko
2021-02-05 14:37 ` ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type Sedat Dilek
2021-02-05 14:41 ` Sedat Dilek
2021-02-05 15:23 ` Sedat Dilek
2021-02-05 15:28 ` Arnaldo Carvalho de Melo
2021-02-05 15:40 ` Sedat Dilek
2021-02-05 17:48 ` Sedat Dilek
2021-02-05 18:53 ` Sedat Dilek
2021-02-05 19:06 ` Sedat Dilek
2021-02-05 19:10 ` Yonghong Song
2021-02-05 19:15 ` Sedat Dilek
2021-02-05 19:20 ` Yonghong Song
2021-02-05 19:24 ` Sedat Dilek
2021-02-05 19:44 ` Sedat Dilek
2021-02-05 19:21 ` Sedat Dilek
2021-02-05 19:30 ` Fāng-ruì Sòng
2021-02-05 19:38 ` Sedat Dilek
2021-02-05 19:24 ` Arnaldo Carvalho de Melo
2021-02-05 20:03 ` Yonghong Song
2021-02-05 20:31 ` Sedat Dilek
2021-02-05 21:54 ` Yonghong Song
2021-02-06 3:34 ` Sedat Dilek
2021-02-06 5:44 ` Sedat Dilek
2021-02-06 5:53 ` Sedat Dilek
2021-02-06 6:26 ` Sedat Dilek
2021-02-06 6:52 ` Sedat Dilek
2021-02-06 8:26 ` Yonghong Song
2021-02-06 8:32 ` Sedat Dilek
2021-02-06 9:32 ` Sedat Dilek
2021-02-06 9:37 ` Sedat Dilek
2021-02-06 9:48 ` Sedat Dilek
2021-02-06 10:16 ` Sedat Dilek
2021-02-06 10:23 ` Sedat Dilek
2021-02-06 11:27 ` Sedat Dilek
2021-02-06 12:54 ` Sedat Dilek
2021-02-06 16:24 ` Mark Wieelard
2021-02-06 17:53 ` Yonghong Song
2021-02-06 18:10 ` Sedat Dilek
2021-02-06 19:17 ` Yonghong Song [this message]
2021-02-06 19:22 ` Sedat Dilek
2021-02-06 19:28 ` Sedat Dilek
2021-02-06 19:32 ` Yonghong Song
2021-02-06 19:44 ` Sedat Dilek
2021-02-06 20:12 ` Yonghong Song
2021-02-06 20:46 ` Sedat Dilek
2021-02-06 7:26 ` Sedat Dilek
2021-02-05 21:10 ` Nick Desaulniers
2021-02-05 21:16 ` Sedat Dilek
2021-02-05 21:40 ` Nick Desaulniers
2021-02-05 21:42 ` Sedat Dilek
2021-02-08 2:44 ` ANNOUNCE: pahole v1.20 (gcc11 DWARF5's default, lots of ELF sections, BTF) Sedat Dilek
2021-02-08 12:32 ` Arnaldo Carvalho de Melo
2021-02-08 12:36 ` Sedat Dilek
2021-02-17 12:08 ` Domenico Andreoli
2021-02-17 12:48 ` Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=64206fbc-656a-5ffd-6e9d-739c8c6f7410@fb.com \
--to=yhs@fb.com \
--cc=acme@kernel.org \
--cc=andriin@fb.com \
--cc=arnaldo.melo@gmail.com \
--cc=berrange@redhat.com \
--cc=bpf@vger.kernel.org \
--cc=cavok@debian.org \
--cc=dwarves@vger.kernel.org \
--cc=jengelh@inai.de \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mark@klomp.org \
--cc=masahiroy@kernel.org \
--cc=omosnace@redhat.com \
--cc=paul@paul-moore.com \
--cc=sedat.dilek@gmail.com \
--cc=tstellar@redhat.com \
--cc=zzam@gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).