* Signedness of char in BTF
@ 2022-07-21 14:31 Lorenz Bauer
2022-07-21 14:54 ` Jose E. Marchesi
2022-07-21 18:35 ` Yonghong Song
0 siblings, 2 replies; 9+ messages in thread
From: Lorenz Bauer @ 2022-07-21 14:31 UTC (permalink / raw)
To: yhs, andrii, bpf
Hi Yonghong and Andrii,
I have some questions re: signedness of chars in BTF. According to [1] BTF_INT_ENCODING() may be one of SIGNED, CHAR or BOOL. If I read [2] correctly the signedness of char is implementation defined. Does this mean that I need to know which implementation generated the BTF to interpret CHAR correctly?
Somewhat related, how to I make clang emit BTF_INT_CHAR in the first place? I've tried with clang-14, but only ever get
[6] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=(none)
[6] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED
The kernel seems to agree that CHAR isn't a thing [3].
Thanks!
Lorenz
1: https://www.kernel.org/doc/html/latest/bpf/btf.html#btf-kind-int
2: https://stackoverflow.com/a/2054941/19544965
3: https://sourcegraph.com/github.com/torvalds/linux@353f7988dd8413c47718f7ca79c030b6fb62cfe5/-/blob/kernel/bpf/btf.c?L2928-2934
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: Signedness of char in BTF 2022-07-21 14:31 Signedness of char in BTF Lorenz Bauer @ 2022-07-21 14:54 ` Jose E. Marchesi 2022-07-21 18:44 ` Yonghong Song 2022-07-21 18:35 ` Yonghong Song 1 sibling, 1 reply; 9+ messages in thread From: Jose E. Marchesi @ 2022-07-21 14:54 UTC (permalink / raw) To: Lorenz Bauer; +Cc: yhs, andrii, bpf > Hi Yonghong and Andrii, > > I have some questions re: signedness of chars in BTF. According to [1] > BTF_INT_ENCODING() may be one of SIGNED, CHAR or BOOL. I have always assumed that the bits in `encoding' are non-exclusive i.e. it is a bitmap, not an enumerated. > If I read [2] correctly the signedness of char is implementation > defined. Does this mean that I need to know which implementation > generated the BTF to interpret CHAR correctly? > > Somewhat related, how to I make clang emit BTF_INT_CHAR in the first > place? I've tried with clang-14, but only ever get > > [6] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=(none) > [6] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED Hm, in GCC we currently generate: [1] int 'unsigned char'(0x00000001U#B) size=0x00000001U#B offset=0x00UB#b bits=0x08UB#b CHAR [2] int 'char'(0x00000001U#B) size=0x00000001U#B offset=0x00UB#b bits=0x08UB#b SIGNED CHAR Which turns out is not correct? We used a signed type for `char' because that was what the LLVM BPF toolchain uses, but then we assumed we had to emit the CHAR bit as well... wrong assumption apparently (I just tried with clang 15 and it doesn't set the CHAR bits for neither `char' nor `unsigned char'). But then what is the CHAR bit for? > The kernel seems to agree that CHAR isn't a thing [3]. > > Thanks! > Lorenz > > 1: https://www.kernel.org/doc/html/latest/bpf/btf.html#btf-kind-int > 2: https://stackoverflow.com/a/2054941/19544965 > 3: > https://sourcegraph.com/github.com/torvalds/linux@353f7988dd8413c47718f7ca79c030b6fb62cfe5/-/blob/kernel/bpf/btf.c?L2928-2934 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Signedness of char in BTF 2022-07-21 14:54 ` Jose E. Marchesi @ 2022-07-21 18:44 ` Yonghong Song 2022-07-21 22:21 ` Jose E. Marchesi 0 siblings, 1 reply; 9+ messages in thread From: Yonghong Song @ 2022-07-21 18:44 UTC (permalink / raw) To: Jose E. Marchesi, Lorenz Bauer; +Cc: andrii, bpf On 7/21/22 7:54 AM, Jose E. Marchesi wrote: > >> Hi Yonghong and Andrii, >> >> I have some questions re: signedness of chars in BTF. According to [1] >> BTF_INT_ENCODING() may be one of SIGNED, CHAR or BOOL. > > I have always assumed that the bits in `encoding' are non-exclusive > i.e. it is a bitmap, not an enumerated. Based on current BTF design, it is enumerated. So signed char is 'signed 1-byte int', unsigned char is 'unsigned 1-byte int' and 'char' could be BTF_INT_CHAR but since in debuginfo any 'char' has a signedness bit, so it is folded into 'signed 1-byte int' or 'unsigned 1-byte int'. > >> If I read [2] correctly the signedness of char is implementation >> defined. Does this mean that I need to know which implementation >> generated the BTF to interpret CHAR correctly? >> >> Somewhat related, how to I make clang emit BTF_INT_CHAR in the first >> place? I've tried with clang-14, but only ever get >> >> [6] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=(none) >> [6] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED > > Hm, in GCC we currently generate: > > [1] int 'unsigned char'(0x00000001U#B) size=0x00000001U#B offset=0x00UB#b bits=0x08UB#b CHAR > [2] int 'char'(0x00000001U#B) size=0x00000001U#B offset=0x00UB#b bits=0x08UB#b SIGNED CHAR > > Which turns out is not correct? > > We used a signed type for `char' because that was what the LLVM BPF > toolchain uses, but then we assumed we had to emit the CHAR bit as > well... wrong assumption apparently (I just tried with clang 15 and it > doesn't set the CHAR bits for neither `char' nor `unsigned char'). > > But then what is the CHAR bit for? This is not generated by llvm or pahole but apparently it may still have some meaning when printing the value, a 'char c' may have a dump like 'c' instead of '0x63'. In kernel/bpf/btf.c, we have /* * BTF_INT_CHAR encoding never seems to be set for * char arrays, so if size is 1 and element is * printable as a char, we'll do that. */ if (elem_size == 1) encoding = BTF_INT_CHAR; > >> The kernel seems to agree that CHAR isn't a thing [3]. >> >> Thanks! >> Lorenz >> >> 1: https://www.kernel.org/doc/html/latest/bpf/btf.html#btf-kind-int >> 2: https://stackoverflow.com/a/2054941/19544965 >> 3: >> https://sourcegraph.com/github.com/torvalds/linux@353f7988dd8413c47718f7ca79c030b6fb62cfe5/-/blob/kernel/bpf/btf.c?L2928-2934 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Signedness of char in BTF 2022-07-21 18:44 ` Yonghong Song @ 2022-07-21 22:21 ` Jose E. Marchesi 2022-07-21 22:52 ` Yonghong Song 0 siblings, 1 reply; 9+ messages in thread From: Jose E. Marchesi @ 2022-07-21 22:21 UTC (permalink / raw) To: Yonghong Song; +Cc: Lorenz Bauer, andrii, bpf, david.faust Hi Yonghong. > On 7/21/22 7:54 AM, Jose E. Marchesi wrote: >> >>> Hi Yonghong and Andrii, >>> >>> I have some questions re: signedness of chars in BTF. According to [1] >>> BTF_INT_ENCODING() may be one of SIGNED, CHAR or BOOL. >> I have always assumed that the bits in `encoding' are non-exclusive >> i.e. it is a bitmap, not an enumerated. > > Based on current BTF design, it is enumerated. So signed char > is 'signed 1-byte int', unsigned char is 'unsigned 1-byte int' > and 'char' could be BTF_INT_CHAR but since in debuginfo > any 'char' has a signedness bit, so it is folded into > 'signed 1-byte int' or 'unsigned 1-byte int'. Ok, we will change GCC so it does the same thing. What about BOOL? I don't think we ever use that bit. Does LLVM generate it for any case? >>> If I read [2] correctly the signedness of char is implementation >>> defined. Does this mean that I need to know which implementation >>> generated the BTF to interpret CHAR correctly? >>> >>> Somewhat related, how to I make clang emit BTF_INT_CHAR in the first >>> place? I've tried with clang-14, but only ever get >>> >>> [6] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=(none) >>> [6] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED >> Hm, in GCC we currently generate: >> [1] int 'unsigned char'(0x00000001U#B) size=0x00000001U#B >> offset=0x00UB#b bits=0x08UB#b CHAR >> [2] int 'char'(0x00000001U#B) size=0x00000001U#B offset=0x00UB#b bits=0x08UB#b SIGNED CHAR >> Which turns out is not correct? >> We used a signed type for `char' because that was what the LLVM BPF >> toolchain uses, but then we assumed we had to emit the CHAR bit as >> well... wrong assumption apparently (I just tried with clang 15 and it >> doesn't set the CHAR bits for neither `char' nor `unsigned char'). >> But then what is the CHAR bit for? > > This is not generated by llvm or pahole but apparently it may still > have some meaning when printing the value, a 'char c' may have > a dump like 'c' instead of '0x63'. In kernel/bpf/btf.c, we have > > /* > * BTF_INT_CHAR encoding never seems to be set for > * char arrays, so if size is 1 and element is > * printable as a char, we'll do that. > */ > if (elem_size == 1) > encoding = BTF_INT_CHAR; > >> >>> The kernel seems to agree that CHAR isn't a thing [3]. >>> >>> Thanks! >>> Lorenz >>> >>> 1: https://www.kernel.org/doc/html/latest/bpf/btf.html#btf-kind-int >>> 2: https://stackoverflow.com/a/2054941/19544965 >>> 3: >>> https://sourcegraph.com/github.com/torvalds/linux@353f7988dd8413c47718f7ca79c030b6fb62cfe5/-/blob/kernel/bpf/btf.c?L2928-2934 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Signedness of char in BTF 2022-07-21 22:21 ` Jose E. Marchesi @ 2022-07-21 22:52 ` Yonghong Song 2022-07-22 11:25 ` Jose E. Marchesi 0 siblings, 1 reply; 9+ messages in thread From: Yonghong Song @ 2022-07-21 22:52 UTC (permalink / raw) To: Jose E. Marchesi; +Cc: Lorenz Bauer, andrii, bpf, david.faust On 7/21/22 3:21 PM, Jose E. Marchesi wrote: > > Hi Yonghong. > >> On 7/21/22 7:54 AM, Jose E. Marchesi wrote: >>> >>>> Hi Yonghong and Andrii, >>>> >>>> I have some questions re: signedness of chars in BTF. According to [1] >>>> BTF_INT_ENCODING() may be one of SIGNED, CHAR or BOOL. >>> I have always assumed that the bits in `encoding' are non-exclusive >>> i.e. it is a bitmap, not an enumerated. >> >> Based on current BTF design, it is enumerated. So signed char >> is 'signed 1-byte int', unsigned char is 'unsigned 1-byte int' >> and 'char' could be BTF_INT_CHAR but since in debuginfo >> any 'char' has a signedness bit, so it is folded into >> 'signed 1-byte int' or 'unsigned 1-byte int'. > > Ok, we will change GCC so it does the same thing. > > What about BOOL? I don't think we ever use that bit. Does LLVM > generate it for any case? The llvm and pahole generate BTF_INT_BOOL when the dwarf type has attribute DW_ATE_boolean. But BTF_INT_BOOL is actually used in libbpf to differentiate configuration values (CONFIG_* = 'y' vs. CONFIG_* = <value>) In llvm, uint8_t BTFEncoding; switch (Encoding) { case dwarf::DW_ATE_boolean: BTFEncoding = BTF::INT_BOOL; break; case dwarf::DW_ATE_signed: case dwarf::DW_ATE_signed_char: BTFEncoding = BTF::INT_SIGNED; break; case dwarf::DW_ATE_unsigned: case dwarf::DW_ATE_unsigned_char: BTFEncoding = 0; break; default: llvm_unreachable("Unknown BTFTypeInt Encoding"); } For a concrete example, [$ ~/tmp1] cat t.c int test(_Bool g) { return g; } [$ ~/tmp1] clang -target bpf -O2 -g -c t.c [$ ~/tmp1] bpftool btf dump file t.o [1] INT '_Bool' size=1 bits_offset=0 nr_bits=8 encoding=BOOL [2] FUNC_PROTO '(anon)' ret_type_id=3 vlen=1 'g' type_id=1 [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [4] FUNC 'test' type_id=2 linkage=global [$ ~/tmp1] > >>>> If I read [2] correctly the signedness of char is implementation >>>> defined. Does this mean that I need to know which implementation >>>> generated the BTF to interpret CHAR correctly? >>>> >>>> Somewhat related, how to I make clang emit BTF_INT_CHAR in the first >>>> place? I've tried with clang-14, but only ever get >>>> >>>> [6] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=(none) >>>> [6] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED >>> Hm, in GCC we currently generate: >>> [1] int 'unsigned char'(0x00000001U#B) size=0x00000001U#B >>> offset=0x00UB#b bits=0x08UB#b CHAR >>> [2] int 'char'(0x00000001U#B) size=0x00000001U#B offset=0x00UB#b bits=0x08UB#b SIGNED CHAR >>> Which turns out is not correct? >>> We used a signed type for `char' because that was what the LLVM BPF >>> toolchain uses, but then we assumed we had to emit the CHAR bit as >>> well... wrong assumption apparently (I just tried with clang 15 and it >>> doesn't set the CHAR bits for neither `char' nor `unsigned char'). >>> But then what is the CHAR bit for? >> >> This is not generated by llvm or pahole but apparently it may still >> have some meaning when printing the value, a 'char c' may have >> a dump like 'c' instead of '0x63'. In kernel/bpf/btf.c, we have >> >> /* >> * BTF_INT_CHAR encoding never seems to be set for >> * char arrays, so if size is 1 and element is >> * printable as a char, we'll do that. >> */ >> if (elem_size == 1) >> encoding = BTF_INT_CHAR; >> >>> >>>> The kernel seems to agree that CHAR isn't a thing [3]. >>>> >>>> Thanks! >>>> Lorenz >>>> >>>> 1: https://www.kernel.org/doc/html/latest/bpf/btf.html#btf-kind-int >>>> 2: https://stackoverflow.com/a/2054941/19544965 >>>> 3: >>>> https://sourcegraph.com/github.com/torvalds/linux@353f7988dd8413c47718f7ca79c030b6fb62cfe5/-/blob/kernel/bpf/btf.c?L2928-2934 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Signedness of char in BTF 2022-07-21 22:52 ` Yonghong Song @ 2022-07-22 11:25 ` Jose E. Marchesi 2022-07-22 15:59 ` Yonghong Song 2022-08-02 17:28 ` Jose E. Marchesi 0 siblings, 2 replies; 9+ messages in thread From: Jose E. Marchesi @ 2022-07-22 11:25 UTC (permalink / raw) To: Yonghong Song; +Cc: Lorenz Bauer, andrii, bpf, david.faust > The llvm and pahole generate BTF_INT_BOOL when the dwarf type has > attribute DW_ATE_boolean. > But BTF_INT_BOOL is actually used in libbpf to differentiate > configuration values (CONFIG_* = 'y' vs. CONFIG_* = <value>) > > In llvm, > uint8_t BTFEncoding; > switch (Encoding) { > case dwarf::DW_ATE_boolean: > BTFEncoding = BTF::INT_BOOL; > break; > case dwarf::DW_ATE_signed: > case dwarf::DW_ATE_signed_char: > BTFEncoding = BTF::INT_SIGNED; > break; > case dwarf::DW_ATE_unsigned: > case dwarf::DW_ATE_unsigned_char: > BTFEncoding = 0; > break; > default: > llvm_unreachable("Unknown BTFTypeInt Encoding"); > } I just sent a patch to make GCC behave the same way: https://gcc.gnu.org/pipermail/gcc-patches/2022-July/598702.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Signedness of char in BTF 2022-07-22 11:25 ` Jose E. Marchesi @ 2022-07-22 15:59 ` Yonghong Song 2022-08-02 17:28 ` Jose E. Marchesi 1 sibling, 0 replies; 9+ messages in thread From: Yonghong Song @ 2022-07-22 15:59 UTC (permalink / raw) To: Jose E. Marchesi; +Cc: Lorenz Bauer, andrii, bpf, david.faust On 7/22/22 4:25 AM, Jose E. Marchesi wrote: > >> The llvm and pahole generate BTF_INT_BOOL when the dwarf type has >> attribute DW_ATE_boolean. >> But BTF_INT_BOOL is actually used in libbpf to differentiate >> configuration values (CONFIG_* = 'y' vs. CONFIG_* = <value>) >> >> In llvm, >> uint8_t BTFEncoding; >> switch (Encoding) { >> case dwarf::DW_ATE_boolean: >> BTFEncoding = BTF::INT_BOOL; >> break; >> case dwarf::DW_ATE_signed: >> case dwarf::DW_ATE_signed_char: >> BTFEncoding = BTF::INT_SIGNED; >> break; >> case dwarf::DW_ATE_unsigned: >> case dwarf::DW_ATE_unsigned_char: >> BTFEncoding = 0; >> break; >> default: >> llvm_unreachable("Unknown BTFTypeInt Encoding"); >> } > > I just sent a patch to make GCC behave the same way: > https://gcc.gnu.org/pipermail/gcc-patches/2022-July/598702.html Sounds good. Thanks! ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Signedness of char in BTF 2022-07-22 11:25 ` Jose E. Marchesi 2022-07-22 15:59 ` Yonghong Song @ 2022-08-02 17:28 ` Jose E. Marchesi 1 sibling, 0 replies; 9+ messages in thread From: Jose E. Marchesi @ 2022-08-02 17:28 UTC (permalink / raw) To: Yonghong Song; +Cc: Lorenz Bauer, andrii, bpf, david.faust >> The llvm and pahole generate BTF_INT_BOOL when the dwarf type has >> attribute DW_ATE_boolean. >> But BTF_INT_BOOL is actually used in libbpf to differentiate >> configuration values (CONFIG_* = 'y' vs. CONFIG_* = <value>) >> >> In llvm, >> uint8_t BTFEncoding; >> switch (Encoding) { >> case dwarf::DW_ATE_boolean: >> BTFEncoding = BTF::INT_BOOL; >> break; >> case dwarf::DW_ATE_signed: >> case dwarf::DW_ATE_signed_char: >> BTFEncoding = BTF::INT_SIGNED; >> break; >> case dwarf::DW_ATE_unsigned: >> case dwarf::DW_ATE_unsigned_char: >> BTFEncoding = 0; >> break; >> default: >> llvm_unreachable("Unknown BTFTypeInt Encoding"); >> } > > I just sent a patch to make GCC behave the same way: > https://gcc.gnu.org/pipermail/gcc-patches/2022-July/598702.html This patch is now applied in GCC master. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Signedness of char in BTF 2022-07-21 14:31 Signedness of char in BTF Lorenz Bauer 2022-07-21 14:54 ` Jose E. Marchesi @ 2022-07-21 18:35 ` Yonghong Song 1 sibling, 0 replies; 9+ messages in thread From: Yonghong Song @ 2022-07-21 18:35 UTC (permalink / raw) To: Lorenz Bauer, andrii, bpf On 7/21/22 7:31 AM, Lorenz Bauer wrote: > Hi Yonghong and Andrii, > > I have some questions re: signedness of chars in BTF. According to [1] BTF_INT_ENCODING() may be one of SIGNED, CHAR or BOOL. If I read [2] correctly the signedness of char is implementation defined. Does this mean that I need to know which implementation generated the BTF to interpret CHAR correctly? > > Somewhat related, how to I make clang emit BTF_INT_CHAR in the first place? I've tried with clang-14, but only ever get > > [6] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=(none) > [6] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED > > The kernel seems to agree that CHAR isn't a thing [3]. clang does not generate BTF_INT_CHAR. BTFTypeInt::BTFTypeInt(uint32_t Encoding, uint32_t SizeInBits, uint32_t OffsetInBits, StringRef TypeName) : Name(TypeName) { // Translate IR int encoding to BTF int encoding. uint8_t BTFEncoding; switch (Encoding) { case dwarf::DW_ATE_boolean: BTFEncoding = BTF::INT_BOOL; break; case dwarf::DW_ATE_signed: case dwarf::DW_ATE_signed_char: BTFEncoding = BTF::INT_SIGNED; break; case dwarf::DW_ATE_unsigned: case dwarf::DW_ATE_unsigned_char: BTFEncoding = 0; /* INT_UNSIGNED */ break; default: llvm_unreachable("Unknown BTFTypeInt Encoding"); } pahole does not generate INT_CHAR type either. in pahole: static int32_t btf_encoder__add_base_type(struct btf_encoder *encoder, const struct base_type *bt, co nst char *name) { const struct btf_type *t; uint8_t encoding = 0; /* unsigned */ uint16_t byte_sz; int32_t id; if (bt->is_signed) { encoding = BTF_INT_SIGNED; } else if (bt->is_bool) { encoding = BTF_INT_BOOL; } else if (bt->float_type && encoder->gen_floats) { /* for floats */ } ... } So for both clang and pahole, CHAR goes to INT_SIGNED or INT_UNSIGNED. The reason is originally BTF tries to mimic CTF but a simplified version, and CTF has CTF_TYPE_INT_CHAR, but later on found BTF_INT_CHAR is not that useful so llvm and pahole doesn't generate it any more. The libbpf and kernel still supports BTF_INT_CHAR and when it is used to print out values it is interpreted as type 'char'. > > Thanks! > Lorenz > > 1: https://www.kernel.org/doc/html/latest/bpf/btf.html#btf-kind-int > 2: https://stackoverflow.com/a/2054941/19544965 > 3: https://sourcegraph.com/github.com/torvalds/linux@353f7988dd8413c47718f7ca79c030b6fb62cfe5/-/blob/kernel/bpf/btf.c?L2928-2934 ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-08-02 17:29 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-07-21 14:31 Signedness of char in BTF Lorenz Bauer 2022-07-21 14:54 ` Jose E. Marchesi 2022-07-21 18:44 ` Yonghong Song 2022-07-21 22:21 ` Jose E. Marchesi 2022-07-21 22:52 ` Yonghong Song 2022-07-22 11:25 ` Jose E. Marchesi 2022-07-22 15:59 ` Yonghong Song 2022-08-02 17:28 ` Jose E. Marchesi 2022-07-21 18:35 ` Yonghong Song
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.