single, comprehensive kernel data types document?

single, comprehensive kernel data types document?

[Robert P. J. Day]

  is there a single, decent online doc that explains the proper data
types (int16_t, int32_t and so on) to use in kernel code? including
the relationship with types to be used in code to be exported to user
space (include/uapi/linux/types.h)?

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

single, comprehensive kernel data types document?

[Greg KH]

On Fri, Apr 15, 2016 at 08:04:53AM -0400, Robert P. J. Day wrote:
> 
>   is there a single, decent online doc that explains the proper data
> types (int16_t, int32_t and so on) to use in kernel code?

First off, never use int16_t and friends, that's not ok :)

Second, it's simple, use:
	u8
	u16
	u32
	u64
and friends in kernel code (s8, s16, and so on for signed values.)

'int' is a return type, and for loops and other things that you know
will fit in that size.

> including the relationship with types to be used in code to be
> exported to user space (include/uapi/linux/types.h)?

For values that cross the user/kernel boundry, add '__' to the front of
the variable:
	__u8
	__u16
	__u32
and so on.  NEVER use 'int' or 'long' crossing that boundry, it's not
going to work properly.

I think one of the chapters in LDD3 describes all of this, you might
want to re-read it for the details.

hope this helps,

greg k-h

single, comprehensive kernel data types document?

[John Chludzinski]

Never use stdint.h? Wasn't that the intent of stdint.h ... for kernel 
code? For embedded code?

---John

On 2016-04-15 08:27, Greg KH wrote:
> On Fri, Apr 15, 2016 at 08:04:53AM -0400, Robert P. J. Day wrote:
>> 
>>   is there a single, decent online doc that explains the proper data
>> types (int16_t, int32_t and so on) to use in kernel code?
> 
> First off, never use int16_t and friends, that's not ok :)
> 
> Second, it's simple, use:
> 	u8
> 	u16
> 	u32
> 	u64
> and friends in kernel code (s8, s16, and so on for signed values.)
> 
> 'int' is a return type, and for loops and other things that you know
> will fit in that size.
> 
>> including the relationship with types to be used in code to be
>> exported to user space (include/uapi/linux/types.h)?
> 
> For values that cross the user/kernel boundry, add '__' to the front of
> the variable:
> 	__u8
> 	__u16
> 	__u32
> and so on.  NEVER use 'int' or 'long' crossing that boundry, it's not
> going to work properly.
> 
> I think one of the chapters in LDD3 describes all of this, you might
> want to re-read it for the details.
> 
> hope this helps,
> 
> greg k-h
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

single, comprehensive kernel data types document?

[Silvan Jegen]

Am 2016-04-15 14:37, schrieb John Chludzinski:
> Never use stdint.h? Wasn't that the intent of stdint.h ... for kernel
> code? For embedded code?

I could be wrong about this but the reason for not using it may be that 
there is no standard C library where the Kernel lives. Thus using types 
from stdint.h would be confusing. Also, the string "u16" is much shorter 
than "uint16_t" and thus must surely be faster! :P


Cheers,

Silvan

single, comprehensive kernel data types document?

[Daniel.]

Hi John

stdint.h is part of standard headers (C99?), I think you shouldn't use
standard headers for kernel when
there are such "kernel headers" for same proprose. If the developers
created that specific headers for
kernel, they should have a good reason for it.

I have a pertinent question on this topic too,

I've been using *aways* u8, u16, u32 in kernel code (driver code) and
*aways* __u8, __u16, __u32
for code that goes to both (usualy ioctl definition headers). What is
happening here is that __u8 from
userspace is being "casted" to u8 in driver during an ioctl call, is
that a problem? This is the "right
way to do it", right?

Also, LDD3 stats this:

It's important to remember that these types are Linux specific, and
using them hinders porting software to other Unix flavors. Systems
with recent compilers support the C99-standard types, such as uint8_t
and uint32_t; if portability is a concern, those types can be used in
favor of the Linux-specific variety [1].

So using C99 standard types for userspace headers (ioctl headers for
example) are okay!?

[1] http://www.makelinux.net/ldd3/chp-11-sect-2

2016-04-15 9:37 GMT-03:00 John Chludzinski <john.chludzinski@vivaldi.net>:
> Never use stdint.h? Wasn't that the intent of stdint.h ... for kernel
> code? For embedded code?
>
> ---John
>
> On 2016-04-15 08:27, Greg KH wrote:
>> On Fri, Apr 15, 2016 at 08:04:53AM -0400, Robert P. J. Day wrote:
>>>
>>>   is there a single, decent online doc that explains the proper data
>>> types (int16_t, int32_t and so on) to use in kernel code?
>>
>> First off, never use int16_t and friends, that's not ok :)
>>
>> Second, it's simple, use:
>>       u8
>>       u16
>>       u32
>>       u64
>> and friends in kernel code (s8, s16, and so on for signed values.)
>>
>> 'int' is a return type, and for loops and other things that you know
>> will fit in that size.
>>
>>> including the relationship with types to be used in code to be
>>> exported to user space (include/uapi/linux/types.h)?
>>
>> For values that cross the user/kernel boundry, add '__' to the front of
>> the variable:
>>       __u8
>>       __u16
>>       __u32
>> and so on.  NEVER use 'int' or 'long' crossing that boundry, it's not
>> going to work properly.
>>
>> I think one of the chapters in LDD3 describes all of this, you might
>> want to re-read it for the details.
>>
>> hope this helps,
>>
>> greg k-h
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies



-- 
"Do or do not. There is no try"
  Yoda Master

single, comprehensive kernel data types document?

[Greg KH]

On Fri, Apr 15, 2016 at 08:37:56AM -0400, John Chludzinski wrote:
> Never use stdint.h?

There is no 'stdint.h' in kernel code, sorry, we don't have that header.

thanks,

greg k-h

single, comprehensive kernel data types document?

[Greg KH]

On Fri, Apr 15, 2016 at 10:09:35AM -0300, Daniel. wrote:
> I've been using *aways* u8, u16, u32 in kernel code (driver code) and
> *aways* __u8, __u16, __u32
> for code that goes to both (usualy ioctl definition headers). What is
> happening here is that __u8 from
> userspace is being "casted" to u8 in driver during an ioctl call, is
> that a problem? This is the "right
> way to do it", right?

Yes, that is correct.

> Also, LDD3 stats this:
> 
> It's important to remember that these types are Linux specific, and
> using them hinders porting software to other Unix flavors. Systems
> with recent compilers support the C99-standard types, such as uint8_t
> and uint32_t; if portability is a concern, those types can be used in
> favor of the Linux-specific variety [1].
> 
> So using C99 standard types for userspace headers (ioctl headers for
> example) are okay!?

No, they aren't, you can not used those types for ioctl headers.

You can get away with using it in kernel-only code, but really, why
would you want to?

thanks,

greg k-h

single, comprehensive kernel data types document?

[Nicholas Mc Guire]

On Fri, Apr 15, 2016 at 05:27:24AM -0700, Greg KH wrote:
> On Fri, Apr 15, 2016 at 08:04:53AM -0400, Robert P. J. Day wrote:
> > 
> >   is there a single, decent online doc that explains the proper data
> > types (int16_t, int32_t and so on) to use in kernel code?
> 
> First off, never use int16_t and friends, that's not ok :)
> 
> Second, it's simple, use:
> 	u8
> 	u16
> 	u32
> 	u64
> and friends in kernel code (s8, s16, and so on for signed values.)
>

I don?t think its quite that simple - as an exampe here is a part of the 
variations of chars that exist in the kernel:

Type              : equivalent types (basis 4.0 x86 64)
char              : signed char, __signed__ char, __s8, s8, int8_t
unsigned char     : unsigned char, u_char, __u8, u8, unchar,  u_int8_t, uint8_t
                    __ticket_t,
                    insn_byte_t,
                    kprobe_opcode_t

Now for new unsigned char objects it might be good enough to simply use
u8 but it does depend on the subsystem and if that has some typedef in 
use or not or readability could really suffer. 

It would however make a lot of sense I belive if a list of types that 
should be used and which should no longer be used were available.
Probably a number of the above typedef might actually be deprecated for 
any new code (and char is actually one of the shorter lists - unsigned
short, unsigned int, unsigned long have atleast 25 equivalent typedefs 
each).

So a typedef cleanup would make a lot of sense for new conributors and it
would also help readability as well as make static code analysis easier...

If the above recommendation {u,s}{8,16,32,64} is somehow official policy
for new code, then it maybe should go into Documentation/CodingStyle ?

thx!
hofrat

single, comprehensive kernel data types document?

[Greg KH]

On Fri, Apr 15, 2016 at 03:59:09PM +0000, Nicholas Mc Guire wrote:
> On Fri, Apr 15, 2016 at 05:27:24AM -0700, Greg KH wrote:
> > On Fri, Apr 15, 2016 at 08:04:53AM -0400, Robert P. J. Day wrote:
> > > 
> > >   is there a single, decent online doc that explains the proper data
> > > types (int16_t, int32_t and so on) to use in kernel code?
> > 
> > First off, never use int16_t and friends, that's not ok :)
> > 
> > Second, it's simple, use:
> > 	u8
> > 	u16
> > 	u32
> > 	u64
> > and friends in kernel code (s8, s16, and so on for signed values.)
> >
> 
> I don?t think its quite that simple - as an exampe here is a part of the 
> variations of chars that exist in the kernel:
> 
> Type              : equivalent types (basis 4.0 x86 64)
> char              : signed char, __signed__ char, __s8, s8, int8_t
> unsigned char     : unsigned char, u_char, __u8, u8, unchar,  u_int8_t, uint8_t
>                     __ticket_t,
>                     insn_byte_t,
>                     kprobe_opcode_t

And that's a mess, which is why not all of those are supposed to cross
the user/kernel boundry.

> Now for new unsigned char objects it might be good enough to simply use
> u8 but it does depend on the subsystem and if that has some typedef in 
> use or not or readability could really suffer. 

One might argue that the subsystem should be fixed up to not use such
crazy typedefs :)

> It would however make a lot of sense I belive if a list of types that 
> should be used and which should no longer be used were available.
> Probably a number of the above typedef might actually be deprecated for 
> any new code (and char is actually one of the shorter lists - unsigned
> short, unsigned int, unsigned long have atleast 25 equivalent typedefs 
> each).
> 
> So a typedef cleanup would make a lot of sense for new conributors and it
> would also help readability as well as make static code analysis easier...
> 
> If the above recommendation {u,s}{8,16,32,64} is somehow official policy
> for new code, then it maybe should go into Documentation/CodingStyle ?

The question was about the user/kernel boundry, and for that, you HAVE
to use the "__" types, otherwise it will be broken.

Within the kernel, yes, you can use lots of different types for the same
"real" variable size, but you shouldn't, just use the well-known and
common types "u8" and you will be fine.  Those other ones are there due
to code being brought in from all over the place, that's what happens
with a codebase of 20 million lines at times :)

So, if someone is looking at doing some cleanup patches, I think you
found a nice place to start...

good luck!

greg k-h

single, comprehensive kernel data types document?

[Daniel.]

2016-04-15 10:55 GMT-03:00 Greg KH <greg@kroah.com>:
> On Fri, Apr 15, 2016 at 10:09:35AM -0300, Daniel. wrote:
>> I've been using *aways* u8, u16, u32 in kernel code (driver code) and
>> *aways* __u8, __u16, __u32
>> for code that goes to both (usualy ioctl definition headers). What is
>> happening here is that __u8 from
>> userspace is being "casted" to u8 in driver during an ioctl call, is
>> that a problem? This is the "right
>> way to do it", right?
>
> Yes, that is correct.
>
>> Also, LDD3 stats this:
>>
>> It's important to remember that these types are Linux specific, and
>> using them hinders porting software to other Unix flavors. Systems
>> with recent compilers support the C99-standard types, such as uint8_t
>> and uint32_t; if portability is a concern, those types can be used in
>> favor of the Linux-specific variety [1].
>>
>> So using C99 standard types for userspace headers (ioctl headers for
>> example) are okay!?
>
> No, they aren't, you can not used those types for ioctl headers.
>
> You can get away with using it in kernel-only code, but really, why
> would you want to?
I don't :),

Thanks for the reply
>
> thanks,
>
> greg k-h

Regards,


-- 
"Do or do not. There is no try"
  Yoda Master

single, comprehensive kernel data types document?

[Valdis.Kletnieks at vt.edu]

On Fri, 15 Apr 2016 09:18:32 -0700, Greg KH said:

> Within the kernel, yes, you can use lots of different types for the same
> "real" variable size, but you shouldn't, just use the well-known and
> common types "u8" and you will be fine.  Those other ones are there due
> to code being brought in from all over the place, that's what happens
> with a codebase of 20 million lines at times :)

What's the current feeling on using typedefs to get more type safety?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160415/09ea186b/attachment.bin 

single, comprehensive kernel data types document?

[Greg KH]

On Fri, Apr 15, 2016 at 01:07:19PM -0400, Valdis.Kletnieks at vt.edu wrote:
> On Fri, 15 Apr 2016 09:18:32 -0700, Greg KH said:
> 
> > Within the kernel, yes, you can use lots of different types for the same
> > "real" variable size, but you shouldn't, just use the well-known and
> > common types "u8" and you will be fine.  Those other ones are there due
> > to code being brought in from all over the place, that's what happens
> > with a codebase of 20 million lines at times :)
> 
> What's the current feeling on using typedefs to get more type safety?

typedefs are evil, never use them.  No non-core-kernel developer should
EVER create a new typedef, except for a function pointer signature.

greg k-h

single, comprehensive kernel data types document?

[Rob Groner]

Sorry if this isn't related, it seemed like it was...

I recently discovered one of our drivers isn't written correctly for
64-bit.  It uses a uint32_t to hold an address. Whoops.

In previous drivers when I've needed to hold an address, I've used an
"unsigned long", as (so far as I could tell) that would give me the
correct number of bytes whether on 32 or 64-bit systems.

Now that I have to fix this driver, I'd rather do whatever the
"standard" method is for storing an address value.

Looking at code in the kernel and linux/types.h, I see "phys_addr_t and
dma_addr_t.  Is that what I want to use?  What if it's a virtual
address?  void *?

Thanks.

Rob

single, comprehensive kernel data types document?

[Greg KH]

On Wed, Apr 20, 2016 at 04:37:07PM -0400, Rob Groner wrote:
> Sorry if this isn't related, it seemed like it was...
> 
> I recently discovered one of our drivers isn't written correctly for
> 64-bit.  It uses a uint32_t to hold an address. Whoops.
> 
> In previous drivers when I've needed to hold an address, I've used an
> "unsigned long", as (so far as I could tell) that would give me the
> correct number of bytes whether on 32 or 64-bit systems.
> 
> Now that I have to fix this driver, I'd rather do whatever the
> "standard" method is for storing an address value.
> 
> Looking at code in the kernel and linux/types.h, I see "phys_addr_t and
> dma_addr_t.  Is that what I want to use?  What if it's a virtual
> address?  void *?

You want to use '__u64' and cast properly within the kernel to a
pointer.

hope this helps,

greg k-h

single, comprehensive kernel data types document?

[Rob Groner]

On Thu, 2016-04-21 at 11:51 +0900, Greg KH wrote:
> On Wed, Apr 20, 2016 at 04:37:07PM -0400, Rob Groner wrote:
> > Sorry if this isn't related, it seemed like it was...
> > 
> > I recently discovered one of our drivers isn't written correctly for
> > 64-bit.  It uses a uint32_t to hold an address. Whoops.
> > 
> > In previous drivers when I've needed to hold an address, I've used an
> > "unsigned long", as (so far as I could tell) that would give me the
> > correct number of bytes whether on 32 or 64-bit systems.
> > 
> > Now that I have to fix this driver, I'd rather do whatever the
> > "standard" method is for storing an address value.
> > 
> > Looking at code in the kernel and linux/types.h, I see "phys_addr_t and
> > dma_addr_t.  Is that what I want to use?  What if it's a virtual
> > address?  void *?
> 
> You want to use '__u64' and cast properly within the kernel to a
> pointer.
> 
> hope this helps,
> 
> greg k-h

Thank you Greg.

I was thinking there was a type that would hold a memory address (like
an allocated DMA buffer address, or a PCI address) that would be the
correct size on a 32-bit or 64-bit system without me having to specify a
size.  If I use __u64 to hold a memory address, won't that be the wrong
size on a 32-bit system?

Rob

single, comprehensive kernel data types document?

[Josh Cartwright]

On Thu, Apr 21, 2016 at 08:51:04AM -0400, Rob Groner wrote:
> On Thu, 2016-04-21 at 11:51 +0900, Greg KH wrote:
> > On Wed, Apr 20, 2016 at 04:37:07PM -0400, Rob Groner wrote:
> > > Sorry if this isn't related, it seemed like it was...
> > > 
> > > I recently discovered one of our drivers isn't written correctly for
> > > 64-bit.  It uses a uint32_t to hold an address. Whoops.
> > > 
> > > In previous drivers when I've needed to hold an address, I've used an
> > > "unsigned long", as (so far as I could tell) that would give me the
> > > correct number of bytes whether on 32 or 64-bit systems.
> > > 
> > > Now that I have to fix this driver, I'd rather do whatever the
> > > "standard" method is for storing an address value.
> > > 
> > > Looking at code in the kernel and linux/types.h, I see "phys_addr_t and
> > > dma_addr_t.  Is that what I want to use?  What if it's a virtual
> > > address?  void *?
> > 
> > You want to use '__u64' and cast properly within the kernel to a
> > pointer.
> > 
> > hope this helps,
> > 
> > greg k-h
> 
> Thank you Greg.
> 
> I was thinking there was a type that would hold a memory address (like
> an allocated DMA buffer address, or a PCI address) that would be the
> correct size on a 32-bit or 64-bit system without me having to specify a
> size.  If I use __u64 to hold a memory address, won't that be the wrong
> size on a 32-bit system?

It will be bigger than is strictly necessary in that specific case, but
is that a problem?

Also, please consider the case where a user is running a 32-bit
userspace and a 64-bit kernel.  Not making datastructures which cross
the user/kernel boundaries make use of the largest fixed-width[1] type
means that the kernel would have to handle this case differently (in
fact, you can find many 'compat' examples in the kernel tree which do
just that, because the types were incorrectly used).  And, that's just
terrible :(.

  Josh

1: Just wanted to add it's not just about size of these datatypes,
   either.  It's about their alignment.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160421/24c44cbd/attachment.bin 

single, comprehensive kernel data types document?

[Greg KH]

On Thu, Apr 21, 2016 at 08:51:04AM -0400, Rob Groner wrote:
> On Thu, 2016-04-21 at 11:51 +0900, Greg KH wrote:
> > On Wed, Apr 20, 2016 at 04:37:07PM -0400, Rob Groner wrote:
> > > Sorry if this isn't related, it seemed like it was...
> > > 
> > > I recently discovered one of our drivers isn't written correctly for
> > > 64-bit.  It uses a uint32_t to hold an address. Whoops.
> > > 
> > > In previous drivers when I've needed to hold an address, I've used an
> > > "unsigned long", as (so far as I could tell) that would give me the
> > > correct number of bytes whether on 32 or 64-bit systems.
> > > 
> > > Now that I have to fix this driver, I'd rather do whatever the
> > > "standard" method is for storing an address value.
> > > 
> > > Looking at code in the kernel and linux/types.h, I see "phys_addr_t and
> > > dma_addr_t.  Is that what I want to use?  What if it's a virtual
> > > address?  void *?
> > 
> > You want to use '__u64' and cast properly within the kernel to a
> > pointer.
> > 
> > hope this helps,
> > 
> > greg k-h
> 
> Thank you Greg.
> 
> I was thinking there was a type that would hold a memory address (like
> an allocated DMA buffer address, or a PCI address) that would be the
> correct size on a 32-bit or 64-bit system without me having to specify a
> size.  If I use __u64 to hold a memory address, won't that be the wrong
> size on a 32-bit system?

As Josh says, this is fine, you just "burn" 4 bytes, but you need to do
that anyway for alignment and you save for not having a 'compat ioctl'
callback to do the pointer fixup that would be required if you were to
use a 32bit value for a pointer.

And also, as Josh says, consider mixed user/kernel sizes, that gets
messy very quickly, so just always make pointers 64bits and all works
just fine.

hope this helps,

greg k-h

single, comprehensive kernel data types document?

[Valdis.Kletnieks at vt.edu]

On Fri, 22 Apr 2016 11:37:13 +0900, Greg KH said:

> As Josh says, this is fine, you just "burn" 4 bytes,

Are there stil any platforms left where that matters?  Sure, when I started
coding, an HP2114 had all of 8K of RAM, so wasting 4 bytes mattered.  But I
don't know of anything that will boot Linux and is still so storaged
constrained that we care.  Heck, even my fairly old wireless router has 128M
of RAM on it - and even THERE the 8M of flash is a bigger constraint :)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160421/8c7bdbeb/attachment.bin 

single, comprehensive kernel data types document?

[Greg KH]

On Thu, Apr 21, 2016 at 11:02:32PM -0400, Valdis.Kletnieks at vt.edu wrote:
> On Fri, 22 Apr 2016 11:37:13 +0900, Greg KH said:
> 
> > As Josh says, this is fine, you just "burn" 4 bytes,
> 
> Are there stil any platforms left where that matters?  Sure, when I started
> coding, an HP2114 had all of 8K of RAM, so wasting 4 bytes mattered.  But I
> don't know of anything that will boot Linux and is still so storaged
> constrained that we care.  Heck, even my fairly old wireless router has 128M
> of RAM on it - and even THERE the 8M of flash is a bigger constraint :)

People cram Linux into machines with only 2M of ram, there are tons of
chips out there that are new and shipping in devices with less than 1M
of ram that would be great to have Linux running in them, but can't at
the moment.

So yes, in some systems it does matter, in others, it doesn't but that's
the beauty of Linux, it runs almost everywhere :)

thanks,

greg k-h