[PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[Bart Van Assche]

The purpose of patch "linux/types.h: enable endian checks for all
sparse builds" was to encourage driver authors to annotate
endianness correctly in their drivers. However, since that patch
went upstream no endianness annotations in drivers have been fixed.
I think that this shows that the followed approach does not work,
probably because several driver authors do not use sparse. For
developers who are not the authors of these drivers it would take
a very significant effort to make these drivers endianness clean.
Examples are drivers/scsi/qla2xxx and drivers/infiniband/hw/nes.
Hence restore the ability to disable sparse endianness checks such
that it becomes again easy to review other sparse diagnostics for
people who want to analyze drivers they are not the author of.

References: commit 05de97003c77 ("linux/types.h: enable endian checks for all sparse builds")
Signed-off-by: Bart Van Assche <bart.vanassche-Sjgp3cTcYWE@public.gmane.org>
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: Michael S. Tsirkin <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Cc: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
---
[v2]: Elaborated patch description

 include/uapi/linux/types.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/uapi/linux/types.h b/include/uapi/linux/types.h
index 41e5914f0a8e..d3dcb0764c45 100644
--- a/include/uapi/linux/types.h
+++ b/include/uapi/linux/types.h
@@ -23,7 +23,11 @@
 #else
 #define __bitwise__
 #endif
+#if !defined(__CHECK_ENDIAN__) || __CHECK_ENDIAN__ != 0
 #define __bitwise __bitwise__
+#else
+#define __bitwise
+#endif
 
 typedef __u16 __bitwise __le16;
 typedef __u16 __bitwise __be16;
-- 
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[Bart Van Assche]

The purpose of patch "linux/types.h: enable endian checks for all
sparse builds" was to encourage driver authors to annotate
endianness correctly in their drivers. However, since that patch
went upstream no endianness annotations in drivers have been fixed.
I think that this shows that the followed approach does not work,
probably because several driver authors do not use sparse. For
developers who are not the authors of these drivers it would take
a very significant effort to make these drivers endianness clean.
Examples are drivers/scsi/qla2xxx and drivers/infiniband/hw/nes.
Hence restore the ability to disable sparse endianness checks such
that it becomes again easy to review other sparse diagnostics for
people who want to analyze drivers they are not the author of.

References: commit 05de97003c77 ("linux/types.h: enable endian checks for all sparse builds")
Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Cc: linux-scsi@vger.kernel.org
Cc: linux-rdma@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
---
[v2]: Elaborated patch description

 include/uapi/linux/types.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/uapi/linux/types.h b/include/uapi/linux/types.h
index 41e5914f0a8e..d3dcb0764c45 100644
--- a/include/uapi/linux/types.h
+++ b/include/uapi/linux/types.h
@@ -23,7 +23,11 @@
 #else
 #define __bitwise__
 #endif
+#if !defined(__CHECK_ENDIAN__) || __CHECK_ENDIAN__ != 0
 #define __bitwise __bitwise__
+#else
+#define __bitwise
+#endif
 
 typedef __u16 __bitwise __le16;
 typedef __u16 __bitwise __be16;
-- 
2.14.2

Re: [PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[Michael S. Tsirkin]

On Mon, Oct 16, 2017 at 10:26:33AM -0700, Bart Van Assche wrote:
> The purpose of patch "linux/types.h: enable endian checks for all
> sparse builds" was to encourage driver authors to annotate
> endianness correctly in their drivers. However, since that patch
> went upstream no endianness annotations in drivers have been fixed.

I doubt that's true. What's the basis for this claim?

> I think that this shows that the followed approach does not work,
> probably because several driver authors do not use sparse. For
> developers who are not the authors of these drivers it would take
> a very significant effort to make these drivers endianness clean.

I'm afraid I still don't see it.  For developers endian-ness is really
easy.  Look at hardware spec make sure code matches.  You can often do
without looking at the spec too, if a given field is always used with
cpu_to_le, mark it __le.  If you don't want to change driver code, you
don't really need to run sparse on it.

> Examples are drivers/scsi/qla2xxx and drivers/infiniband/hw/nes.

These seem to be actively maintained. So post a patch, maintainers
can look at the spec to help make sure annotations are right.

> Hence restore the ability to disable sparse endianness checks such
> that it becomes again easy to review other sparse diagnostics for
> people who want to analyze drivers they are not the author of.

What are these diagnostics that are important to analyze for people
unable to make even trivial driver changes? White-listing these
as opposed to black-listing endian-ness might be a better idea.

> References: commit 05de97003c77 ("linux/types.h: enable endian checks for all sparse builds")
> Signed-off-by: Bart Van Assche <bart.vanassche-Sjgp3cTcYWE@public.gmane.org>
> Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: Michael S. Tsirkin <mst-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Cc: Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> Cc: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
> ---
> [v2]: Elaborated patch description
> 
>  include/uapi/linux/types.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/include/uapi/linux/types.h b/include/uapi/linux/types.h
> index 41e5914f0a8e..d3dcb0764c45 100644
> --- a/include/uapi/linux/types.h
> +++ b/include/uapi/linux/types.h
> @@ -23,7 +23,11 @@
>  #else
>  #define __bitwise__
>  #endif
> +#if !defined(__CHECK_ENDIAN__) || __CHECK_ENDIAN__ != 0
>  #define __bitwise __bitwise__
> +#else
> +#define __bitwise
> +#endif
>  
>  typedef __u16 __bitwise __le16;
>  typedef __u16 __bitwise __be16;
> -- 
> 2.14.2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[Michael S. Tsirkin]

On Mon, Oct 16, 2017 at 10:26:33AM -0700, Bart Van Assche wrote:
> The purpose of patch "linux/types.h: enable endian checks for all
> sparse builds" was to encourage driver authors to annotate
> endianness correctly in their drivers. However, since that patch
> went upstream no endianness annotations in drivers have been fixed.

I doubt that's true. What's the basis for this claim?

> I think that this shows that the followed approach does not work,
> probably because several driver authors do not use sparse. For
> developers who are not the authors of these drivers it would take
> a very significant effort to make these drivers endianness clean.

I'm afraid I still don't see it.  For developers endian-ness is really
easy.  Look at hardware spec make sure code matches.  You can often do
without looking at the spec too, if a given field is always used with
cpu_to_le, mark it __le.  If you don't want to change driver code, you
don't really need to run sparse on it.

> Examples are drivers/scsi/qla2xxx and drivers/infiniband/hw/nes.

These seem to be actively maintained. So post a patch, maintainers
can look at the spec to help make sure annotations are right.

> Hence restore the ability to disable sparse endianness checks such
> that it becomes again easy to review other sparse diagnostics for
> people who want to analyze drivers they are not the author of.

What are these diagnostics that are important to analyze for people
unable to make even trivial driver changes? White-listing these
as opposed to black-listing endian-ness might be a better idea.

> References: commit 05de97003c77 ("linux/types.h: enable endian checks for all sparse builds")
> Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
> Cc: linux-scsi@vger.kernel.org
> Cc: linux-rdma@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Christoph Hellwig <hch@infradead.org>
> Cc: Leon Romanovsky <leon@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> ---
> [v2]: Elaborated patch description
> 
>  include/uapi/linux/types.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/include/uapi/linux/types.h b/include/uapi/linux/types.h
> index 41e5914f0a8e..d3dcb0764c45 100644
> --- a/include/uapi/linux/types.h
> +++ b/include/uapi/linux/types.h
> @@ -23,7 +23,11 @@
>  #else
>  #define __bitwise__
>  #endif
> +#if !defined(__CHECK_ENDIAN__) || __CHECK_ENDIAN__ != 0
>  #define __bitwise __bitwise__
> +#else
> +#define __bitwise
> +#endif
>  
>  typedef __u16 __bitwise __le16;
>  typedef __u16 __bitwise __be16;
> -- 
> 2.14.2

Re: [PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[Bart Van Assche]

On Mon, 2017-10-16 at 22:57 +0300, Michael S. Tsirkin wrote:
> On Mon, Oct 16, 2017 at 10:26:33AM -0700, Bart Van Assche wrote:
> > I think that this shows that the followed approach does not work,
> > probably because several driver authors do not use sparse. For
> > developers who are not the authors of these drivers it would take
> > a very significant effort to make these drivers endianness clean.
> 
> I'm afraid I still don't see it.  For developers endian-ness is really
> easy.  Look at hardware spec make sure code matches.  You can often do
> without looking at the spec too, if a given field is always used with
> cpu_to_le, mark it __le.  If you don't want to change driver code, you
> don't really need to run sparse on it.

You seem to assume that all drivers that are not yet endianness clean do
not contain any endianness conversion bugs. I severely doubt that that
assumption is correct. It is likely that it is not possible to make several
kernel drivers endianness clean due to endianness conversion bugs in such
drivers.

> > Examples are drivers/scsi/qla2xxx and drivers/infiniband/hw/nes.
> 
> These seem to be actively maintained. So post a patch, maintainers
> can look at the spec to help make sure annotations are right.

I don't have the time to delve deep in these two and the many other kernel
drivers that are not endianness clean. So please stop telling *me* that *I*
have to fix the endianness annotations in these drivers.

BTW, I think it should be mentioned here that you have tried to fix the
endianness annotations in the qla2xxx driver but once you noticed how 
complicated that task was that you gave up half-way. See also Michael
Tsirkin, [PATCH] scsi/qla2xxx: label endian-ness for many fields, 9 Dec
2016 (https://www.spinics.net/lists/linux-scsi/msg102739.html).

> > Hence restore the ability to disable sparse endianness checks such
> > that it becomes again easy to review other sparse diagnostics for
> > people who want to analyze drivers they are not the author of.
> 
> What are these diagnostics that are important to analyze?

E.g. a spin_unlock() that is missing from an error path. Sparse can detect
such errors.

> White-listing these as opposed to black-listing endian-ness might be a
> better idea.

As explained in a previous e-mail, any approach that suppresses endianness
error messages automatically makes it easier for driver authors to ignore
endianness error messages. This is why I prefer that if endianness error
messages are suppressed that this happens manually. Hence the patch at the
start of this e-mail thread that restores __CHECK_ENDIAN__.

Bart.

Re: [PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

On Mon, Oct 16, 2017 at 09:26:36PM +0000, Bart Van Assche wrote:
> On Mon, 2017-10-16 at 22:57 +0300, Michael S. Tsirkin wrote:
> > On Mon, Oct 16, 2017 at 10:26:33AM -0700, Bart Van Assche wrote:

<...>

>
> > > Examples are drivers/scsi/qla2xxx and drivers/infiniband/hw/nes.
> >
> > These seem to be actively maintained. So post a patch, maintainers
> > can look at the spec to help make sure annotations are right.
>
> I don't have the time to delve deep in these two and the many other kernel
> drivers that are not endianness clean. So please stop telling *me* that *I*
> have to fix the endianness annotations in these drivers.

With all my respect, but authors of drivers/infiniband/hw/nes are not
showing any attempts to make that driver endianness clean. It works for them
and they have no worries about static analyzer tools output, so why Bart
should do it?

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2] linux/types.h: Restore the ability to disable sparse endianness checks

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

On Mon, Oct 16, 2017 at 09:26:36PM +0000, Bart Van Assche wrote:
> On Mon, 2017-10-16 at 22:57 +0300, Michael S. Tsirkin wrote:
> > On Mon, Oct 16, 2017 at 10:26:33AM -0700, Bart Van Assche wrote:

<...>

>
> > > Examples are drivers/scsi/qla2xxx and drivers/infiniband/hw/nes.
> >
> > These seem to be actively maintained. So post a patch, maintainers
> > can look at the spec to help make sure annotations are right.
>
> I don't have the time to delve deep in these two and the many other kernel
> drivers that are not endianness clean. So please stop telling *me* that *I*
> have to fix the endianness annotations in these drivers.

With all my respect, but authors of drivers/infiniband/hw/nes are not
showing any attempts to make that driver endianness clean. It works for them
and they have no worries about static analyzer tools output, so why Bart
should do it?

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]