Re: [PATCH v4 1/2] kernel.h: Introduce const_max() for VLA removal

From: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
To: Kees Cook <keescook@chromium.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	Rasmus Villemoes <linux@rasmusvillemoes.dk>,
	Randy Dunlap <rdunlap@infradead.org>,
	Ingo Molnar <mingo@kernel.org>,
	David Laight <David.Laight@aculab.com>,
	Ian Abbott <abbotti@mev.co.uk>,
	linux-input <linux-input@vger.kernel.org>,
	linux-btrfs <linux-btrfs@vger.kernel.org>,
	Network Development <netdev@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Kernel Hardening <kernel-hardening@lists.openwall.com>
Subject: Re: [PATCH v4 1/2] kernel.h: Introduce const_max() for VLA removal
Date: Fri, 16 Mar 2018 00:17:50 +0100	[thread overview]
Message-ID: <CANiq72m7GEZS=Wkzg5KLkTnzW8vYz8X90OwVs4r5vcCCAp1-Pg@mail.gmail.com> (raw)
In-Reply-To: <CANiq72nEsno3JH=c_Xaf9gn1pJdM=ni6Z0ZUDqcEF07FT+SBSw@mail.gmail.com>

On Fri, Mar 16, 2018 at 12:08 AM, Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
> On Thu, Mar 15, 2018 at 11:58 PM, Miguel Ojeda
> <miguel.ojeda.sandonis@gmail.com> wrote:
>> On Thu, Mar 15, 2018 at 11:46 PM, Kees Cook <keescook@chromium.org> wrote:
>>>
>>> By using this eye-bleed:
>>>
>>> size_t __error_not_const_arg(void) \
>>> __compiletime_error("const_max() used with non-compile-time constant arg");
>>> size_t __error_not_positive_arg(void) \
>>> __compiletime_error("const_max() used with negative arg");
>>> #define const_max(x, y)                                                 \
>>>         __builtin_choose_expr(__builtin_constant_p(x) &&                \
>>>                               __builtin_constant_p(y),                  \
>>>                 __builtin_choose_expr((x) >= 0 && (y) >= 0,             \
>>>                                       (typeof(x))(x) > (typeof(y))(y) ? \
>>>                                         (x) : (y),                      \
>>>                                       __error_not_positive_arg()),      \
>>>                 __error_not_const_arg())
>>>
>>
>> I was writing it like this:
>>
>> #define const_max(a, b) \
>>     ({ \
>>         if ((a) < 0) \
>>             __const_max_called_with_negative_value(); \
>>         if ((b) < 0) \
>>             __const_max_called_with_negative_value(); \
>>         if (!__builtin_types_compatible_p(typeof(a), typeof(b))) \
>>             __const_max_called_with_incompatible_types(); \
>>         __builtin_choose_expr((a) > (b), (a), (b)); \
>> })
>
> The full one, using your naming convention:
>
> #define const_max(x, y)                                          \
>     ({                                                           \
>         if (!__builtin_constant_p(x))                            \
>             __error_not_const_arg();                             \
>         if (!__builtin_constant_p(y))                            \
>             __error_not_const_arg();                             \
>         if (!__builtin_types_compatible_p(typeof(x), typeof(y))) \
>             __error_incompatible_types();                        \
>         if ((x) < 0)                                             \
>             __error_not_positive_arg();                          \
>         if ((y) < 0)                                             \
>             __error_not_positive_arg();                          \
>         __builtin_choose_expr((x) > (y), (x), (y));              \
>     })
>

Nevermind... gcc doesn't take that as a constant expr, even if it
compiles as one at -O0.