PATCH: avoid possible integer overflow with cmp_range() in kernel/range.c

PATCH: avoid possible integer overflow with cmp_range() in kernel/range.c

[Louis Langholtz]

The cmp_range function (in kernel/range.c) is returning the difference between two s64 values (actually coming from u64 typed variables) in an int which can overflow (depending on the size of int). This function is used as a compare function for linux's sort function (in lib/sort.c). Linux's sort function however only cares if the compare function returns a value less than, equal to, or greater than zero.

As sort doesn't need the actual difference, this overflow potential is avoided with the following patch (against linux kernel 3.18 code from Linus's git repo and commit 0541881502a1276149889fe468662ff6a8fc8f6d):

commit 641362d32fef0cfd7b12e1821c1139d75dd23330
Author: Lou Langholtz <lou_langholtz@me.com>
Date:   Mon Nov 24 09:31:01 2014 -0700

    Avoid overflow possibility

diff --git a/kernel/range.c b/kernel/range.c
index 322ea8e..86337e2 100644
--- a/kernel/range.c
+++ b/kernel/range.c
@@ -113,12 +113,17 @@ static int cmp_range(const void *x1, const void *x2)
 {
        const struct range *r1 = x1;
        const struct range *r2 = x2;
-       s64 start1, start2;
+       u64 start1, start2;
 
        start1 = r1->start;
        start2 = r2->start;
 
-       return start1 - start2;
+       /* avoid any overflow possibilities and don't just return start1 - start2 */
+       if (start1 > start2)
+               return 1;
+       if (start2 > start1)
+               return -1;
+       return 0;
 }
 
 int clean_sort_range(struct range *range, int az)

Re: PATCH: avoid possible integer overflow with cmp_range() in kernel/range.c

[Yinghai Lu]

tile should be:

[PATCH] x86: ....

On Tue, Nov 25, 2014 at 11:14 AM, Louis Langholtz <lou_langholtz@me.com> wrote:
> The cmp_range function (in kernel/range.c) is returning the difference between two s64 values (actually coming from u64 typed variables) in an int which can overflow (depending on the size of int). This function is used as a compare function for linux's sort function (in lib/sort.c). Linux's sort function however only cares if the compare function returns a value less than, equal to, or greater than zero.
>
> As sort doesn't need the actual difference, this overflow potential is avoided with the following patch (against linux kernel 3.18 code from Linus's git repo and commit 0541881502a1276149889fe468662ff6a8fc8f6d):
>
> commit 641362d32fef0cfd7b12e1821c1139d75dd23330
> Author: Lou Langholtz <lou_langholtz@me.com>
> Date:   Mon Nov 24 09:31:01 2014 -0700

do not need 6 lines.

>
>     Avoid overflow possibility

You need
Signed-off-by: ....

Please check Documentation/SubmittingPatches for more info.

>
> diff --git a/kernel/range.c b/kernel/range.c
> index 322ea8e..86337e2 100644
> --- a/kernel/range.c
> +++ b/kernel/range.c
> @@ -113,12 +113,17 @@ static int cmp_range(const void *x1, const void *x2)
>  {
>         const struct range *r1 = x1;
>         const struct range *r2 = x2;
> -       s64 start1, start2;
> +       u64 start1, start2;
>
>         start1 = r1->start;
>         start2 = r2->start;
>
> -       return start1 - start2;
> +       /* avoid any overflow possibilities and don't just return start1 - start2 */
> +       if (start1 > start2)
> +               return 1;
> +       if (start2 > start1)
> +               return -1;
> +       return 0;
>  }
>
>  int clean_sort_range(struct range *range, int az)
>