[PATCH] atm: he: fix sign-extension overflow on large shift

[PATCH] atm: he: fix sign-extension overflow on large shift

[Colin King]

From: Colin Ian King <colin.king@canonical.com>

Shifting the 1 by exp by an int can lead to sign-extension overlow when
exp is 31 since 1 is an signed int and sign-extending this result to an
unsigned long long will set the upper 32 bits.  Fix this by shifting an
unsigned long.

Detected by cppcheck:
(warning) Shifting signed 32-bit value by 31 bits is undefined behaviour

Signed-off-by: Colin Ian King <colin.king@canonical.com>
---
 drivers/atm/he.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/atm/he.c b/drivers/atm/he.c
index 2e9d1cfe3aeb..211607986134 100644
--- a/drivers/atm/he.c
+++ b/drivers/atm/he.c
@@ -718,7 +718,7 @@ static int he_init_cs_block_rcm(struct he_dev *he_dev)
 			instead of '/ 512', use '>> 9' to prevent a call
 			to divdu3 on x86 platforms
 		*/
-		rate_cps = (unsigned long long) (1 << exp) * (man + 512) >> 9;
+		rate_cps = (unsigned long long) (1UL << exp) * (man + 512) >> 9;
 
 		if (rate_cps < 10)
 			rate_cps = 10;	/* 2.2.1 minimum payload rate is 10 cps */
-- 
2.19.1

Re: [PATCH] atm: he: fix sign-extension overflow on large shift

[Dan Carpenter]

On Tue, Jan 15, 2019 at 06:03:38PM +0000, Colin King wrote:
> From: Colin Ian King <colin.king@canonical.com>
> 
> Shifting the 1 by exp by an int can lead to sign-extension overlow when
> exp is 31 since 1 is an signed int and sign-extending this result to an
> unsigned long long will set the upper 32 bits.  Fix this by shifting an
> unsigned long.

You could just cast it to unsigned.

	rate_cps = (unsigned long long) (1U << exp) * (man + 512) >> 9;

I don't think it makes a difference at runtime because we just
pick buf = 4 either way...

regards,
dan carpenter

Re: [PATCH] atm: he: fix sign-extension overflow on large shift

[David Miller]

From: Colin King <colin.king@canonical.com>
Date: Tue, 15 Jan 2019 18:03:38 +0000

> From: Colin Ian King <colin.king@canonical.com>
> 
> Shifting the 1 by exp by an int can lead to sign-extension overlow when
> exp is 31 since 1 is an signed int and sign-extending this result to an
> unsigned long long will set the upper 32 bits.  Fix this by shifting an
> unsigned long.
> 
> Detected by cppcheck:
> (warning) Shifting signed 32-bit value by 31 bits is undefined behaviour
> 
> Signed-off-by: Colin Ian King <colin.king@canonical.com>

Applied.