[PATCH v2] v4l2-dv-timings: fix overflow in gtf timings calculation

[PATCH v2] v4l2-dv-timings: fix overflow in gtf timings calculation

[Prashant Laddha]

Please find version v2 for the overflow fix in gtf timing calculation.
v2 incorporates comments from Hans on version 1 of this patch.

Regards,
Prashant

Prashant Laddha (1):
  v4l2-dv-timings: fix overflow in gtf timings calculation

 drivers/media/v4l2-core/v4l2-dv-timings.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

Changes compared to v1:
1. Use div_u64() for 64 bit division
2. Use u64 instead 
-- 
1.9.1

[PATCH v2] v4l2-dv-timings: fix overflow in gtf timings calculation

[Prashant Laddha]

The intermediate calculation in the expression for hblank can exceed
32 bit signed range. This overflow can lead to negative values for
hblank. Typecasting intermediate variable to higher precision.

Cc: Hans Verkuil <hans.verkuil@cisco.com>
Cc: Martin Bugge <marbugge@cisco.com>
Signed-off-by: Prashant Laddha <prladdha@cisco.com>
---
 drivers/media/v4l2-core/v4l2-dv-timings.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-dv-timings.c b/drivers/media/v4l2-core/v4l2-dv-timings.c
index 86e11d1..3e14dc4 100644
--- a/drivers/media/v4l2-core/v4l2-dv-timings.c
+++ b/drivers/media/v4l2-core/v4l2-dv-timings.c
@@ -25,6 +25,7 @@
 #include <linux/videodev2.h>
 #include <linux/v4l2-dv-timings.h>
 #include <media/v4l2-dv-timings.h>
+#include <linux/math64.h>
 
 MODULE_AUTHOR("Hans Verkuil");
 MODULE_DESCRIPTION("V4L2 DV Timings Helper Functions");
@@ -572,16 +573,25 @@ bool v4l2_detect_gtf(unsigned frame_height,
 	image_width = (image_width + GTF_CELL_GRAN/2) & ~(GTF_CELL_GRAN - 1);
 
 	/* Horizontal */
-	if (default_gtf)
-		h_blank = ((image_width * GTF_D_C_PRIME * hfreq) -
-					(image_width * GTF_D_M_PRIME * 1000) +
-			(hfreq * (100 - GTF_D_C_PRIME) + GTF_D_M_PRIME * 1000) / 2) /
-			(hfreq * (100 - GTF_D_C_PRIME) + GTF_D_M_PRIME * 1000);
-	else
-		h_blank = ((image_width * GTF_S_C_PRIME * hfreq) -
-					(image_width * GTF_S_M_PRIME * 1000) +
-			(hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000) / 2) /
-			(hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000);
+	if (default_gtf) {
+		u64 num;
+		u64 den;
+
+		num = ((image_width * GTF_D_C_PRIME * (u64)hfreq) -
+		      ((u64)image_width * GTF_D_M_PRIME * 1000));
+		den = ((u64)hfreq * (100 - GTF_D_C_PRIME) +
+		      GTF_D_M_PRIME * 1000);
+		h_blank = div_u64((num + (den >> 1)), den);
+	} else {
+		u64 num;
+		u64 den;
+
+		num = ((image_width * GTF_S_C_PRIME * (u64)hfreq) -
+		      ((u64)image_width * GTF_S_M_PRIME * 1000));
+		den = ((u64)hfreq * (100 - GTF_S_C_PRIME) +
+		      GTF_S_M_PRIME * 1000);
+		h_blank = div_u64((num + (den >> 1)), den);
+	}
 
 	h_blank = ((h_blank + GTF_CELL_GRAN) / (2 * GTF_CELL_GRAN)) *
 		  (2 * GTF_CELL_GRAN);
-- 
1.9.1