drm: Use u64 for intermediate dotclock calculations

We have reached the era where monitor bandwidths now exceed 31bits in
frequency calculations, though as we stored them in kHz units we are
safe from overflow in the modelines for some time.

[   48.723720] UBSAN: Undefined behaviour in ../drivers/gpu/drm/drm_modes.c:325:49
[   48.726943] signed integer overflow:
[   48.728503] 2240 * 1000000 cannot be represented in type 'int'

Reported-by: Martin Liška <marxin.liska@gmail.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=98372
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20161021141540.26837-1-chris@chris-wilson.co.uk

drivers/gpu/drm/drm_modes.c

@@ -165,6 +165,7 @@ struct drm_display_mode *drm_cvt_mode(struct drm_device *dev, int hdisplay,
 	unsigned int vfieldrate, hperiod;
 	int hdisplay_rnd, hmargin, vdisplay_rnd, vmargin, vsync;
 	int interlace;
+	u64 tmp;
 
 	/* allocate the drm_display_mode structure. If failure, we will
 	 * return directly
@@ -322,8 +323,11 @@ struct drm_display_mode *drm_cvt_mode(struct drm_device *dev, int hdisplay,
 		drm_mode->vsync_end = drm_mode->vsync_start + vsync;
 	}
 	/* 15/13. Find pixel clock frequency (kHz for xf86) */
-	drm_mode->clock = drm_mode->htotal * HV_FACTOR * 1000 / hperiod;
-	drm_mode->clock -= drm_mode->clock % CVT_CLOCK_STEP;
+	tmp = drm_mode->htotal; /* perform intermediate calcs in u64 */
+	tmp *= HV_FACTOR * 1000;
+	do_div(tmp, hperiod);
+	tmp -= drm_mode->clock % CVT_CLOCK_STEP;
+	drm_mode->clock = tmp;
 	/* 18/16. Find actual vertical frame frequency */
 	/* ignore - just set the mode flag for interlaced */
 	if (interlaced) {