From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33991) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X0TbB-0006X3-1j for qemu-devel@nongnu.org; Fri, 27 Jun 2014 06:41:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X0Tb6-0002YP-2Z for qemu-devel@nongnu.org; Fri, 27 Jun 2014 06:41:40 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:57744 helo=mx01.kamp.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X0Tb5-0002Y3-NZ for qemu-devel@nongnu.org; Fri, 27 Jun 2014 06:41:36 -0400 From: Peter Lieven Date: Fri, 27 Jun 2014 12:41:11 +0200 Message-Id: <1403865671-16238-1-git-send-email-pl@kamp.de> Subject: [Qemu-devel] [PATCH] ui/vnc: avoid memory corruption if width % VNC_DIRTY_PIXELS_PER_BIT != 0 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Peter Lieven , kraxel@redhat.com, qemu-stable@nongnu.org during resolution change in Windows 7 it happens sometimes that Windows changes to an intermediate resolution where server_stride % cmp_bytes != 0 (in vnc_refresh_server_surface). The problem that causes memory corruption is where the guest fb is copied to the server fb. It could be easily fixed by truncating cmp_bytes in vnc_refresh_server_surface. But by looking at the code it seems that none of the encoders called in vnc_send_framebuffer_update really cares about w > pixman_image_get_width(vd->server). This patch will therefore remove all DIV_ROUND_UPs for now to avoid corruption or illegal reads. I think there are really almost no real resultions out there where width % 16 != 0. If we really find some we might need to either decrease VNC_DIRTY_PIXELS_PER_BIT or make it dynamic depending on the resolution. Cc: qemu-stable@nongnu.org Signed-off-by: Peter Lieven --- ui/vnc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ui/vnc.c b/ui/vnc.c index 14a86c3..9e37d47 100644 --- a/ui/vnc.c +++ b/ui/vnc.c @@ -577,7 +577,7 @@ void *vnc_server_fb_ptr(VncDisplay *vd, int x, int y) memset(bitmap, 0x00, sizeof(bitmap));\ for (y = 0; y < h; y++) {\ bitmap_set(bitmap[y], 0,\ - DIV_ROUND_UP(w, VNC_DIRTY_PIXELS_PER_BIT));\ + w / VNC_DIRTY_PIXELS_PER_BIT);\ } \ } @@ -2738,7 +2738,7 @@ static int vnc_refresh_server_surface(VncDisplay *vd) } guest_ptr += x * cmp_bytes; - for (; x < DIV_ROUND_UP(width, VNC_DIRTY_PIXELS_PER_BIT); + for (; x < width / VNC_DIRTY_PIXELS_PER_BIT; x++, guest_ptr += cmp_bytes, server_ptr += cmp_bytes) { if (!test_and_clear_bit(x, vd->guest.dirty[y])) { continue; -- 1.7.9.5