Did some testing with your patches:
Test 1:
Xen-Version: Compiled a xen with the 077 commit, your first patch and your 3. (latest) patch for testing.
Passed-through devices: only vga
Result: At some late point in the boot of the domU, the dom0 (the server itself) restarted itself. This is repeatable. I added the qemu-log and what I was able to capture from the xl dmesg (I only hit 'xl dmesg' numerous times until the server restarted and might not have gotten the end of it, but since there is no xl dmesg log i'm aware off, I couldn't think of another way. If you have any, please tell me). From the qemu logĀ I would guess the restart was caused by some problem in the vram mapping?