[Qemu-devel] [Bug 1497479] [NEW] memory corruption with migrate/savevm in TCG mode

[Qemu-devel] [Bug 1497479] [NEW] memory corruption with migrate/savevm in TCG mode

[Pavel Boldin]

Public bug reported:

[ISSUE]

QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
enabling the global dirty pages log and entering the final stage of
saving the VM.

[DESCRIPTION]

The situation is the following:
1. TLB misses is the only way for page dirtying in the TCG mode.
2. If TLB is hit by a running VM during the execution of the `ram_save_iterate' by migration thread (e.g. if VM is mostly idling) then some pages are missing in the dirty log.
3. These pages are then not migrated during `ram_save_complete'.
4. This makes memory content in a saved VM state differ from the actual VM memory.
5. If the affected area includes some Kernel data structures such as trees or lists this can cause Kernel to Oops after loading the saved state.

[SOLUTION]

A proposed solution is to flush TLB when `log_global_start' is called.
Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

[LINKS]

Ubuntu bug: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

** Affects: qemu
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1497479

Title:
  memory corruption with migrate/savevm in TCG mode

Status in QEMU:
  New

Bug description:
  [ISSUE]

  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.

  [DESCRIPTION]

  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
  2. If TLB is hit by a running VM during the execution of the `ram_save_iterate' by migration thread (e.g. if VM is mostly idling) then some pages are missing in the dirty log.
  3. These pages are then not migrated during `ram_save_complete'.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
  5. If the affected area includes some Kernel data structures such as trees or lists this can cause Kernel to Oops after loading the saved state.

  [SOLUTION]

  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

  [LINKS]

  Ubuntu bug:
  https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1497479/+subscriptions

[Qemu-devel] [Bug 1497479] Re: memory corruption with migrate/savevm in TCG mode

[Pavel Boldin]

** Description changed:

  [ISSUE]
  
  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.
  
  [DESCRIPTION]
  
  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
- 2. If TLB is hit by a running VM during the execution of the `ram_save_iterate' by migration thread (e.g. if VM is mostly idling) then some pages are missing in the dirty log.
- 3. These pages are then not migrated during `ram_save_complete'.
+ 2. If TLB is always hit by a running VM code during the execution of the `ram_save_iterate' by migration thread then these pages are missing in the dirty log. The TLB is always hit for instance when the VM is mostly idling and the Kernel only handles APIC timer interrupts.
+ 3. These pages are then missed during `ram_save_complete' stage.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
- 5. If the affected area includes some Kernel data structures such as trees or lists this can cause Kernel to Oops after loading the saved state.
+ 5. If the affected memory pages contain some Kernel data structures these can be corrupted by this memory inconsistency, causing Kernel to Oops after loading the saved state.
  
  [SOLUTION]
  
  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch
  
  [LINKS]
  
  Ubuntu bug: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1497479

Title:
  memory corruption with migrate/savevm in TCG mode

Status in QEMU:
  New

Bug description:
  [ISSUE]

  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.

  [DESCRIPTION]

  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
  2. If TLB is always hit by a running VM code during the execution of the `ram_save_iterate' by migration thread then these pages are missing in the dirty log. The TLB is always hit for instance when the VM is mostly idling and the Kernel only handles APIC timer interrupts.
  3. These pages are then missed during `ram_save_complete' stage.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
  5. If the affected memory pages contain some Kernel data structures these can be corrupted by this memory inconsistency, causing Kernel to Oops after loading the saved state.

  [SOLUTION]

  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

  [LINKS]

  Ubuntu bug:
  https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1497479/+subscriptions

[Qemu-devel] [Bug 1497479] Re: memory corruption with migrate/savevm in TCG mode

[Serge Hallyn]

Hi,

is this a duplicate of 1493049?  (Should they be merged?)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1497479

Title:
  memory corruption with migrate/savevm in TCG mode

Status in QEMU:
  New

Bug description:
  [ISSUE]

  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.

  [DESCRIPTION]

  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
  2. If TLB is always hit by a running VM code during the execution of the `ram_save_iterate' by migration thread then these pages are missing in the dirty log. The TLB is always hit for instance when the VM is mostly idling and the Kernel only handles APIC timer interrupts.
  3. These pages are then missed during `ram_save_complete' stage.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
  5. If the affected memory pages contain some Kernel data structures these can be corrupted by this memory inconsistency, causing Kernel to Oops after loading the saved state.

  [SOLUTION]

  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

  [LINKS]

  Ubuntu bug:
  https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1497479/+subscriptions

[Qemu-devel] [Bug 1497479] Re: memory corruption with migrate/savevm in TCG mode

[Pavel Boldin]

Hi,

This one is for QEMU master, 1493049 is for Ubuntu packages.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1497479

Title:
  memory corruption with migrate/savevm in TCG mode

Status in QEMU:
  New

Bug description:
  [ISSUE]

  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.

  [DESCRIPTION]

  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
  2. If TLB is always hit by a running VM code during the execution of the `ram_save_iterate' by migration thread then these pages are missing in the dirty log. The TLB is always hit for instance when the VM is mostly idling and the Kernel only handles APIC timer interrupts.
  3. These pages are then missed during `ram_save_complete' stage.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
  5. If the affected memory pages contain some Kernel data structures these can be corrupted by this memory inconsistency, causing Kernel to Oops after loading the saved state.

  [SOLUTION]

  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

  [LINKS]

  Ubuntu bug:
  https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1497479/+subscriptions

Re: [Qemu-devel] [Bug 1497479] Re: memory corruption with migrate/savevm in TCG mode

[Serge Hallyn]

Generally combining them is still better - but if it helps you to
keep things straight then no problem, sorry for the noise - thanks.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1497479

Title:
  memory corruption with migrate/savevm in TCG mode

Status in QEMU:
  New

Bug description:
  [ISSUE]

  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.

  [DESCRIPTION]

  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
  2. If TLB is always hit by a running VM code during the execution of the `ram_save_iterate' by migration thread then these pages are missing in the dirty log. The TLB is always hit for instance when the VM is mostly idling and the Kernel only handles APIC timer interrupts.
  3. These pages are then missed during `ram_save_complete' stage.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
  5. If the affected memory pages contain some Kernel data structures these can be corrupted by this memory inconsistency, causing Kernel to Oops after loading the saved state.

  [SOLUTION]

  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

  [LINKS]

  Ubuntu bug:
  https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1497479/+subscriptions

[Qemu-devel] [Bug 1497479] Re: memory corruption with migrate/savevm in TCG mode

[Thomas Huth]

Looking through old bug tickets... can you still reproduce this issue
with the latest version of QEMU? Or could we close this ticket nowadays?
If you still can reproduce the issue, please send your patch to the
qemu-devel mailing list for discussion (we generally do not take patches
from the bugtracker). See https://wiki.qemu.org/Contribute/SubmitAPatch
for details.


** Changed in: qemu
       Status: New => Incomplete

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1497479

Title:
  memory corruption with migrate/savevm in TCG mode

Status in QEMU:
  Incomplete

Bug description:
  [ISSUE]

  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.

  [DESCRIPTION]

  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
  2. If TLB is always hit by a running VM code during the execution of the `ram_save_iterate' by migration thread then these pages are missing in the dirty log. The TLB is always hit for instance when the VM is mostly idling and the Kernel only handles APIC timer interrupts.
  3. These pages are then missed during `ram_save_complete' stage.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
  5. If the affected memory pages contain some Kernel data structures these can be corrupted by this memory inconsistency, causing Kernel to Oops after loading the saved state.

  [SOLUTION]

  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

  [LINKS]

  Ubuntu bug:
  https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1497479/+subscriptions

[Qemu-devel] [Bug 1497479] Re: memory corruption with migrate/savevm in TCG mode

[Launchpad Bug Tracker]

[Expired for QEMU because there has been no activity for 60 days.]

** Changed in: qemu
       Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1497479

Title:
  memory corruption with migrate/savevm in TCG mode

Status in QEMU:
  Expired

Bug description:
  [ISSUE]

  QEMU releases 2.3.1 and lower are forgetting to flush TLBs before
  enabling the global dirty pages log and entering the final stage of
  saving the VM.

  [DESCRIPTION]

  The situation is the following:
  1. TLB misses is the only way for page dirtying in the TCG mode.
  2. If TLB is always hit by a running VM code during the execution of the `ram_save_iterate' by migration thread then these pages are missing in the dirty log. The TLB is always hit for instance when the VM is mostly idling and the Kernel only handles APIC timer interrupts.
  3. These pages are then missed during `ram_save_complete' stage.
  4. This makes memory content in a saved VM state differ from the actual VM memory.
  5. If the affected memory pages contain some Kernel data structures these can be corrupted by this memory inconsistency, causing Kernel to Oops after loading the saved state.

  [SOLUTION]

  A proposed solution is to flush TLB when `log_global_start' is called.
  Here is the patch: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049/+attachment/4459905/+files/tcg-commit-on-log-global-start.patch

  [LINKS]

  Ubuntu bug:
  https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1493049

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1497479/+subscriptions