[PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, Zhen-Hua]

This patchset is an update of Bill Sumner's patchset, implements a fix for:
If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
when a panic happens, the kdump kernel will boot with these faults:

    dmar: DRHD: handling fault status reg 102
    dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
    DMAR:[fault reason 01] Present bit in root entry is clear

    dmar: DRHD: handling fault status reg 2
    dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
    INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear

On some system, the interrupt remapping fault will also happen even if the 
intel_iommu is not set to on, because the interrupt remapping will be enabled 
when x2apic is needed by the system.

The cause of the DMA fault is described in Bill's original version, and the 
INTR-Remap fault is caused by a similar reason. In short, the initialization 
of vt-d drivers causes the in-flight DMA and interrupt requests get wrong 
response.

To fix this problem, we modifies the behaviors of the intel vt-d in the 
crashdump kernel:

For DMA Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the translation, keep it enabled.
3. Use the old root entry table, do not rewrite the RTA register.
4. Malloc and use new context entry table, copy data from the old ones that
   used by the old kernel.
5. Keep using the old page tables before driver is loaded.
6. After device driver is loaded, when it issues the first dma_map command, 
   free the dmar_domain structure for this device, and generate a new one, so 
   that the device can be assigned a new and empty page table. 
7. When a new context entry table is generated, we also save its address to 
   the old root entry table.

For Interrupt Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the interrupt remapping, keep it enabled.
3. Use the old interrupt remapping table, do not rewrite the IRTA register.
4. When ioapic entry is setup, the interrupt remapping table is changed, and 
   the updated data will be stored to the old interrupt remapping table.

Advantages of this approach:
1. All manipulation of the IO-device is done by the Linux device-driver
   for that device.
2. This approach behaves in a manner very similar to operation without an
   active iommu.
3. Any activity between the IO-device and its RMRR areas is handled by the
   device-driver in the same manner as during a non-kdump boot.
4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
   This supports the practice of creating a special kdump kernel without
   drivers for any devices that are not required for taking a crashdump. 
5. Minimal code-changes among the existing mainline intel vt-d code.

Summary of changes in this patch set:
1. Added some useful function for root entry table in code intel-iommu.c
2. Added new members to struct root_entry and struct irte;
3. Functions to load old root entry table to iommu->root_entry from the memory 
   of old kernel.
4. Functions to malloc new context entry table and copy the data from the old
   ones to the malloced new ones.
5. Functions to enable support for DMA remapping in kdump kernel.
6. Functions to load old irte data from the old kernel to the kdump kernel.
7. Some code changes that support other behaviours that have been listed.
8. In the new functions, use physical address as "unsigned long" type, not 
   pointers.

Original version by Bill Sumner:
    https://lkml.org/lkml/2014/1/10/518
    https://lkml.org/lkml/2014/4/15/716
    https://lkml.org/lkml/2014/4/24/836

Zhenhua's updates:
    https://lkml.org/lkml/2014/10/21/134
    https://lkml.org/lkml/2014/12/15/121
    https://lkml.org/lkml/2014/12/22/53
    https://lkml.org/lkml/2015/1/6/1166
    https://lkml.org/lkml/2015/1/12/35

Changelog[v9]:
    1. Add new function iommu_attach_domain_with_id.
    2. Do not copy old page tables, keep using the old ones.
    3. Remove functions:
           intel_iommu_did_to_domain_values_entry
           intel_iommu_get_dids_from_old_kernel
           device_to_domain_id
           copy_page_addr
           copy_page_table
           copy_context_entry
           copy_context_entry_table
    4. Add new function device_to_existing_context_entry.

Changelog[v8]:
    1. Add a missing __iommu_flush_cache in function copy_page_table.

Changelog[v7]:
    1. Use __iommu_flush_cache to flush the data to hardware.

Changelog[v6]:
    1. Use "unsigned long" as type of physical address.
    2. Use new function unmap_device_dma to unmap the old dma.
    3. Some small incorrect bits order for aw shift.

Changelog[v5]:
    1. Do not disable and re-enable traslation and interrupt remapping. 
    2. Use old root entry table.
    3. Use old interrupt remapping table.
    4. New functions to copy data from old kernel, and save to old kernel mem.
    5. New functions to save updated root entry table and irte table.
    6. Use intel_unmap to unmap the old dma;
    7. Allocate new pages while driver is being loaded.

Changelog[v4]:
    1. Cut off the patches that move some defines and functions to new files.
    2. Reduce the numbers of patches to five, make it more easier to read.
    3. Changed the name of functions, make them consistent with current context
       get/set functions.
    4. Add change to function __iommu_attach_domain.

Changelog[v3]:
    1. Commented-out "#define DEBUG 1" to eliminate debug messages.
    2. Updated the comments about changes in each version.
    3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
          struct as recommended by Baoquan He [bhe@redhat.com]
          init_iova_domain(&domain->iovad, DMA_32BIT_PFN);

Changelog[v2]:
    The following series implements a fix for:
    A kdump problem about DMA that has been discussed for a long time. That is,
    when a kernel panics and boots into the kdump kernel, DMA started by the
    panicked kernel is not stopped before the kdump kernel is booted and the
    kdump kernel disables the IOMMU while this DMA continues.  This causes the
    IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
    them as physical memory addresses -- which causes the DMA to either:
        (1) generate DMAR errors or 
        (2) generate PCI SERR errors or 
        (3) transfer data to or from incorrect areas of memory. Often this 
            causes the dump to fail.

Changelog[v1]:
    The original version.

Changed in this version:
1. Do not disable and re-enable traslation and interrupt remapping. 
2. Use old root entry table.
3. Use old interrupt remapping table.
4. Use "unsigned long" as physical address.
5. Use intel_unmap to unmap the old dma;

Baoquan He <bhe@redhat.com> helps testing this patchset.
Takao Indoh <indou.takao@jp.fujitsu.com> gives valuable suggestions.

Li, Zhen-Hua (10):
  iommu/vt-d: New function to attach domain with id
  iommu/vt-d: Items required for kdump
  iommu/vt-d: Function to get old context entry
  iommu/vt-d: functions to copy data from old mem
  iommu/vt-d: Add functions to load and save old re
  iommu/vt-d: datatypes and functions used for kdump
  iommu/vt-d: enable kdump support in iommu module
  iommu/vt-d: assign new page table for dma_map
  iommu/vt-d: Copy functions for irte
  iommu/vt-d: Use old irte in kdump kernel

 drivers/iommu/intel-iommu.c         | 535 ++++++++++++++++++++++++++++++++++--
 drivers/iommu/intel_irq_remapping.c | 105 ++++++-
 include/linux/intel-iommu.h         |  18 ++
 3 files changed, 628 insertions(+), 30 deletions(-)

-- 
2.0.0-rc0

[PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, Zhen-Hua]

This patchset is an update of Bill Sumner's patchset, implements a fix for:
If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
when a panic happens, the kdump kernel will boot with these faults:

    dmar: DRHD: handling fault status reg 102
    dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
    DMAR:[fault reason 01] Present bit in root entry is clear

    dmar: DRHD: handling fault status reg 2
    dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
    INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear

On some system, the interrupt remapping fault will also happen even if the 
intel_iommu is not set to on, because the interrupt remapping will be enabled 
when x2apic is needed by the system.

The cause of the DMA fault is described in Bill's original version, and the 
INTR-Remap fault is caused by a similar reason. In short, the initialization 
of vt-d drivers causes the in-flight DMA and interrupt requests get wrong 
response.

To fix this problem, we modifies the behaviors of the intel vt-d in the 
crashdump kernel:

For DMA Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the translation, keep it enabled.
3. Use the old root entry table, do not rewrite the RTA register.
4. Malloc and use new context entry table, copy data from the old ones that
   used by the old kernel.
5. Keep using the old page tables before driver is loaded.
6. After device driver is loaded, when it issues the first dma_map command, 
   free the dmar_domain structure for this device, and generate a new one, so 
   that the device can be assigned a new and empty page table. 
7. When a new context entry table is generated, we also save its address to 
   the old root entry table.

For Interrupt Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the interrupt remapping, keep it enabled.
3. Use the old interrupt remapping table, do not rewrite the IRTA register.
4. When ioapic entry is setup, the interrupt remapping table is changed, and 
   the updated data will be stored to the old interrupt remapping table.

Advantages of this approach:
1. All manipulation of the IO-device is done by the Linux device-driver
   for that device.
2. This approach behaves in a manner very similar to operation without an
   active iommu.
3. Any activity between the IO-device and its RMRR areas is handled by the
   device-driver in the same manner as during a non-kdump boot.
4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
   This supports the practice of creating a special kdump kernel without
   drivers for any devices that are not required for taking a crashdump. 
5. Minimal code-changes among the existing mainline intel vt-d code.

Summary of changes in this patch set:
1. Added some useful function for root entry table in code intel-iommu.c
2. Added new members to struct root_entry and struct irte;
3. Functions to load old root entry table to iommu->root_entry from the memory 
   of old kernel.
4. Functions to malloc new context entry table and copy the data from the old
   ones to the malloced new ones.
5. Functions to enable support for DMA remapping in kdump kernel.
6. Functions to load old irte data from the old kernel to the kdump kernel.
7. Some code changes that support other behaviours that have been listed.
8. In the new functions, use physical address as "unsigned long" type, not 
   pointers.

Original version by Bill Sumner:
    https://lkml.org/lkml/2014/1/10/518
    https://lkml.org/lkml/2014/4/15/716
    https://lkml.org/lkml/2014/4/24/836

Zhenhua's updates:
    https://lkml.org/lkml/2014/10/21/134
    https://lkml.org/lkml/2014/12/15/121
    https://lkml.org/lkml/2014/12/22/53
    https://lkml.org/lkml/2015/1/6/1166
    https://lkml.org/lkml/2015/1/12/35

Changelog[v9]:
    1. Add new function iommu_attach_domain_with_id.
    2. Do not copy old page tables, keep using the old ones.
    3. Remove functions:
           intel_iommu_did_to_domain_values_entry
           intel_iommu_get_dids_from_old_kernel
           device_to_domain_id
           copy_page_addr
           copy_page_table
           copy_context_entry
           copy_context_entry_table
    4. Add new function device_to_existing_context_entry.

Changelog[v8]:
    1. Add a missing __iommu_flush_cache in function copy_page_table.

Changelog[v7]:
    1. Use __iommu_flush_cache to flush the data to hardware.

Changelog[v6]:
    1. Use "unsigned long" as type of physical address.
    2. Use new function unmap_device_dma to unmap the old dma.
    3. Some small incorrect bits order for aw shift.

Changelog[v5]:
    1. Do not disable and re-enable traslation and interrupt remapping. 
    2. Use old root entry table.
    3. Use old interrupt remapping table.
    4. New functions to copy data from old kernel, and save to old kernel mem.
    5. New functions to save updated root entry table and irte table.
    6. Use intel_unmap to unmap the old dma;
    7. Allocate new pages while driver is being loaded.

Changelog[v4]:
    1. Cut off the patches that move some defines and functions to new files.
    2. Reduce the numbers of patches to five, make it more easier to read.
    3. Changed the name of functions, make them consistent with current context
       get/set functions.
    4. Add change to function __iommu_attach_domain.

Changelog[v3]:
    1. Commented-out "#define DEBUG 1" to eliminate debug messages.
    2. Updated the comments about changes in each version.
    3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
          struct as recommended by Baoquan He [bhe@redhat.com]
          init_iova_domain(&domain->iovad, DMA_32BIT_PFN);

Changelog[v2]:
    The following series implements a fix for:
    A kdump problem about DMA that has been discussed for a long time. That is,
    when a kernel panics and boots into the kdump kernel, DMA started by the
    panicked kernel is not stopped before the kdump kernel is booted and the
    kdump kernel disables the IOMMU while this DMA continues.  This causes the
    IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
    them as physical memory addresses -- which causes the DMA to either:
        (1) generate DMAR errors or 
        (2) generate PCI SERR errors or 
        (3) transfer data to or from incorrect areas of memory. Often this 
            causes the dump to fail.

Changelog[v1]:
    The original version.

Changed in this version:
1. Do not disable and re-enable traslation and interrupt remapping. 
2. Use old root entry table.
3. Use old interrupt remapping table.
4. Use "unsigned long" as physical address.
5. Use intel_unmap to unmap the old dma;

Baoquan He <bhe@redhat.com> helps testing this patchset.
Takao Indoh <indou.takao@jp.fujitsu.com> gives valuable suggestions.

Li, Zhen-Hua (10):
  iommu/vt-d: New function to attach domain with id
  iommu/vt-d: Items required for kdump
  iommu/vt-d: Function to get old context entry
  iommu/vt-d: functions to copy data from old mem
  iommu/vt-d: Add functions to load and save old re
  iommu/vt-d: datatypes and functions used for kdump
  iommu/vt-d: enable kdump support in iommu module
  iommu/vt-d: assign new page table for dma_map
  iommu/vt-d: Copy functions for irte
  iommu/vt-d: Use old irte in kdump kernel

 drivers/iommu/intel-iommu.c         | 535 ++++++++++++++++++++++++++++++++++--
 drivers/iommu/intel_irq_remapping.c | 105 ++++++-
 include/linux/intel-iommu.h         |  18 ++
 3 files changed, 628 insertions(+), 30 deletions(-)

-- 
2.0.0-rc0

[PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, Zhen-Hua]

This patchset is an update of Bill Sumner's patchset, implements a fix for:
If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
when a panic happens, the kdump kernel will boot with these faults:

    dmar: DRHD: handling fault status reg 102
    dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
    DMAR:[fault reason 01] Present bit in root entry is clear

    dmar: DRHD: handling fault status reg 2
    dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
    INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear

On some system, the interrupt remapping fault will also happen even if the 
intel_iommu is not set to on, because the interrupt remapping will be enabled 
when x2apic is needed by the system.

The cause of the DMA fault is described in Bill's original version, and the 
INTR-Remap fault is caused by a similar reason. In short, the initialization 
of vt-d drivers causes the in-flight DMA and interrupt requests get wrong 
response.

To fix this problem, we modifies the behaviors of the intel vt-d in the 
crashdump kernel:

For DMA Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the translation, keep it enabled.
3. Use the old root entry table, do not rewrite the RTA register.
4. Malloc and use new context entry table, copy data from the old ones that
   used by the old kernel.
5. Keep using the old page tables before driver is loaded.
6. After device driver is loaded, when it issues the first dma_map command, 
   free the dmar_domain structure for this device, and generate a new one, so 
   that the device can be assigned a new and empty page table. 
7. When a new context entry table is generated, we also save its address to 
   the old root entry table.

For Interrupt Remapping:
1. To accept the vt-d hardware in an active state,
2. Do not disable and re-enable the interrupt remapping, keep it enabled.
3. Use the old interrupt remapping table, do not rewrite the IRTA register.
4. When ioapic entry is setup, the interrupt remapping table is changed, and 
   the updated data will be stored to the old interrupt remapping table.

Advantages of this approach:
1. All manipulation of the IO-device is done by the Linux device-driver
   for that device.
2. This approach behaves in a manner very similar to operation without an
   active iommu.
3. Any activity between the IO-device and its RMRR areas is handled by the
   device-driver in the same manner as during a non-kdump boot.
4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
   This supports the practice of creating a special kdump kernel without
   drivers for any devices that are not required for taking a crashdump. 
5. Minimal code-changes among the existing mainline intel vt-d code.

Summary of changes in this patch set:
1. Added some useful function for root entry table in code intel-iommu.c
2. Added new members to struct root_entry and struct irte;
3. Functions to load old root entry table to iommu->root_entry from the memory 
   of old kernel.
4. Functions to malloc new context entry table and copy the data from the old
   ones to the malloced new ones.
5. Functions to enable support for DMA remapping in kdump kernel.
6. Functions to load old irte data from the old kernel to the kdump kernel.
7. Some code changes that support other behaviours that have been listed.
8. In the new functions, use physical address as "unsigned long" type, not 
   pointers.

Original version by Bill Sumner:
    https://lkml.org/lkml/2014/1/10/518
    https://lkml.org/lkml/2014/4/15/716
    https://lkml.org/lkml/2014/4/24/836

Zhenhua's updates:
    https://lkml.org/lkml/2014/10/21/134
    https://lkml.org/lkml/2014/12/15/121
    https://lkml.org/lkml/2014/12/22/53
    https://lkml.org/lkml/2015/1/6/1166
    https://lkml.org/lkml/2015/1/12/35

Changelog[v9]:
    1. Add new function iommu_attach_domain_with_id.
    2. Do not copy old page tables, keep using the old ones.
    3. Remove functions:
           intel_iommu_did_to_domain_values_entry
           intel_iommu_get_dids_from_old_kernel
           device_to_domain_id
           copy_page_addr
           copy_page_table
           copy_context_entry
           copy_context_entry_table
    4. Add new function device_to_existing_context_entry.

Changelog[v8]:
    1. Add a missing __iommu_flush_cache in function copy_page_table.

Changelog[v7]:
    1. Use __iommu_flush_cache to flush the data to hardware.

Changelog[v6]:
    1. Use "unsigned long" as type of physical address.
    2. Use new function unmap_device_dma to unmap the old dma.
    3. Some small incorrect bits order for aw shift.

Changelog[v5]:
    1. Do not disable and re-enable traslation and interrupt remapping. 
    2. Use old root entry table.
    3. Use old interrupt remapping table.
    4. New functions to copy data from old kernel, and save to old kernel mem.
    5. New functions to save updated root entry table and irte table.
    6. Use intel_unmap to unmap the old dma;
    7. Allocate new pages while driver is being loaded.

Changelog[v4]:
    1. Cut off the patches that move some defines and functions to new files.
    2. Reduce the numbers of patches to five, make it more easier to read.
    3. Changed the name of functions, make them consistent with current context
       get/set functions.
    4. Add change to function __iommu_attach_domain.

Changelog[v3]:
    1. Commented-out "#define DEBUG 1" to eliminate debug messages.
    2. Updated the comments about changes in each version.
    3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
          struct as recommended by Baoquan He [bhe@redhat.com]
          init_iova_domain(&domain->iovad, DMA_32BIT_PFN);

Changelog[v2]:
    The following series implements a fix for:
    A kdump problem about DMA that has been discussed for a long time. That is,
    when a kernel panics and boots into the kdump kernel, DMA started by the
    panicked kernel is not stopped before the kdump kernel is booted and the
    kdump kernel disables the IOMMU while this DMA continues.  This causes the
    IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
    them as physical memory addresses -- which causes the DMA to either:
        (1) generate DMAR errors or 
        (2) generate PCI SERR errors or 
        (3) transfer data to or from incorrect areas of memory. Often this 
            causes the dump to fail.

Changelog[v1]:
    The original version.

Changed in this version:
1. Do not disable and re-enable traslation and interrupt remapping. 
2. Use old root entry table.
3. Use old interrupt remapping table.
4. Use "unsigned long" as physical address.
5. Use intel_unmap to unmap the old dma;

Baoquan He <bhe@redhat.com> helps testing this patchset.
Takao Indoh <indou.takao@jp.fujitsu.com> gives valuable suggestions.

Li, Zhen-Hua (10):
  iommu/vt-d: New function to attach domain with id
  iommu/vt-d: Items required for kdump
  iommu/vt-d: Function to get old context entry
  iommu/vt-d: functions to copy data from old mem
  iommu/vt-d: Add functions to load and save old re
  iommu/vt-d: datatypes and functions used for kdump
  iommu/vt-d: enable kdump support in iommu module
  iommu/vt-d: assign new page table for dma_map
  iommu/vt-d: Copy functions for irte
  iommu/vt-d: Use old irte in kdump kernel

 drivers/iommu/intel-iommu.c         | 535 ++++++++++++++++++++++++++++++++++--
 drivers/iommu/intel_irq_remapping.c | 105 ++++++-
 include/linux/intel-iommu.h         |  18 ++
 3 files changed, 628 insertions(+), 30 deletions(-)

-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 01/10] iommu/vt-d: New function to attach domain with id

[Li, Zhen-Hua]

Allow specification of the domain-id for the new domain.
This patch only adds a new function iommu_attach_domain_with_id, it is like
the function iommu_attach_domain(), only adding a parameter "did".

Bill Sumner:
    (In older versions) Add new 'did' parameter to iommu_attach_domain();
    The caller of this function.

Li, Zhenhua:
    New function iommu_attach_domain_with_id(), instead of updating funtion
    iommu_attach_domain();

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index ae4c1a8..76674a1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1558,6 +1558,16 @@ static int iommu_attach_domain(struct dmar_domain *domain,
 	return num;
 }
 
+static int iommu_attach_domain_with_id(struct dmar_domain *domain,
+			       struct intel_iommu *iommu,
+			       int domain_number)
+{
+	if (domain_number >= 0)
+		return domain_number;
+
+	return iommu_attach_domain(domain, iommu);
+}
+
 static int iommu_attach_vm_domain(struct dmar_domain *domain,
 				  struct intel_iommu *iommu)
 {
@@ -2225,6 +2235,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	u16 dma_alias;
 	unsigned long flags;
 	u8 bus, devfn;
+	int did = -1;   /* Default to "no domain_id supplied" */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2258,7 +2269,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
-	domain->id = iommu_attach_domain(domain, iommu);
+	domain->id = iommu_attach_domain_with_id(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
 		return NULL;
-- 
2.0.0-rc0

[PATCH v9 01/10] iommu/vt-d: New function to attach domain with id

[Li, Zhen-Hua]

Allow specification of the domain-id for the new domain.
This patch only adds a new function iommu_attach_domain_with_id, it is like
the function iommu_attach_domain(), only adding a parameter "did".

Bill Sumner:
    (In older versions) Add new 'did' parameter to iommu_attach_domain();
    The caller of this function.

Li, Zhenhua:
    New function iommu_attach_domain_with_id(), instead of updating funtion
    iommu_attach_domain();

Signed-off-by: Bill Sumner <billsumnerlinux-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index ae4c1a8..76674a1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1558,6 +1558,16 @@ static int iommu_attach_domain(struct dmar_domain *domain,
 	return num;
 }
 
+static int iommu_attach_domain_with_id(struct dmar_domain *domain,
+			       struct intel_iommu *iommu,
+			       int domain_number)
+{
+	if (domain_number >= 0)
+		return domain_number;
+
+	return iommu_attach_domain(domain, iommu);
+}
+
 static int iommu_attach_vm_domain(struct dmar_domain *domain,
 				  struct intel_iommu *iommu)
 {
@@ -2225,6 +2235,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	u16 dma_alias;
 	unsigned long flags;
 	u8 bus, devfn;
+	int did = -1;   /* Default to "no domain_id supplied" */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2258,7 +2269,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
-	domain->id = iommu_attach_domain(domain, iommu);
+	domain->id = iommu_attach_domain_with_id(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
 		return NULL;
-- 
2.0.0-rc0

[PATCH v9 01/10] iommu/vt-d: New function to attach domain with id

[Li, Zhen-Hua]

Allow specification of the domain-id for the new domain.
This patch only adds a new function iommu_attach_domain_with_id, it is like
the function iommu_attach_domain(), only adding a parameter "did".

Bill Sumner:
    (In older versions) Add new 'did' parameter to iommu_attach_domain();
    The caller of this function.

Li, Zhenhua:
    New function iommu_attach_domain_with_id(), instead of updating funtion
    iommu_attach_domain();

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index ae4c1a8..76674a1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1558,6 +1558,16 @@ static int iommu_attach_domain(struct dmar_domain *domain,
 	return num;
 }
 
+static int iommu_attach_domain_with_id(struct dmar_domain *domain,
+			       struct intel_iommu *iommu,
+			       int domain_number)
+{
+	if (domain_number >= 0)
+		return domain_number;
+
+	return iommu_attach_domain(domain, iommu);
+}
+
 static int iommu_attach_vm_domain(struct dmar_domain *domain,
 				  struct intel_iommu *iommu)
 {
@@ -2225,6 +2235,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	u16 dma_alias;
 	unsigned long flags;
 	u8 bus, devfn;
+	int did = -1;   /* Default to "no domain_id supplied" */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2258,7 +2269,7 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
-	domain->id = iommu_attach_domain(domain, iommu);
+	domain->id = iommu_attach_domain_with_id(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
 		return NULL;
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 02/10] iommu/vt-d: Items required for kdump

[Li, Zhen-Hua]

Add context entry functions needed for kdump.

Bill Sumner:
    Original version;

Li, Zhenhua:
    Changed the name of new functions, make them consistent with current
    context get/set functions.
    Remove the structure dve which is not used in new version.

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 56 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 76674a1..577d5de 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
 #include <linux/pci-ats.h>
 #include <linux/memblock.h>
 #include <linux/dma-contiguous.h>
+#include <linux/crash_dump.h>
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -211,6 +212,12 @@ get_context_addr_from_root(struct root_entry *root)
 		NULL);
 }
 
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+	return  root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
 /*
  * low 64 bits:
  * 0: present
@@ -231,6 +238,32 @@ static inline bool context_present(struct context_entry *context)
 {
 	return (context->lo & 1);
 }
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+	return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+	return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+	return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+	return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+	return((c->hi >> 8) & 0xffff);
+}
+
 static inline void context_set_present(struct context_entry *context)
 {
 	context->lo |= 1;
@@ -316,6 +349,29 @@ static inline int first_pte_in_page(struct dma_pte *pte)
 	return !((unsigned long)pte & ~VTD_PAGE_MASK);
 }
 
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating.
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Allocate pages for new context entry, copy data from old context entries
+ *    in the old kernel to the new ones.
+ *
+ */
+
+
+#endif /* CONFIG_CRASH_DUMP */
+
 /*
  * This domain is a statically identity mapping domain.
  *	1. This domain creats a static 1:1 mapping to all usable memory.
-- 
2.0.0-rc0

[PATCH v9 02/10] iommu/vt-d: Items required for kdump

[Li, Zhen-Hua]

Add context entry functions needed for kdump.

Bill Sumner:
    Original version;

Li, Zhenhua:
    Changed the name of new functions, make them consistent with current
    context get/set functions.
    Remove the structure dve which is not used in new version.

Signed-off-by: Bill Sumner <billsumnerlinux-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 56 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 76674a1..577d5de 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
 #include <linux/pci-ats.h>
 #include <linux/memblock.h>
 #include <linux/dma-contiguous.h>
+#include <linux/crash_dump.h>
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -211,6 +212,12 @@ get_context_addr_from_root(struct root_entry *root)
 		NULL);
 }
 
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+	return  root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
 /*
  * low 64 bits:
  * 0: present
@@ -231,6 +238,32 @@ static inline bool context_present(struct context_entry *context)
 {
 	return (context->lo & 1);
 }
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+	return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+	return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+	return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+	return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+	return((c->hi >> 8) & 0xffff);
+}
+
 static inline void context_set_present(struct context_entry *context)
 {
 	context->lo |= 1;
@@ -316,6 +349,29 @@ static inline int first_pte_in_page(struct dma_pte *pte)
 	return !((unsigned long)pte & ~VTD_PAGE_MASK);
 }
 
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating.
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Allocate pages for new context entry, copy data from old context entries
+ *    in the old kernel to the new ones.
+ *
+ */
+
+
+#endif /* CONFIG_CRASH_DUMP */
+
 /*
  * This domain is a statically identity mapping domain.
  *	1. This domain creats a static 1:1 mapping to all usable memory.
-- 
2.0.0-rc0

[PATCH v9 02/10] iommu/vt-d: Items required for kdump

[Li, Zhen-Hua]

Add context entry functions needed for kdump.

Bill Sumner:
    Original version;

Li, Zhenhua:
    Changed the name of new functions, make them consistent with current
    context get/set functions.
    Remove the structure dve which is not used in new version.

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 56 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 76674a1..577d5de 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
 #include <linux/pci-ats.h>
 #include <linux/memblock.h>
 #include <linux/dma-contiguous.h>
+#include <linux/crash_dump.h>
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -211,6 +212,12 @@ get_context_addr_from_root(struct root_entry *root)
 		NULL);
 }
 
+static inline unsigned long
+get_context_phys_from_root(struct root_entry *root)
+{
+	return  root_present(root) ? (root->val & VTD_PAGE_MASK) : 0;
+}
+
 /*
  * low 64 bits:
  * 0: present
@@ -231,6 +238,32 @@ static inline bool context_present(struct context_entry *context)
 {
 	return (context->lo & 1);
 }
+
+static inline int context_fault_enable(struct context_entry *c)
+{
+	return((c->lo >> 1) & 0x1);
+}
+
+static inline int context_translation_type(struct context_entry *c)
+{
+	return((c->lo >> 2) & 0x3);
+}
+
+static inline u64 context_address_root(struct context_entry *c)
+{
+	return((c->lo >> VTD_PAGE_SHIFT));
+}
+
+static inline int context_address_width(struct context_entry *c)
+{
+	return((c->hi >> 0) & 0x7);
+}
+
+static inline int context_domain_id(struct context_entry *c)
+{
+	return((c->hi >> 8) & 0xffff);
+}
+
 static inline void context_set_present(struct context_entry *context)
 {
 	context->lo |= 1;
@@ -316,6 +349,29 @@ static inline int first_pte_in_page(struct dma_pte *pte)
 	return !((unsigned long)pte & ~VTD_PAGE_MASK);
 }
 
+
+#ifdef CONFIG_CRASH_DUMP
+
+/*
+ * Fix Crashdump failure caused by leftover DMA through a hardware IOMMU
+ *
+ * Fixes the crashdump kernel to deal with an active iommu and legacy
+ * DMA from the (old) panicked kernel in a manner similar to how legacy
+ * DMA is handled when no hardware iommu was in use by the old kernel --
+ * allow the legacy DMA to continue into its current buffers.
+ *
+ * In the crashdump kernel, this code:
+ * 1. skips disabling the IOMMU's translating.
+ * 2. Do not re-enable IOMMU's translating.
+ * 3. In kdump kernel, use the old root entry table.
+ * 4. Allocate pages for new context entry, copy data from old context entries
+ *    in the old kernel to the new ones.
+ *
+ */
+
+
+#endif /* CONFIG_CRASH_DUMP */
+
 /*
  * This domain is a statically identity mapping domain.
  *	1. This domain creats a static 1:1 mapping to all usable memory.
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 03/10] iommu/vt-d: Function to get old context entry

[Li, Zhen-Hua]

Interface for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 577d5de..f7dbe70 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -369,6 +369,10 @@ static inline int first_pte_in_page(struct dma_pte *pte)
  *
  */
 
+static struct context_entry *device_to_existing_context_entry(
+				struct intel_iommu *iommu,
+				u8 bus, u8 devfn);
+
 
 #endif /* CONFIG_CRASH_DUMP */
 
@@ -4796,3 +4800,26 @@ static void __init check_tylersburg_isoch(void)
 	printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is 16; your BIOS set %d\n",
 	       vtisochctrl);
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static struct context_entry *device_to_existing_context_entry(
+				struct intel_iommu *iommu,
+				u8 bus, u8 devfn)
+{
+	struct root_entry *root;
+	struct context_entry *context;
+	struct context_entry *ret;
+	unsigned long flags;
+
+	ret = NULL;
+	spin_lock_irqsave(&iommu->lock, flags);
+	root = &iommu->root_entry[bus];
+	context = get_context_addr_from_root(root);
+	if (context && context_present(context+devfn))
+		ret = &context[devfn];
+	spin_unlock_irqrestore(&iommu->lock, flags);
+	return ret;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0

[PATCH v9 03/10] iommu/vt-d: Function to get old context entry

[Li, Zhen-Hua]

Interface for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.

Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 577d5de..f7dbe70 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -369,6 +369,10 @@ static inline int first_pte_in_page(struct dma_pte *pte)
  *
  */
 
+static struct context_entry *device_to_existing_context_entry(
+				struct intel_iommu *iommu,
+				u8 bus, u8 devfn);
+
 
 #endif /* CONFIG_CRASH_DUMP */
 
@@ -4796,3 +4800,26 @@ static void __init check_tylersburg_isoch(void)
 	printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is 16; your BIOS set %d\n",
 	       vtisochctrl);
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static struct context_entry *device_to_existing_context_entry(
+				struct intel_iommu *iommu,
+				u8 bus, u8 devfn)
+{
+	struct root_entry *root;
+	struct context_entry *context;
+	struct context_entry *ret;
+	unsigned long flags;
+
+	ret = NULL;
+	spin_lock_irqsave(&iommu->lock, flags);
+	root = &iommu->root_entry[bus];
+	context = get_context_addr_from_root(root);
+	if (context && context_present(context+devfn))
+		ret = &context[devfn];
+	spin_unlock_irqrestore(&iommu->lock, flags);
+	return ret;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0

[PATCH v9 03/10] iommu/vt-d: Function to get old context entry

[Li, Zhen-Hua]

Interface for when a new domain in the crashdump kernel needs some
values from the panicked kernel's context entries.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 577d5de..f7dbe70 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -369,6 +369,10 @@ static inline int first_pte_in_page(struct dma_pte *pte)
  *
  */
 
+static struct context_entry *device_to_existing_context_entry(
+				struct intel_iommu *iommu,
+				u8 bus, u8 devfn);
+
 
 #endif /* CONFIG_CRASH_DUMP */
 
@@ -4796,3 +4800,26 @@ static void __init check_tylersburg_isoch(void)
 	printk(KERN_WARNING "DMAR: Recommended TLB entries for ISOCH unit is 16; your BIOS set %d\n",
 	       vtisochctrl);
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static struct context_entry *device_to_existing_context_entry(
+				struct intel_iommu *iommu,
+				u8 bus, u8 devfn)
+{
+	struct root_entry *root;
+	struct context_entry *context;
+	struct context_entry *ret;
+	unsigned long flags;
+
+	ret = NULL;
+	spin_lock_irqsave(&iommu->lock, flags);
+	root = &iommu->root_entry[bus];
+	context = get_context_addr_from_root(root);
+	if (context && context_present(context+devfn))
+		ret = &context[devfn];
+	spin_unlock_irqrestore(&iommu->lock, flags);
+	return ret;
+}
+
+#endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 04/10] iommu/vt-d: functions to copy data from old mem

[Li, Zhen-Hua]

Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.

To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Check if pfn is ram:
        if (page_is_ram(pfn))

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
Signed-off-by: Takao Indoh <indou.takao@jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 102 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   9 ++++
 2 files changed, 111 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f7dbe70..7f3484a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -374,6 +374,17 @@ static struct context_entry *device_to_existing_context_entry(
 				u8 bus, u8 devfn);
 
 
+/*
+ * A structure used to store the address allocated by ioremap();
+ * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
+ */
+struct iommu_remapped_entry {
+	struct list_head list;
+	void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4822,4 +4833,95 @@ static struct context_entry *device_to_existing_context_entry(
 	return ret;
 }
 
+/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = from >> VTD_PAGE_SHIFT;
+	offset = from & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+	} else{
+
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)from, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(to, virt_mem, size);
+
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = to >> VTD_PAGE_SHIFT;
+	offset = to & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+	} else{
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)to, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(virt_mem, from, size);
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+	struct iommu_remapped_entry *mem_entry, *tmp;
+
+	mutex_lock(&__iommu_mem_list_lock);
+	list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+		iounmap(mem_entry->mem);
+		list_del(&mem_entry->list);
+		kfree(mem_entry);
+	}
+	mutex_unlock(&__iommu_mem_list_lock);
+	return 0;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
 #include <linux/iova.h>
 #include <linux/io.h>
 #include <linux/dma_remapping.h>
+#include <linux/crash_dump.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
 
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
 
 extern const struct attribute_group *intel_iommu_groups[];
 
+#ifdef CONFIG_CRASH_DUMP
+extern int __iommu_load_from_oldmem(void *to, unsigned long from,
+					unsigned long size);
+extern int __iommu_save_to_oldmem(unsigned long to, void *from,
+					unsigned long size);
+extern int __iommu_free_mapped_mem(void);
+#endif /* CONFIG_CRASH_DUMP */
+
 #endif
-- 
2.0.0-rc0

[PATCH v9 04/10] iommu/vt-d: functions to copy data from old mem

[Li, Zhen-Hua]

Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.

To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Check if pfn is ram:
        if (page_is_ram(pfn))

Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
Signed-off-by: Takao Indoh <indou.takao-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 102 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   9 ++++
 2 files changed, 111 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f7dbe70..7f3484a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -374,6 +374,17 @@ static struct context_entry *device_to_existing_context_entry(
 				u8 bus, u8 devfn);
 
 
+/*
+ * A structure used to store the address allocated by ioremap();
+ * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
+ */
+struct iommu_remapped_entry {
+	struct list_head list;
+	void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4822,4 +4833,95 @@ static struct context_entry *device_to_existing_context_entry(
 	return ret;
 }
 
+/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = from >> VTD_PAGE_SHIFT;
+	offset = from & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+	} else{
+
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)from, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(to, virt_mem, size);
+
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = to >> VTD_PAGE_SHIFT;
+	offset = to & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+	} else{
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)to, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(virt_mem, from, size);
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+	struct iommu_remapped_entry *mem_entry, *tmp;
+
+	mutex_lock(&__iommu_mem_list_lock);
+	list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+		iounmap(mem_entry->mem);
+		list_del(&mem_entry->list);
+		kfree(mem_entry);
+	}
+	mutex_unlock(&__iommu_mem_list_lock);
+	return 0;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
 #include <linux/iova.h>
 #include <linux/io.h>
 #include <linux/dma_remapping.h>
+#include <linux/crash_dump.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
 
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
 
 extern const struct attribute_group *intel_iommu_groups[];
 
+#ifdef CONFIG_CRASH_DUMP
+extern int __iommu_load_from_oldmem(void *to, unsigned long from,
+					unsigned long size);
+extern int __iommu_save_to_oldmem(unsigned long to, void *from,
+					unsigned long size);
+extern int __iommu_free_mapped_mem(void);
+#endif /* CONFIG_CRASH_DUMP */
+
 #endif
-- 
2.0.0-rc0

[PATCH v9 04/10] iommu/vt-d: functions to copy data from old mem

[Li, Zhen-Hua]

Add some functions to copy the data from old kernel.
These functions are used to copy context tables and page tables.

To avoid calling iounmap between spin_lock_irqsave and spin_unlock_irqrestore,
use a link here, store the pointers , and then use iounmap to free them in
another place.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Check if pfn is ram:
        if (page_is_ram(pfn))

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
Signed-off-by: Takao Indoh <indou.takao@jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 102 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   9 ++++
 2 files changed, 111 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f7dbe70..7f3484a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -374,6 +374,17 @@ static struct context_entry *device_to_existing_context_entry(
 				u8 bus, u8 devfn);
 
 
+/*
+ * A structure used to store the address allocated by ioremap();
+ * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
+ */
+struct iommu_remapped_entry {
+	struct list_head list;
+	void __iomem *mem;
+};
+static LIST_HEAD(__iommu_remapped_mem);
+static DEFINE_MUTEX(__iommu_mem_list_lock);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4822,4 +4833,95 @@ static struct context_entry *device_to_existing_context_entry(
 	return ret;
 }
 
+/*
+ * Copy memory from a physically-addressed area into a virtually-addressed area
+ */
+int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = from >> VTD_PAGE_SHIFT;
+	offset = from & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
+	} else{
+
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)from, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(to, virt_mem, size);
+
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Copy memory from a virtually-addressed area into a physically-addressed area
+ */
+int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
+{
+	unsigned long pfn;		/* Page Frame Number */
+	size_t csize = (size_t)size;	/* Num(bytes to copy) */
+	unsigned long offset;		/* Lower 12 bits of to */
+	void __iomem *virt_mem;
+	struct iommu_remapped_entry *mapped;
+
+	pfn = to >> VTD_PAGE_SHIFT;
+	offset = to & (~VTD_PAGE_MASK);
+
+	if (page_is_ram(pfn)) {
+		memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
+	} else{
+		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
+				GFP_KERNEL);
+		if (!mapped)
+			return -ENOMEM;
+
+		virt_mem = ioremap_cache((unsigned long)to, size);
+		if (!virt_mem) {
+			kfree(mapped);
+			return -ENOMEM;
+		}
+		memcpy(virt_mem, from, size);
+		mutex_lock(&__iommu_mem_list_lock);
+		mapped->mem = virt_mem;
+		list_add_tail(&mapped->list, &__iommu_remapped_mem);
+		mutex_unlock(&__iommu_mem_list_lock);
+	}
+	return size;
+}
+
+/*
+ * Free the mapped memory for ioremap;
+ */
+int __iommu_free_mapped_mem(void)
+{
+	struct iommu_remapped_entry *mem_entry, *tmp;
+
+	mutex_lock(&__iommu_mem_list_lock);
+	list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
+		iounmap(mem_entry->mem);
+		list_del(&mem_entry->list);
+		kfree(mem_entry);
+	}
+	mutex_unlock(&__iommu_mem_list_lock);
+	return 0;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a65208a..8ffa523 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -26,6 +26,7 @@
 #include <linux/iova.h>
 #include <linux/io.h>
 #include <linux/dma_remapping.h>
+#include <linux/crash_dump.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
 
@@ -368,4 +369,12 @@ extern int dmar_ir_support(void);
 
 extern const struct attribute_group *intel_iommu_groups[];
 
+#ifdef CONFIG_CRASH_DUMP
+extern int __iommu_load_from_oldmem(void *to, unsigned long from,
+					unsigned long size);
+extern int __iommu_save_to_oldmem(unsigned long to, void *from,
+					unsigned long size);
+extern int __iommu_free_mapped_mem(void);
+#endif /* CONFIG_CRASH_DUMP */
+
 #endif
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 05/10] iommu/vt-d: Add functions to load and save old re

[Li, Zhen-Hua]

Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.

We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Add __iommu_flush_cache.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
Signed-off-by: Takao Indoh <indou.takao@jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  5 +++++
 2 files changed, 57 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 7f3484a..1cb9780 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -373,6 +373,9 @@ static struct context_entry *device_to_existing_context_entry(
 				struct intel_iommu *iommu,
 				u8 bus, u8 devfn);
 
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index);
 
 /*
  * A structure used to store the address allocated by ioremap();
@@ -4924,4 +4927,53 @@ int __iommu_free_mapped_mem(void)
 	return 0;
 }
 
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+	memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+	u8 start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+
+	if (index < -1 || index >= ROOT_ENTRY_NR)
+		return;
+
+	if (index == -1) {
+		start = 0;
+		size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+	} else {
+		start = index * sizeof(struct root_entry);
+		size = sizeof(struct root_entry);
+	}
+	to = iommu->root_entry_old_virt;
+	from = iommu->root_entry;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
 	spinlock_t	lock; /* protect context, domain ids */
 	struct root_entry *root_entry; /* virtual address */
 
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *root_entry_old_virt; /* mapped from old root entry */
+	unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
 	struct iommu_flush flush;
 #endif
 	struct q_inval  *qi;            /* Queued invalidation info */
-- 
2.0.0-rc0

[PATCH v9 05/10] iommu/vt-d: Add functions to load and save old re

[Li, Zhen-Hua]

Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.

We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Add __iommu_flush_cache.

Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
Signed-off-by: Takao Indoh <indou.takao-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  5 +++++
 2 files changed, 57 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 7f3484a..1cb9780 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -373,6 +373,9 @@ static struct context_entry *device_to_existing_context_entry(
 				struct intel_iommu *iommu,
 				u8 bus, u8 devfn);
 
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index);
 
 /*
  * A structure used to store the address allocated by ioremap();
@@ -4924,4 +4927,53 @@ int __iommu_free_mapped_mem(void)
 	return 0;
 }
 
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+	memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+	u8 start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+
+	if (index < -1 || index >= ROOT_ENTRY_NR)
+		return;
+
+	if (index == -1) {
+		start = 0;
+		size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+	} else {
+		start = index * sizeof(struct root_entry);
+		size = sizeof(struct root_entry);
+	}
+	to = iommu->root_entry_old_virt;
+	from = iommu->root_entry;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
 	spinlock_t	lock; /* protect context, domain ids */
 	struct root_entry *root_entry; /* virtual address */
 
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *root_entry_old_virt; /* mapped from old root entry */
+	unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
 	struct iommu_flush flush;
 #endif
 	struct q_inval  *qi;            /* Queued invalidation info */
-- 
2.0.0-rc0

[PATCH v9 05/10] iommu/vt-d: Add functions to load and save old re

[Li, Zhen-Hua]

Add functions to load root entry table from old kernel, and to save updated
root entry table.
Add two member in struct intel_iommu, to store the RTA in old kernel, and
the mapped virt address of it.

We use the old RTA in dump kernel, and when the iommu->root_entry is used as
a cache in kdump kernel, its phys address will not be save to RTA register,
but when its data is changed, we will save the new data to old root entry table.

Li, Zhen-hua:
    The functions and logics.

Takao Indoh:
    Add __iommu_flush_cache.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
Signed-off-by: Takao Indoh <indou.takao@jp.fujitsu.com>
---
 drivers/iommu/intel-iommu.c | 52 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  5 +++++
 2 files changed, 57 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 7f3484a..1cb9780 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -373,6 +373,9 @@ static struct context_entry *device_to_existing_context_entry(
 				struct intel_iommu *iommu,
 				u8 bus, u8 devfn);
 
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
+
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index);
 
 /*
  * A structure used to store the address allocated by ioremap();
@@ -4924,4 +4927,53 @@ int __iommu_free_mapped_mem(void)
 	return 0;
 }
 
+/*
+ * Load the old root entry table to new root entry table.
+ */
+static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+	memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
+
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+}
+
+/*
+ * When the data in new root entry table is changed, this function
+ * must be called to save the updated data to old root entry table.
+ */
+static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
+{
+	u8 start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->root_entry)
+		|| (!iommu->root_entry_old_virt)
+		|| (!iommu->root_entry_old_phys))
+		return;
+
+	if (index < -1 || index >= ROOT_ENTRY_NR)
+		return;
+
+	if (index == -1) {
+		start = 0;
+		size = ROOT_ENTRY_NR * sizeof(struct root_entry);
+	} else {
+		start = index * sizeof(struct root_entry);
+		size = sizeof(struct root_entry);
+	}
+	to = iommu->root_entry_old_virt;
+	from = iommu->root_entry;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8ffa523..8e29b97 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -329,6 +329,11 @@ struct intel_iommu {
 	spinlock_t	lock; /* protect context, domain ids */
 	struct root_entry *root_entry; /* virtual address */
 
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *root_entry_old_virt; /* mapped from old root entry */
+	unsigned long root_entry_old_phys; /* root entry in old kernel */
+#endif
+
 	struct iommu_flush flush;
 #endif
 	struct q_inval  *qi;            /* Queued invalidation info */
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 06/10] iommu/vt-d: datatypes and functions used for kdump

[Li, Zhen-Hua]

Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.

Functions:
    Use old root entry table, and load the old data to root_entry as cache.
    Malloc new context table and copy old context table to the new one.

Bill Sumner:
    Original version, the creation of the data types and functions.

Li, Zhenhua:
    Update the caller of context_get_* and context_put*, use context_*
    and context_set_* for replacement.
    Update the name of the function that loads root entry table.
    Use new function to copy old context entry tables and page tables.
    Use "unsigned long" for physical address.
    Remove the functions to copy page table in Bill's version.
    Remove usage of dve and ppap in Bill's version.

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 113 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1cb9780..44f3369 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,18 @@ struct iommu_remapped_entry {
 static LIST_HEAD(__iommu_remapped_mem);
 static DEFINE_MUTEX(__iommu_mem_list_lock);
 
+/* ========================================================================
+ * Copy iommu translation tables from old kernel into new  kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ * ------------------------------------------------------------------------
+ */
+
+static int copy_root_entry_table(struct intel_iommu *iommu);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd);
+
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4976,4 +4988,105 @@ static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
 	__iommu_flush_cache(iommu, to + start, size);
 }
 
+/*
+ * Load root entry tables from old kernel.
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu)
+{
+	u32 bus;				/* Index: root-entry-table */
+	struct root_entry  *re;			/* Virt(iterator: new table) */
+	unsigned long context_old_phys;		/* Phys(context table entry) */
+	struct context_entry *context_new_virt;	/* Virt(new context_entry) */
+
+	/*
+	 * A new root entry table has been allocated ,
+	 * we need copy re from old kernel to the new allocated one.
+	 */
+
+	if (!iommu->root_entry_old_phys)
+		return -ENOMEM;
+
+	for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
+		if (!root_present(re))
+			continue;
+
+		context_old_phys = get_context_phys_from_root(re);
+
+		if (!context_old_phys)
+			continue;
+
+		context_new_virt =
+			(struct context_entry *)alloc_pgtable_page(iommu->node);
+
+		if (!context_new_virt)
+			return -ENOMEM;
+
+		__iommu_load_from_oldmem(context_new_virt,
+					context_old_phys,
+					VTD_PAGE_SIZE);
+
+		__iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
+
+		set_root_value(re, virt_to_phys(context_new_virt));
+	}
+
+	return 0;
+}
+
+/*
+ * Interface to the "load translation tables" set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
+{
+	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
+	unsigned long long q;		/* quadword scratch */
+	int ret = 0;			/* Integer return code */
+	unsigned long flags;
+
+	iommu = drhd->iommu;
+	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
+	if (!q)
+		return -1;
+
+	spin_lock_irqsave(&iommu->lock, flags);
+
+	/* Load the root-entry table from the old kernel
+	 * foreach context_entry_table in root_entry
+	 *   Copy each entry table from old kernel
+	 */
+	if (!iommu->root_entry) {
+		iommu->root_entry =
+			(struct root_entry *)alloc_pgtable_page(iommu->node);
+		if (!iommu->root_entry) {
+			spin_unlock_irqrestore(&iommu->lock, flags);
+			return -ENOMEM;
+		}
+	}
+
+	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
+	if (!iommu->root_entry_old_phys) {
+		pr_err("Could not read old root entry address.");
+		return -1;
+	}
+
+	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
+						VTD_PAGE_SIZE);
+	if (!iommu->root_entry_old_virt) {
+		pr_err("Could not map the old root entry.");
+		return -ENOMEM;
+	}
+
+	__iommu_load_old_root_entry(iommu);
+	ret = copy_root_entry_table(iommu);
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+	__iommu_update_old_root_entry(iommu, -1);
+
+	spin_unlock_irqrestore(&iommu->lock, flags);
+
+	__iommu_free_mapped_mem();
+
+	return ret;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0

[PATCH v9 06/10] iommu/vt-d: datatypes and functions used for kdump

[Li, Zhen-Hua]

Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.

Functions:
    Use old root entry table, and load the old data to root_entry as cache.
    Malloc new context table and copy old context table to the new one.

Bill Sumner:
    Original version, the creation of the data types and functions.

Li, Zhenhua:
    Update the caller of context_get_* and context_put*, use context_*
    and context_set_* for replacement.
    Update the name of the function that loads root entry table.
    Use new function to copy old context entry tables and page tables.
    Use "unsigned long" for physical address.
    Remove the functions to copy page table in Bill's version.
    Remove usage of dve and ppap in Bill's version.

Signed-off-by: Bill Sumner <billsumnerlinux-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 113 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1cb9780..44f3369 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,18 @@ struct iommu_remapped_entry {
 static LIST_HEAD(__iommu_remapped_mem);
 static DEFINE_MUTEX(__iommu_mem_list_lock);
 
+/* ========================================================================
+ * Copy iommu translation tables from old kernel into new  kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ * ------------------------------------------------------------------------
+ */
+
+static int copy_root_entry_table(struct intel_iommu *iommu);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd);
+
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4976,4 +4988,105 @@ static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
 	__iommu_flush_cache(iommu, to + start, size);
 }
 
+/*
+ * Load root entry tables from old kernel.
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu)
+{
+	u32 bus;				/* Index: root-entry-table */
+	struct root_entry  *re;			/* Virt(iterator: new table) */
+	unsigned long context_old_phys;		/* Phys(context table entry) */
+	struct context_entry *context_new_virt;	/* Virt(new context_entry) */
+
+	/*
+	 * A new root entry table has been allocated ,
+	 * we need copy re from old kernel to the new allocated one.
+	 */
+
+	if (!iommu->root_entry_old_phys)
+		return -ENOMEM;
+
+	for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
+		if (!root_present(re))
+			continue;
+
+		context_old_phys = get_context_phys_from_root(re);
+
+		if (!context_old_phys)
+			continue;
+
+		context_new_virt =
+			(struct context_entry *)alloc_pgtable_page(iommu->node);
+
+		if (!context_new_virt)
+			return -ENOMEM;
+
+		__iommu_load_from_oldmem(context_new_virt,
+					context_old_phys,
+					VTD_PAGE_SIZE);
+
+		__iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
+
+		set_root_value(re, virt_to_phys(context_new_virt));
+	}
+
+	return 0;
+}
+
+/*
+ * Interface to the "load translation tables" set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
+{
+	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
+	unsigned long long q;		/* quadword scratch */
+	int ret = 0;			/* Integer return code */
+	unsigned long flags;
+
+	iommu = drhd->iommu;
+	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
+	if (!q)
+		return -1;
+
+	spin_lock_irqsave(&iommu->lock, flags);
+
+	/* Load the root-entry table from the old kernel
+	 * foreach context_entry_table in root_entry
+	 *   Copy each entry table from old kernel
+	 */
+	if (!iommu->root_entry) {
+		iommu->root_entry =
+			(struct root_entry *)alloc_pgtable_page(iommu->node);
+		if (!iommu->root_entry) {
+			spin_unlock_irqrestore(&iommu->lock, flags);
+			return -ENOMEM;
+		}
+	}
+
+	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
+	if (!iommu->root_entry_old_phys) {
+		pr_err("Could not read old root entry address.");
+		return -1;
+	}
+
+	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
+						VTD_PAGE_SIZE);
+	if (!iommu->root_entry_old_virt) {
+		pr_err("Could not map the old root entry.");
+		return -ENOMEM;
+	}
+
+	__iommu_load_old_root_entry(iommu);
+	ret = copy_root_entry_table(iommu);
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+	__iommu_update_old_root_entry(iommu, -1);
+
+	spin_unlock_irqrestore(&iommu->lock, flags);
+
+	__iommu_free_mapped_mem();
+
+	return ret;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0

[PATCH v9 06/10] iommu/vt-d: datatypes and functions used for kdump

[Li, Zhen-Hua]

Populate it with support functions to copy iommu translation tables from
from the panicked kernel into the kdump kernel in the event of a crash.

Functions:
    Use old root entry table, and load the old data to root_entry as cache.
    Malloc new context table and copy old context table to the new one.

Bill Sumner:
    Original version, the creation of the data types and functions.

Li, Zhenhua:
    Update the caller of context_get_* and context_put*, use context_*
    and context_set_* for replacement.
    Update the name of the function that loads root entry table.
    Use new function to copy old context entry tables and page tables.
    Use "unsigned long" for physical address.
    Remove the functions to copy page table in Bill's version.
    Remove usage of dve and ppap in Bill's version.

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 113 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1cb9780..44f3369 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -388,6 +388,18 @@ struct iommu_remapped_entry {
 static LIST_HEAD(__iommu_remapped_mem);
 static DEFINE_MUTEX(__iommu_mem_list_lock);
 
+/* ========================================================================
+ * Copy iommu translation tables from old kernel into new  kernel.
+ * Entry to this set of functions is: intel_iommu_load_translation_tables()
+ * ------------------------------------------------------------------------
+ */
+
+static int copy_root_entry_table(struct intel_iommu *iommu);
+
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd);
+
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev);
+
 #endif /* CONFIG_CRASH_DUMP */
 
 /*
@@ -4976,4 +4988,105 @@ static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
 	__iommu_flush_cache(iommu, to + start, size);
 }
 
+/*
+ * Load root entry tables from old kernel.
+ */
+static int copy_root_entry_table(struct intel_iommu *iommu)
+{
+	u32 bus;				/* Index: root-entry-table */
+	struct root_entry  *re;			/* Virt(iterator: new table) */
+	unsigned long context_old_phys;		/* Phys(context table entry) */
+	struct context_entry *context_new_virt;	/* Virt(new context_entry) */
+
+	/*
+	 * A new root entry table has been allocated ,
+	 * we need copy re from old kernel to the new allocated one.
+	 */
+
+	if (!iommu->root_entry_old_phys)
+		return -ENOMEM;
+
+	for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
+		if (!root_present(re))
+			continue;
+
+		context_old_phys = get_context_phys_from_root(re);
+
+		if (!context_old_phys)
+			continue;
+
+		context_new_virt =
+			(struct context_entry *)alloc_pgtable_page(iommu->node);
+
+		if (!context_new_virt)
+			return -ENOMEM;
+
+		__iommu_load_from_oldmem(context_new_virt,
+					context_old_phys,
+					VTD_PAGE_SIZE);
+
+		__iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
+
+		set_root_value(re, virt_to_phys(context_new_virt));
+	}
+
+	return 0;
+}
+
+/*
+ * Interface to the "load translation tables" set of functions
+ * from mainline code.
+ */
+static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
+{
+	struct intel_iommu *iommu;	/* Virt(iommu hardware registers) */
+	unsigned long long q;		/* quadword scratch */
+	int ret = 0;			/* Integer return code */
+	unsigned long flags;
+
+	iommu = drhd->iommu;
+	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
+	if (!q)
+		return -1;
+
+	spin_lock_irqsave(&iommu->lock, flags);
+
+	/* Load the root-entry table from the old kernel
+	 * foreach context_entry_table in root_entry
+	 *   Copy each entry table from old kernel
+	 */
+	if (!iommu->root_entry) {
+		iommu->root_entry =
+			(struct root_entry *)alloc_pgtable_page(iommu->node);
+		if (!iommu->root_entry) {
+			spin_unlock_irqrestore(&iommu->lock, flags);
+			return -ENOMEM;
+		}
+	}
+
+	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
+	if (!iommu->root_entry_old_phys) {
+		pr_err("Could not read old root entry address.");
+		return -1;
+	}
+
+	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
+						VTD_PAGE_SIZE);
+	if (!iommu->root_entry_old_virt) {
+		pr_err("Could not map the old root entry.");
+		return -ENOMEM;
+	}
+
+	__iommu_load_old_root_entry(iommu);
+	ret = copy_root_entry_table(iommu);
+	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
+	__iommu_update_old_root_entry(iommu, -1);
+
+	spin_unlock_irqrestore(&iommu->lock, flags);
+
+	__iommu_free_mapped_mem();
+
+	return ret;
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module

[Li, Zhen-Hua]

Modify the operation of the following functions when called during crash dump:
    device_to_context_entry
    free_context_table
    get_domain_for_dev
    init_dmars
    intel_iommu_init

Bill Sumner:
    Original version.

Zhenhua:
    The name of new calling functions.
    Do not disable and re-enable TE in kdump kernel.
    Use the did and gaw from old context entry;

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 118 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 103 insertions(+), 15 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 44f3369..312f06b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -841,6 +841,11 @@ static struct context_entry * device_to_context_entry(struct intel_iommu *iommu,
 		set_root_value(root, phy_addr);
 		set_root_present(root);
 		__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel())
+			__iommu_update_old_root_entry(iommu, bus);
+#endif
 	}
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	return &context[devfn];
@@ -892,7 +897,8 @@ static void free_context_table(struct intel_iommu *iommu)
 
 	spin_lock_irqsave(&iommu->lock, flags);
 	if (!iommu->root_entry) {
-		goto out;
+		spin_unlock_irqrestore(&iommu->lock, flags);
+		return;
 	}
 	for (i = 0; i < ROOT_ENTRY_NR; i++) {
 		root = &iommu->root_entry[i];
@@ -900,10 +906,23 @@ static void free_context_table(struct intel_iommu *iommu)
 		if (context)
 			free_pgtable_page(context);
 	}
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		iommu->root_entry_old_phys = 0;
+		root = iommu->root_entry_old_virt;
+		iommu->root_entry_old_virt = NULL;
+	}
+#endif
 	free_pgtable_page(iommu->root_entry);
 	iommu->root_entry = NULL;
-out:
+
 	spin_unlock_irqrestore(&iommu->lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		iounmap(root);
+#endif
 }
 
 static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2322,6 +2341,9 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	unsigned long flags;
 	u8 bus, devfn;
 	int did = -1;   /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+	struct context_entry *ce = NULL;
+#endif /* CONFIG_CRASH_DUMP */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2355,6 +2377,22 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		/*
+		 * if this device had a did in the old kernel
+		 * use its values instead of generating new ones
+		 */
+		ce = device_to_existing_context_entry(iommu, bus, devfn);
+
+		if (ce) {
+			did = context_domain_id(ce);
+			gaw = agaw_to_width(context_address_width(ce));
+		}
+	}
+#endif /* CONFIG_CRASH_DUMP */
+
 	domain->id = iommu_attach_domain_with_id(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
@@ -2889,14 +2927,33 @@ static int __init init_dmars(void)
 		if (ret)
 			goto free_iommu;
 
-		/*
-		 * TBD:
-		 * we could share the same root & context tables
-		 * among all IOMMU's. Need to Split it later.
-		 */
-		ret = iommu_alloc_root_entry(iommu);
-		if (ret)
-			goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			pr_info("IOMMU Copying translate tables from panicked kernel\n");
+			ret = intel_iommu_load_translation_tables(drhd);
+			if (ret) {
+				pr_err("IOMMU: Copy translate tables failed\n");
+
+				/* Best to stop trying */
+				goto free_iommu;
+			}
+			pr_info("IOMMU: root_cache:0x%12.12llx phys:0x%12.12llx\n",
+				(u64)iommu->root_entry,
+				(u64)iommu->root_entry_old_phys);
+		} else {
+#endif /* CONFIG_CRASH_DUMP */
+			/*
+			 * TBD:
+			 * we could share the same root & context tables
+			 * among all IOMMU's. Need to Split it later.
+			 */
+			ret = iommu_alloc_root_entry(iommu);
+			if (ret)
+				goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
+
 		if (!ecap_pass_through(iommu->ecap))
 			hw_pass_through = 0;
 	}
@@ -2913,6 +2970,16 @@ static int __init init_dmars(void)
 
 	check_tylersburg_isoch();
 
+#ifdef CONFIG_CRASH_DUMP
+	/*
+	 * In the crashdump kernel: Skip setting-up new domains for
+	 * si, rmrr, and the isa bus on the expectation that these
+	 * translations were copied from the old kernel.
+	 */
+	if (is_kdump_kernel())
+		goto skip_new_domains_for_si_rmrr_isa;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * If pass through is not set or not enabled, setup context entries for
 	 * identity mappings for rmrr, gfx, and isa and may fall back to static
@@ -2953,6 +3020,10 @@ static int __init init_dmars(void)
 
 	iommu_prepare_isa();
 
+#ifdef CONFIG_CRASH_DUMP
+skip_new_domains_for_si_rmrr_isa:;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * for each drhd
 	 *   enable fault log
@@ -2981,7 +3052,15 @@ static int __init init_dmars(void)
 
 		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
 		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
-		iommu_enable_translation(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			if (!(iommu->gcmd & DMA_GCMD_TE))
+				iommu_enable_translation(iommu);
+		} else
+#endif
+			iommu_enable_translation(iommu);
+
 		iommu_disable_protect_mem_regions(iommu);
 	}
 
@@ -4266,12 +4345,21 @@ int __init intel_iommu_init(void)
 		goto out_free_dmar;
 	}
 
+#ifdef CONFIG_CRASH_DUMP
 	/*
-	 * Disable translation if already enabled prior to OS handover.
+	 * If (This is the crash kernel)
+	 *    Skip disabling the iommu hardware translations
 	 */
-	for_each_active_iommu(iommu, drhd)
-		if (iommu->gcmd & DMA_GCMD_TE)
-			iommu_disable_translation(iommu);
+	if (is_kdump_kernel()) {
+		pr_info("IOMMU Skip disabling iommu hardware translations\n");
+	} else
+#endif /* CONFIG_CRASH_DUMP */
+		/*
+		 * Disable translation if already enabled prior to OS handover.
+		 */
+		for_each_active_iommu(iommu, drhd)
+			if (iommu->gcmd & DMA_GCMD_TE)
+				iommu_disable_translation(iommu);
 
 	if (dmar_dev_scope_init() < 0) {
 		if (force_on)
-- 
2.0.0-rc0

[PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module

[Li, Zhen-Hua]

Modify the operation of the following functions when called during crash dump:
    device_to_context_entry
    free_context_table
    get_domain_for_dev
    init_dmars
    intel_iommu_init

Bill Sumner:
    Original version.

Zhenhua:
    The name of new calling functions.
    Do not disable and re-enable TE in kdump kernel.
    Use the did and gaw from old context entry;

Signed-off-by: Bill Sumner <billsumnerlinux-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 118 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 103 insertions(+), 15 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 44f3369..312f06b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -841,6 +841,11 @@ static struct context_entry * device_to_context_entry(struct intel_iommu *iommu,
 		set_root_value(root, phy_addr);
 		set_root_present(root);
 		__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel())
+			__iommu_update_old_root_entry(iommu, bus);
+#endif
 	}
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	return &context[devfn];
@@ -892,7 +897,8 @@ static void free_context_table(struct intel_iommu *iommu)
 
 	spin_lock_irqsave(&iommu->lock, flags);
 	if (!iommu->root_entry) {
-		goto out;
+		spin_unlock_irqrestore(&iommu->lock, flags);
+		return;
 	}
 	for (i = 0; i < ROOT_ENTRY_NR; i++) {
 		root = &iommu->root_entry[i];
@@ -900,10 +906,23 @@ static void free_context_table(struct intel_iommu *iommu)
 		if (context)
 			free_pgtable_page(context);
 	}
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		iommu->root_entry_old_phys = 0;
+		root = iommu->root_entry_old_virt;
+		iommu->root_entry_old_virt = NULL;
+	}
+#endif
 	free_pgtable_page(iommu->root_entry);
 	iommu->root_entry = NULL;
-out:
+
 	spin_unlock_irqrestore(&iommu->lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		iounmap(root);
+#endif
 }
 
 static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2322,6 +2341,9 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	unsigned long flags;
 	u8 bus, devfn;
 	int did = -1;   /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+	struct context_entry *ce = NULL;
+#endif /* CONFIG_CRASH_DUMP */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2355,6 +2377,22 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		/*
+		 * if this device had a did in the old kernel
+		 * use its values instead of generating new ones
+		 */
+		ce = device_to_existing_context_entry(iommu, bus, devfn);
+
+		if (ce) {
+			did = context_domain_id(ce);
+			gaw = agaw_to_width(context_address_width(ce));
+		}
+	}
+#endif /* CONFIG_CRASH_DUMP */
+
 	domain->id = iommu_attach_domain_with_id(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
@@ -2889,14 +2927,33 @@ static int __init init_dmars(void)
 		if (ret)
 			goto free_iommu;
 
-		/*
-		 * TBD:
-		 * we could share the same root & context tables
-		 * among all IOMMU's. Need to Split it later.
-		 */
-		ret = iommu_alloc_root_entry(iommu);
-		if (ret)
-			goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			pr_info("IOMMU Copying translate tables from panicked kernel\n");
+			ret = intel_iommu_load_translation_tables(drhd);
+			if (ret) {
+				pr_err("IOMMU: Copy translate tables failed\n");
+
+				/* Best to stop trying */
+				goto free_iommu;
+			}
+			pr_info("IOMMU: root_cache:0x%12.12llx phys:0x%12.12llx\n",
+				(u64)iommu->root_entry,
+				(u64)iommu->root_entry_old_phys);
+		} else {
+#endif /* CONFIG_CRASH_DUMP */
+			/*
+			 * TBD:
+			 * we could share the same root & context tables
+			 * among all IOMMU's. Need to Split it later.
+			 */
+			ret = iommu_alloc_root_entry(iommu);
+			if (ret)
+				goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
+
 		if (!ecap_pass_through(iommu->ecap))
 			hw_pass_through = 0;
 	}
@@ -2913,6 +2970,16 @@ static int __init init_dmars(void)
 
 	check_tylersburg_isoch();
 
+#ifdef CONFIG_CRASH_DUMP
+	/*
+	 * In the crashdump kernel: Skip setting-up new domains for
+	 * si, rmrr, and the isa bus on the expectation that these
+	 * translations were copied from the old kernel.
+	 */
+	if (is_kdump_kernel())
+		goto skip_new_domains_for_si_rmrr_isa;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * If pass through is not set or not enabled, setup context entries for
 	 * identity mappings for rmrr, gfx, and isa and may fall back to static
@@ -2953,6 +3020,10 @@ static int __init init_dmars(void)
 
 	iommu_prepare_isa();
 
+#ifdef CONFIG_CRASH_DUMP
+skip_new_domains_for_si_rmrr_isa:;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * for each drhd
 	 *   enable fault log
@@ -2981,7 +3052,15 @@ static int __init init_dmars(void)
 
 		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
 		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
-		iommu_enable_translation(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			if (!(iommu->gcmd & DMA_GCMD_TE))
+				iommu_enable_translation(iommu);
+		} else
+#endif
+			iommu_enable_translation(iommu);
+
 		iommu_disable_protect_mem_regions(iommu);
 	}
 
@@ -4266,12 +4345,21 @@ int __init intel_iommu_init(void)
 		goto out_free_dmar;
 	}
 
+#ifdef CONFIG_CRASH_DUMP
 	/*
-	 * Disable translation if already enabled prior to OS handover.
+	 * If (This is the crash kernel)
+	 *    Skip disabling the iommu hardware translations
 	 */
-	for_each_active_iommu(iommu, drhd)
-		if (iommu->gcmd & DMA_GCMD_TE)
-			iommu_disable_translation(iommu);
+	if (is_kdump_kernel()) {
+		pr_info("IOMMU Skip disabling iommu hardware translations\n");
+	} else
+#endif /* CONFIG_CRASH_DUMP */
+		/*
+		 * Disable translation if already enabled prior to OS handover.
+		 */
+		for_each_active_iommu(iommu, drhd)
+			if (iommu->gcmd & DMA_GCMD_TE)
+				iommu_disable_translation(iommu);
 
 	if (dmar_dev_scope_init() < 0) {
 		if (force_on)
-- 
2.0.0-rc0

[PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module

[Li, Zhen-Hua]

Modify the operation of the following functions when called during crash dump:
    device_to_context_entry
    free_context_table
    get_domain_for_dev
    init_dmars
    intel_iommu_init

Bill Sumner:
    Original version.

Zhenhua:
    The name of new calling functions.
    Do not disable and re-enable TE in kdump kernel.
    Use the did and gaw from old context entry;

Signed-off-by: Bill Sumner <billsumnerlinux@gmail.com>
Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 118 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 103 insertions(+), 15 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 44f3369..312f06b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -841,6 +841,11 @@ static struct context_entry * device_to_context_entry(struct intel_iommu *iommu,
 		set_root_value(root, phy_addr);
 		set_root_present(root);
 		__iommu_flush_cache(iommu, root, sizeof(*root));
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel())
+			__iommu_update_old_root_entry(iommu, bus);
+#endif
 	}
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	return &context[devfn];
@@ -892,7 +897,8 @@ static void free_context_table(struct intel_iommu *iommu)
 
 	spin_lock_irqsave(&iommu->lock, flags);
 	if (!iommu->root_entry) {
-		goto out;
+		spin_unlock_irqrestore(&iommu->lock, flags);
+		return;
 	}
 	for (i = 0; i < ROOT_ENTRY_NR; i++) {
 		root = &iommu->root_entry[i];
@@ -900,10 +906,23 @@ static void free_context_table(struct intel_iommu *iommu)
 		if (context)
 			free_pgtable_page(context);
 	}
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		iommu->root_entry_old_phys = 0;
+		root = iommu->root_entry_old_virt;
+		iommu->root_entry_old_virt = NULL;
+	}
+#endif
 	free_pgtable_page(iommu->root_entry);
 	iommu->root_entry = NULL;
-out:
+
 	spin_unlock_irqrestore(&iommu->lock, flags);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		iounmap(root);
+#endif
 }
 
 static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain,
@@ -2322,6 +2341,9 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	unsigned long flags;
 	u8 bus, devfn;
 	int did = -1;   /* Default to "no domain_id supplied" */
+#ifdef CONFIG_CRASH_DUMP
+	struct context_entry *ce = NULL;
+#endif /* CONFIG_CRASH_DUMP */
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2355,6 +2377,22 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	domain = alloc_domain(0);
 	if (!domain)
 		return NULL;
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel()) {
+		/*
+		 * if this device had a did in the old kernel
+		 * use its values instead of generating new ones
+		 */
+		ce = device_to_existing_context_entry(iommu, bus, devfn);
+
+		if (ce) {
+			did = context_domain_id(ce);
+			gaw = agaw_to_width(context_address_width(ce));
+		}
+	}
+#endif /* CONFIG_CRASH_DUMP */
+
 	domain->id = iommu_attach_domain_with_id(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
@@ -2889,14 +2927,33 @@ static int __init init_dmars(void)
 		if (ret)
 			goto free_iommu;
 
-		/*
-		 * TBD:
-		 * we could share the same root & context tables
-		 * among all IOMMU's. Need to Split it later.
-		 */
-		ret = iommu_alloc_root_entry(iommu);
-		if (ret)
-			goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			pr_info("IOMMU Copying translate tables from panicked kernel\n");
+			ret = intel_iommu_load_translation_tables(drhd);
+			if (ret) {
+				pr_err("IOMMU: Copy translate tables failed\n");
+
+				/* Best to stop trying */
+				goto free_iommu;
+			}
+			pr_info("IOMMU: root_cache:0x%12.12llx phys:0x%12.12llx\n",
+				(u64)iommu->root_entry,
+				(u64)iommu->root_entry_old_phys);
+		} else {
+#endif /* CONFIG_CRASH_DUMP */
+			/*
+			 * TBD:
+			 * we could share the same root & context tables
+			 * among all IOMMU's. Need to Split it later.
+			 */
+			ret = iommu_alloc_root_entry(iommu);
+			if (ret)
+				goto free_iommu;
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
+
 		if (!ecap_pass_through(iommu->ecap))
 			hw_pass_through = 0;
 	}
@@ -2913,6 +2970,16 @@ static int __init init_dmars(void)
 
 	check_tylersburg_isoch();
 
+#ifdef CONFIG_CRASH_DUMP
+	/*
+	 * In the crashdump kernel: Skip setting-up new domains for
+	 * si, rmrr, and the isa bus on the expectation that these
+	 * translations were copied from the old kernel.
+	 */
+	if (is_kdump_kernel())
+		goto skip_new_domains_for_si_rmrr_isa;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * If pass through is not set or not enabled, setup context entries for
 	 * identity mappings for rmrr, gfx, and isa and may fall back to static
@@ -2953,6 +3020,10 @@ static int __init init_dmars(void)
 
 	iommu_prepare_isa();
 
+#ifdef CONFIG_CRASH_DUMP
+skip_new_domains_for_si_rmrr_isa:;
+#endif /* CONFIG_CRASH_DUMP */
+
 	/*
 	 * for each drhd
 	 *   enable fault log
@@ -2981,7 +3052,15 @@ static int __init init_dmars(void)
 
 		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
 		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
-		iommu_enable_translation(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			if (!(iommu->gcmd & DMA_GCMD_TE))
+				iommu_enable_translation(iommu);
+		} else
+#endif
+			iommu_enable_translation(iommu);
+
 		iommu_disable_protect_mem_regions(iommu);
 	}
 
@@ -4266,12 +4345,21 @@ int __init intel_iommu_init(void)
 		goto out_free_dmar;
 	}
 
+#ifdef CONFIG_CRASH_DUMP
 	/*
-	 * Disable translation if already enabled prior to OS handover.
+	 * If (This is the crash kernel)
+	 *    Skip disabling the iommu hardware translations
 	 */
-	for_each_active_iommu(iommu, drhd)
-		if (iommu->gcmd & DMA_GCMD_TE)
-			iommu_disable_translation(iommu);
+	if (is_kdump_kernel()) {
+		pr_info("IOMMU Skip disabling iommu hardware translations\n");
+	} else
+#endif /* CONFIG_CRASH_DUMP */
+		/*
+		 * Disable translation if already enabled prior to OS handover.
+		 */
+		for_each_active_iommu(iommu, drhd)
+			if (iommu->gcmd & DMA_GCMD_TE)
+				iommu_disable_translation(iommu);
 
 	if (dmar_dev_scope_init() < 0) {
 		if (force_on)
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 08/10] iommu/vt-d: assign new page table for dma_map

[Li, Zhen-Hua]

When a device driver issues the first dma_map command for a device, we
assign a new and empty page-table, thus removing all mappings from the
old kernel for the device.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 54 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 46 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 312f06b..6b691d4 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
+#include <linux/dma-mapping.h>
 
 #include "irq_remapping.h"
 
@@ -3121,14 +3122,30 @@ static struct dmar_domain *__get_valid_domain_for_dev(struct device *dev)
 		return NULL;
 	}
 
-	/* make sure context mapping is ok */
-	if (unlikely(!domain_context_mapped(dev))) {
-		ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
-		if (ret) {
-			printk(KERN_ERR "Domain context map for %s failed",
-			       dev_name(dev));
-			return NULL;
-		}
+	/* if in kdump kernel, we need to unmap the mapped dma pages,
+	 * detach this device first.
+	 */
+	if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unmap_device_dma(domain, dev);
+			domain = get_domain_for_dev(dev,
+				DEFAULT_DOMAIN_ADDRESS_WIDTH);
+			if (!domain) {
+				pr_err("Allocating domain for %s failed",
+				       dev_name(dev));
+				return NULL;
+			}
+		} else
+#endif
+			return domain;
+	}
+
+	ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+	if (ret) {
+		pr_err("Domain context map for %s failed",
+		       dev_name(dev));
+		return NULL;
 	}
 
 	return domain;
@@ -5177,4 +5194,25 @@ static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
 	return ret;
 }
 
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+	struct intel_iommu *iommu;
+	struct context_entry *ce;
+	struct iova *iova;
+	u8 bus, devfn;
+	phys_addr_t phys_addr;
+	dma_addr_t dev_addr;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	ce = device_to_context_entry(iommu, bus, devfn);
+	phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+	dev_addr = phys_to_dma(dev, phys_addr);
+
+	iova = find_iova(&domain->iovad, IOVA_PFN(dev_addr));
+	if (iova)
+		intel_unmap(dev, dev_addr);
+
+	domain_remove_one_dev_info(domain, dev);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0

[PATCH v9 08/10] iommu/vt-d: assign new page table for dma_map

[Li, Zhen-Hua]

When a device driver issues the first dma_map command for a device, we
assign a new and empty page-table, thus removing all mappings from the
old kernel for the device.

Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 54 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 46 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 312f06b..6b691d4 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
+#include <linux/dma-mapping.h>
 
 #include "irq_remapping.h"
 
@@ -3121,14 +3122,30 @@ static struct dmar_domain *__get_valid_domain_for_dev(struct device *dev)
 		return NULL;
 	}
 
-	/* make sure context mapping is ok */
-	if (unlikely(!domain_context_mapped(dev))) {
-		ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
-		if (ret) {
-			printk(KERN_ERR "Domain context map for %s failed",
-			       dev_name(dev));
-			return NULL;
-		}
+	/* if in kdump kernel, we need to unmap the mapped dma pages,
+	 * detach this device first.
+	 */
+	if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unmap_device_dma(domain, dev);
+			domain = get_domain_for_dev(dev,
+				DEFAULT_DOMAIN_ADDRESS_WIDTH);
+			if (!domain) {
+				pr_err("Allocating domain for %s failed",
+				       dev_name(dev));
+				return NULL;
+			}
+		} else
+#endif
+			return domain;
+	}
+
+	ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+	if (ret) {
+		pr_err("Domain context map for %s failed",
+		       dev_name(dev));
+		return NULL;
 	}
 
 	return domain;
@@ -5177,4 +5194,25 @@ static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
 	return ret;
 }
 
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+	struct intel_iommu *iommu;
+	struct context_entry *ce;
+	struct iova *iova;
+	u8 bus, devfn;
+	phys_addr_t phys_addr;
+	dma_addr_t dev_addr;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	ce = device_to_context_entry(iommu, bus, devfn);
+	phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+	dev_addr = phys_to_dma(dev, phys_addr);
+
+	iova = find_iova(&domain->iovad, IOVA_PFN(dev_addr));
+	if (iova)
+		intel_unmap(dev, dev_addr);
+
+	domain_remove_one_dev_info(domain, dev);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0

[PATCH v9 08/10] iommu/vt-d: assign new page table for dma_map

[Li, Zhen-Hua]

When a device driver issues the first dma_map command for a device, we
assign a new and empty page-table, thus removing all mappings from the
old kernel for the device.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel-iommu.c | 54 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 46 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 312f06b..6b691d4 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -44,6 +44,7 @@
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
+#include <linux/dma-mapping.h>
 
 #include "irq_remapping.h"
 
@@ -3121,14 +3122,30 @@ static struct dmar_domain *__get_valid_domain_for_dev(struct device *dev)
 		return NULL;
 	}
 
-	/* make sure context mapping is ok */
-	if (unlikely(!domain_context_mapped(dev))) {
-		ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
-		if (ret) {
-			printk(KERN_ERR "Domain context map for %s failed",
-			       dev_name(dev));
-			return NULL;
-		}
+	/* if in kdump kernel, we need to unmap the mapped dma pages,
+	 * detach this device first.
+	 */
+	if (likely(domain_context_mapped(dev))) {
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unmap_device_dma(domain, dev);
+			domain = get_domain_for_dev(dev,
+				DEFAULT_DOMAIN_ADDRESS_WIDTH);
+			if (!domain) {
+				pr_err("Allocating domain for %s failed",
+				       dev_name(dev));
+				return NULL;
+			}
+		} else
+#endif
+			return domain;
+	}
+
+	ret = domain_context_mapping(domain, dev, CONTEXT_TT_MULTI_LEVEL);
+	if (ret) {
+		pr_err("Domain context map for %s failed",
+		       dev_name(dev));
+		return NULL;
 	}
 
 	return domain;
@@ -5177,4 +5194,25 @@ static int intel_iommu_load_translation_tables(struct dmar_drhd_unit *drhd)
 	return ret;
 }
 
+static void unmap_device_dma(struct dmar_domain *domain, struct device *dev)
+{
+	struct intel_iommu *iommu;
+	struct context_entry *ce;
+	struct iova *iova;
+	u8 bus, devfn;
+	phys_addr_t phys_addr;
+	dma_addr_t dev_addr;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	ce = device_to_context_entry(iommu, bus, devfn);
+	phys_addr = context_address_root(ce) << VTD_PAGE_SHIFT;
+	dev_addr = phys_to_dma(dev, phys_addr);
+
+	iova = find_iova(&domain->iovad, IOVA_PFN(dev_addr));
+	if (iova)
+		intel_unmap(dev, dev_addr);
+
+	domain_remove_one_dev_info(domain, dev);
+}
+
 #endif /* CONFIG_CRASH_DUMP */
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 09/10] iommu/vt-d: Copy functions for irte

[Li, Zhen-Hua]

Functions to copy the irte data from the old kernel into the kdump kernel.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 62 +++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h         |  4 +++
 2 files changed, 66 insertions(+)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 14de1ab..20c060b 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
 #include <linux/irq.h>
 #include <linux/intel-iommu.h>
 #include <linux/acpi.h>
+#include <linux/crash_dump.h>
 #include <asm/io_apic.h>
 #include <asm/smp.h>
 #include <asm/cpu.h>
@@ -17,6 +18,11 @@
 
 #include "irq_remapping.h"
 
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
 struct ioapic_scope {
 	struct intel_iommu *iommu;
 	unsigned int id;
@@ -1302,3 +1308,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool insert)
 
 	return ret;
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	memcpy(iommu->ir_table->base,
+		iommu->ir_table->base_old_virt,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	__iommu_flush_cache(iommu, iommu->ir_table->base,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+	int start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+		return -1;
+
+	if (index == -1) {
+		start = 0;
+		size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+	} else {
+		start = index * sizeof(struct irte);
+		size = sizeof(struct irte);
+	}
+
+	to = iommu->ir_table->base_old_virt;
+	from = iommu->ir_table->base;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+
+	return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
 struct ir_table {
 	struct irte *base;
 	unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *base_old_virt;
+	unsigned long base_old_phys;
+#endif
 };
 #endif
 
-- 
2.0.0-rc0

[PATCH v9 09/10] iommu/vt-d: Copy functions for irte

[Li, Zhen-Hua]

Functions to copy the irte data from the old kernel into the kdump kernel.

Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel_irq_remapping.c | 62 +++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h         |  4 +++
 2 files changed, 66 insertions(+)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 14de1ab..20c060b 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
 #include <linux/irq.h>
 #include <linux/intel-iommu.h>
 #include <linux/acpi.h>
+#include <linux/crash_dump.h>
 #include <asm/io_apic.h>
 #include <asm/smp.h>
 #include <asm/cpu.h>
@@ -17,6 +18,11 @@
 
 #include "irq_remapping.h"
 
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
 struct ioapic_scope {
 	struct intel_iommu *iommu;
 	unsigned int id;
@@ -1302,3 +1308,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool insert)
 
 	return ret;
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	memcpy(iommu->ir_table->base,
+		iommu->ir_table->base_old_virt,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	__iommu_flush_cache(iommu, iommu->ir_table->base,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+	int start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+		return -1;
+
+	if (index == -1) {
+		start = 0;
+		size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+	} else {
+		start = index * sizeof(struct irte);
+		size = sizeof(struct irte);
+	}
+
+	to = iommu->ir_table->base_old_virt;
+	from = iommu->ir_table->base;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+
+	return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
 struct ir_table {
 	struct irte *base;
 	unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *base_old_virt;
+	unsigned long base_old_phys;
+#endif
 };
 #endif
 
-- 
2.0.0-rc0

[PATCH v9 09/10] iommu/vt-d: Copy functions for irte

[Li, Zhen-Hua]

Functions to copy the irte data from the old kernel into the kdump kernel.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 62 +++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h         |  4 +++
 2 files changed, 66 insertions(+)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 14de1ab..20c060b 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
 #include <linux/irq.h>
 #include <linux/intel-iommu.h>
 #include <linux/acpi.h>
+#include <linux/crash_dump.h>
 #include <asm/io_apic.h>
 #include <asm/smp.h>
 #include <asm/cpu.h>
@@ -17,6 +18,11 @@
 
 #include "irq_remapping.h"
 
+#ifdef CONFIG_CRASH_DUMP
+static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
+#endif /* CONFIG_CRASH_DUMP */
+
 struct ioapic_scope {
 	struct intel_iommu *iommu;
 	unsigned int id;
@@ -1302,3 +1308,59 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool insert)
 
 	return ret;
 }
+
+#ifdef CONFIG_CRASH_DUMP
+
+static int __iommu_load_old_irte(struct intel_iommu *iommu)
+{
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	memcpy(iommu->ir_table->base,
+		iommu->ir_table->base_old_virt,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	__iommu_flush_cache(iommu, iommu->ir_table->base,
+		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+
+	return 0;
+}
+
+static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
+{
+	int start;
+	unsigned long size;
+	void __iomem *to;
+	void *from;
+
+	if ((!iommu)
+		|| (!iommu->ir_table)
+		|| (!iommu->ir_table->base)
+		|| (!iommu->ir_table->base_old_phys)
+		|| (!iommu->ir_table->base_old_virt))
+		return -1;
+
+	if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
+		return -1;
+
+	if (index == -1) {
+		start = 0;
+		size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
+	} else {
+		start = index * sizeof(struct irte);
+		size = sizeof(struct irte);
+	}
+
+	to = iommu->ir_table->base_old_virt;
+	from = iommu->ir_table->base;
+	memcpy(to + start, from + start, size);
+
+	__iommu_flush_cache(iommu, to + start, size);
+
+	return 0;
+}
+#endif /* CONFIG_CRASH_DUMP */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 8e29b97..76c6ea5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -290,6 +290,10 @@ struct q_inval {
 struct ir_table {
 	struct irte *base;
 	unsigned long *bitmap;
+#ifdef CONFIG_CRASH_DUMP
+	void __iomem *base_old_virt;
+	unsigned long base_old_phys;
+#endif
 };
 #endif
 
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v9 10/10] iommu/vt-d: Use old irte in kdump kernel

[Li, Zhen-Hua]

Fix the intr-remapping fault.

[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear

Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 43 +++++++++++++++++++++++++++++++------
 1 file changed, 37 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 20c060b..4e5a02d 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -199,6 +199,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
 
 	set_64bit(&irte->low, irte_modified->low);
 	set_64bit(&irte->high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, index);
+#endif
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -260,6 +265,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
 	bitmap_release_region(iommu->ir_table->bitmap, index,
 			      irq_iommu->irte_mask);
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, -1);
+#endif
+
 	return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
 }
 
@@ -662,11 +672,20 @@ static int __init intel_enable_irq_remapping(void)
 		 */
 		dmar_fault(-1, iommu);
 
-		/*
-		 * Disable intr remapping and queued invalidation, if already
-		 * enabled prior to OS handover.
-		 */
-		iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			/* Do notdisable irq and then re-enable again. */
+		} else {
+#endif
+			/*
+			 * Disable intr remapping and queued invalidation,
+			 * if already enabled prior to OS handover.
+			 */
+			iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
 
 		dmar_disable_qi(iommu);
 	}
@@ -702,7 +721,19 @@ static int __init intel_enable_irq_remapping(void)
 	 * Setup Interrupt-remapping for all the DRHD's now.
 	 */
 	for_each_iommu(iommu, drhd) {
-		iommu_set_irq_remapping(iommu, eim);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unsigned long long q;
+
+			q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+			iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+			iommu->ir_table->base_old_virt = ioremap_cache(
+				iommu->ir_table->base_old_phys,
+				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+			__iommu_load_old_irte(iommu);
+		} else
+#endif
+			iommu_set_irq_remapping(iommu, eim);
 		setup = 1;
 	}
 
-- 
2.0.0-rc0

[PATCH v9 10/10] iommu/vt-d: Use old irte in kdump kernel

[Li, Zhen-Hua]

Fix the intr-remapping fault.

[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear

Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.

Signed-off-by: Li, Zhen-Hua <zhen-hual-VXdhtT5mjnY@public.gmane.org>
---
 drivers/iommu/intel_irq_remapping.c | 43 +++++++++++++++++++++++++++++++------
 1 file changed, 37 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 20c060b..4e5a02d 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -199,6 +199,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
 
 	set_64bit(&irte->low, irte_modified->low);
 	set_64bit(&irte->high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, index);
+#endif
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -260,6 +265,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
 	bitmap_release_region(iommu->ir_table->bitmap, index,
 			      irq_iommu->irte_mask);
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, -1);
+#endif
+
 	return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
 }
 
@@ -662,11 +672,20 @@ static int __init intel_enable_irq_remapping(void)
 		 */
 		dmar_fault(-1, iommu);
 
-		/*
-		 * Disable intr remapping and queued invalidation, if already
-		 * enabled prior to OS handover.
-		 */
-		iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			/* Do notdisable irq and then re-enable again. */
+		} else {
+#endif
+			/*
+			 * Disable intr remapping and queued invalidation,
+			 * if already enabled prior to OS handover.
+			 */
+			iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
 
 		dmar_disable_qi(iommu);
 	}
@@ -702,7 +721,19 @@ static int __init intel_enable_irq_remapping(void)
 	 * Setup Interrupt-remapping for all the DRHD's now.
 	 */
 	for_each_iommu(iommu, drhd) {
-		iommu_set_irq_remapping(iommu, eim);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unsigned long long q;
+
+			q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+			iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+			iommu->ir_table->base_old_virt = ioremap_cache(
+				iommu->ir_table->base_old_phys,
+				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+			__iommu_load_old_irte(iommu);
+		} else
+#endif
+			iommu_set_irq_remapping(iommu, eim);
 		setup = 1;
 	}
 
-- 
2.0.0-rc0

[PATCH v9 10/10] iommu/vt-d: Use old irte in kdump kernel

[Li, Zhen-Hua]

Fix the intr-remapping fault.

[1.594890] dmar: DRHD: handling fault status reg 2
[1.594894] dmar: INTR-REMAP: Request device [[41:00.0] fault index 4d
[1.594894] INTR-REMAP:[fault reason 34] Present field in the IRTE entry
is clear

Use old irte in kdump kernel, do not disable and re-enable interrupt
remapping.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
---
 drivers/iommu/intel_irq_remapping.c | 43 +++++++++++++++++++++++++++++++------
 1 file changed, 37 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 20c060b..4e5a02d 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -199,6 +199,11 @@ static int modify_irte(int irq, struct irte *irte_modified)
 
 	set_64bit(&irte->low, irte_modified->low);
 	set_64bit(&irte->high, irte_modified->high);
+
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, index);
+#endif
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -260,6 +265,11 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
 	bitmap_release_region(iommu->ir_table->bitmap, index,
 			      irq_iommu->irte_mask);
 
+#ifdef CONFIG_CRASH_DUMP
+	if (is_kdump_kernel())
+		__iommu_update_old_irte(iommu, -1);
+#endif
+
 	return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
 }
 
@@ -662,11 +672,20 @@ static int __init intel_enable_irq_remapping(void)
 		 */
 		dmar_fault(-1, iommu);
 
-		/*
-		 * Disable intr remapping and queued invalidation, if already
-		 * enabled prior to OS handover.
-		 */
-		iommu_disable_irq_remapping(iommu);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			/* Do notdisable irq and then re-enable again. */
+		} else {
+#endif
+			/*
+			 * Disable intr remapping and queued invalidation,
+			 * if already enabled prior to OS handover.
+			 */
+			iommu_disable_irq_remapping(iommu);
+
+#ifdef CONFIG_CRASH_DUMP
+		}
+#endif
 
 		dmar_disable_qi(iommu);
 	}
@@ -702,7 +721,19 @@ static int __init intel_enable_irq_remapping(void)
 	 * Setup Interrupt-remapping for all the DRHD's now.
 	 */
 	for_each_iommu(iommu, drhd) {
-		iommu_set_irq_remapping(iommu, eim);
+#ifdef CONFIG_CRASH_DUMP
+		if (is_kdump_kernel()) {
+			unsigned long long q;
+
+			q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+			iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
+			iommu->ir_table->base_old_virt = ioremap_cache(
+				iommu->ir_table->base_old_phys,
+				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+			__iommu_load_old_irte(iommu);
+		} else
+#endif
+			iommu_set_irq_remapping(iommu, eim);
 		setup = 1;
 	}
 
-- 
2.0.0-rc0


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module

[Joerg Roedel]

On Thu, Mar 19, 2015 at 01:36:25PM +0800, Li, Zhen-Hua wrote:
> +#ifdef CONFIG_CRASH_DUMP
> +		if (is_kdump_kernel())
> +			__iommu_update_old_root_entry(iommu, bus);
> +#endif

All the is_kdump_kernel checks in this patch (and maybe in other patches
too) should really be checks whether translation on the IOMMU was
enabled or not when the kernel booted. You might also boot from a kernel
that had translation disabled into a kdump kernel that wants to enable
it. In this case these checks would break.

Speaking of booting from kernels with translation disabled, there is a
valid use of is_kdump_enabled(), to omit iommu initialization in the
kdump kernel when translation was disabled before. But the other checks
should depend on the state the iommu had when booting the kdump kernel.


	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

Hi Zhen-Hua,

On Thu, Mar 19, 2015 at 01:36:18PM +0800, Li, Zhen-Hua wrote:
> This patchset is an update of Bill Sumner's patchset, implements a fix for:
> If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
> when a panic happens, the kdump kernel will boot with these faults:

I reviewed this patch-set and it is getting closer to a point where it
could be merged. I found a few white-space errors in the review, please
do a check_patch run on the next round and fix these. Besides that and
given some third-party testing and reviews I think we can look forward
to merge it early after the merge-window for v4.1, to give it enough
testing in -next too.


	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

Hi Zhen-Hua,

On Thu, Mar 19, 2015 at 01:36:18PM +0800, Li, Zhen-Hua wrote:
> This patchset is an update of Bill Sumner's patchset, implements a fix for:
> If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
> when a panic happens, the kdump kernel will boot with these faults:

I reviewed this patch-set and it is getting closer to a point where it
could be merged. I found a few white-space errors in the review, please
do a check_patch run on the next round and fix these. Besides that and
given some third-party testing and reviews I think we can look forward
to merge it early after the merge-window for v4.1, to give it enough
testing in -next too.


	Joerg

Re: [PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module

[Li, ZhenHua]

Hi Joerg,
Thinking about it carefully, I think you suggestions are very helpful, 
and the checks should be:
* All these things should be done in the second kernel, not only the 
kdump kernel. Sometimes user may use kexec manually start a new kernel.
* Copying those tables should only happen in the second kernel, only in 
this situation: iommu is enabled in both first and second kernel.

So, I think we can do this:
1. Use a new variable iommu_enabled_in_last_kernel;
2. When iommu module is loaded, it means the iommu is enabled in current 
kernel.  Then we check DMA_GSTS_TES:
    * if DMA_GSTS_TES is set, then it should be the second kernel(kexec 
starts this kernel manually, or it is kdump kernel), and the first 
kernel used iommu tables.  Here we need to copy these tables; then set 
iommu_enabled_in_last_kernel to 1.
    * if DMA_GSTS_TES is NOT set, then we do not need to care about 
these tables.

3. Replace all
#ifdef CONFIG_CRASH_DUMP
	if (is_kdump_kernel()) {
             // Do some thing
         }
#endif

To:
if (iommu_enabled_in_last_kernel) {
      // Do some thing
}

Do you agree with these?

Thanks
Zhenhua

On 04/02/2015 07:06 PM, Joerg Roedel wrote:
> On Thu, Mar 19, 2015 at 01:36:25PM +0800, Li, Zhen-Hua wrote:
>> +#ifdef CONFIG_CRASH_DUMP
>> +		if (is_kdump_kernel())
>> +			__iommu_update_old_root_entry(iommu, bus);
>> +#endif
>
> All the is_kdump_kernel checks in this patch (and maybe in other patches
> too) should really be checks whether translation on the IOMMU was
> enabled or not when the kernel booted. You might also boot from a kernel
> that had translation disabled into a kdump kernel that wants to enable
> it. In this case these checks would break.
>
> Speaking of booting from kernels with translation disabled, there is a
> valid use of is_kdump_enabled(), to omit iommu initialization in the
> kdump kernel when translation was disabled before. But the other checks
> should depend on the state the iommu had when booting the kdump kernel.
>
>
> 	Joerg
>

Re: [PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module

[Li, ZhenHua]

Hi Joerg,
Thinking about it carefully, I think you suggestions are very helpful, 
and the checks should be:
* All these things should be done in the second kernel, not only the 
kdump kernel. Sometimes user may use kexec manually start a new kernel.
* Copying those tables should only happen in the second kernel, only in 
this situation: iommu is enabled in both first and second kernel.

So, I think we can do this:
1. Use a new variable iommu_enabled_in_last_kernel;
2. When iommu module is loaded, it means the iommu is enabled in current 
kernel.  Then we check DMA_GSTS_TES:
    * if DMA_GSTS_TES is set, then it should be the second kernel(kexec 
starts this kernel manually, or it is kdump kernel), and the first 
kernel used iommu tables.  Here we need to copy these tables; then set 
iommu_enabled_in_last_kernel to 1.
    * if DMA_GSTS_TES is NOT set, then we do not need to care about 
these tables.

3. Replace all
#ifdef CONFIG_CRASH_DUMP
	if (is_kdump_kernel()) {
             // Do some thing
         }
#endif

To:
if (iommu_enabled_in_last_kernel) {
      // Do some thing
}

Do you agree with these?

Thanks
Zhenhua

On 04/02/2015 07:06 PM, Joerg Roedel wrote:
> On Thu, Mar 19, 2015 at 01:36:25PM +0800, Li, Zhen-Hua wrote:
>> +#ifdef CONFIG_CRASH_DUMP
>> +		if (is_kdump_kernel())
>> +			__iommu_update_old_root_entry(iommu, bus);
>> +#endif
>
> All the is_kdump_kernel checks in this patch (and maybe in other patches
> too) should really be checks whether translation on the IOMMU was
> enabled or not when the kernel booted. You might also boot from a kernel
> that had translation disabled into a kdump kernel that wants to enable
> it. In this case these checks would break.
>
> Speaking of booting from kernels with translation disabled, there is a
> valid use of is_kdump_enabled(), to omit iommu initialization in the
> kdump kernel when translation was disabled before. But the other checks
> should depend on the state the iommu had when booting the kdump kernel.
>
>
> 	Joerg
>

Re: [PATCH v9 07/10] iommu/vt-d: enable kdump support in iommu module

[Li, ZhenHua]

Hi Joerg,
Thinking about it carefully, I think you suggestions are very helpful, 
and the checks should be:
* All these things should be done in the second kernel, not only the 
kdump kernel. Sometimes user may use kexec manually start a new kernel.
* Copying those tables should only happen in the second kernel, only in 
this situation: iommu is enabled in both first and second kernel.

So, I think we can do this:
1. Use a new variable iommu_enabled_in_last_kernel;
2. When iommu module is loaded, it means the iommu is enabled in current 
kernel.  Then we check DMA_GSTS_TES:
    * if DMA_GSTS_TES is set, then it should be the second kernel(kexec 
starts this kernel manually, or it is kdump kernel), and the first 
kernel used iommu tables.  Here we need to copy these tables; then set 
iommu_enabled_in_last_kernel to 1.
    * if DMA_GSTS_TES is NOT set, then we do not need to care about 
these tables.

3. Replace all
#ifdef CONFIG_CRASH_DUMP
	if (is_kdump_kernel()) {
             // Do some thing
         }
#endif

To:
if (iommu_enabled_in_last_kernel) {
      // Do some thing
}

Do you agree with these?

Thanks
Zhenhua

On 04/02/2015 07:06 PM, Joerg Roedel wrote:
> On Thu, Mar 19, 2015 at 01:36:25PM +0800, Li, Zhen-Hua wrote:
>> +#ifdef CONFIG_CRASH_DUMP
>> +		if (is_kdump_kernel())
>> +			__iommu_update_old_root_entry(iommu, bus);
>> +#endif
>
> All the is_kdump_kernel checks in this patch (and maybe in other patches
> too) should really be checks whether translation on the IOMMU was
> enabled or not when the kernel booted. You might also boot from a kernel
> that had translation disabled into a kdump kernel that wants to enable
> it. In this case these checks would break.
>
> Speaking of booting from kernels with translation disabled, there is a
> valid use of is_kdump_enabled(), to omit iommu initialization in the
> kdump kernel when translation was disabled before. But the other checks
> should depend on the state the iommu had when booting the kdump kernel.
>
>
> 	Joerg
>


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

Hi Joerg,
This is quite strange. I checked the patches from patch 01 to 10 using 
./scripts/checkpatch.pl under the kernel source directory, but got 0 
errors and 0 warning.  Only some white spaces in the cover letter 00, 
but is could not be checked by this script.

But I checked the intel-iommu.c using "checkpatch.pl -f", found too many 
warnings and errors. Maybe we need a new patch to fix them.

Thanks
Zhenhua

On 04/02/2015 07:11 PM, Joerg Roedel wrote:
> Hi Zhen-Hua,
>
> On Thu, Mar 19, 2015 at 01:36:18PM +0800, Li, Zhen-Hua wrote:
>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>> when a panic happens, the kdump kernel will boot with these faults:
>
> I reviewed this patch-set and it is getting closer to a point where it
> could be merged. I found a few white-space errors in the review, please
> do a check_patch run on the next round and fix these. Besides that and
> given some third-party testing and reviews I think we can look forward
> to merge it early after the merge-window for v4.1, to give it enough
> testing in -next too.
>
>
> 	Joerg
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

Hi Joerg,
This is quite strange. I checked the patches from patch 01 to 10 using 
./scripts/checkpatch.pl under the kernel source directory, but got 0 
errors and 0 warning.  Only some white spaces in the cover letter 00, 
but is could not be checked by this script.

But I checked the intel-iommu.c using "checkpatch.pl -f", found too many 
warnings and errors. Maybe we need a new patch to fix them.

Thanks
Zhenhua

On 04/02/2015 07:11 PM, Joerg Roedel wrote:
> Hi Zhen-Hua,
>
> On Thu, Mar 19, 2015 at 01:36:18PM +0800, Li, Zhen-Hua wrote:
>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>> when a panic happens, the kdump kernel will boot with these faults:
>
> I reviewed this patch-set and it is getting closer to a point where it
> could be merged. I found a few white-space errors in the review, please
> do a check_patch run on the next round and fix these. Besides that and
> given some third-party testing and reviews I think we can look forward
> to merge it early after the merge-window for v4.1, to give it enough
> testing in -next too.
>
>
> 	Joerg
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

Hi Joerg,
This is quite strange. I checked the patches from patch 01 to 10 using 
./scripts/checkpatch.pl under the kernel source directory, but got 0 
errors and 0 warning.  Only some white spaces in the cover letter 00, 
but is could not be checked by this script.

But I checked the intel-iommu.c using "checkpatch.pl -f", found too many 
warnings and errors. Maybe we need a new patch to fix them.

Thanks
Zhenhua

On 04/02/2015 07:11 PM, Joerg Roedel wrote:
> Hi Zhen-Hua,
>
> On Thu, Mar 19, 2015 at 01:36:18PM +0800, Li, Zhen-Hua wrote:
>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>> when a panic happens, the kdump kernel will boot with these faults:
>
> I reviewed this patch-set and it is getting closer to a point where it
> could be merged. I found a few white-space errors in the review, please
> do a check_patch run on the next round and fix these. Besides that and
> given some third-party testing and reviews I think we can look forward
> to merge it early after the merge-window for v4.1, to give it enough
> testing in -next too.
>
>
> 	Joerg
>


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 03/19/15 at 01:36pm, Li, Zhen-Hua wrote:
> This patchset is an update of Bill Sumner's patchset, implements a fix for:
> If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
> when a panic happens, the kdump kernel will boot with these faults:
> 
>     dmar: DRHD: handling fault status reg 102
>     dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>     DMAR:[fault reason 01] Present bit in root entry is clear
> 
>     dmar: DRHD: handling fault status reg 2
>     dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>     INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
> 
> On some system, the interrupt remapping fault will also happen even if the 
> intel_iommu is not set to on, because the interrupt remapping will be enabled 
> when x2apic is needed by the system.
> 
> The cause of the DMA fault is described in Bill's original version, and the 
> INTR-Remap fault is caused by a similar reason. In short, the initialization 
> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong 
> response.
> 
> To fix this problem, we modifies the behaviors of the intel vt-d in the 
> crashdump kernel:
> 
> For DMA Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the translation, keep it enabled.
> 3. Use the old root entry table, do not rewrite the RTA register.
> 4. Malloc and use new context entry table, copy data from the old ones that
>    used by the old kernel.
> 5. Keep using the old page tables before driver is loaded.
> 6. After device driver is loaded, when it issues the first dma_map command, 
>    free the dmar_domain structure for this device, and generate a new one, so 
>    that the device can be assigned a new and empty page table. 
> 7. When a new context entry table is generated, we also save its address to 
>    the old root entry table.
> 
> For Interrupt Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
> 4. When ioapic entry is setup, the interrupt remapping table is changed, and 
>    the updated data will be stored to the old interrupt remapping table.
> 
> Advantages of this approach:
> 1. All manipulation of the IO-device is done by the Linux device-driver
>    for that device.
> 2. This approach behaves in a manner very similar to operation without an
>    active iommu.
> 3. Any activity between the IO-device and its RMRR areas is handled by the
>    device-driver in the same manner as during a non-kdump boot.
> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>    This supports the practice of creating a special kdump kernel without
>    drivers for any devices that are not required for taking a crashdump. 
> 5. Minimal code-changes among the existing mainline intel vt-d code.
> 
> Summary of changes in this patch set:
> 1. Added some useful function for root entry table in code intel-iommu.c
> 2. Added new members to struct root_entry and struct irte;
> 3. Functions to load old root entry table to iommu->root_entry from the memory 
>    of old kernel.
> 4. Functions to malloc new context entry table and copy the data from the old
>    ones to the malloced new ones.
> 5. Functions to enable support for DMA remapping in kdump kernel.
> 6. Functions to load old irte data from the old kernel to the kdump kernel.
> 7. Some code changes that support other behaviours that have been listed.
> 8. In the new functions, use physical address as "unsigned long" type, not 
>    pointers.
> 
> Original version by Bill Sumner:
>     https://lkml.org/lkml/2014/1/10/518
>     https://lkml.org/lkml/2014/4/15/716
>     https://lkml.org/lkml/2014/4/24/836
> 
> Zhenhua's updates:
>     https://lkml.org/lkml/2014/10/21/134
>     https://lkml.org/lkml/2014/12/15/121
>     https://lkml.org/lkml/2014/12/22/53
>     https://lkml.org/lkml/2015/1/6/1166
>     https://lkml.org/lkml/2015/1/12/35
> 
> Changelog[v9]:
>     1. Add new function iommu_attach_domain_with_id.
>     2. Do not copy old page tables, keep using the old ones.
>     3. Remove functions:
>            intel_iommu_did_to_domain_values_entry
>            intel_iommu_get_dids_from_old_kernel
>            device_to_domain_id
>            copy_page_addr
>            copy_page_table
>            copy_context_entry
>            copy_context_entry_table
>     4. Add new function device_to_existing_context_entry.
> 
> Changelog[v8]:
>     1. Add a missing __iommu_flush_cache in function copy_page_table.
> 
> Changelog[v7]:
>     1. Use __iommu_flush_cache to flush the data to hardware.
> 
> Changelog[v6]:
>     1. Use "unsigned long" as type of physical address.
>     2. Use new function unmap_device_dma to unmap the old dma.
>     3. Some small incorrect bits order for aw shift.
> 
> Changelog[v5]:
>     1. Do not disable and re-enable traslation and interrupt remapping. 
>     2. Use old root entry table.
>     3. Use old interrupt remapping table.
>     4. New functions to copy data from old kernel, and save to old kernel mem.
>     5. New functions to save updated root entry table and irte table.
>     6. Use intel_unmap to unmap the old dma;
>     7. Allocate new pages while driver is being loaded.
> 
> Changelog[v4]:
>     1. Cut off the patches that move some defines and functions to new files.
>     2. Reduce the numbers of patches to five, make it more easier to read.
>     3. Changed the name of functions, make them consistent with current context
>        get/set functions.
>     4. Add change to function __iommu_attach_domain.
> 
> Changelog[v3]:
>     1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>     2. Updated the comments about changes in each version.
>     3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>           struct as recommended by Baoquan He [bhe@redhat.com]
>           init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
> 
> Changelog[v2]:
>     The following series implements a fix for:
>     A kdump problem about DMA that has been discussed for a long time. That is,
>     when a kernel panics and boots into the kdump kernel, DMA started by the
>     panicked kernel is not stopped before the kdump kernel is booted and the
>     kdump kernel disables the IOMMU while this DMA continues.  This causes the
>     IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>     them as physical memory addresses -- which causes the DMA to either:
>         (1) generate DMAR errors or 
>         (2) generate PCI SERR errors or 
>         (3) transfer data to or from incorrect areas of memory. Often this 
>             causes the dump to fail.
> 
> Changelog[v1]:
>     The original version.
> 
> Changed in this version:
> 1. Do not disable and re-enable traslation and interrupt remapping. 
> 2. Use old root entry table.
> 3. Use old interrupt remapping table.
> 4. Use "unsigned long" as physical address.
> 5. Use intel_unmap to unmap the old dma;
> 
> Baoquan He <bhe@redhat.com> helps testing this patchset.
> Takao Indoh <indou.takao@jp.fujitsu.com> gives valuable suggestions.
> 
> Li, Zhen-Hua (10):
>   iommu/vt-d: New function to attach domain with id
>   iommu/vt-d: Items required for kdump
>   iommu/vt-d: Function to get old context entry
>   iommu/vt-d: functions to copy data from old mem
>   iommu/vt-d: Add functions to load and save old re
>   iommu/vt-d: datatypes and functions used for kdump
>   iommu/vt-d: enable kdump support in iommu module
>   iommu/vt-d: assign new page table for dma_map
>   iommu/vt-d: Copy functions for irte
>   iommu/vt-d: Use old irte in kdump kernel
> 
>  drivers/iommu/intel-iommu.c         | 535 ++++++++++++++++++++++++++++++++++--
>  drivers/iommu/intel_irq_remapping.c | 105 ++++++-
>  include/linux/intel-iommu.h         |  18 ++
>  3 files changed, 628 insertions(+), 30 deletions(-)
> 
> -- 
> 2.0.0-rc0
> 

Zhenhua,

I will review the patchset recently, sorry for jumping in late.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 03/19/15 at 01:36pm, Li, Zhen-Hua wrote:
> This patchset is an update of Bill Sumner's patchset, implements a fix for:
> If a kernel boots with intel_iommu=on on a system that supports intel vt-d, 
> when a panic happens, the kdump kernel will boot with these faults:
> 
>     dmar: DRHD: handling fault status reg 102
>     dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fff80000
>     DMAR:[fault reason 01] Present bit in root entry is clear
> 
>     dmar: DRHD: handling fault status reg 2
>     dmar: INTR-REMAP: Request device [[61:00.0] fault index 42
>     INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
> 
> On some system, the interrupt remapping fault will also happen even if the 
> intel_iommu is not set to on, because the interrupt remapping will be enabled 
> when x2apic is needed by the system.
> 
> The cause of the DMA fault is described in Bill's original version, and the 
> INTR-Remap fault is caused by a similar reason. In short, the initialization 
> of vt-d drivers causes the in-flight DMA and interrupt requests get wrong 
> response.
> 
> To fix this problem, we modifies the behaviors of the intel vt-d in the 
> crashdump kernel:
> 
> For DMA Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the translation, keep it enabled.
> 3. Use the old root entry table, do not rewrite the RTA register.
> 4. Malloc and use new context entry table, copy data from the old ones that
>    used by the old kernel.
> 5. Keep using the old page tables before driver is loaded.
> 6. After device driver is loaded, when it issues the first dma_map command, 
>    free the dmar_domain structure for this device, and generate a new one, so 
>    that the device can be assigned a new and empty page table. 
> 7. When a new context entry table is generated, we also save its address to 
>    the old root entry table.
> 
> For Interrupt Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the interrupt remapping, keep it enabled.
> 3. Use the old interrupt remapping table, do not rewrite the IRTA register.
> 4. When ioapic entry is setup, the interrupt remapping table is changed, and 
>    the updated data will be stored to the old interrupt remapping table.
> 
> Advantages of this approach:
> 1. All manipulation of the IO-device is done by the Linux device-driver
>    for that device.
> 2. This approach behaves in a manner very similar to operation without an
>    active iommu.
> 3. Any activity between the IO-device and its RMRR areas is handled by the
>    device-driver in the same manner as during a non-kdump boot.
> 4. If an IO-device has no driver in the kdump kernel, it is simply left alone.
>    This supports the practice of creating a special kdump kernel without
>    drivers for any devices that are not required for taking a crashdump. 
> 5. Minimal code-changes among the existing mainline intel vt-d code.
> 
> Summary of changes in this patch set:
> 1. Added some useful function for root entry table in code intel-iommu.c
> 2. Added new members to struct root_entry and struct irte;
> 3. Functions to load old root entry table to iommu->root_entry from the memory 
>    of old kernel.
> 4. Functions to malloc new context entry table and copy the data from the old
>    ones to the malloced new ones.
> 5. Functions to enable support for DMA remapping in kdump kernel.
> 6. Functions to load old irte data from the old kernel to the kdump kernel.
> 7. Some code changes that support other behaviours that have been listed.
> 8. In the new functions, use physical address as "unsigned long" type, not 
>    pointers.
> 
> Original version by Bill Sumner:
>     https://lkml.org/lkml/2014/1/10/518
>     https://lkml.org/lkml/2014/4/15/716
>     https://lkml.org/lkml/2014/4/24/836
> 
> Zhenhua's updates:
>     https://lkml.org/lkml/2014/10/21/134
>     https://lkml.org/lkml/2014/12/15/121
>     https://lkml.org/lkml/2014/12/22/53
>     https://lkml.org/lkml/2015/1/6/1166
>     https://lkml.org/lkml/2015/1/12/35
> 
> Changelog[v9]:
>     1. Add new function iommu_attach_domain_with_id.
>     2. Do not copy old page tables, keep using the old ones.
>     3. Remove functions:
>            intel_iommu_did_to_domain_values_entry
>            intel_iommu_get_dids_from_old_kernel
>            device_to_domain_id
>            copy_page_addr
>            copy_page_table
>            copy_context_entry
>            copy_context_entry_table
>     4. Add new function device_to_existing_context_entry.
> 
> Changelog[v8]:
>     1. Add a missing __iommu_flush_cache in function copy_page_table.
> 
> Changelog[v7]:
>     1. Use __iommu_flush_cache to flush the data to hardware.
> 
> Changelog[v6]:
>     1. Use "unsigned long" as type of physical address.
>     2. Use new function unmap_device_dma to unmap the old dma.
>     3. Some small incorrect bits order for aw shift.
> 
> Changelog[v5]:
>     1. Do not disable and re-enable traslation and interrupt remapping. 
>     2. Use old root entry table.
>     3. Use old interrupt remapping table.
>     4. New functions to copy data from old kernel, and save to old kernel mem.
>     5. New functions to save updated root entry table and irte table.
>     6. Use intel_unmap to unmap the old dma;
>     7. Allocate new pages while driver is being loaded.
> 
> Changelog[v4]:
>     1. Cut off the patches that move some defines and functions to new files.
>     2. Reduce the numbers of patches to five, make it more easier to read.
>     3. Changed the name of functions, make them consistent with current context
>        get/set functions.
>     4. Add change to function __iommu_attach_domain.
> 
> Changelog[v3]:
>     1. Commented-out "#define DEBUG 1" to eliminate debug messages.
>     2. Updated the comments about changes in each version.
>     3. Fixed: one-line added to Copy-Translations patch to initialize the iovad
>           struct as recommended by Baoquan He [bhe@redhat.com]
>           init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
> 
> Changelog[v2]:
>     The following series implements a fix for:
>     A kdump problem about DMA that has been discussed for a long time. That is,
>     when a kernel panics and boots into the kdump kernel, DMA started by the
>     panicked kernel is not stopped before the kdump kernel is booted and the
>     kdump kernel disables the IOMMU while this DMA continues.  This causes the
>     IOMMU to stop translating the DMA addresses as IOVAs and begin to treat
>     them as physical memory addresses -- which causes the DMA to either:
>         (1) generate DMAR errors or 
>         (2) generate PCI SERR errors or 
>         (3) transfer data to or from incorrect areas of memory. Often this 
>             causes the dump to fail.
> 
> Changelog[v1]:
>     The original version.
> 
> Changed in this version:
> 1. Do not disable and re-enable traslation and interrupt remapping. 
> 2. Use old root entry table.
> 3. Use old interrupt remapping table.
> 4. Use "unsigned long" as physical address.
> 5. Use intel_unmap to unmap the old dma;
> 
> Baoquan He <bhe@redhat.com> helps testing this patchset.
> Takao Indoh <indou.takao@jp.fujitsu.com> gives valuable suggestions.
> 
> Li, Zhen-Hua (10):
>   iommu/vt-d: New function to attach domain with id
>   iommu/vt-d: Items required for kdump
>   iommu/vt-d: Function to get old context entry
>   iommu/vt-d: functions to copy data from old mem
>   iommu/vt-d: Add functions to load and save old re
>   iommu/vt-d: datatypes and functions used for kdump
>   iommu/vt-d: enable kdump support in iommu module
>   iommu/vt-d: assign new page table for dma_map
>   iommu/vt-d: Copy functions for irte
>   iommu/vt-d: Use old irte in kdump kernel
> 
>  drivers/iommu/intel-iommu.c         | 535 ++++++++++++++++++++++++++++++++++--
>  drivers/iommu/intel_irq_remapping.c | 105 ++++++-
>  include/linux/intel-iommu.h         |  18 ++
>  3 files changed, 628 insertions(+), 30 deletions(-)
> 
> -- 
> 2.0.0-rc0
> 

Zhenhua,

I will review the patchset recently, sorry for jumping in late.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

On 04/03/2015 04:28 PM, Dave Young wrote:
> On 03/19/15 at 01:36pm, Li, Zhen-Hua wrote:
>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>> when a panic happens, the kdump kernel will boot with these faults:
> Zhenhua,
>
> I will review the patchset recently, sorry for jumping in late.
>
> Thanks
> Dave
>
Hi Dave,
Thanks for your review. And please also take a look at the plan I sent 
in another mail:

Change use of
      if (is_kdump_kernel()) { }

To:
     if (iommu_enabled_in_last_kernel) { }

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

On 04/03/2015 04:28 PM, Dave Young wrote:
> On 03/19/15 at 01:36pm, Li, Zhen-Hua wrote:
>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>> when a panic happens, the kdump kernel will boot with these faults:
> Zhenhua,
>
> I will review the patchset recently, sorry for jumping in late.
>
> Thanks
> Dave
>
Hi Dave,
Thanks for your review. And please also take a look at the plan I sent 
in another mail:

Change use of
      if (is_kdump_kernel()) { }

To:
     if (iommu_enabled_in_last_kernel) { }

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

On 04/03/2015 04:28 PM, Dave Young wrote:
> On 03/19/15 at 01:36pm, Li, Zhen-Hua wrote:
>> This patchset is an update of Bill Sumner's patchset, implements a fix for:
>> If a kernel boots with intel_iommu=on on a system that supports intel vt-d,
>> when a panic happens, the kdump kernel will boot with these faults:
> Zhenhua,
>
> I will review the patchset recently, sorry for jumping in late.
>
> Thanks
> Dave
>
Hi Dave,
Thanks for your review. And please also take a look at the plan I sent 
in another mail:

Change use of
      if (is_kdump_kernel()) { }

To:
     if (iommu_enabled_in_last_kernel) { }


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

> To fix this problem, we modifies the behaviors of the intel vt-d in the 
> crashdump kernel:
> 
> For DMA Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the translation, keep it enabled.
> 3. Use the old root entry table, do not rewrite the RTA register.
> 4. Malloc and use new context entry table, copy data from the old ones that
>    used by the old kernel.

Have not read all the patches, but I have a question, not sure this has been
answered before. Old memory is not reliable, what if the old memory get corrupted
before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
cause problems.

Hope I'm wrong though.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

> To fix this problem, we modifies the behaviors of the intel vt-d in the 
> crashdump kernel:
> 
> For DMA Remapping:
> 1. To accept the vt-d hardware in an active state,
> 2. Do not disable and re-enable the translation, keep it enabled.
> 3. Use the old root entry table, do not rewrite the RTA register.
> 4. Malloc and use new context entry table, copy data from the old ones that
>    used by the old kernel.

Have not read all the patches, but I have a question, not sure this has been
answered before. Old memory is not reliable, what if the old memory get corrupted
before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
cause problems.

Hope I'm wrong though.

Thanks
Dave



_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

Hi Dave,

There may be some possibilities that the old iommu data is corrupted by
some other modules. Currently we do not have a better solution for the
dmar faults.

But I think when this happens, we need to fix the module that corrupted
the old iommu data. I once met a similar problem in normal kernel, the
queue used by the qi_* functions was written again by another module.
The fix was in that module, not in iommu module.


Thanks
Zhenhua

On 04/03/2015 04:40 PM, Dave Young wrote:
>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>> crashdump kernel:
>>
>> For DMA Remapping:
>> 1. To accept the vt-d hardware in an active state,
>> 2. Do not disable and re-enable the translation, keep it enabled.
>> 3. Use the old root entry table, do not rewrite the RTA register.
>> 4. Malloc and use new context entry table, copy data from the old ones that
>>     used by the old kernel.
>
> Have not read all the patches, but I have a question, not sure this has been
> answered before. Old memory is not reliable, what if the old memory get corrupted
> before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
> cause problems.
>
> Hope I'm wrong though.
>
> Thanks
> Dave
>
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

Hi Dave,

There may be some possibilities that the old iommu data is corrupted by
some other modules. Currently we do not have a better solution for the
dmar faults.

But I think when this happens, we need to fix the module that corrupted
the old iommu data. I once met a similar problem in normal kernel, the
queue used by the qi_* functions was written again by another module.
The fix was in that module, not in iommu module.


Thanks
Zhenhua

On 04/03/2015 04:40 PM, Dave Young wrote:
>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>> crashdump kernel:
>>
>> For DMA Remapping:
>> 1. To accept the vt-d hardware in an active state,
>> 2. Do not disable and re-enable the translation, keep it enabled.
>> 3. Use the old root entry table, do not rewrite the RTA register.
>> 4. Malloc and use new context entry table, copy data from the old ones that
>>     used by the old kernel.
>
> Have not read all the patches, but I have a question, not sure this has been
> answered before. Old memory is not reliable, what if the old memory get corrupted
> before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
> cause problems.
>
> Hope I'm wrong though.
>
> Thanks
> Dave
>
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

Hi Dave,

There may be some possibilities that the old iommu data is corrupted by
some other modules. Currently we do not have a better solution for the
dmar faults.

But I think when this happens, we need to fix the module that corrupted
the old iommu data. I once met a similar problem in normal kernel, the
queue used by the qi_* functions was written again by another module.
The fix was in that module, not in iommu module.


Thanks
Zhenhua

On 04/03/2015 04:40 PM, Dave Young wrote:
>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>> crashdump kernel:
>>
>> For DMA Remapping:
>> 1. To accept the vt-d hardware in an active state,
>> 2. Do not disable and re-enable the translation, keep it enabled.
>> 3. Use the old root entry table, do not rewrite the RTA register.
>> 4. Malloc and use new context entry table, copy data from the old ones that
>>     used by the old kernel.
>
> Have not read all the patches, but I have a question, not sure this has been
> answered before. Old memory is not reliable, what if the old memory get corrupted
> before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
> cause problems.
>
> Hope I'm wrong though.
>
> Thanks
> Dave
>
>


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> Hi Dave,
> 
> There may be some possibilities that the old iommu data is corrupted by
> some other modules. Currently we do not have a better solution for the
> dmar faults.
> 
> But I think when this happens, we need to fix the module that corrupted
> the old iommu data. I once met a similar problem in normal kernel, the
> queue used by the qi_* functions was written again by another module.
> The fix was in that module, not in iommu module.

It is too late, there will be no chance to save vmcore then.

Also if it is possible to continue corrupt other area of oldmem because
of using old iommu tables then it will cause more problems.

So I think the tables at least need some verifycation before being used.

> 
> 
> Thanks
> Zhenhua
> 
> On 04/03/2015 04:40 PM, Dave Young wrote:
> >>To fix this problem, we modifies the behaviors of the intel vt-d in the
> >>crashdump kernel:
> >>
> >>For DMA Remapping:
> >>1. To accept the vt-d hardware in an active state,
> >>2. Do not disable and re-enable the translation, keep it enabled.
> >>3. Use the old root entry table, do not rewrite the RTA register.
> >>4. Malloc and use new context entry table, copy data from the old ones that
> >>    used by the old kernel.
> >
> >Have not read all the patches, but I have a question, not sure this has been
> >answered before. Old memory is not reliable, what if the old memory get corrupted
> >before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
> >cause problems.
> >
> >Hope I'm wrong though.
> >
> >Thanks
> >Dave
> >
> >
> 

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> Hi Dave,
> 
> There may be some possibilities that the old iommu data is corrupted by
> some other modules. Currently we do not have a better solution for the
> dmar faults.
> 
> But I think when this happens, we need to fix the module that corrupted
> the old iommu data. I once met a similar problem in normal kernel, the
> queue used by the qi_* functions was written again by another module.
> The fix was in that module, not in iommu module.

It is too late, there will be no chance to save vmcore then.

Also if it is possible to continue corrupt other area of oldmem because
of using old iommu tables then it will cause more problems.

So I think the tables at least need some verifycation before being used.

> 
> 
> Thanks
> Zhenhua
> 
> On 04/03/2015 04:40 PM, Dave Young wrote:
> >>To fix this problem, we modifies the behaviors of the intel vt-d in the
> >>crashdump kernel:
> >>
> >>For DMA Remapping:
> >>1. To accept the vt-d hardware in an active state,
> >>2. Do not disable and re-enable the translation, keep it enabled.
> >>3. Use the old root entry table, do not rewrite the RTA register.
> >>4. Malloc and use new context entry table, copy data from the old ones that
> >>    used by the old kernel.
> >
> >Have not read all the patches, but I have a question, not sure this has been
> >answered before. Old memory is not reliable, what if the old memory get corrupted
> >before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
> >cause problems.
> >
> >Hope I'm wrong though.
> >
> >Thanks
> >Dave
> >
> >
> 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, Zhen-Hua]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 2043 bytes --]

The hardware will do some verification, but not completely.  If people think the OS should also do this, then it should be another patchset, I think.

Thanks
Zhenhua 

> ÔÚ 2015Äê4ÔÂ3ÈÕ£¬17:21£¬Dave Young <dyoung@redhat.com> дµÀ£º
> 
>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>> Hi Dave,
>> 
>> There may be some possibilities that the old iommu data is corrupted by
>> some other modules. Currently we do not have a better solution for the
>> dmar faults.
>> 
>> But I think when this happens, we need to fix the module that corrupted
>> the old iommu data. I once met a similar problem in normal kernel, the
>> queue used by the qi_* functions was written again by another module.
>> The fix was in that module, not in iommu module.
> 
> It is too late, there will be no chance to save vmcore then.
> 
> Also if it is possible to continue corrupt other area of oldmem because
> of using old iommu tables then it will cause more problems.
> 
> So I think the tables at least need some verifycation before being used.
> 
>> 
>> 
>> Thanks
>> Zhenhua
>> 
>> On 04/03/2015 04:40 PM, Dave Young wrote:
>>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>>> crashdump kernel:
>>>> 
>>>> For DMA Remapping:
>>>> 1. To accept the vt-d hardware in an active state,
>>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>> 4. Malloc and use new context entry table, copy data from the old ones that
>>>>   used by the old kernel.
>>> 
>>> Have not read all the patches, but I have a question, not sure this has been
>>> answered before. Old memory is not reliable, what if the old memory get corrupted
>>> before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
>>> cause problems.
>>> 
>>> Hope I'm wrong though.
>>> 
>>> Thanks
>>> Dave
>> 
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, Zhen-Hua]

VGhlIGhhcmR3YXJlIHdpbGwgZG8gc29tZSB2ZXJpZmljYXRpb24sIGJ1dCBub3QgY29tcGxldGVs
eS4gIElmIHBlb3BsZSB0aGluayB0aGUgT1Mgc2hvdWxkIGFsc28gZG8gdGhpcywgdGhlbiBpdCBz
aG91bGQgYmUgYW5vdGhlciBwYXRjaHNldCwgSSB0aGluay4NCg0KVGhhbmtzDQpaaGVuaHVhIA0K
DQo+INTaIDIwMTXE6jTUwjPI1aOsMTc6MjGjrERhdmUgWW91bmcgPGR5b3VuZ0ByZWRoYXQuY29t
PiDQtLXAo7oNCj4gDQo+PiBPbiAwNC8wMy8xNSBhdCAwNTowMXBtLCBMaSwgWmhlbkh1YSB3cm90
ZToNCj4+IEhpIERhdmUsDQo+PiANCj4+IFRoZXJlIG1heSBiZSBzb21lIHBvc3NpYmlsaXRpZXMg
dGhhdCB0aGUgb2xkIGlvbW11IGRhdGEgaXMgY29ycnVwdGVkIGJ5DQo+PiBzb21lIG90aGVyIG1v
ZHVsZXMuIEN1cnJlbnRseSB3ZSBkbyBub3QgaGF2ZSBhIGJldHRlciBzb2x1dGlvbiBmb3IgdGhl
DQo+PiBkbWFyIGZhdWx0cy4NCj4+IA0KPj4gQnV0IEkgdGhpbmsgd2hlbiB0aGlzIGhhcHBlbnMs
IHdlIG5lZWQgdG8gZml4IHRoZSBtb2R1bGUgdGhhdCBjb3JydXB0ZWQNCj4+IHRoZSBvbGQgaW9t
bXUgZGF0YS4gSSBvbmNlIG1ldCBhIHNpbWlsYXIgcHJvYmxlbSBpbiBub3JtYWwga2VybmVsLCB0
aGUNCj4+IHF1ZXVlIHVzZWQgYnkgdGhlIHFpXyogZnVuY3Rpb25zIHdhcyB3cml0dGVuIGFnYWlu
IGJ5IGFub3RoZXIgbW9kdWxlLg0KPj4gVGhlIGZpeCB3YXMgaW4gdGhhdCBtb2R1bGUsIG5vdCBp
biBpb21tdSBtb2R1bGUuDQo+IA0KPiBJdCBpcyB0b28gbGF0ZSwgdGhlcmUgd2lsbCBiZSBubyBj
aGFuY2UgdG8gc2F2ZSB2bWNvcmUgdGhlbi4NCj4gDQo+IEFsc28gaWYgaXQgaXMgcG9zc2libGUg
dG8gY29udGludWUgY29ycnVwdCBvdGhlciBhcmVhIG9mIG9sZG1lbSBiZWNhdXNlDQo+IG9mIHVz
aW5nIG9sZCBpb21tdSB0YWJsZXMgdGhlbiBpdCB3aWxsIGNhdXNlIG1vcmUgcHJvYmxlbXMuDQo+
IA0KPiBTbyBJIHRoaW5rIHRoZSB0YWJsZXMgYXQgbGVhc3QgbmVlZCBzb21lIHZlcmlmeWNhdGlv
biBiZWZvcmUgYmVpbmcgdXNlZC4NCj4gDQo+PiANCj4+IA0KPj4gVGhhbmtzDQo+PiBaaGVuaHVh
DQo+PiANCj4+IE9uIDA0LzAzLzIwMTUgMDQ6NDAgUE0sIERhdmUgWW91bmcgd3JvdGU6DQo+Pj4+
IFRvIGZpeCB0aGlzIHByb2JsZW0sIHdlIG1vZGlmaWVzIHRoZSBiZWhhdmlvcnMgb2YgdGhlIGlu
dGVsIHZ0LWQgaW4gdGhlDQo+Pj4+IGNyYXNoZHVtcCBrZXJuZWw6DQo+Pj4+IA0KPj4+PiBGb3Ig
RE1BIFJlbWFwcGluZzoNCj4+Pj4gMS4gVG8gYWNjZXB0IHRoZSB2dC1kIGhhcmR3YXJlIGluIGFu
IGFjdGl2ZSBzdGF0ZSwNCj4+Pj4gMi4gRG8gbm90IGRpc2FibGUgYW5kIHJlLWVuYWJsZSB0aGUg
dHJhbnNsYXRpb24sIGtlZXAgaXQgZW5hYmxlZC4NCj4+Pj4gMy4gVXNlIHRoZSBvbGQgcm9vdCBl
bnRyeSB0YWJsZSwgZG8gbm90IHJld3JpdGUgdGhlIFJUQSByZWdpc3Rlci4NCj4+Pj4gNC4gTWFs
bG9jIGFuZCB1c2UgbmV3IGNvbnRleHQgZW50cnkgdGFibGUsIGNvcHkgZGF0YSBmcm9tIHRoZSBv
bGQgb25lcyB0aGF0DQo+Pj4+ICAgdXNlZCBieSB0aGUgb2xkIGtlcm5lbC4NCj4+PiANCj4+PiBI
YXZlIG5vdCByZWFkIGFsbCB0aGUgcGF0Y2hlcywgYnV0IEkgaGF2ZSBhIHF1ZXN0aW9uLCBub3Qg
c3VyZSB0aGlzIGhhcyBiZWVuDQo+Pj4gYW5zd2VyZWQgYmVmb3JlLiBPbGQgbWVtb3J5IGlzIG5v
dCByZWxpYWJsZSwgd2hhdCBpZiB0aGUgb2xkIG1lbW9yeSBnZXQgY29ycnVwdGVkDQo+Pj4gYmVm
b3JlIHBhbmljPyBJcyBpdCBzYWZlIHRvIGNvbnRpbnVlIHVzaW5nIGl0IGluIDJuZCBrZXJuZWws
IEkgd29ycnkgdGhhdCBpdCB3aWxsDQo+Pj4gY2F1c2UgcHJvYmxlbXMuDQo+Pj4gDQo+Pj4gSG9w
ZSBJJ20gd3JvbmcgdGhvdWdoLg0KPj4+IA0KPj4+IFRoYW5rcw0KPj4+IERhdmUNCj4+IA0K

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, Zhen-Hua]

The hardware will do some verification, but not completely.  If people think the OS should also do this, then it should be another patchset, I think.

Thanks
Zhenhua 

> 在 2015年4月3日，17:21，Dave Young <dyoung@redhat.com> 写道：
> 
>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>> Hi Dave,
>> 
>> There may be some possibilities that the old iommu data is corrupted by
>> some other modules. Currently we do not have a better solution for the
>> dmar faults.
>> 
>> But I think when this happens, we need to fix the module that corrupted
>> the old iommu data. I once met a similar problem in normal kernel, the
>> queue used by the qi_* functions was written again by another module.
>> The fix was in that module, not in iommu module.
> 
> It is too late, there will be no chance to save vmcore then.
> 
> Also if it is possible to continue corrupt other area of oldmem because
> of using old iommu tables then it will cause more problems.
> 
> So I think the tables at least need some verifycation before being used.
> 
>> 
>> 
>> Thanks
>> Zhenhua
>> 
>> On 04/03/2015 04:40 PM, Dave Young wrote:
>>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>>> crashdump kernel:
>>>> 
>>>> For DMA Remapping:
>>>> 1. To accept the vt-d hardware in an active state,
>>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>> 4. Malloc and use new context entry table, copy data from the old ones that
>>>>   used by the old kernel.
>>> 
>>> Have not read all the patches, but I have a question, not sure this has been
>>> answered before. Old memory is not reliable, what if the old memory get corrupted
>>> before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
>>> cause problems.
>>> 
>>> Hope I'm wrong though.
>>> 
>>> Thanks
>>> Dave
>> 

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, Zhen-Hua]

The hardware will do some verification, but not completely.  If people think the OS should also do this, then it should be another patchset, I think.

Thanks
Zhenhua 

> 在 2015年4月3日，17:21，Dave Young <dyoung@redhat.com> 写道：
> 
>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>> Hi Dave,
>> 
>> There may be some possibilities that the old iommu data is corrupted by
>> some other modules. Currently we do not have a better solution for the
>> dmar faults.
>> 
>> But I think when this happens, we need to fix the module that corrupted
>> the old iommu data. I once met a similar problem in normal kernel, the
>> queue used by the qi_* functions was written again by another module.
>> The fix was in that module, not in iommu module.
> 
> It is too late, there will be no chance to save vmcore then.
> 
> Also if it is possible to continue corrupt other area of oldmem because
> of using old iommu tables then it will cause more problems.
> 
> So I think the tables at least need some verifycation before being used.
> 
>> 
>> 
>> Thanks
>> Zhenhua
>> 
>> On 04/03/2015 04:40 PM, Dave Young wrote:
>>>> To fix this problem, we modifies the behaviors of the intel vt-d in the
>>>> crashdump kernel:
>>>> 
>>>> For DMA Remapping:
>>>> 1. To accept the vt-d hardware in an active state,
>>>> 2. Do not disable and re-enable the translation, keep it enabled.
>>>> 3. Use the old root entry table, do not rewrite the RTA register.
>>>> 4. Malloc and use new context entry table, copy data from the old ones that
>>>>   used by the old kernel.
>>> 
>>> Have not read all the patches, but I have a question, not sure this has been
>>> answered before. Old memory is not reliable, what if the old memory get corrupted
>>> before panic? Is it safe to continue using it in 2nd kernel, I worry that it will
>>> cause problems.
>>> 
>>> Hope I'm wrong though.
>>> 
>>> Thanks
>>> Dave
>> 
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Baoquan He]

On 04/03/15 at 05:21pm, Dave Young wrote:
> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> > Hi Dave,
> > 
> > There may be some possibilities that the old iommu data is corrupted by
> > some other modules. Currently we do not have a better solution for the
> > dmar faults.
> > 
> > But I think when this happens, we need to fix the module that corrupted
> > the old iommu data. I once met a similar problem in normal kernel, the
> > queue used by the qi_* functions was written again by another module.
> > The fix was in that module, not in iommu module.
> 
> It is too late, there will be no chance to save vmcore then.
> 
> Also if it is possible to continue corrupt other area of oldmem because
> of using old iommu tables then it will cause more problems.
> 
> So I think the tables at least need some verifycation before being used.
> 

Yes, it's a good thinking anout this and verification is also an
interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
and then verify this again when panic happens in purgatory. This checks
whether any code stomps into region reserved for kexec/kernel and corrupt
the loaded kernel.

If this is decided to do it should be an enhancement to current
patchset but not a approach change. Since this patchset is going very
close to point as maintainers expected maybe this can be merged firstly,
then think about enhancement. After all without this patchset vt-d often
raised error message, hung.

By the way I tested this patchset it works very well on my HP z420 work
station.

Thanks
Baoquan

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Baoquan He]

On 04/03/15 at 05:21pm, Dave Young wrote:
> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> > Hi Dave,
> > 
> > There may be some possibilities that the old iommu data is corrupted by
> > some other modules. Currently we do not have a better solution for the
> > dmar faults.
> > 
> > But I think when this happens, we need to fix the module that corrupted
> > the old iommu data. I once met a similar problem in normal kernel, the
> > queue used by the qi_* functions was written again by another module.
> > The fix was in that module, not in iommu module.
> 
> It is too late, there will be no chance to save vmcore then.
> 
> Also if it is possible to continue corrupt other area of oldmem because
> of using old iommu tables then it will cause more problems.
> 
> So I think the tables at least need some verifycation before being used.
> 

Yes, it's a good thinking anout this and verification is also an
interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
and then verify this again when panic happens in purgatory. This checks
whether any code stomps into region reserved for kexec/kernel and corrupt
the loaded kernel.

If this is decided to do it should be an enhancement to current
patchset but not a approach change. Since this patchset is going very
close to point as maintainers expected maybe this can be merged firstly,
then think about enhancement. After all without this patchset vt-d often
raised error message, hung.

By the way I tested this patchset it works very well on my HP z420 work
station.

Thanks
Baoquan

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Baoquan He]

On 04/03/15 at 05:21pm, Dave Young wrote:
> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> > Hi Dave,
> > 
> > There may be some possibilities that the old iommu data is corrupted by
> > some other modules. Currently we do not have a better solution for the
> > dmar faults.
> > 
> > But I think when this happens, we need to fix the module that corrupted
> > the old iommu data. I once met a similar problem in normal kernel, the
> > queue used by the qi_* functions was written again by another module.
> > The fix was in that module, not in iommu module.
> 
> It is too late, there will be no chance to save vmcore then.
> 
> Also if it is possible to continue corrupt other area of oldmem because
> of using old iommu tables then it will cause more problems.
> 
> So I think the tables at least need some verifycation before being used.
> 

Yes, it's a good thinking anout this and verification is also an
interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
and then verify this again when panic happens in purgatory. This checks
whether any code stomps into region reserved for kexec/kernel and corrupt
the loaded kernel.

If this is decided to do it should be an enhancement to current
patchset but not a approach change. Since this patchset is going very
close to point as maintainers expected maybe this can be merged firstly,
then think about enhancement. After all without this patchset vt-d often
raised error message, hung.

By the way I tested this patchset it works very well on my HP z420 work
station.

Thanks
Baoquan

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/05/15 at 09:54am, Baoquan He wrote:
> On 04/03/15 at 05:21pm, Dave Young wrote:
> > On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> > > Hi Dave,
> > > 
> > > There may be some possibilities that the old iommu data is corrupted by
> > > some other modules. Currently we do not have a better solution for the
> > > dmar faults.
> > > 
> > > But I think when this happens, we need to fix the module that corrupted
> > > the old iommu data. I once met a similar problem in normal kernel, the
> > > queue used by the qi_* functions was written again by another module.
> > > The fix was in that module, not in iommu module.
> > 
> > It is too late, there will be no chance to save vmcore then.
> > 
> > Also if it is possible to continue corrupt other area of oldmem because
> > of using old iommu tables then it will cause more problems.
> > 
> > So I think the tables at least need some verifycation before being used.
> > 
> 
> Yes, it's a good thinking anout this and verification is also an
> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> and then verify this again when panic happens in purgatory. This checks
> whether any code stomps into region reserved for kexec/kernel and corrupt
> the loaded kernel.
> 
> If this is decided to do it should be an enhancement to current
> patchset but not a approach change. Since this patchset is going very
> close to point as maintainers expected maybe this can be merged firstly,
> then think about enhancement. After all without this patchset vt-d often
> raised error message, hung.

It does not convince me, we should do it right at the beginning instead of
introduce something wrong.

I wonder why the old dma can not be remap to a specific page in kdump kernel
so that it will not corrupt more memory. But I may missed something, I will
looking for old threads and catch up.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/05/15 at 09:54am, Baoquan He wrote:
> On 04/03/15 at 05:21pm, Dave Young wrote:
> > On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> > > Hi Dave,
> > > 
> > > There may be some possibilities that the old iommu data is corrupted by
> > > some other modules. Currently we do not have a better solution for the
> > > dmar faults.
> > > 
> > > But I think when this happens, we need to fix the module that corrupted
> > > the old iommu data. I once met a similar problem in normal kernel, the
> > > queue used by the qi_* functions was written again by another module.
> > > The fix was in that module, not in iommu module.
> > 
> > It is too late, there will be no chance to save vmcore then.
> > 
> > Also if it is possible to continue corrupt other area of oldmem because
> > of using old iommu tables then it will cause more problems.
> > 
> > So I think the tables at least need some verifycation before being used.
> > 
> 
> Yes, it's a good thinking anout this and verification is also an
> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> and then verify this again when panic happens in purgatory. This checks
> whether any code stomps into region reserved for kexec/kernel and corrupt
> the loaded kernel.
> 
> If this is decided to do it should be an enhancement to current
> patchset but not a approach change. Since this patchset is going very
> close to point as maintainers expected maybe this can be merged firstly,
> then think about enhancement. After all without this patchset vt-d often
> raised error message, hung.

It does not convince me, we should do it right at the beginning instead of
introduce something wrong.

I wonder why the old dma can not be remap to a specific page in kdump kernel
so that it will not corrupt more memory. But I may missed something, I will
looking for old threads and catch up.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/03/15 at 02:05pm, Li, Zhen-Hua wrote:
> The hardware will do some verification, but not completely.  If people think the OS should also do this, then it should be another patchset, I think.

If there is chance to corrupt more memory I think it is not a right way.
We should think about a better solution instead of fix it later.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/03/15 at 02:05pm, Li, Zhen-Hua wrote:
> The hardware will do some verification, but not completely.  If people think the OS should also do this, then it should be another patchset, I think.

If there is chance to corrupt more memory I think it is not a right way.
We should think about a better solution instead of fix it later.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/03/15 at 02:05pm, Li, Zhen-Hua wrote:
> The hardware will do some verification, but not completely.  If people think the OS should also do this, then it should be another patchset, I think.

If there is chance to corrupt more memory I think it is not a right way.
We should think about a better solution instead of fix it later.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/07/15 at 11:46am, Dave Young wrote:
> On 04/05/15 at 09:54am, Baoquan He wrote:
> > On 04/03/15 at 05:21pm, Dave Young wrote:
> > > On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> > > > Hi Dave,
> > > > 
> > > > There may be some possibilities that the old iommu data is corrupted by
> > > > some other modules. Currently we do not have a better solution for the
> > > > dmar faults.
> > > > 
> > > > But I think when this happens, we need to fix the module that corrupted
> > > > the old iommu data. I once met a similar problem in normal kernel, the
> > > > queue used by the qi_* functions was written again by another module.
> > > > The fix was in that module, not in iommu module.
> > > 
> > > It is too late, there will be no chance to save vmcore then.
> > > 
> > > Also if it is possible to continue corrupt other area of oldmem because
> > > of using old iommu tables then it will cause more problems.
> > > 
> > > So I think the tables at least need some verifycation before being used.
> > > 
> > 
> > Yes, it's a good thinking anout this and verification is also an
> > interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> > and then verify this again when panic happens in purgatory. This checks
> > whether any code stomps into region reserved for kexec/kernel and corrupt
> > the loaded kernel.
> > 
> > If this is decided to do it should be an enhancement to current
> > patchset but not a approach change. Since this patchset is going very
> > close to point as maintainers expected maybe this can be merged firstly,
> > then think about enhancement. After all without this patchset vt-d often
> > raised error message, hung.
> 
> It does not convince me, we should do it right at the beginning instead of
> introduce something wrong.
> 
> I wonder why the old dma can not be remap to a specific page in kdump kernel
> so that it will not corrupt more memory. But I may missed something, I will
> looking for old threads and catch up.

I have read the old discussion, above way was dropped because it could corrupt
filesystem. Apologize about late commenting.

But current solution sounds bad to me because of using old memory which is not
reliable. 

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/07/15 at 11:46am, Dave Young wrote:
> On 04/05/15 at 09:54am, Baoquan He wrote:
> > On 04/03/15 at 05:21pm, Dave Young wrote:
> > > On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> > > > Hi Dave,
> > > > 
> > > > There may be some possibilities that the old iommu data is corrupted by
> > > > some other modules. Currently we do not have a better solution for the
> > > > dmar faults.
> > > > 
> > > > But I think when this happens, we need to fix the module that corrupted
> > > > the old iommu data. I once met a similar problem in normal kernel, the
> > > > queue used by the qi_* functions was written again by another module.
> > > > The fix was in that module, not in iommu module.
> > > 
> > > It is too late, there will be no chance to save vmcore then.
> > > 
> > > Also if it is possible to continue corrupt other area of oldmem because
> > > of using old iommu tables then it will cause more problems.
> > > 
> > > So I think the tables at least need some verifycation before being used.
> > > 
> > 
> > Yes, it's a good thinking anout this and verification is also an
> > interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> > and then verify this again when panic happens in purgatory. This checks
> > whether any code stomps into region reserved for kexec/kernel and corrupt
> > the loaded kernel.
> > 
> > If this is decided to do it should be an enhancement to current
> > patchset but not a approach change. Since this patchset is going very
> > close to point as maintainers expected maybe this can be merged firstly,
> > then think about enhancement. After all without this patchset vt-d often
> > raised error message, hung.
> 
> It does not convince me, we should do it right at the beginning instead of
> introduce something wrong.
> 
> I wonder why the old dma can not be remap to a specific page in kdump kernel
> so that it will not corrupt more memory. But I may missed something, I will
> looking for old threads and catch up.

I have read the old discussion, above way was dropped because it could corrupt
filesystem. Apologize about late commenting.

But current solution sounds bad to me because of using old memory which is not
reliable. 

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

On 04/07/2015 05:08 PM, Dave Young wrote:
> On 04/07/15 at 11:46am, Dave Young wrote:
>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>> Hi Dave,
>>>>>
>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>> some other modules. Currently we do not have a better solution for the
>>>>> dmar faults.
>>>>>
>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>> queue used by the qi_* functions was written again by another module.
>>>>> The fix was in that module, not in iommu module.
>>>>
>>>> It is too late, there will be no chance to save vmcore then.
>>>>
>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>> of using old iommu tables then it will cause more problems.
>>>>
>>>> So I think the tables at least need some verifycation before being used.
>>>>
>>>
>>> Yes, it's a good thinking anout this and verification is also an
>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>> and then verify this again when panic happens in purgatory. This checks
>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>> the loaded kernel.
>>>
>>> If this is decided to do it should be an enhancement to current
>>> patchset but not a approach change. Since this patchset is going very
>>> close to point as maintainers expected maybe this can be merged firstly,
>>> then think about enhancement. After all without this patchset vt-d often
>>> raised error message, hung.
>>
>> It does not convince me, we should do it right at the beginning instead of
>> introduce something wrong.
>>
>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>> so that it will not corrupt more memory. But I may missed something, I will
>> looking for old threads and catch up.
>
> I have read the old discussion, above way was dropped because it could corrupt
> filesystem. Apologize about late commenting.
>
> But current solution sounds bad to me because of using old memory which is not
> reliable.
>
> Thanks
> Dave
>
Seems we do not have a better solution for the dmar faults.  But I 
believe we can find out how to verify the iommu data which is located in 
old memory.

Thanks
Zhenhua

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Li, ZhenHua]

On 04/07/2015 05:08 PM, Dave Young wrote:
> On 04/07/15 at 11:46am, Dave Young wrote:
>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>> Hi Dave,
>>>>>
>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>> some other modules. Currently we do not have a better solution for the
>>>>> dmar faults.
>>>>>
>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>> queue used by the qi_* functions was written again by another module.
>>>>> The fix was in that module, not in iommu module.
>>>>
>>>> It is too late, there will be no chance to save vmcore then.
>>>>
>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>> of using old iommu tables then it will cause more problems.
>>>>
>>>> So I think the tables at least need some verifycation before being used.
>>>>
>>>
>>> Yes, it's a good thinking anout this and verification is also an
>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>> and then verify this again when panic happens in purgatory. This checks
>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>> the loaded kernel.
>>>
>>> If this is decided to do it should be an enhancement to current
>>> patchset but not a approach change. Since this patchset is going very
>>> close to point as maintainers expected maybe this can be merged firstly,
>>> then think about enhancement. After all without this patchset vt-d often
>>> raised error message, hung.
>>
>> It does not convince me, we should do it right at the beginning instead of
>> introduce something wrong.
>>
>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>> so that it will not corrupt more memory. But I may missed something, I will
>> looking for old threads and catch up.
>
> I have read the old discussion, above way was dropped because it could corrupt
> filesystem. Apologize about late commenting.
>
> But current solution sounds bad to me because of using old memory which is not
> reliable.
>
> Thanks
> Dave
>
Seems we do not have a better solution for the dmar faults.  But I 
believe we can find out how to verify the iommu data which is located in 
old memory.

Thanks
Zhenhua


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 04/06/2015 11:46 PM, Dave Young wrote:
> On 04/05/15 at 09:54am, Baoquan He wrote:
>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>> Hi Dave,
>>>>
>>>> There may be some possibilities that the old iommu data is corrupted by
>>>> some other modules. Currently we do not have a better solution for the
>>>> dmar faults.
>>>>
>>>> But I think when this happens, we need to fix the module that corrupted
>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>> queue used by the qi_* functions was written again by another module.
>>>> The fix was in that module, not in iommu module.
>>>
>>> It is too late, there will be no chance to save vmcore then.
>>>
>>> Also if it is possible to continue corrupt other area of oldmem because
>>> of using old iommu tables then it will cause more problems.
>>>
>>> So I think the tables at least need some verifycation before being used.
>>>
>>
>> Yes, it's a good thinking anout this and verification is also an
>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>> and then verify this again when panic happens in purgatory. This checks
>> whether any code stomps into region reserved for kexec/kernel and corrupt
>> the loaded kernel.
>>
>> If this is decided to do it should be an enhancement to current
>> patchset but not a approach change. Since this patchset is going very
>> close to point as maintainers expected maybe this can be merged firstly,
>> then think about enhancement. After all without this patchset vt-d often
>> raised error message, hung.
>
> It does not convince me, we should do it right at the beginning instead of
> introduce something wrong.
>
> I wonder why the old dma can not be remap to a specific page in kdump kernel
> so that it will not corrupt more memory. But I may missed something, I will
> looking for old threads and catch up.
>
> Thanks
> Dave
>
The (only) issue is not corruption, but once the iommu is re-configured, the old,
not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
in the kexec kernel.

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 04/06/2015 11:46 PM, Dave Young wrote:
> On 04/05/15 at 09:54am, Baoquan He wrote:
>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>> Hi Dave,
>>>>
>>>> There may be some possibilities that the old iommu data is corrupted by
>>>> some other modules. Currently we do not have a better solution for the
>>>> dmar faults.
>>>>
>>>> But I think when this happens, we need to fix the module that corrupted
>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>> queue used by the qi_* functions was written again by another module.
>>>> The fix was in that module, not in iommu module.
>>>
>>> It is too late, there will be no chance to save vmcore then.
>>>
>>> Also if it is possible to continue corrupt other area of oldmem because
>>> of using old iommu tables then it will cause more problems.
>>>
>>> So I think the tables at least need some verifycation before being used.
>>>
>>
>> Yes, it's a good thinking anout this and verification is also an
>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>> and then verify this again when panic happens in purgatory. This checks
>> whether any code stomps into region reserved for kexec/kernel and corrupt
>> the loaded kernel.
>>
>> If this is decided to do it should be an enhancement to current
>> patchset but not a approach change. Since this patchset is going very
>> close to point as maintainers expected maybe this can be merged firstly,
>> then think about enhancement. After all without this patchset vt-d often
>> raised error message, hung.
>
> It does not convince me, we should do it right at the beginning instead of
> introduce something wrong.
>
> I wonder why the old dma can not be remap to a specific page in kdump kernel
> so that it will not corrupt more memory. But I may missed something, I will
> looking for old threads and catch up.
>
> Thanks
> Dave
>
The (only) issue is not corruption, but once the iommu is re-configured, the old,
not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
in the kexec kernel.

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 04/06/2015 11:46 PM, Dave Young wrote:
> On 04/05/15 at 09:54am, Baoquan He wrote:
>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>> Hi Dave,
>>>>
>>>> There may be some possibilities that the old iommu data is corrupted by
>>>> some other modules. Currently we do not have a better solution for the
>>>> dmar faults.
>>>>
>>>> But I think when this happens, we need to fix the module that corrupted
>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>> queue used by the qi_* functions was written again by another module.
>>>> The fix was in that module, not in iommu module.
>>>
>>> It is too late, there will be no chance to save vmcore then.
>>>
>>> Also if it is possible to continue corrupt other area of oldmem because
>>> of using old iommu tables then it will cause more problems.
>>>
>>> So I think the tables at least need some verifycation before being used.
>>>
>>
>> Yes, it's a good thinking anout this and verification is also an
>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>> and then verify this again when panic happens in purgatory. This checks
>> whether any code stomps into region reserved for kexec/kernel and corrupt
>> the loaded kernel.
>>
>> If this is decided to do it should be an enhancement to current
>> patchset but not a approach change. Since this patchset is going very
>> close to point as maintainers expected maybe this can be merged firstly,
>> then think about enhancement. After all without this patchset vt-d often
>> raised error message, hung.
>
> It does not convince me, we should do it right at the beginning instead of
> introduce something wrong.
>
> I wonder why the old dma can not be remap to a specific page in kdump kernel
> so that it will not corrupt more memory. But I may missed something, I will
> looking for old threads and catch up.
>
> Thanks
> Dave
>
The (only) issue is not corruption, but once the iommu is re-configured, the old,
not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
in the kexec kernel.


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/07/15 at 05:55pm, Li, ZhenHua wrote:
> On 04/07/2015 05:08 PM, Dave Young wrote:
> >On 04/07/15 at 11:46am, Dave Young wrote:
> >>On 04/05/15 at 09:54am, Baoquan He wrote:
> >>>On 04/03/15 at 05:21pm, Dave Young wrote:
> >>>>On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> >>>>>Hi Dave,
> >>>>>
> >>>>>There may be some possibilities that the old iommu data is corrupted by
> >>>>>some other modules. Currently we do not have a better solution for the
> >>>>>dmar faults.
> >>>>>
> >>>>>But I think when this happens, we need to fix the module that corrupted
> >>>>>the old iommu data. I once met a similar problem in normal kernel, the
> >>>>>queue used by the qi_* functions was written again by another module.
> >>>>>The fix was in that module, not in iommu module.
> >>>>
> >>>>It is too late, there will be no chance to save vmcore then.
> >>>>
> >>>>Also if it is possible to continue corrupt other area of oldmem because
> >>>>of using old iommu tables then it will cause more problems.
> >>>>
> >>>>So I think the tables at least need some verifycation before being used.
> >>>>
> >>>
> >>>Yes, it's a good thinking anout this and verification is also an
> >>>interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> >>>and then verify this again when panic happens in purgatory. This checks
> >>>whether any code stomps into region reserved for kexec/kernel and corrupt
> >>>the loaded kernel.
> >>>
> >>>If this is decided to do it should be an enhancement to current
> >>>patchset but not a approach change. Since this patchset is going very
> >>>close to point as maintainers expected maybe this can be merged firstly,
> >>>then think about enhancement. After all without this patchset vt-d often
> >>>raised error message, hung.
> >>
> >>It does not convince me, we should do it right at the beginning instead of
> >>introduce something wrong.
> >>
> >>I wonder why the old dma can not be remap to a specific page in kdump kernel
> >>so that it will not corrupt more memory. But I may missed something, I will
> >>looking for old threads and catch up.
> >
> >I have read the old discussion, above way was dropped because it could corrupt
> >filesystem. Apologize about late commenting.
> >
> >But current solution sounds bad to me because of using old memory which is not
> >reliable.
> >
> >Thanks
> >Dave
> >
> Seems we do not have a better solution for the dmar faults.  But I believe
> we can find out how to verify the iommu data which is located in old memory.

That will be great, thanks.

So there's two things:
1) make sure old pg tables are right, this is what we were talking about.
2) avoid writing old memory, I suppose only dma read could corrupt filesystem,
right? So how about for any dma writes just create a scratch page in 2nd kernel
memory. Only using old page table for dma read.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/07/15 at 05:55pm, Li, ZhenHua wrote:
> On 04/07/2015 05:08 PM, Dave Young wrote:
> >On 04/07/15 at 11:46am, Dave Young wrote:
> >>On 04/05/15 at 09:54am, Baoquan He wrote:
> >>>On 04/03/15 at 05:21pm, Dave Young wrote:
> >>>>On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> >>>>>Hi Dave,
> >>>>>
> >>>>>There may be some possibilities that the old iommu data is corrupted by
> >>>>>some other modules. Currently we do not have a better solution for the
> >>>>>dmar faults.
> >>>>>
> >>>>>But I think when this happens, we need to fix the module that corrupted
> >>>>>the old iommu data. I once met a similar problem in normal kernel, the
> >>>>>queue used by the qi_* functions was written again by another module.
> >>>>>The fix was in that module, not in iommu module.
> >>>>
> >>>>It is too late, there will be no chance to save vmcore then.
> >>>>
> >>>>Also if it is possible to continue corrupt other area of oldmem because
> >>>>of using old iommu tables then it will cause more problems.
> >>>>
> >>>>So I think the tables at least need some verifycation before being used.
> >>>>
> >>>
> >>>Yes, it's a good thinking anout this and verification is also an
> >>>interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> >>>and then verify this again when panic happens in purgatory. This checks
> >>>whether any code stomps into region reserved for kexec/kernel and corrupt
> >>>the loaded kernel.
> >>>
> >>>If this is decided to do it should be an enhancement to current
> >>>patchset but not a approach change. Since this patchset is going very
> >>>close to point as maintainers expected maybe this can be merged firstly,
> >>>then think about enhancement. After all without this patchset vt-d often
> >>>raised error message, hung.
> >>
> >>It does not convince me, we should do it right at the beginning instead of
> >>introduce something wrong.
> >>
> >>I wonder why the old dma can not be remap to a specific page in kdump kernel
> >>so that it will not corrupt more memory. But I may missed something, I will
> >>looking for old threads and catch up.
> >
> >I have read the old discussion, above way was dropped because it could corrupt
> >filesystem. Apologize about late commenting.
> >
> >But current solution sounds bad to me because of using old memory which is not
> >reliable.
> >
> >Thanks
> >Dave
> >
> Seems we do not have a better solution for the dmar faults.  But I believe
> we can find out how to verify the iommu data which is located in old memory.

That will be great, thanks.

So there's two things:
1) make sure old pg tables are right, this is what we were talking about.
2) avoid writing old memory, I suppose only dma read could corrupt filesystem,
right? So how about for any dma writes just create a scratch page in 2nd kernel
memory. Only using old page table for dma read.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/07/15 at 05:55pm, Li, ZhenHua wrote:
> On 04/07/2015 05:08 PM, Dave Young wrote:
> >On 04/07/15 at 11:46am, Dave Young wrote:
> >>On 04/05/15 at 09:54am, Baoquan He wrote:
> >>>On 04/03/15 at 05:21pm, Dave Young wrote:
> >>>>On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> >>>>>Hi Dave,
> >>>>>
> >>>>>There may be some possibilities that the old iommu data is corrupted by
> >>>>>some other modules. Currently we do not have a better solution for the
> >>>>>dmar faults.
> >>>>>
> >>>>>But I think when this happens, we need to fix the module that corrupted
> >>>>>the old iommu data. I once met a similar problem in normal kernel, the
> >>>>>queue used by the qi_* functions was written again by another module.
> >>>>>The fix was in that module, not in iommu module.
> >>>>
> >>>>It is too late, there will be no chance to save vmcore then.
> >>>>
> >>>>Also if it is possible to continue corrupt other area of oldmem because
> >>>>of using old iommu tables then it will cause more problems.
> >>>>
> >>>>So I think the tables at least need some verifycation before being used.
> >>>>
> >>>
> >>>Yes, it's a good thinking anout this and verification is also an
> >>>interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> >>>and then verify this again when panic happens in purgatory. This checks
> >>>whether any code stomps into region reserved for kexec/kernel and corrupt
> >>>the loaded kernel.
> >>>
> >>>If this is decided to do it should be an enhancement to current
> >>>patchset but not a approach change. Since this patchset is going very
> >>>close to point as maintainers expected maybe this can be merged firstly,
> >>>then think about enhancement. After all without this patchset vt-d often
> >>>raised error message, hung.
> >>
> >>It does not convince me, we should do it right at the beginning instead of
> >>introduce something wrong.
> >>
> >>I wonder why the old dma can not be remap to a specific page in kdump kernel
> >>so that it will not corrupt more memory. But I may missed something, I will
> >>looking for old threads and catch up.
> >
> >I have read the old discussion, above way was dropped because it could corrupt
> >filesystem. Apologize about late commenting.
> >
> >But current solution sounds bad to me because of using old memory which is not
> >reliable.
> >
> >Thanks
> >Dave
> >
> Seems we do not have a better solution for the dmar faults.  But I believe
> we can find out how to verify the iommu data which is located in old memory.

That will be great, thanks.

So there's two things:
1) make sure old pg tables are right, this is what we were talking about.
2) avoid writing old memory, I suppose only dma read could corrupt filesystem,
right? So how about for any dma writes just create a scratch page in 2nd kernel
memory. Only using old page table for dma read.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

On Fri, Apr 03, 2015 at 04:40:31PM +0800, Dave Young wrote:
> Have not read all the patches, but I have a question, not sure this
> has been answered before. Old memory is not reliable, what if the old
> memory get corrupted before panic? Is it safe to continue using it in
> 2nd kernel, I worry that it will cause problems.

Yes, the old memory could be corrupted, and there are more failure cases
left which we have no way of handling yet (if iommu data structures are
in kdump backup areas).

The question is what to do if we find some of the old data structures
corrupted, hand how far should the tests go. Should we also check the
page-tables, for example? I think if some of the data structures for a
device are corrupted it probably already failed in the old kernel and
things won't get worse in the new one.

So checking is not strictly necessary in the first version of these
patches (unless we find a valid failure scenario). Once we have some
good plan on what to do if we find corruption, we can add checking of
course.


Regards,

	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/04/2015 07:05 AM, Joerg Roedel wrote:
> On Fri, Apr 03, 2015 at 04:40:31PM +0800, Dave Young wrote:
>> Have not read all the patches, but I have a question, not sure this
>> has been answered before. Old memory is not reliable, what if the old
>> memory get corrupted before panic? Is it safe to continue using it in
>> 2nd kernel, I worry that it will cause problems.
>
> Yes, the old memory could be corrupted, and there are more failure cases
> left which we have no way of handling yet (if iommu data structures are
> in kdump backup areas).
>
> The question is what to do if we find some of the old data structures
> corrupted, hand how far should the tests go. Should we also check the
> page-tables, for example? I think if some of the data structures for a
> device are corrupted it probably already failed in the old kernel and
> things won't get worse in the new one.
>
> So checking is not strictly necessary in the first version of these
> patches (unless we find a valid failure scenario). Once we have some
> good plan on what to do if we find corruption, we can add checking of
> course.
>
>
> Regards,
>
> 	Joerg
>

Agreed.  This is a significant improvement over what we (don') have.

Corruption related to IOMMU must occur within the host, and it must be
a software corruption, b/c the IOMMU inherently protects itself by protecting
all of memory from errant DMAs.  Therefore, if the only IOMMU corruptor is
in the host, it's likely the entire host kernel crash dump will either be
useless, or corrupted by the security breach, at which point,
this is just another scenario of a failed crash dump that will never be taken.

The kernel can't protect the mapping tables, which are the most likely area to
be corrupted, b/c it'd (minimally) have to be per-device (to avoid locking
& coherency issues), and would require significant
overhead to keep/update a checksum-like scheme on (potentially) 4 levels of page tables.

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/04/2015 07:05 AM, Joerg Roedel wrote:
> On Fri, Apr 03, 2015 at 04:40:31PM +0800, Dave Young wrote:
>> Have not read all the patches, but I have a question, not sure this
>> has been answered before. Old memory is not reliable, what if the old
>> memory get corrupted before panic? Is it safe to continue using it in
>> 2nd kernel, I worry that it will cause problems.
>
> Yes, the old memory could be corrupted, and there are more failure cases
> left which we have no way of handling yet (if iommu data structures are
> in kdump backup areas).
>
> The question is what to do if we find some of the old data structures
> corrupted, hand how far should the tests go. Should we also check the
> page-tables, for example? I think if some of the data structures for a
> device are corrupted it probably already failed in the old kernel and
> things won't get worse in the new one.
>
> So checking is not strictly necessary in the first version of these
> patches (unless we find a valid failure scenario). Once we have some
> good plan on what to do if we find corruption, we can add checking of
> course.
>
>
> Regards,
>
> 	Joerg
>

Agreed.  This is a significant improvement over what we (don') have.

Corruption related to IOMMU must occur within the host, and it must be
a software corruption, b/c the IOMMU inherently protects itself by protecting
all of memory from errant DMAs.  Therefore, if the only IOMMU corruptor is
in the host, it's likely the entire host kernel crash dump will either be
useless, or corrupted by the security breach, at which point,
this is just another scenario of a failed crash dump that will never be taken.

The kernel can't protect the mapping tables, which are the most likely area to
be corrupted, b/c it'd (minimally) have to be per-device (to avoid locking
& coherency issues), and would require significant
overhead to keep/update a checksum-like scheme on (potentially) 4 levels of page tables.



_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/04/15 at 01:05pm, Joerg Roedel wrote:
> On Fri, Apr 03, 2015 at 04:40:31PM +0800, Dave Young wrote:
> > Have not read all the patches, but I have a question, not sure this
> > has been answered before. Old memory is not reliable, what if the old
> > memory get corrupted before panic? Is it safe to continue using it in
> > 2nd kernel, I worry that it will cause problems.
> 
> Yes, the old memory could be corrupted, and there are more failure cases
> left which we have no way of handling yet (if iommu data structures are
> in kdump backup areas).
> 
> The question is what to do if we find some of the old data structures
> corrupted, hand how far should the tests go. Should we also check the
> page-tables, for example? I think if some of the data structures for a
> device are corrupted it probably already failed in the old kernel and
> things won't get worse in the new one.

I agree that we can do nothing with the old corrupted data, but I worry
about the future corruption after using the old corrupted data. I wonder
if we can mark all the oldmem as readonly so that we can lower the risk.
Is it resonable?

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/04/15 at 01:05pm, Joerg Roedel wrote:
> On Fri, Apr 03, 2015 at 04:40:31PM +0800, Dave Young wrote:
> > Have not read all the patches, but I have a question, not sure this
> > has been answered before. Old memory is not reliable, what if the old
> > memory get corrupted before panic? Is it safe to continue using it in
> > 2nd kernel, I worry that it will cause problems.
> 
> Yes, the old memory could be corrupted, and there are more failure cases
> left which we have no way of handling yet (if iommu data structures are
> in kdump backup areas).
> 
> The question is what to do if we find some of the old data structures
> corrupted, hand how far should the tests go. Should we also check the
> page-tables, for example? I think if some of the data structures for a
> device are corrupted it probably already failed in the old kernel and
> things won't get worse in the new one.

I agree that we can do nothing with the old corrupted data, but I worry
about the future corruption after using the old corrupted data. I wonder
if we can mark all the oldmem as readonly so that we can lower the risk.
Is it resonable?

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

On Tue, May 05, 2015 at 02:09:31PM +0800, Dave Young wrote:
> I agree that we can do nothing with the old corrupted data, but I worry
> about the future corruption after using the old corrupted data. I wonder
> if we can mark all the oldmem as readonly so that we can lower the risk.
> Is it resonable?

Do you mean marking it read-only for the devices? That will very likely
cause DMAR faults, re-introducing the problem this patch-set tries to
fix.


	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

On Tue, May 05, 2015 at 02:09:31PM +0800, Dave Young wrote:
> I agree that we can do nothing with the old corrupted data, but I worry
> about the future corruption after using the old corrupted data. I wonder
> if we can mark all the oldmem as readonly so that we can lower the risk.
> Is it resonable?

Do you mean marking it read-only for the devices? That will very likely
cause DMAR faults, re-introducing the problem this patch-set tries to
fix.


	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/05/15 at 05:23pm, Joerg Roedel wrote:
> On Tue, May 05, 2015 at 02:09:31PM +0800, Dave Young wrote:
> > I agree that we can do nothing with the old corrupted data, but I worry
> > about the future corruption after using the old corrupted data. I wonder
> > if we can mark all the oldmem as readonly so that we can lower the risk.
> > Is it resonable?
> 
> Do you mean marking it read-only for the devices? That will very likely
> cause DMAR faults, re-introducing the problem this patch-set tries to
> fix.

I means to block all dma write to oldmem, I believe it will cause DMA error.
But all other DMA reading requests will continue and work. This will avoid
future possible corruption. It will solve half of the problem at least?

For the original problem, the key issue is dmar faults cause kdump kernel
hang so that vmcore can not be saved. I do not know the reason why it hangs
I think it is acceptable if kdump kernel boot ok with some dma errors..

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/05/15 at 05:23pm, Joerg Roedel wrote:
> On Tue, May 05, 2015 at 02:09:31PM +0800, Dave Young wrote:
> > I agree that we can do nothing with the old corrupted data, but I worry
> > about the future corruption after using the old corrupted data. I wonder
> > if we can mark all the oldmem as readonly so that we can lower the risk.
> > Is it resonable?
> 
> Do you mean marking it read-only for the devices? That will very likely
> cause DMAR faults, re-introducing the problem this patch-set tries to
> fix.

I means to block all dma write to oldmem, I believe it will cause DMA error.
But all other DMA reading requests will continue and work. This will avoid
future possible corruption. It will solve half of the problem at least?

For the original problem, the key issue is dmar faults cause kdump kernel
hang so that vmcore can not be saved. I do not know the reason why it hangs
I think it is acceptable if kdump kernel boot ok with some dma errors..

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

On Wed, May 06, 2015 at 09:46:49AM +0800, Dave Young wrote:
> For the original problem, the key issue is dmar faults cause kdump kernel
> hang so that vmcore can not be saved. I do not know the reason why it hangs
> I think it is acceptable if kdump kernel boot ok with some dma errors..

It hangs because some devices can't handle the DMAR faults and the kdump
kernel can't initialize them and will hang itself. For that it doesn't
matter whether the fault was caused by a read or write request.


	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

On Wed, May 06, 2015 at 09:46:49AM +0800, Dave Young wrote:
> For the original problem, the key issue is dmar faults cause kdump kernel
> hang so that vmcore can not be saved. I do not know the reason why it hangs
> I think it is acceptable if kdump kernel boot ok with some dma errors..

It hangs because some devices can't handle the DMAR faults and the kdump
kernel can't initialize them and will hang itself. For that it doesn't
matter whether the fault was caused by a read or write request.


	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/06/15 at 10:16am, Joerg Roedel wrote:
> On Wed, May 06, 2015 at 09:46:49AM +0800, Dave Young wrote:
> > For the original problem, the key issue is dmar faults cause kdump kernel
> > hang so that vmcore can not be saved. I do not know the reason why it hangs
> > I think it is acceptable if kdump kernel boot ok with some dma errors..
> 
> It hangs because some devices can't handle the DMAR faults and the kdump
> kernel can't initialize them and will hang itself. For that it doesn't
> matter whether the fault was caused by a read or write request.

Ok, thanks for explanation. so it explains sometimes kdump kernel boot ok
with faults, sometimes it hangs instead.

Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/06/15 at 10:16am, Joerg Roedel wrote:
> On Wed, May 06, 2015 at 09:46:49AM +0800, Dave Young wrote:
> > For the original problem, the key issue is dmar faults cause kdump kernel
> > hang so that vmcore can not be saved. I do not know the reason why it hangs
> > I think it is acceptable if kdump kernel boot ok with some dma errors..
> 
> It hangs because some devices can't handle the DMAR faults and the kdump
> kernel can't initialize them and will hang itself. For that it doesn't
> matter whether the fault was caused by a read or write request.

Ok, thanks for explanation. so it explains sometimes kdump kernel boot ok
with faults, sometimes it hangs instead.

Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/06/15 at 10:16am, Joerg Roedel wrote:
> On Wed, May 06, 2015 at 09:46:49AM +0800, Dave Young wrote:
> > For the original problem, the key issue is dmar faults cause kdump kernel
> > hang so that vmcore can not be saved. I do not know the reason why it hangs
> > I think it is acceptable if kdump kernel boot ok with some dma errors..
> 
> It hangs because some devices can't handle the DMAR faults and the kdump
> kernel can't initialize them and will hang itself. For that it doesn't
> matter whether the fault was caused by a read or write request.

Ok, thanks for explanation. so it explains sometimes kdump kernel boot ok
with faults, sometimes it hangs instead.

Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/04/15 at 01:05pm, Joerg Roedel wrote:
> On Fri, Apr 03, 2015 at 04:40:31PM +0800, Dave Young wrote:
> > Have not read all the patches, but I have a question, not sure this
> > has been answered before. Old memory is not reliable, what if the old
> > memory get corrupted before panic? Is it safe to continue using it in
> > 2nd kernel, I worry that it will cause problems.
> 
> Yes, the old memory could be corrupted, and there are more failure cases
> left which we have no way of handling yet (if iommu data structures are
> in kdump backup areas).
> 
> The question is what to do if we find some of the old data structures
> corrupted, hand how far should the tests go. Should we also check the
> page-tables, for example? I think if some of the data structures for a
> device are corrupted it probably already failed in the old kernel and
> things won't get worse in the new one.

Joreg, I can not find the last reply from you, so just reply here about
my worries here.

I said that the patchset will cause more problems, let me explain about
it more here:

Suppose page table was corrupted, ie. original mapping iova1 -> page 1
it was changed to iova1 -> page 2 accidently while crash happening,
thus future dma will read/write page 2 instead page 1, right?

so the behavior changes like:
originally, dmar faults happen, but kdump kernel may boot ok with these
faults, and vmcore can be saved.
with the patchset, dmar faults does not happen, dma translation will be
handled, but dma write could corrupt unrelevant memory.

This might be corner case, but who knows because kernel paniced we can
not assume old page table is right.

But seems you all think it is safe, but let us understand each other
first then go to a solution.

Today we talked with Zhenhua about the problem I think both of us are clear
about the problems. Just he think it can be left as future work.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/04/15 at 01:05pm, Joerg Roedel wrote:
> On Fri, Apr 03, 2015 at 04:40:31PM +0800, Dave Young wrote:
> > Have not read all the patches, but I have a question, not sure this
> > has been answered before. Old memory is not reliable, what if the old
> > memory get corrupted before panic? Is it safe to continue using it in
> > 2nd kernel, I worry that it will cause problems.
> 
> Yes, the old memory could be corrupted, and there are more failure cases
> left which we have no way of handling yet (if iommu data structures are
> in kdump backup areas).
> 
> The question is what to do if we find some of the old data structures
> corrupted, hand how far should the tests go. Should we also check the
> page-tables, for example? I think if some of the data structures for a
> device are corrupted it probably already failed in the old kernel and
> things won't get worse in the new one.

Joreg, I can not find the last reply from you, so just reply here about
my worries here.

I said that the patchset will cause more problems, let me explain about
it more here:

Suppose page table was corrupted, ie. original mapping iova1 -> page 1
it was changed to iova1 -> page 2 accidently while crash happening,
thus future dma will read/write page 2 instead page 1, right?

so the behavior changes like:
originally, dmar faults happen, but kdump kernel may boot ok with these
faults, and vmcore can be saved.
with the patchset, dmar faults does not happen, dma translation will be
handled, but dma write could corrupt unrelevant memory.

This might be corner case, but who knows because kernel paniced we can
not assume old page table is right.

But seems you all think it is safe, but let us understand each other
first then go to a solution.

Today we talked with Zhenhua about the problem I think both of us are clear
about the problems. Just he think it can be left as future work.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/07/15 at 10:12am, Don Dutile wrote:
> On 04/06/2015 11:46 PM, Dave Young wrote:
> >On 04/05/15 at 09:54am, Baoquan He wrote:
> >>On 04/03/15 at 05:21pm, Dave Young wrote:
> >>>On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> >>>>Hi Dave,
> >>>>
> >>>>There may be some possibilities that the old iommu data is corrupted by
> >>>>some other modules. Currently we do not have a better solution for the
> >>>>dmar faults.
> >>>>
> >>>>But I think when this happens, we need to fix the module that corrupted
> >>>>the old iommu data. I once met a similar problem in normal kernel, the
> >>>>queue used by the qi_* functions was written again by another module.
> >>>>The fix was in that module, not in iommu module.
> >>>
> >>>It is too late, there will be no chance to save vmcore then.
> >>>
> >>>Also if it is possible to continue corrupt other area of oldmem because
> >>>of using old iommu tables then it will cause more problems.
> >>>
> >>>So I think the tables at least need some verifycation before being used.
> >>>
> >>
> >>Yes, it's a good thinking anout this and verification is also an
> >>interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> >>and then verify this again when panic happens in purgatory. This checks
> >>whether any code stomps into region reserved for kexec/kernel and corrupt
> >>the loaded kernel.
> >>
> >>If this is decided to do it should be an enhancement to current
> >>patchset but not a approach change. Since this patchset is going very
> >>close to point as maintainers expected maybe this can be merged firstly,
> >>then think about enhancement. After all without this patchset vt-d often
> >>raised error message, hung.
> >
> >It does not convince me, we should do it right at the beginning instead of
> >introduce something wrong.
> >
> >I wonder why the old dma can not be remap to a specific page in kdump kernel
> >so that it will not corrupt more memory. But I may missed something, I will
> >looking for old threads and catch up.
> >
> >Thanks
> >Dave
> >
> The (only) issue is not corruption, but once the iommu is re-configured, the old,
> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
> in the kexec kernel.
> 

Don, so if iommu is not reconfigured then these faults will not happen?

Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:

intel_iommu_init()
{
...

       dmar_table_init();

       disable active iommu translations;

       if (no_iommu || dmar_disabled)
                goto out_free_dmar;

...
}

Any reason not move no_iommu check to the begining of intel_iommu_init function?

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 04/07/15 at 10:12am, Don Dutile wrote:
> On 04/06/2015 11:46 PM, Dave Young wrote:
> >On 04/05/15 at 09:54am, Baoquan He wrote:
> >>On 04/03/15 at 05:21pm, Dave Young wrote:
> >>>On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> >>>>Hi Dave,
> >>>>
> >>>>There may be some possibilities that the old iommu data is corrupted by
> >>>>some other modules. Currently we do not have a better solution for the
> >>>>dmar faults.
> >>>>
> >>>>But I think when this happens, we need to fix the module that corrupted
> >>>>the old iommu data. I once met a similar problem in normal kernel, the
> >>>>queue used by the qi_* functions was written again by another module.
> >>>>The fix was in that module, not in iommu module.
> >>>
> >>>It is too late, there will be no chance to save vmcore then.
> >>>
> >>>Also if it is possible to continue corrupt other area of oldmem because
> >>>of using old iommu tables then it will cause more problems.
> >>>
> >>>So I think the tables at least need some verifycation before being used.
> >>>
> >>
> >>Yes, it's a good thinking anout this and verification is also an
> >>interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> >>and then verify this again when panic happens in purgatory. This checks
> >>whether any code stomps into region reserved for kexec/kernel and corrupt
> >>the loaded kernel.
> >>
> >>If this is decided to do it should be an enhancement to current
> >>patchset but not a approach change. Since this patchset is going very
> >>close to point as maintainers expected maybe this can be merged firstly,
> >>then think about enhancement. After all without this patchset vt-d often
> >>raised error message, hung.
> >
> >It does not convince me, we should do it right at the beginning instead of
> >introduce something wrong.
> >
> >I wonder why the old dma can not be remap to a specific page in kdump kernel
> >so that it will not corrupt more memory. But I may missed something, I will
> >looking for old threads and catch up.
> >
> >Thanks
> >Dave
> >
> The (only) issue is not corruption, but once the iommu is re-configured, the old,
> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
> in the kexec kernel.
> 

Don, so if iommu is not reconfigured then these faults will not happen?

Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:

intel_iommu_init()
{
...

       dmar_table_init();

       disable active iommu translations;

       if (no_iommu || dmar_disabled)
                goto out_free_dmar;

...
}

Any reason not move no_iommu check to the begining of intel_iommu_init function?

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 10:00 AM, Dave Young wrote:
> On 04/07/15 at 10:12am, Don Dutile wrote:
>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>> Hi Dave,
>>>>>>
>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>> dmar faults.
>>>>>>
>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>> The fix was in that module, not in iommu module.
>>>>>
>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>
>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>> of using old iommu tables then it will cause more problems.
>>>>>
>>>>> So I think the tables at least need some verifycation before being used.
>>>>>
>>>>
>>>> Yes, it's a good thinking anout this and verification is also an
>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>> and then verify this again when panic happens in purgatory. This checks
>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>> the loaded kernel.
>>>>
>>>> If this is decided to do it should be an enhancement to current
>>>> patchset but not a approach change. Since this patchset is going very
>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>> then think about enhancement. After all without this patchset vt-d often
>>>> raised error message, hung.
>>>
>>> It does not convince me, we should do it right at the beginning instead of
>>> introduce something wrong.
>>>
>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>> so that it will not corrupt more memory. But I may missed something, I will
>>> looking for old threads and catch up.
>>>
>>> Thanks
>>> Dave
>>>
>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>> in the kexec kernel.
>>
>
> Don, so if iommu is not reconfigured then these faults will not happen?
>
Well, if iommu is not reconfigured, then if the crash isn't caused by
an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.

> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>
> intel_iommu_init()
> {
> ...
>
>         dmar_table_init();
>
>         disable active iommu translations;
>
>         if (no_iommu || dmar_disabled)
>                  goto out_free_dmar;
>
> ...
> }
>
> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>
What does that do/help?

> Thanks
> Dave
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 10:00 AM, Dave Young wrote:
> On 04/07/15 at 10:12am, Don Dutile wrote:
>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>> Hi Dave,
>>>>>>
>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>> dmar faults.
>>>>>>
>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>> The fix was in that module, not in iommu module.
>>>>>
>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>
>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>> of using old iommu tables then it will cause more problems.
>>>>>
>>>>> So I think the tables at least need some verifycation before being used.
>>>>>
>>>>
>>>> Yes, it's a good thinking anout this and verification is also an
>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>> and then verify this again when panic happens in purgatory. This checks
>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>> the loaded kernel.
>>>>
>>>> If this is decided to do it should be an enhancement to current
>>>> patchset but not a approach change. Since this patchset is going very
>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>> then think about enhancement. After all without this patchset vt-d often
>>>> raised error message, hung.
>>>
>>> It does not convince me, we should do it right at the beginning instead of
>>> introduce something wrong.
>>>
>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>> so that it will not corrupt more memory. But I may missed something, I will
>>> looking for old threads and catch up.
>>>
>>> Thanks
>>> Dave
>>>
>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>> in the kexec kernel.
>>
>
> Don, so if iommu is not reconfigured then these faults will not happen?
>
Well, if iommu is not reconfigured, then if the crash isn't caused by
an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.

> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>
> intel_iommu_init()
> {
> ...
>
>         dmar_table_init();
>
>         disable active iommu translations;
>
>         if (no_iommu || dmar_disabled)
>                  goto out_free_dmar;
>
> ...
> }
>
> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>
What does that do/help?

> Thanks
> Dave
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 10:00 AM, Dave Young wrote:
> On 04/07/15 at 10:12am, Don Dutile wrote:
>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>> Hi Dave,
>>>>>>
>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>> dmar faults.
>>>>>>
>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>> The fix was in that module, not in iommu module.
>>>>>
>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>
>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>> of using old iommu tables then it will cause more problems.
>>>>>
>>>>> So I think the tables at least need some verifycation before being used.
>>>>>
>>>>
>>>> Yes, it's a good thinking anout this and verification is also an
>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>> and then verify this again when panic happens in purgatory. This checks
>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>> the loaded kernel.
>>>>
>>>> If this is decided to do it should be an enhancement to current
>>>> patchset but not a approach change. Since this patchset is going very
>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>> then think about enhancement. After all without this patchset vt-d often
>>>> raised error message, hung.
>>>
>>> It does not convince me, we should do it right at the beginning instead of
>>> introduce something wrong.
>>>
>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>> so that it will not corrupt more memory. But I may missed something, I will
>>> looking for old threads and catch up.
>>>
>>> Thanks
>>> Dave
>>>
>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>> in the kexec kernel.
>>
>
> Don, so if iommu is not reconfigured then these faults will not happen?
>
Well, if iommu is not reconfigured, then if the crash isn't caused by
an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.

> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>
> intel_iommu_init()
> {
> ...
>
>         dmar_table_init();
>
>         disable active iommu translations;
>
>         if (no_iommu || dmar_disabled)
>                  goto out_free_dmar;
>
> ...
> }
>
> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>
What does that do/help?

> Thanks
> Dave
>


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/07/15 at 10:25am, Don Dutile wrote:
> On 05/07/2015 10:00 AM, Dave Young wrote:
> >On 04/07/15 at 10:12am, Don Dutile wrote:
> >>On 04/06/2015 11:46 PM, Dave Young wrote:
> >>>On 04/05/15 at 09:54am, Baoquan He wrote:
> >>>>On 04/03/15 at 05:21pm, Dave Young wrote:
> >>>>>On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> >>>>>>Hi Dave,
> >>>>>>
> >>>>>>There may be some possibilities that the old iommu data is corrupted by
> >>>>>>some other modules. Currently we do not have a better solution for the
> >>>>>>dmar faults.
> >>>>>>
> >>>>>>But I think when this happens, we need to fix the module that corrupted
> >>>>>>the old iommu data. I once met a similar problem in normal kernel, the
> >>>>>>queue used by the qi_* functions was written again by another module.
> >>>>>>The fix was in that module, not in iommu module.
> >>>>>
> >>>>>It is too late, there will be no chance to save vmcore then.
> >>>>>
> >>>>>Also if it is possible to continue corrupt other area of oldmem because
> >>>>>of using old iommu tables then it will cause more problems.
> >>>>>
> >>>>>So I think the tables at least need some verifycation before being used.
> >>>>>
> >>>>
> >>>>Yes, it's a good thinking anout this and verification is also an
> >>>>interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> >>>>and then verify this again when panic happens in purgatory. This checks
> >>>>whether any code stomps into region reserved for kexec/kernel and corrupt
> >>>>the loaded kernel.
> >>>>
> >>>>If this is decided to do it should be an enhancement to current
> >>>>patchset but not a approach change. Since this patchset is going very
> >>>>close to point as maintainers expected maybe this can be merged firstly,
> >>>>then think about enhancement. After all without this patchset vt-d often
> >>>>raised error message, hung.
> >>>
> >>>It does not convince me, we should do it right at the beginning instead of
> >>>introduce something wrong.
> >>>
> >>>I wonder why the old dma can not be remap to a specific page in kdump kernel
> >>>so that it will not corrupt more memory. But I may missed something, I will
> >>>looking for old threads and catch up.
> >>>
> >>>Thanks
> >>>Dave
> >>>
> >>The (only) issue is not corruption, but once the iommu is re-configured, the old,
> >>not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
> >>will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
> >>in the kexec kernel.
> >>
> >
> >Don, so if iommu is not reconfigured then these faults will not happen?
> >
> Well, if iommu is not reconfigured, then if the crash isn't caused by
> an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
> them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.

So NMI_IOCK is one reason to cause kernel hang, I think I'm still not clear about
what does re-configured means though. DMAR faults will happen originally this is the old
behavior but we are removing the faults by alowing DMA continuing into old memory
space.

> 
> >Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
> >
> >intel_iommu_init()
> >{
> >...
> >
> >        dmar_table_init();
> >
> >        disable active iommu translations;
> >
> >        if (no_iommu || dmar_disabled)
> >                 goto out_free_dmar;
> >
> >...
> >}
> >
> >Any reason not move no_iommu check to the begining of intel_iommu_init function?
> >
> What does that do/help?

Just do not know why the previous handling is necessary with iommu=off, shouldn't
we do noting and return earlier?

Also there is a guess, dmar faults appears after iommu_init, so not sure if the codes
before dmar_disabled checking have some effect about enabling the faults messages.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/07/15 at 10:25am, Don Dutile wrote:
> On 05/07/2015 10:00 AM, Dave Young wrote:
> >On 04/07/15 at 10:12am, Don Dutile wrote:
> >>On 04/06/2015 11:46 PM, Dave Young wrote:
> >>>On 04/05/15 at 09:54am, Baoquan He wrote:
> >>>>On 04/03/15 at 05:21pm, Dave Young wrote:
> >>>>>On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
> >>>>>>Hi Dave,
> >>>>>>
> >>>>>>There may be some possibilities that the old iommu data is corrupted by
> >>>>>>some other modules. Currently we do not have a better solution for the
> >>>>>>dmar faults.
> >>>>>>
> >>>>>>But I think when this happens, we need to fix the module that corrupted
> >>>>>>the old iommu data. I once met a similar problem in normal kernel, the
> >>>>>>queue used by the qi_* functions was written again by another module.
> >>>>>>The fix was in that module, not in iommu module.
> >>>>>
> >>>>>It is too late, there will be no chance to save vmcore then.
> >>>>>
> >>>>>Also if it is possible to continue corrupt other area of oldmem because
> >>>>>of using old iommu tables then it will cause more problems.
> >>>>>
> >>>>>So I think the tables at least need some verifycation before being used.
> >>>>>
> >>>>
> >>>>Yes, it's a good thinking anout this and verification is also an
> >>>>interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
> >>>>and then verify this again when panic happens in purgatory. This checks
> >>>>whether any code stomps into region reserved for kexec/kernel and corrupt
> >>>>the loaded kernel.
> >>>>
> >>>>If this is decided to do it should be an enhancement to current
> >>>>patchset but not a approach change. Since this patchset is going very
> >>>>close to point as maintainers expected maybe this can be merged firstly,
> >>>>then think about enhancement. After all without this patchset vt-d often
> >>>>raised error message, hung.
> >>>
> >>>It does not convince me, we should do it right at the beginning instead of
> >>>introduce something wrong.
> >>>
> >>>I wonder why the old dma can not be remap to a specific page in kdump kernel
> >>>so that it will not corrupt more memory. But I may missed something, I will
> >>>looking for old threads and catch up.
> >>>
> >>>Thanks
> >>>Dave
> >>>
> >>The (only) issue is not corruption, but once the iommu is re-configured, the old,
> >>not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
> >>will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
> >>in the kexec kernel.
> >>
> >
> >Don, so if iommu is not reconfigured then these faults will not happen?
> >
> Well, if iommu is not reconfigured, then if the crash isn't caused by
> an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
> them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.

So NMI_IOCK is one reason to cause kernel hang, I think I'm still not clear about
what does re-configured means though. DMAR faults will happen originally this is the old
behavior but we are removing the faults by alowing DMA continuing into old memory
space.

> 
> >Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
> >
> >intel_iommu_init()
> >{
> >...
> >
> >        dmar_table_init();
> >
> >        disable active iommu translations;
> >
> >        if (no_iommu || dmar_disabled)
> >                 goto out_free_dmar;
> >
> >...
> >}
> >
> >Any reason not move no_iommu check to the begining of intel_iommu_init function?
> >
> What does that do/help?

Just do not know why the previous handling is necessary with iommu=off, shouldn't
we do noting and return earlier?

Also there is a guess, dmar faults appears after iommu_init, so not sure if the codes
before dmar_disabled checking have some effect about enabling the faults messages.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 10:00 AM, Dave Young wrote:
> On 04/07/15 at 10:12am, Don Dutile wrote:
>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>> Hi Dave,
>>>>>>
>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>> dmar faults.
>>>>>>
>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>> The fix was in that module, not in iommu module.
>>>>>
>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>
>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>> of using old iommu tables then it will cause more problems.
>>>>>
>>>>> So I think the tables at least need some verifycation before being used.
>>>>>
>>>>
>>>> Yes, it's a good thinking anout this and verification is also an
>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>> and then verify this again when panic happens in purgatory. This checks
>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>> the loaded kernel.
>>>>
>>>> If this is decided to do it should be an enhancement to current
>>>> patchset but not a approach change. Since this patchset is going very
>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>> then think about enhancement. After all without this patchset vt-d often
>>>> raised error message, hung.
>>>
>>> It does not convince me, we should do it right at the beginning instead of
>>> introduce something wrong.
>>>
>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>> so that it will not corrupt more memory. But I may missed something, I will
>>> looking for old threads and catch up.
>>>
>>> Thanks
>>> Dave
>>>
>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>> in the kexec kernel.
>>
>
> Don, so if iommu is not reconfigured then these faults will not happen?
>
> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>
> intel_iommu_init()
> {
> ...
>
>         dmar_table_init();
>
>         disable active iommu translations;
>
>         if (no_iommu || dmar_disabled)
>                  goto out_free_dmar;
>
> ...
> }
>
> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>
> Thanks
> Dave
Looks like you could.

> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 10:00 AM, Dave Young wrote:
> On 04/07/15 at 10:12am, Don Dutile wrote:
>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>> Hi Dave,
>>>>>>
>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>> dmar faults.
>>>>>>
>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>> The fix was in that module, not in iommu module.
>>>>>
>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>
>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>> of using old iommu tables then it will cause more problems.
>>>>>
>>>>> So I think the tables at least need some verifycation before being used.
>>>>>
>>>>
>>>> Yes, it's a good thinking anout this and verification is also an
>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>> and then verify this again when panic happens in purgatory. This checks
>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>> the loaded kernel.
>>>>
>>>> If this is decided to do it should be an enhancement to current
>>>> patchset but not a approach change. Since this patchset is going very
>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>> then think about enhancement. After all without this patchset vt-d often
>>>> raised error message, hung.
>>>
>>> It does not convince me, we should do it right at the beginning instead of
>>> introduce something wrong.
>>>
>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>> so that it will not corrupt more memory. But I may missed something, I will
>>> looking for old threads and catch up.
>>>
>>> Thanks
>>> Dave
>>>
>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>> in the kexec kernel.
>>
>
> Don, so if iommu is not reconfigured then these faults will not happen?
>
> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>
> intel_iommu_init()
> {
> ...
>
>         dmar_table_init();
>
>         disable active iommu translations;
>
>         if (no_iommu || dmar_disabled)
>                  goto out_free_dmar;
>
> ...
> }
>
> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>
> Thanks
> Dave
Looks like you could.

> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 09:21 PM, Dave Young wrote:
> On 05/07/15 at 10:25am, Don Dutile wrote:
>> On 05/07/2015 10:00 AM, Dave Young wrote:
>>> On 04/07/15 at 10:12am, Don Dutile wrote:
>>>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>>>> Hi Dave,
>>>>>>>>
>>>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>>>> dmar faults.
>>>>>>>>
>>>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>>>> The fix was in that module, not in iommu module.
>>>>>>>
>>>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>>>
>>>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>>>> of using old iommu tables then it will cause more problems.
>>>>>>>
>>>>>>> So I think the tables at least need some verifycation before being used.
>>>>>>>
>>>>>>
>>>>>> Yes, it's a good thinking anout this and verification is also an
>>>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>>>> and then verify this again when panic happens in purgatory. This checks
>>>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>>>> the loaded kernel.
>>>>>>
>>>>>> If this is decided to do it should be an enhancement to current
>>>>>> patchset but not a approach change. Since this patchset is going very
>>>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>>>> then think about enhancement. After all without this patchset vt-d often
>>>>>> raised error message, hung.
>>>>>
>>>>> It does not convince me, we should do it right at the beginning instead of
>>>>> introduce something wrong.
>>>>>
>>>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>>>> so that it will not corrupt more memory. But I may missed something, I will
>>>>> looking for old threads and catch up.
>>>>>
>>>>> Thanks
>>>>> Dave
>>>>>
>>>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>>>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>>>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>>>> in the kexec kernel.
>>>>
>>>
>>> Don, so if iommu is not reconfigured then these faults will not happen?
>>>
>> Well, if iommu is not reconfigured, then if the crash isn't caused by
>> an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
>> them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.
>
> So NMI_IOCK is one reason to cause kernel hang, I think I'm still not clear about
> what does re-configured means though. DMAR faults will happen originally this is the old
> behavior but we are removing the faults by alowing DMA continuing into old memory
> space.
>
A flood of faults occur when the 2nd kernel (re-)configures the IOMMU because
the second kernel effectively clears/disable all DMA except RMRRs, so any DMA from 1st kernel will flood
the system with faults.  Its the flood of dmar faults that eventually wedges &/or crashes the system
while trying to take a kdump.

>>
>>> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>>>
>>> intel_iommu_init()
>>> {
>>> ...
>>>
>>>         dmar_table_init();
>>>
>>>         disable active iommu translations;
>>>
>>>         if (no_iommu || dmar_disabled)
>>>                  goto out_free_dmar;
>>>
>>> ...
>>> }
>>>
>>> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>>>
>> What does that do/help?
>
> Just do not know why the previous handling is necessary with iommu=off, shouldn't
> we do noting and return earlier?
>
> Also there is a guess, dmar faults appears after iommu_init, so not sure if the codes
> before dmar_disabled checking have some effect about enabling the faults messages.
>
> Thanks
> Dave
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 09:21 PM, Dave Young wrote:
> On 05/07/15 at 10:25am, Don Dutile wrote:
>> On 05/07/2015 10:00 AM, Dave Young wrote:
>>> On 04/07/15 at 10:12am, Don Dutile wrote:
>>>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>>>> Hi Dave,
>>>>>>>>
>>>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>>>> dmar faults.
>>>>>>>>
>>>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>>>> The fix was in that module, not in iommu module.
>>>>>>>
>>>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>>>
>>>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>>>> of using old iommu tables then it will cause more problems.
>>>>>>>
>>>>>>> So I think the tables at least need some verifycation before being used.
>>>>>>>
>>>>>>
>>>>>> Yes, it's a good thinking anout this and verification is also an
>>>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>>>> and then verify this again when panic happens in purgatory. This checks
>>>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>>>> the loaded kernel.
>>>>>>
>>>>>> If this is decided to do it should be an enhancement to current
>>>>>> patchset but not a approach change. Since this patchset is going very
>>>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>>>> then think about enhancement. After all without this patchset vt-d often
>>>>>> raised error message, hung.
>>>>>
>>>>> It does not convince me, we should do it right at the beginning instead of
>>>>> introduce something wrong.
>>>>>
>>>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>>>> so that it will not corrupt more memory. But I may missed something, I will
>>>>> looking for old threads and catch up.
>>>>>
>>>>> Thanks
>>>>> Dave
>>>>>
>>>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>>>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>>>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>>>> in the kexec kernel.
>>>>
>>>
>>> Don, so if iommu is not reconfigured then these faults will not happen?
>>>
>> Well, if iommu is not reconfigured, then if the crash isn't caused by
>> an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
>> them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.
>
> So NMI_IOCK is one reason to cause kernel hang, I think I'm still not clear about
> what does re-configured means though. DMAR faults will happen originally this is the old
> behavior but we are removing the faults by alowing DMA continuing into old memory
> space.
>
A flood of faults occur when the 2nd kernel (re-)configures the IOMMU because
the second kernel effectively clears/disable all DMA except RMRRs, so any DMA from 1st kernel will flood
the system with faults.  Its the flood of dmar faults that eventually wedges &/or crashes the system
while trying to take a kdump.

>>
>>> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>>>
>>> intel_iommu_init()
>>> {
>>> ...
>>>
>>>         dmar_table_init();
>>>
>>>         disable active iommu translations;
>>>
>>>         if (no_iommu || dmar_disabled)
>>>                  goto out_free_dmar;
>>>
>>> ...
>>> }
>>>
>>> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>>>
>> What does that do/help?
>
> Just do not know why the previous handling is necessary with iommu=off, shouldn't
> we do noting and return earlier?
>
> Also there is a guess, dmar faults appears after iommu_init, so not sure if the codes
> before dmar_disabled checking have some effect about enabling the faults messages.
>
> Thanks
> Dave
>

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Don Dutile]

On 05/07/2015 09:21 PM, Dave Young wrote:
> On 05/07/15 at 10:25am, Don Dutile wrote:
>> On 05/07/2015 10:00 AM, Dave Young wrote:
>>> On 04/07/15 at 10:12am, Don Dutile wrote:
>>>> On 04/06/2015 11:46 PM, Dave Young wrote:
>>>>> On 04/05/15 at 09:54am, Baoquan He wrote:
>>>>>> On 04/03/15 at 05:21pm, Dave Young wrote:
>>>>>>> On 04/03/15 at 05:01pm, Li, ZhenHua wrote:
>>>>>>>> Hi Dave,
>>>>>>>>
>>>>>>>> There may be some possibilities that the old iommu data is corrupted by
>>>>>>>> some other modules. Currently we do not have a better solution for the
>>>>>>>> dmar faults.
>>>>>>>>
>>>>>>>> But I think when this happens, we need to fix the module that corrupted
>>>>>>>> the old iommu data. I once met a similar problem in normal kernel, the
>>>>>>>> queue used by the qi_* functions was written again by another module.
>>>>>>>> The fix was in that module, not in iommu module.
>>>>>>>
>>>>>>> It is too late, there will be no chance to save vmcore then.
>>>>>>>
>>>>>>> Also if it is possible to continue corrupt other area of oldmem because
>>>>>>> of using old iommu tables then it will cause more problems.
>>>>>>>
>>>>>>> So I think the tables at least need some verifycation before being used.
>>>>>>>
>>>>>>
>>>>>> Yes, it's a good thinking anout this and verification is also an
>>>>>> interesting idea. kexec/kdump do a sha256 calculation on loaded kernel
>>>>>> and then verify this again when panic happens in purgatory. This checks
>>>>>> whether any code stomps into region reserved for kexec/kernel and corrupt
>>>>>> the loaded kernel.
>>>>>>
>>>>>> If this is decided to do it should be an enhancement to current
>>>>>> patchset but not a approach change. Since this patchset is going very
>>>>>> close to point as maintainers expected maybe this can be merged firstly,
>>>>>> then think about enhancement. After all without this patchset vt-d often
>>>>>> raised error message, hung.
>>>>>
>>>>> It does not convince me, we should do it right at the beginning instead of
>>>>> introduce something wrong.
>>>>>
>>>>> I wonder why the old dma can not be remap to a specific page in kdump kernel
>>>>> so that it will not corrupt more memory. But I may missed something, I will
>>>>> looking for old threads and catch up.
>>>>>
>>>>> Thanks
>>>>> Dave
>>>>>
>>>> The (only) issue is not corruption, but once the iommu is re-configured, the old,
>>>> not-stopped-yet, dma engines will use iova's that will generate dmar faults, which
>>>> will be enabled when the iommu is re-configured (even to a single/simple paging scheme)
>>>> in the kexec kernel.
>>>>
>>>
>>> Don, so if iommu is not reconfigured then these faults will not happen?
>>>
>> Well, if iommu is not reconfigured, then if the crash isn't caused by
>> an IOMMU fault (some systems have firmware-first catch the IOMMU fault & convert
>> them into NMI_IOCK), then the DMA's will continue into the old kernel memory space.
>
> So NMI_IOCK is one reason to cause kernel hang, I think I'm still not clear about
> what does re-configured means though. DMAR faults will happen originally this is the old
> behavior but we are removing the faults by alowing DMA continuing into old memory
> space.
>
A flood of faults occur when the 2nd kernel (re-)configures the IOMMU because
the second kernel effectively clears/disable all DMA except RMRRs, so any DMA from 1st kernel will flood
the system with faults.  Its the flood of dmar faults that eventually wedges &/or crashes the system
while trying to take a kdump.

>>
>>> Baoquan and me has a confusion below today about iommu=off/intel_iommu=off:
>>>
>>> intel_iommu_init()
>>> {
>>> ...
>>>
>>>         dmar_table_init();
>>>
>>>         disable active iommu translations;
>>>
>>>         if (no_iommu || dmar_disabled)
>>>                  goto out_free_dmar;
>>>
>>> ...
>>> }
>>>
>>> Any reason not move no_iommu check to the begining of intel_iommu_init function?
>>>
>> What does that do/help?
>
> Just do not know why the previous handling is necessary with iommu=off, shouldn't
> we do noting and return earlier?
>
> Also there is a guess, dmar faults appears after iommu_init, so not sure if the codes
> before dmar_disabled checking have some effect about enabling the faults messages.
>
> Thanks
> Dave
>


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Joerg Roedel]

On Thu, May 07, 2015 at 09:56:00PM +0800, Dave Young wrote:
> Joreg, I can not find the last reply from you, so just reply here about
> my worries here.
> 
> I said that the patchset will cause more problems, let me explain about
> it more here:
> 
> Suppose page table was corrupted, ie. original mapping iova1 -> page 1
> it was changed to iova1 -> page 2 accidently while crash happening,
> thus future dma will read/write page 2 instead page 1, right?

When the page-table is corrupted then it is a left-over from the old
kernel. When the kdump kernel boots the situation can at least not get
worse. For the page tables it is also hard to detect wrong mappings (if
this would be possible the hardware could already do it), so any checks
we could do there are of limited use anyway.


	Joerg

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/11/15 at 12:11pm, Joerg Roedel wrote:
> On Thu, May 07, 2015 at 09:56:00PM +0800, Dave Young wrote:
> > Joreg, I can not find the last reply from you, so just reply here about
> > my worries here.
> > 
> > I said that the patchset will cause more problems, let me explain about
> > it more here:
> > 
> > Suppose page table was corrupted, ie. original mapping iova1 -> page 1
> > it was changed to iova1 -> page 2 accidently while crash happening,
> > thus future dma will read/write page 2 instead page 1, right?
> 
> When the page-table is corrupted then it is a left-over from the old
> kernel. When the kdump kernel boots the situation can at least not get
> worse. For the page tables it is also hard to detect wrong mappings (if
> this would be possible the hardware could already do it), so any checks
> we could do there are of limited use anyway.

Joerg, since both of you guys do not think it is a problem I will object it 
any more though I still do not like reusing the old page tables. So let's
leave it as a future issue.

Thanks
Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/11/15 at 12:11pm, Joerg Roedel wrote:
> On Thu, May 07, 2015 at 09:56:00PM +0800, Dave Young wrote:
> > Joreg, I can not find the last reply from you, so just reply here about
> > my worries here.
> > 
> > I said that the patchset will cause more problems, let me explain about
> > it more here:
> > 
> > Suppose page table was corrupted, ie. original mapping iova1 -> page 1
> > it was changed to iova1 -> page 2 accidently while crash happening,
> > thus future dma will read/write page 2 instead page 1, right?
> 
> When the page-table is corrupted then it is a left-over from the old
> kernel. When the kdump kernel boots the situation can at least not get
> worse. For the page tables it is also hard to detect wrong mappings (if
> this would be possible the hardware could already do it), so any checks
> we could do there are of limited use anyway.

Joerg, since both of you guys do not think it is a problem I will object it 
any more though I still do not like reusing the old page tables. So let's
leave it as a future issue.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/12/15 at 01:57pm, Dave Young wrote:
> On 05/11/15 at 12:11pm, Joerg Roedel wrote:
> > On Thu, May 07, 2015 at 09:56:00PM +0800, Dave Young wrote:
> > > Joreg, I can not find the last reply from you, so just reply here about
> > > my worries here.
> > > 
> > > I said that the patchset will cause more problems, let me explain about
> > > it more here:
> > > 
> > > Suppose page table was corrupted, ie. original mapping iova1 -> page 1
> > > it was changed to iova1 -> page 2 accidently while crash happening,
> > > thus future dma will read/write page 2 instead page 1, right?
> > 
> > When the page-table is corrupted then it is a left-over from the old
> > kernel. When the kdump kernel boots the situation can at least not get
> > worse. For the page tables it is also hard to detect wrong mappings (if
> > this would be possible the hardware could already do it), so any checks
> > we could do there are of limited use anyway.
> 
> Joerg, since both of you guys do not think it is a problem I will object it 

s/will object/will not object

> any more though I still do not like reusing the old page tables. So let's
> leave it as a future issue.
> 
> Thanks
> Dave

Re: [PATCH v9 0/10] iommu/vt-d: Fix intel vt-d faults in kdump kernel

[Dave Young]

On 05/12/15 at 01:57pm, Dave Young wrote:
> On 05/11/15 at 12:11pm, Joerg Roedel wrote:
> > On Thu, May 07, 2015 at 09:56:00PM +0800, Dave Young wrote:
> > > Joreg, I can not find the last reply from you, so just reply here about
> > > my worries here.
> > > 
> > > I said that the patchset will cause more problems, let me explain about
> > > it more here:
> > > 
> > > Suppose page table was corrupted, ie. original mapping iova1 -> page 1
> > > it was changed to iova1 -> page 2 accidently while crash happening,
> > > thus future dma will read/write page 2 instead page 1, right?
> > 
> > When the page-table is corrupted then it is a left-over from the old
> > kernel. When the kdump kernel boots the situation can at least not get
> > worse. For the page tables it is also hard to detect wrong mappings (if
> > this would be possible the hardware could already do it), so any checks
> > we could do there are of limited use anyway.
> 
> Joerg, since both of you guys do not think it is a problem I will object it 

s/will object/will not object

> any more though I still do not like reusing the old page tables. So let's
> leave it as a future issue.
> 
> Thanks
> Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec