From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alok Kataria Subject: swiotlb detection should be memory hotplug aware ? Date: Fri, 12 Mar 2010 18:07:12 -0800 Message-ID: <1268446032.13691.69.camel@ank32> Reply-To: akataria@vmware.com Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Return-path: Received: from smtp-outbound-1.vmware.com ([65.115.85.69]:33046 "EHLO smtp-outbound-1.vmware.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753996Ab0CMCHN (ORCPT ); Fri, 12 Mar 2010 21:07:13 -0500 Sender: linux-acpi-owner@vger.kernel.org List-Id: linux-acpi@vger.kernel.org To: Len Brown , Andi Kleen , the arch/x86 maintainers Cc: linux-acpi , LKML , Petr Vandrovec Hi, Looking at the current code swiotlb is initialized for 64bit kernels only when the max_pfn value is greater than 4G (MAX_DMA32_PFN value). So in cases when the initial memory is less than 4GB the kernel boots without enabling swiotlb, when we hotadd memory to such a kernel and go beyond the 4G limit, swiotlb is still disabled. As a result when any 32bit devices start using this newly added memory beyond 4G, the kernel starts spitting error messages like below or in some cases it causes kernel panics. <3>[ 815.921504] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff <3>[ 815.944860] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff <3>[ 815.968808] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff <3>[ 815.992821] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff <3>[ 816.016796] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff For systems which have no HW-IOMMU but are capable of memory hotadd this can be a potential problem. IMO, there can be few possible solutions to this. 1. Enable swiotlb for all 64bit kernels which have memory hot-add support. 2. Instead of checking the max_pfn value in pci_swiotlb_detect, check for max_hotpluggable_pfn (or some such) value. Though I don't see such a value readily available. I could parse the SRAT and get hotplug memory information but that will make swiotlb detection logic a little too complex. A quick look around srat_xx.c files and the acpi_memhotplug module didn't find any useful API that could be used directly either. So was wondering if any of you are aware of an easy way to get such information ? Let me know if you have any other ideas as well. Thanks in advance, Alok