From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753210AbcHQUd5 (ORCPT ); Wed, 17 Aug 2016 16:33:57 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:36178 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752435AbcHQUd4 (ORCPT ); Wed, 17 Aug 2016 16:33:56 -0400 Date: Wed, 17 Aug 2016 16:32:38 -0400 From: Konrad Rzeszutek Wilk To: xen-devel@lists.xensource.com, chuck.anderson@oracle.com, Boris Ostrovsky , david.vrabel@citrix.com, jgross@suse.com, jbeulich@suse.com, stefan.bader@canonical.com Cc: linux-kernel@vger.kernel.org Subject: XSA 154 and ISA region (640K -> 1MB) WB cache instead of UC Message-ID: <20160817203238.GA9408@char.us.oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.6.2 (2016-07-01) X-Source-IP: userv0021.oracle.com [156.151.31.71] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hey Jan, et. al., One of the interesting things about XSA 154 fix ("x86: enforce consistent cachability of MMIO mappings") is that when certain applications (mcelog) are trying to map /dev/mmap and lurk in ISA regions - we get: [ 49.399053] WARNING: CPU: 0 PID: 2471 at arch/x86/mm/pat.c:913 untrack_pfn+0x93/0xc0() [ 49.399055] Modules linked in: bnx2fc fcoe libfcoe libfc 8021q mrp garp stp llc bonding dm_multipath vfat fat iTCO_wdt iTCO_vendor_support pcspkr ipmi_devintf ipmi_si ipmi_msghandler sb_edac edac_core i2c_i801 i2c_core lpc_ich mfd_core shpchp ioatdma sg ext4 jbd2 mbcache sr_mod cdrom sd_mod usb_storage ahci libahci megaraid_sas qla2xxx scsi_transport_fc crc32c_intel be2iscsi bnx2i cnic uio cxgb4i cxgb4 cxgb3i libcxgbi ipv6 cxgb3 libiscsi_tcp qla4xxx iscsi_boot_sysfs libiscsi scsi_transport_iscsi ixgbe dca ptp pps_core vxlan udp_tunnel ip6_udp_tunnel mdio dm_mirror dm_region_hash dm_log dm_mod [ 49.399131] CPU: 0 PID: 2471 Comm: mcelog Not tainted 4.1 [ 49.399134] Hardware name: Oracle Corporation SUN SERVER X4-2 /ASSY,MB,X4-2, 1U , BIOS 25030100 04/15/2015 [ 49.399138] 0000000000000000 ffff880074673c28 ffffffff816c66f0 0000000000000000 [ 49.399143] 0000000000000391 ffff880074673c68 ffffffff81084745 ffff880074673c78 [ 49.399148] ffff88014b625db0 0000000000000000 ffff880074673d58 00007f39290ab000 [ 49.399152] Call Trace: [ 49.399166] [] dump_stack+0x63/0x83 [ 49.399175] [] warn_slowpath_common+0x95/0xe0 [ 49.399180] [] warn_slowpath_null+0x1a/0x20 [ 49.399183] [] untrack_pfn+0x93/0xc0 [ 49.399190] [] unmap_single_vma+0xa9/0x100 [ 49.399194] [] unmap_vmas+0x54/0xa0 [ 49.399199] [] exit_mmap+0x9a/0x150 [ 49.399204] [] mmput+0x73/0x110 [ 49.399208] [] dup_mm+0x105/0x110 [ 49.399213] [] copy_process+0x11ed/0x1240 [ 49.399218] [] do_fork+0x79/0x280 [ 49.399226] [] ? syscall_trace_enter_phase1+0x153/0x180 [ 49.399231] [] SyS_clone+0x16/0x20 [ 49.399235] [] system_call_fastpath+0x12/0x71 [ 49.399239] ---[ end trace a61cd3d271a53a54 ]--- The reason is that Linux kernel assumes that the range from 640KB -> 1MB can be mapped as write-back (see is_new_memtype_allowed and x86_platform.is_untracked_pat_range). But we enforce the uncached mode and Linux complains. With the mmio-relax=1, Linux gets its way and is happy. With the patch below: diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 70a38c1..e5ff5a5 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -288,6 +287,12 @@ static void __init xen_banner(void) version >> 16, version & 0xffff, extra.extraversion, xen_feature(XENFEAT_mmu_pt_update_preserve_ad) ? " (preserve-AD)" : ""); } + +static bool xen_ignore(u64 s, u64 e) +{ + return false; +} + /* Check if running on Xen version (major, minor) or later */ bool xen_running_on_version_or_later(unsigned int major, unsigned int minor) @@ -1563,7 +1570,7 @@ asmlinkage __visible void __init xen_start_kernel(void) x86_init.resources.memory_setup = xen_memory_setup; x86_init.oem.arch_setup = xen_arch_setup; x86_init.oem.banner = xen_banner; - + x86_platform.is_untracked_pat_range = xen_ignore; xen_init_time_ops(); /* Things work much better - as we don't treat the 640KB->1MB region specially. Anyhow what I am wondering: a) Should we add a edge case in the hypervisor to allow multiple mappings for this region? I am thinking no.. but it sounds like mapping ISA region as WB is safe even in baremetal? b) Or would it be better to let Linux do its thing and treat 640KB->1MB as uncached instead of writeback? Looking at the kernel it assumes that WB is ok for 640KB->1MB. The comment says: " /* Low ISA region is always mapped WB in page table. No need to track *" which is probably true on baremetal. But with Xen PV: 856 /* 857 * In domU, the ISA region is normal, usable memory, but we 858 * reserve ISA memory anyway because too many things poke 859 * about in there. 860 */ 861 e820_add_region(ISA_START_ADDRESS, ISA_END_ADDRESS - ISA_START_ADDRESS, 862 E820_RESERVED); which would imply we don't have any page table mappings. And then the quick fix I provided above looks like the right solution? CC-ing Boris, Daniel, Juergen, Steve, and Chuck. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: XSA 154 and ISA region (640K -> 1MB) WB cache instead of UC Date: Wed, 17 Aug 2016 16:32:38 -0400 Message-ID: <20160817203238.GA9408@char.us.oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: xen-devel@lists.xensource.com, chuck.anderson@oracle.com, Boris Ostrovsky , david.vrabel@citrix.com, jgross@suse.com, jbeulich@suse.com, stefan.bader@canonical.com Cc: linux-kernel@vger.kernel.org List-Id: xen-devel@lists.xenproject.org SGV5IEphbiwgZXQuIGFsLiwKCk9uZSBvZiB0aGUgaW50ZXJlc3RpbmcgdGhpbmdzIGFib3V0IFhT QSAxNTQgZml4ICgieDg2OiBlbmZvcmNlIGNvbnNpc3RlbnQKY2FjaGFiaWxpdHkgb2YgTU1JTyBt YXBwaW5ncyIpIGlzIHRoYXQgd2hlbiBjZXJ0YWluIGFwcGxpY2F0aW9ucyAobWNlbG9nKQphcmUg dHJ5aW5nIHRvIG1hcCAvZGV2L21tYXAgYW5kIGx1cmsgaW4gSVNBIHJlZ2lvbnMgLSB3ZSBnZXQ6 CgpbICAgNDkuMzk5MDUzXSBXQVJOSU5HOiBDUFU6IDAgUElEOiAyNDcxIGF0IGFyY2gveDg2L21t L3BhdC5jOjkxMyB1bnRyYWNrX3BmbisweDkzLzB4YzAoKQpbICAgNDkuMzk5MDU1XSBNb2R1bGVz IGxpbmtlZCBpbjogYm54MmZjIGZjb2UgbGliZmNvZSBsaWJmYyA4MDIxcSBtcnAgZ2FycCBzdHAg bGxjIGJvbmRpbmcgZG1fbXVsdGlwYXRoIHZmYXQgZmF0IGlUQ09fd2R0IGlUQ09fdmVuZG9yX3N1 cHBvcnQgcGNzcGtyIGlwbWlfZGV2aW50ZiBpcG1pX3NpIGlwbWlfbXNnaGFuZGxlciBzYl9lZGFj IGVkYWNfY29yZSBpMmNfaTgwMSBpMmNfY29yZSBscGNfaWNoIG1mZF9jb3JlIHNocGNocCBpb2F0 ZG1hIHNnIGV4dDQgamJkMiBtYmNhY2hlIHNyX21vZCBjZHJvbSBzZF9tb2QgdXNiX3N0b3JhZ2Ug YWhjaSBsaWJhaGNpIG1lZ2FyYWlkX3NhcyBxbGEyeHh4IHNjc2lfdHJhbnNwb3J0X2ZjIGNyYzMy Y19pbnRlbCBiZTJpc2NzaSBibngyaSBjbmljIHVpbyBjeGdiNGkgY3hnYjQgY3hnYjNpIGxpYmN4 Z2JpIGlwdjYgY3hnYjMgbGliaXNjc2lfdGNwIHFsYTR4eHggaXNjc2lfYm9vdF9zeXNmcyBsaWJp c2NzaSBzY3NpX3RyYW5zcG9ydF9pc2NzaSBpeGdiZSBkY2EgcHRwIHBwc19jb3JlIHZ4bGFuIHVk cF90dW5uZWwgaXA2X3VkcF90dW5uZWwgbWRpbyBkbV9taXJyb3IgZG1fcmVnaW9uX2hhc2ggZG1f bG9nIGRtX21vZApbICAgNDkuMzk5MTMxXSBDUFU6IDAgUElEOiAyNDcxIENvbW06IG1jZWxvZyBO b3QgdGFpbnRlZCA0LjEKWyAgIDQ5LjM5OTEzNF0gSGFyZHdhcmUgbmFtZTogT3JhY2xlIENvcnBv cmF0aW9uIFNVTiBTRVJWRVIgWDQtMiAgICAgICAvQVNTWSxNQixYNC0yLCAxVSAgICAgICwgQklP UyAyNTAzMDEwMCAwNC8xNS8yMDE1ClsgICA0OS4zOTkxMzhdICAwMDAwMDAwMDAwMDAwMDAwIGZm ZmY4ODAwNzQ2NzNjMjggZmZmZmZmZmY4MTZjNjZmMCAwMDAwMDAwMDAwMDAwMDAwClsgICA0OS4z OTkxNDNdICAwMDAwMDAwMDAwMDAwMzkxIGZmZmY4ODAwNzQ2NzNjNjggZmZmZmZmZmY4MTA4NDc0 NSBmZmZmODgwMDc0NjczYzc4ClsgICA0OS4zOTkxNDhdICBmZmZmODgwMTRiNjI1ZGIwIDAwMDAw MDAwMDAwMDAwMDAgZmZmZjg4MDA3NDY3M2Q1OCAwMDAwN2YzOTI5MGFiMDAwClsgICA0OS4zOTkx NTJdIENhbGwgVHJhY2U6ClsgICA0OS4zOTkxNjZdICBbPGZmZmZmZmZmODE2YzY2ZjA+XSBkdW1w X3N0YWNrKzB4NjMvMHg4MwpbICAgNDkuMzk5MTc1XSAgWzxmZmZmZmZmZjgxMDg0NzQ1Pl0gd2Fy bl9zbG93cGF0aF9jb21tb24rMHg5NS8weGUwClsgICA0OS4zOTkxODBdICBbPGZmZmZmZmZmODEw ODQ3YWE+XSB3YXJuX3Nsb3dwYXRoX251bGwrMHgxYS8weDIwClsgICA0OS4zOTkxODNdICBbPGZm ZmZmZmZmODEwNzI1ZjM+XSB1bnRyYWNrX3BmbisweDkzLzB4YzAKWyAgIDQ5LjM5OTE5MF0gIFs8 ZmZmZmZmZmY4MTFiOTBmOT5dIHVubWFwX3NpbmdsZV92bWErMHhhOS8weDEwMApbICAgNDkuMzk5 MTk0XSAgWzxmZmZmZmZmZjgxMWI5NjQ0Pl0gdW5tYXBfdm1hcysweDU0LzB4YTAKWyAgIDQ5LjM5 OTE5OV0gIFs8ZmZmZmZmZmY4MTFiZjBkYT5dIGV4aXRfbW1hcCsweDlhLzB4MTUwClsgICA0OS4z OTkyMDRdICBbPGZmZmZmZmZmODEwODI1ZDM+XSBtbXB1dCsweDczLzB4MTEwClsgICA0OS4zOTky MDhdICBbPGZmZmZmZmZmODEwODI3NzU+XSBkdXBfbW0rMHgxMDUvMHgxMTAKWyAgIDQ5LjM5OTIx M10gIFs8ZmZmZmZmZmY4MTA4M2IxZD5dIGNvcHlfcHJvY2VzcysweDExZWQvMHgxMjQwClsgICA0 OS4zOTkyMThdICBbPGZmZmZmZmZmODEwODQwMDk+XSBkb19mb3JrKzB4NzkvMHgyODAKWyAgIDQ5 LjM5OTIyNl0gIFs8ZmZmZmZmZmY4MTAyNTlkMz5dID8gc3lzY2FsbF90cmFjZV9lbnRlcl9waGFz ZTErMHgxNTMvMHgxODAKWyAgIDQ5LjM5OTIzMV0gIFs8ZmZmZmZmZmY4MTA4NDIyNj5dIFN5U19j bG9uZSsweDE2LzB4MjAKWyAgIDQ5LjM5OTIzNV0gIFs8ZmZmZmZmZmY4MTZjYjNlZT5dIHN5c3Rl bV9jYWxsX2Zhc3RwYXRoKzB4MTIvMHg3MQpbICAgNDkuMzk5MjM5XSAtLS1bIGVuZCB0cmFjZSBh NjFjZDNkMjcxYTUzYTU0IF0tLS0KClRoZSByZWFzb24gaXMgdGhhdCBMaW51eCBrZXJuZWwgYXNz dW1lcyB0aGF0IHRoZSByYW5nZSBmcm9tIDY0MEtCIC0+IDFNQiBjYW4KYmUgbWFwcGVkIGFzIHdy aXRlLWJhY2sgKHNlZSBpc19uZXdfbWVtdHlwZV9hbGxvd2VkIGFuZCB4ODZfcGxhdGZvcm0uaXNf dW50cmFja2VkX3BhdF9yYW5nZSkuCkJ1dCB3ZSBlbmZvcmNlIHRoZSB1bmNhY2hlZCBtb2RlIGFu ZCBMaW51eCBjb21wbGFpbnMuCgpXaXRoIHRoZSBtbWlvLXJlbGF4PTEsIExpbnV4IGdldHMgaXRz IHdheSBhbmQgaXMgaGFwcHkuCgpXaXRoIHRoZSBwYXRjaCBiZWxvdzoKCmRpZmYgLS1naXQgYS9h cmNoL3g4Ni94ZW4vZW5saWdodGVuLmMgYi9hcmNoL3g4Ni94ZW4vZW5saWdodGVuLmMKaW5kZXgg NzBhMzhjMS4uZTVmZjVhNSAxMDA2NDQKLS0tIGEvYXJjaC94ODYveGVuL2VubGlnaHRlbi5jCisr KyBiL2FyY2gveDg2L3hlbi9lbmxpZ2h0ZW4uYwpAQCAtMjg4LDYgKzI4NywxMiBAQCBzdGF0aWMg dm9pZCBfX2luaXQgeGVuX2Jhbm5lcih2b2lkKQogCSAgICAgICB2ZXJzaW9uID4+IDE2LCB2ZXJz aW9uICYgMHhmZmZmLCBleHRyYS5leHRyYXZlcnNpb24sCiAJICAgICAgIHhlbl9mZWF0dXJlKFhF TkZFQVRfbW11X3B0X3VwZGF0ZV9wcmVzZXJ2ZV9hZCkgPyAiIChwcmVzZXJ2ZS1BRCkiIDogIiIp OwogfQorCitzdGF0aWMgYm9vbCB4ZW5faWdub3JlKHU2NCBzLCB1NjQgZSkKK3sKKwlyZXR1cm4g ZmFsc2U7Cit9CisKIC8qIENoZWNrIGlmIHJ1bm5pbmcgb24gWGVuIHZlcnNpb24gKG1ham9yLCBt aW5vcikgb3IgbGF0ZXIgKi8KIGJvb2wKIHhlbl9ydW5uaW5nX29uX3ZlcnNpb25fb3JfbGF0ZXIo dW5zaWduZWQgaW50IG1ham9yLCB1bnNpZ25lZCBpbnQgbWlub3IpCkBAIC0xNTYzLDcgKzE1NzAs NyBAQCBhc21saW5rYWdlIF9fdmlzaWJsZSB2b2lkIF9faW5pdCB4ZW5fc3RhcnRfa2VybmVsKHZv aWQpCiAJCXg4Nl9pbml0LnJlc291cmNlcy5tZW1vcnlfc2V0dXAgPSB4ZW5fbWVtb3J5X3NldHVw OwogCXg4Nl9pbml0Lm9lbS5hcmNoX3NldHVwID0geGVuX2FyY2hfc2V0dXA7CiAJeDg2X2luaXQu b2VtLmJhbm5lciA9IHhlbl9iYW5uZXI7Ci0KKwl4ODZfcGxhdGZvcm0uaXNfdW50cmFja2VkX3Bh dF9yYW5nZSA9IHhlbl9pZ25vcmU7CiAJeGVuX2luaXRfdGltZV9vcHMoKTsKIAogCS8qCgpUaGlu Z3Mgd29yayBtdWNoIGJldHRlciAtIGFzIHdlIGRvbid0IHRyZWF0IHRoZSA2NDBLQi0+MU1CIHJl Z2lvbiBzcGVjaWFsbHkuCgpBbnlob3cgd2hhdCBJIGFtIHdvbmRlcmluZzoKCiBhKSBTaG91bGQg d2UgYWRkIGEgZWRnZSBjYXNlIGluIHRoZSBoeXBlcnZpc29yIHRvIGFsbG93IG11bHRpcGxlIG1h cHBpbmdzCiAgIGZvciB0aGlzIHJlZ2lvbj8gSSBhbSB0aGlua2luZyBuby4uIGJ1dCBpdCBzb3Vu ZHMgbGlrZSBtYXBwaW5nIElTQSByZWdpb24KICAgYXMgV0IgaXMgc2FmZSBldmVuIGluIGJhcmVt ZXRhbD8KCiBiKSBPciB3b3VsZCBpdCBiZSBiZXR0ZXIgdG8gbGV0IExpbnV4IGRvIGl0cyB0aGlu ZyBhbmQgdHJlYXQgNjQwS0ItPjFNQgogICBhcyB1bmNhY2hlZCBpbnN0ZWFkIG9mIHdyaXRlYmFj az8KCiAgIExvb2tpbmcgYXQgdGhlIGtlcm5lbCBpdCBhc3N1bWVzIHRoYXQgV0IgaXMgb2sgZm9y IDY0MEtCLT4xTUIuCiAgIFRoZSBjb21tZW50IHNheXM6CiAgICIgLyogTG93IElTQSByZWdpb24g aXMgYWx3YXlzIG1hcHBlZCBXQiBpbiBwYWdlIHRhYmxlLiBObyBuZWVkIHRvIHRyYWNrICoiCgog ICB3aGljaCBpcyBwcm9iYWJseSB0cnVlIG9uIGJhcmVtZXRhbC4gQnV0IHdpdGggWGVuIFBWOgoK IDg1NiAgICAgICAgIC8qICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogODU3ICAgICAgICAgICogSW4gZG9tVSwgdGhl IElTQSByZWdpb24gaXMgbm9ybWFsLCB1c2FibGUgbWVtb3J5LCBidXQgd2UgICAgICAgICAgICAg CiA4NTggICAgICAgICAgKiByZXNlcnZlIElTQSBtZW1vcnkgYW55d2F5IGJlY2F1c2UgdG9vIG1h bnkgdGhpbmdzIHBva2UgICAgICAgICAgICAgICAKIDg1OSAgICAgICAgICAqIGFib3V0IGluIHRo ZXJlLiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IAogODYwICAgICAgICAgICovICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiA4NjEgICAgICAgICBlODIwX2FkZF9yZWdp b24oSVNBX1NUQVJUX0FERFJFU1MsIElTQV9FTkRfQUREUkVTUyAtIElTQV9TVEFSVF9BRERSRVNT LCAKIDg2MiAgICAgICAgICAgICAgICAgICAgICAgICBFODIwX1JFU0VSVkVEKTsgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAKCiAgIHdoaWNoIHdvdWxkIGltcGx5IHdlIGRvbid0IGhh dmUgYW55IHBhZ2UgdGFibGUgbWFwcGluZ3MuCgogICBBbmQgdGhlbiB0aGUgcXVpY2sgZml4IEkg cHJvdmlkZWQgYWJvdmUgbG9va3MgbGlrZSB0aGUgcmlnaHQgc29sdXRpb24/CgoKQ0MtaW5nIEJv cmlzLCBEYW5pZWwsIEp1ZXJnZW4sIFN0ZXZlLCBhbmQgQ2h1Y2suCgpfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpYZW4tZGV2ZWwgbWFpbGluZyBsaXN0Clhl bi1kZXZlbEBsaXN0cy54ZW4ub3JnCmh0dHBzOi8vbGlzdHMueGVuLm9yZy94ZW4tZGV2ZWwK