From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753090AbcCPPot (ORCPT ); Wed, 16 Mar 2016 11:44:49 -0400 Received: from mail-bl2on0121.outbound.protection.outlook.com ([65.55.169.121]:60758 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750994AbcCPPoq (ORCPT ); Wed, 16 Mar 2016 11:44:46 -0400 Authentication-Results: amacapital.net; dkim=none (message not signed) header.d=none;amacapital.net; dmarc=none action=none header.from=stratus.com; Subject: Re: [PATCH 2/2] x86/mtrr: Refactor PAT initialization code To: Toshi Kani , "Luis R. Rodriguez" References: <1457671546-13486-1-git-send-email-toshi.kani@hpe.com> <1457671546-13486-3-git-send-email-toshi.kani@hpe.com> <20160311090140.GA4873@gmail.com> <20160311091301.GA11595@gmail.com> <1457721266.6393.123.camel@hpe.com> <20160314225048.GE25147@wotan.suse.de> <1458002240.6393.320.camel@hpe.com> CC: Ingo Molnar , "bp@suse.de" , "hpa@zytor.com" , "tglx@linutronix.de" , "jgross@suse.com" , "paul.gortmaker@windriver.com" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , Andy Lutomirski From: Joe Lawrence Message-ID: <56E97F66.9080701@stratus.com> Date: Wed, 16 Mar 2016 11:44:38 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <1458002240.6393.320.camel@hpe.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [198.97.41.12] X-ClientProxiedBy: BN3PR0401CA0015.namprd04.prod.outlook.com (25.162.159.153) To BLUPR0801MB580.namprd08.prod.outlook.com (10.141.253.140) X-MS-Office365-Filtering-Correlation-Id: ca81cc37-8dc9-44bd-729f-08d34db1e50b X-Microsoft-Exchange-Diagnostics: 1;BLUPR0801MB580;2:ocWaNttPFStwXT932opZBRQY1KFTn3zZ2jw5dJFs777F1WVHTT9vCAwkt1PKck3dkW9FrxQPrekLO49HpgeAZfOm82tct6TqBo5QBXbS4rRzQgNm2VEiXS7ZeV37Q60CioUUvbdvRZf8ARHCuz/rXgkbCBt/GYgW6xM+iMZ0t/lJMSMGAShHRs88aqE62mcy;3:ntluDtvFGEBxG+z6je82IVaJD2vGaP6Hf2Sk9+VSu02jfxtPX9TRrDPX3FY5CBZaCWV08BLXzHuHttLio1AsGhHdO+68zIUKQB9m1FpcUZxIxo5pgp04jmKYB/B2mdHP;25:l1XUYG8LLHoshEosfAUByepA3CnE5Sk90pms8jC3WJOnd59vGTphxft1hk0GSABdtyJyVlIAfTih0wLRgSvJ/VeSWOdz2MCEoUIz4QSzdSYDwwjlwIVxSLHmGyE4qp/WgHnZK4ZbjjTpgUFmPJr+hJl/v+uTJbRYiYeKN5UWLu7AmRO8xa0MrG2LKptX02XgbCLZgdQY8l4BqTWaxgeVn4uSTHCS98QvgMw8MBF121gQxcICSya0GA2G+cUcaUZa2hW5LLJb54FyFbHFvuaAl3DELgyrkBKvXw1Tle1emnNRtcGpQbE8R3r/hWzccu1qcWgIN9pCBWZlDcZdnOrTUA== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0801MB580; X-Microsoft-Exchange-Diagnostics: 1;BLUPR0801MB580;20:nCP/e3UpxMtJuhg5Y+XKimgd4PaqpWfQMk7P2a3CyOH5bzZaiR/nSx6zWQ2eEEcYxEQHrC38HeLIisgAc3M4mwb6WL7VLMRJvjZJbJomrRZuQXs013H+Zst2/fAVvmYNwFMrV3BG15iwPcf8w0AVt9VXRtsP4X8xA7QHKDAzl85Ax4Lg2+GeRPKPzcj4Y/83gnypagwnHgJz5UrJYZvsadzJ9z8GKVmnccXkF008RizgUJ4DIXaBWJUlnch/TLeFKNaDrK3s9h+UaaBV/sX9Uh1gIaUb0LrzFxukb20i8ETkLD+YlT4JnAnk2c9lQXdnN2ZGdLza0E0h4nz6eLNQt6t2p3wUqNPUFtpWzdkRKAgAX5Lc7ZrS51egNxJ2+rJUJQ9f/Q6qqRnxcKcGMj558nCJ14tPQBKPJkBM5KjAx4pcT4O0FS/EQkUorcf19yZT3lQu99gm0cc309ABrIkAK9yqUQxC+TgDim8uvngfehd7ecnxrCrWfAlZGGNxCQhp;4:SCHQQ+WAqXJcNO8mr38/vrw/mUykeLv33WQsFwy8x1KniezHxdumhgVOysTDT7QohFXDNv34ZwGstPSybbFGUqPTI39xqTkJTbWA2NQIIxLFnDJXl4Rv3ksti4SpCybv7KLssfVYD3YXHF/QPR02LIReLZbkatgkob1SgpWkQefYr4IIEf70idj/tECLTRgwsYXWGsgZ6gN5fdhjFDiwCrLHmL4Mx4VDdJifoYD1o6rOYfrzKLuNDSoaheK8bn0pPeXnzbo/J34Q6bt4aUKV5Lm9lgLRyLcuLmcoJQULXOK3R1+x4bZQAn4siYmhLPDVB7p9TPkzWUjc5zSds5riI1MXPlM2OdFYKmq8VTKqiz1n/iCYOAQpCa4nfgzjyOlI X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:BLUPR0801MB580;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0801MB580; X-Forefront-PRVS: 08831F51DC X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(377454003)(24454002)(479174004)(5004730100002)(86362001)(5008740100001)(83506001)(5001770100001)(80316001)(65806001)(65956001)(47776003)(4001350100001)(189998001)(66066001)(33656002)(36756003)(92566002)(87266999)(93886004)(54356999)(50466002)(50986999)(76176999)(23676002)(65816999)(3846002)(42186005)(64126003)(586003)(2950100001)(77096005)(1096002)(4326007)(2906002)(230700001)(81166005)(6116002);DIR:OUT;SFP:1102;SCL:1;SRVR:BLUPR0801MB580;H:jlaw-desktop.mno.stratus.com;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTFVQUjA4MDFNQjU4MDsyMzp2ZFF5T2F6Y3hvaVE0WmtkazU5MmpaaWFX?= =?utf-8?B?dGlIei9TZXRkS2RJWEV4M2lRTGVTOFRPSFBnMVpXWG9Bck5OQmh1NlFWQUN4?= =?utf-8?B?SFpFMVhiSW9DcENCVmlFMVZ0ellSa2tWcTNOa2JYTjB5RkhzZzB4dE5tZnI4?= =?utf-8?B?QWRJTmE1UnlPdEZXUXhFT1NHU0xrNlZ5VGtuYk9NWnJSTjJlYmtxQVN5UElZ?= =?utf-8?B?M3J0d2t5TjhES3Q3bkVCUUNKL1hzbXg4eUIyTGtDVUkySCt5VXRnU1BFeXBa?= =?utf-8?B?M2wyTUxXNmxZeTN4NWRocXZ0eVBiL2F6RS9jZTF3NlErbVJhZWV6aW5ZeXFT?= =?utf-8?B?QmZUTE96aWk1eTNWRmlrMVpPalhtUlRHMU5ybGVTcmFRbWdUM3lDSkI3RkIv?= =?utf-8?B?M3habHN1MzRUa1VmdTB6M0xWbWY2SHdLaWVOWnU2YWJEaFQ5Z2Q0UHRvU21P?= =?utf-8?B?VmloaGp3VVBITThHblF1MEtJaTRZc0lRbmxNM3F0L2RxQnpOTk9wQ1IwSmVE?= =?utf-8?B?Y3hYelBBTkEvVUtLRU15WHZqZ0wrUnpwWmdHNHlYbDBKRGRSYU1mY2dtUFcy?= =?utf-8?B?REFOekJsU3ZWQ1FvT0tyWUdnSGE4VWJQSGZFSm1sSXB2azlaUktPSEtVRnVU?= =?utf-8?B?OGdsVUpCVDJSd0dLUmUwUXp1bmx6U0Q4YWNSWW5qVFRCTnUrT1ladXlsMjlW?= =?utf-8?B?b1cxYjZ4cC83eWNQWTN0OXcvMGNmc0hCZnpWdUt5UEFPVlV2NkEyYlltOGE3?= =?utf-8?B?QmVPMDAxMFlhZEE3bXdqdGtGOVMva292K0tIRk5HZ0lJWXU1VWZUVkZGRTVw?= =?utf-8?B?T29CTCs3NkJPV00vS0tpc3BKNVB5blUvTTF2dWhTMGVYS1h2Q3VuMjNZUzlv?= =?utf-8?B?Z2tKb2dYcFJPeGJTby9udXpBRko0bDRnM215aS84dmhIanZMb0RyUWR2Umtt?= =?utf-8?B?TXBrMmVHd0N1M1hKNW50eG1xY3NFbGFSc1JRemF4cVk5ZGFwaDMwaUxmMDRr?= =?utf-8?B?SmdXTHhBWC8yYW5OTXdQbGlKU0xSM0lqRWhjdWhzeVRtbnhxSVZtVzNmY3Nm?= =?utf-8?B?UHhEOFFQbmRUMXByNXRaUWtEL3FJNEJxL1JoelBZVzdDMnJjYmtZWWRUdkE1?= =?utf-8?B?SW5vWkhSSnZOMy9RWnBsNEU3ZUdGN28wQlpXWXRiRDBuSCtwQkREOU1laFMr?= =?utf-8?B?eHdLZHRLcGhhMFNNMVVnM09iN3dKTHVxblVvT3NQaVdkZkp0NE1qOEtEaEpz?= =?utf-8?B?OTRPSVBJOVRXQ1IzM1RCdEQ4ZDR0VEwvbFVhTmYvdjBsdk05dmpFWjM1NmN2?= =?utf-8?B?bHAvOEx0Z2dJYmk5WVlheGZnK3FXSTU5eFFkNWNPd2dIbHlMTS9rdHdlQ042?= =?utf-8?B?WGh6YmxsQld5bzUyTGh4cGtzYlpibnpBQ2w3M2Z0QVJGMEtBTmJIYjlaaWJH?= =?utf-8?Q?8VQQvvQDk8+kiGFCvDSMLVi9dacd?= X-Microsoft-Exchange-Diagnostics: 1;BLUPR0801MB580;5:PCVXBlRu94Q0UBOhilmfM1TtIdPFWGGt1qX7eLGAKf7EamdJQzoFpPlqNCp8gDtfBAlGsq1zRhiJnhmbxj4BmGiRaztKis34laEtj9GrMRAZH3hGVxcP6Ff6DTxgDNcyi4waVVIS6hMSom2r4SQJzw==;24:i6dKsqqg3cVc02t8hQ2YpYEirdZohS3vo7cOhgplTE9t0ay54U4SfflOd2R2SbwNpilFGanuKXzaWuTAmv1MLEKB9Iwk5UKGJz4cXhjTQho=;20:BvCVqAP4QCFMFb1d1I2Xya/SIcM79k2BCcYMwRcyxqnoiFGzXUElATHpiGeX66dX8jiNSCVLyA6z1aID/rHYeKyucv7Yj1TB87CMrawEvbajQWKuWsGC6rxTnPju98I4BDgajrcDeynPUpDjZb3dpjTs5VzzUIZmx4q5kMlJpMg= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: stratus.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2016 15:44:43.1967 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0801MB580 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/14/2016 08:37 PM, Toshi Kani wrote: [... snip ...] >> Joe at Stratus also hit this issue but on a system where MTRR is enabled. >> He sent his report only to me as he thought it was caused by the >> ioremap_wc() changes and his driver was one that got it. In his case >> though he modified the driver significantly, and upon inspection of that >> code saw how it used a secondary backup PCI device for failover for a >> framebuffer device... The changes to the driver in place are rather >> complex though and as such it made no sense to further review unless he >> moved his changes upstream. It is still worth noting this issue has been >> seeing elsehwere, but the root cause is still not known. The error Joe >> got is: >> >> x86/PAT: Xorg:37506 map pfn expected mapping type uncached-minus for [mem >> 0x9f000000-0x9f7fffff], got write-combining >> >> Even though the driver is custom (and actually I even saw another >> unrelated proprietary driver loaded) I figured its worth noting others >> have seen this error without MTRR being disabled. > > The error message looks the same. So, this could be the same issue if WC > is redirected to UC without disabling PAT properly on his env. I need a > whole dmesg output to confirm if this is the case. Another way to hit this > error is that the driver called remap_pfn_range() with UC to a range where > WC map was set by ioremap_wc() already. As mentioned in the other thread, our driver is very custom and performs some page table parlor tricks to failover the frame buffer from one PCI adapter to another. I need some time to fully digest these customizations before I can really be of much help here. My current theory is that we have bug when do: ioremap_wc( backup adapter fb ) iounmap ( backup adapter fb ) << I'm unclear why this occurs ioremap_wc( primary adapter fb ) to failover, we stop_machine and remap the framebuffer (heavily summarized: iterate through its pages, lookup_address to get a ptep, get its __pgprot and then rewrite the pte to the new physical address and the old pgprot_val. A tlb flush on the way out of stop_machine). Since this is all out-of-tree custom work, I'm not asking for any support. If we're too far off the reservation to be useful to the conversation here, no worries. I'm just trying to help if there is an upstream bug. >> The second thread you referred to seems to say that if you built-in the >> code the error does not come up. What the hell. Joe, can you try building >> your driver built-in to see if you also see this go away? Even though I >> don't want to support your custom hacked up driver I do want to know if >> your issue goes away with built-in as well. > > I do not have sufficient info to support this case, and do not have > technical explanation for it, either. Luis -- I tried last night to reproduce (built as a module) without any luck. Going back through my test logs, it looks like the warning is relatively sporadic and may take a few days to hit. Without having a better repro case, I think my warning is caused by that dodgy page table work mentioned above (or at least timing related). Let me see if I can try to provoke the warning into happening more frequently first. Regards, -- Joe