From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752411AbdFGTPR (ORCPT ); Wed, 7 Jun 2017 15:15:17 -0400 Received: from mail-cys01nam02on0044.outbound.protection.outlook.com ([104.47.37.44]:50592 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752378AbdFGTPJ (ORCPT ); Wed, 7 Jun 2017 15:15:09 -0400 Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=amd.com; From: Tom Lendacky Subject: [PATCH v6 10/34] x86, x86/mm, x86/xen, olpc: Use __va() against just the physical address in cr3 To: linux-arch@vger.kernel.org, linux-efi@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, x86@kernel.org, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: Rik van Riel , Radim =?utf-8?b?S3LEjW3DocWZ?= , Toshimitsu Kani , Arnd Bergmann , Jonathan Corbet , Matt Fleming , "Michael S. Tsirkin" , Joerg Roedel , Konrad Rzeszutek Wilk , Paolo Bonzini , Larry Woodman , Brijesh Singh , Ingo Molnar , Borislav Petkov , Andy Lutomirski , "H. Peter Anvin" , Andrey Ryabinin , Alexander Potapenko , Dave Young , Thomas Gleixner , Dmitry Vyukov Date: Wed, 07 Jun 2017 14:14:53 -0500 Message-ID: <20170607191453.28645.92256.stgit@tlendack-t1.amdoffice.net> In-Reply-To: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net> References: <20170607191309.28645.15241.stgit@tlendack-t1.amdoffice.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: MWHPR15CA0040.namprd15.prod.outlook.com (10.173.226.154) To BN6PR12MB1137.namprd12.prod.outlook.com (10.168.226.139) X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN6PR12MB1137: X-MS-Office365-Filtering-Correlation-Id: 2f3ab3da-bad6-40da-8ff6-08d4add97f93 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(48565401081)(201703131423075)(201703031133081);SRVR:BN6PR12MB1137; X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1137;3:MUD+McupZzrYtCLVLQrofWDk1yeU/MdJb3CZs3ncJ69qBMR+D6/INO1KsCyfjmuxtNHVN4h8VWttGexG1004Omka8t7QW5Gf/Ebj7va83f+vw4UzQ3tnzdiaZFSeoEqWThbWTr9/37mXTknjBy99PleYT9Q5u94aDlkQiKZS0gQruosHG/MHc8rC9jZE74Tg8JbTH65KMLSzA/Yndhu6J6r1GeAJbosPDGkHFfeLea/2y6gWnrksunMfg1lcK+9+IxFmhhxYyARc1EuQhRkeoUZI6KqsqGx6e80rqF+xkNJGoOV6YIP2vk8ryxPnaFjO5maVALs6hRVDCrs8H5IWtPG915V/hlMU5vJUcpld9eY=;25:Frd83cG9VZGdHsKX8E14w7aao1VoAAyauaulvaZ4rF79avOPFLGmkbrtsxpK1CQpaDZyegLSAXjjHBT5/SV4a/EXpuMtEw2MZvTexLNZGsJrrAlXMvXvb5VGrqxrkSDpZp7Hx23KCyXu0t2CLaOrszs+4q2otejtCa1TJPG0/54cWt+WoYAGWXwqAzlaMlvAJmNIADDtukV06cxySoWwRCKTMXyOzGHomgeZKwuuNvvXVLJglHBSI+IkQot1JNBJnYBleqTMRgDT/xlchzMGPiaEy7h6tHg8uN5XOF/kvI4yexHtsYp7oNdEJCvMnLbvfjwzFEOwySG0Vu01QXstuKuFwD+2h18O4ewEoF47UsPpOwyGv3R2YkRtRPoSj9Zczv5Yk1MRXGnTyqFf3y0RfzSt8LMCBIeMJRzXtj4dBsIGzhFzOcZjrWu4E9a6gBcSdI9VwjAomCziShlKt4kBwYS8emWys3bEOmf/H/TbnrU= X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1137;31:/PDIaACw0F307tMQYmCs7nn6fHGinWl16m7P9zHm7UBeZ3pqaUcGmBoD5MkH72TKGR0CO4u/g8Fo4LmrMapckTucSzzhpVf12gdLXixIr+s/wzY6rv+FWHmm7jB21WzvHEhUKXJ8zhsf/L34gZSHJdajYA+oY+DHOKKX6p8mSuvtOwKJ3sRCK02yCNimzJFhl2Twad2OUYgVNDtGZM/f9Wi9hzAuh1j+JJrOnVcRvNk=;20:0lvYZMlVWeCV0YNKpZWQUldhR40sTIXu51o+C/QO3x8A/M4e3JF1zTsJvz198OejlT6qhDHuj74f5J/GH2fVqASOjkBIJasA/7PPJuTNY52OHPlJzOk2mnx03j6jEnXaJxINf/l9tTpIc1R2ChxkvcQhhIYwzPlikeS/HCnbmx7ff1g9Bd6MBNK+e/qvpsyf+mt6HfV9/NcUrxqFj6Quzxcd+nm/uYowVfVOyBT158ZuHkCfT70AiM+q7sXXWVFmNSS/1G74rTSbRHb98PzKxKODD0vEKCvonuDX/Uo3mdzD9xEJUE8cJQGR2FfJ72E089smF853/gUkI4h38zuKVeTxtHPEHBHmvg5+1w+ofw2HIEjEfKULu4A/w8DpR0OnrvX1ieey9Vvg0k6wQlJ13JZF/GJSsXSmdzm94GP0I5cisiZNga0/bzW1+5Dv2u5Xp2hDQ6QSgJm8/6DcIYKyBYSj9lxFpg5+Fv2vfJG0QRyfXLPAt4FKcgsQYS7jNIT4 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110)(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(100000703101)(100105400095)(3002001)(10201501046)(6055026)(6041248)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123560025)(20161123564025)(20161123562025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:BN6PR12MB1137;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:BN6PR12MB1137; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjZQUjEyTUIxMTM3OzQ6MFBXYnVQb0V4di9vK3UvRE5oY1RRcjNvMGdW?= =?utf-8?B?VFFBMXdCWkUwWm40dDN4UCt6ZVJuTTFrR1lLQzJJd05lVmp2MmM3Tk1sLytQ?= =?utf-8?B?VWdYZkU0V0ZWRVA4Wmh0Y1VVTFBpWEJFZ2VaNkZWc2diV21hZEZLNlE2VDJD?= =?utf-8?B?bFZVY1ZmRUNGMmR1SXh4cnpjYkU4c3NHcnRNZERVVU52STMzdnczOUJpVU9s?= =?utf-8?B?Tyt5SUxrNjFhWFBGSy84c241aVdhMHJ3dldjajQxL2RucHYxMi9jL0ZweTVG?= =?utf-8?B?ZGI2VzJqK0VTVGZmc1dOa0xwWFdKZ2FxbGgvN3BacUR2aGpWKzVTOUw3MS9O?= =?utf-8?B?NUlJOG9yaXdWaTJYWHBRTDY5RFBnVGpMbER1WjhDaG5rTUdTVTI2dE1rWHl0?= =?utf-8?B?V3hEVUVnM3VLbm1yUFQxYndodGZ6ZnYxR3lPcGlKS1VFMzY4OHhib2MzN1Ew?= =?utf-8?B?ZGVPN3Z1U3ZUamZXNGtydkFWZzZsNG5hSUp1SHNmTDVWZlZlNjVMeDFzV0Va?= =?utf-8?B?OU5CS3gyOWR6aC91dVB6QldrVU50ejhjVEtTNU44dkhSUThUSm85UmdqcmJw?= =?utf-8?B?UFNkNnVkV0RTb2hhSzVXck5NcmxDU0dLc21MOHdDaEJwMFVmYlZWaEpvbjIx?= =?utf-8?B?Y3RReFlhc0J2RWJ4NDUyZk9tZ0RyZ0ZkTjh5UDBWYzdralhNTGV2RDZ3Y2Iy?= =?utf-8?B?WEFYbmcvbSs3WURibHJHdmJkSXUwZHVFWmtnVWJnbS9ZRDRna0szd0VhU3ps?= =?utf-8?B?TGRLUVVRazduenZrOTZSRHg1L1J6akRZOGFoNXNFY1FoZFdDdVcvMkNPWENi?= =?utf-8?B?R0VRNkYwMzFhUnVtTEprZ0xYRUxaT0lPREdyb2t4WlZ0MWxUVDBhVnBXaUcy?= =?utf-8?B?dURadnhiTDUvd01keGhEU0VSMTFMdnY5bjZFTnBsVlZwLzFVNm5MejNQS2pk?= =?utf-8?B?VkQ0eUY4TkVtRGx4L0FBVFl5dGU2c0REaEE5ZVpNZ29LNlRBVnFJSUVGcFJ4?= =?utf-8?B?VDFRQ3N6bUJOcGU2cnJudlY1cHNxTkp5dE10TndSUG8wRVBlWnRqZnhsTkM3?= =?utf-8?B?cjFSR0ZTUUUyWU1EVm5xZFRuc0QwYlltOXk1eU5HcTVHK0NjWnhNckg3THI2?= =?utf-8?B?VVQyUFZ2djRBZGk0bGpCM1FRUU1DM0szSEhiSFk1QXlwRldoZCsrY1d5Qitl?= =?utf-8?B?VE5aMEdrQlJKOTkyVEtJbmo3MVNXZHZjak1DYmdFRTk2c1VhSkNMZnYrMzVZ?= =?utf-8?B?T0pRRWxIdUZHT0o4dUFaUFBMSjl2WHAydCtaM011L2oxYkVvREVMTE96Y1BX?= =?utf-8?B?UkNkMUd6UkRCS29OQ21oOER6VWFGclUxM010b2cyZVJOZGNUS2lleXlsQzla?= =?utf-8?B?VEtvd2krUWdVWU4wVzk3Y2RNU0M5cnhjOURsQjBqMDZCdlRnV3dBc0xGVEYw?= =?utf-8?B?a3hQUzJCcmtYMXdsNDRNcWhKQ1dDZjVoUXM3aTlLR3VkOUZGNjgxZUFxNjRa?= =?utf-8?B?b2ZYS29YVGdoMllJY0R0eGtUdlRrcUVjQVI4Tjlkc3Y4aG90SlRvc1NKWUcv?= =?utf-8?B?UVdQa1llZEhRMDRPVVZuRk5jY0pESmdlQk5OR1RLT0JjVE5SOWN1U2FNZUd2?= =?utf-8?B?R2pVMWRsWHhnUzhqWWVSakpXZ2tacGtKM09tUFhjZ2hFSEJ5aUxEcFE9PQ==?= X-Forefront-PRVS: 03319F6FEF X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(39400400002)(39840400002)(39410400002)(39450400003)(39850400002)(39860400002)(7416002)(305945005)(7736002)(7406005)(97746001)(47776003)(54356999)(76176999)(230700001)(23676002)(86362001)(575784001)(50986999)(5660300001)(66066001)(81166006)(72206003)(103116003)(2906002)(50466002)(478600001)(6506006)(8676002)(1076002)(110136004)(38730400002)(53936002)(55016002)(6666003)(189998001)(53416004)(33646002)(4001350100001)(9686003)(54906002)(42186005)(2950100002)(4326008)(3846002)(6116002)(25786009)(83506001)(921003)(1121003);DIR:OUT;SFP:1101;SCL:1;SRVR:BN6PR12MB1137;H:tlendack-t1.amdoffice.net;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjZQUjEyTUIxMTM3OzIzOklSenBLYkhyZUtwaHlYWGJ4RFRlS2xNV0Zp?= =?utf-8?B?by9hcVpGeStGeW84WkJEMkdTV2xkOVlLZ1JLNlFzRWZYYzZicUg1NlNIL1lH?= =?utf-8?B?Tm5paXJCYWlIbFNLb0xKWGlNYkVrWG4vclgzVnNnRU50V2NzeFQ2ZDVzRHBC?= =?utf-8?B?VUM0eGh6Sm9FSUU0aUlzMlg1RlkzU2JWb1pUM3pQQVBCdlFKS3Q0UG5IOWRr?= =?utf-8?B?Q0Y5NVkyWHRRZHR2VWcvMUNoYzhQZS9uU2pMNms0Q09UZlNZWmt1MHhEbTcr?= =?utf-8?B?RjRvMmJQaHRLV2l2VHVFQ3RhNnRFbXUyU0lLeUlwWVZwQkhoR1Zsb01LaU52?= =?utf-8?B?NFk2dzR4akFqYmYrSXQ3SVdJemdmMFBuM1RyOFhBTFVBUTJENTdWdWlIRjdr?= =?utf-8?B?MGRoSzBKZHVFSStXQWxlS1A5bHJMb0xyU3h3eXNHSmp6ZUN4SEloSk82aTZ5?= =?utf-8?B?QmZRK3gwSHF0WnViekNleGJLL01EZVRKbmxmUVd5QzNMTVNQSWtjLzBNMFBk?= =?utf-8?B?bCsrV041QldTSTFJY3BicjRmM2t1dHB2cFNUa0hkYkF6dTlkRGpmSmFhNURT?= =?utf-8?B?M0tPamxrb2dQNWF0MzBkYW8yek4xajhUZXhNbWpKdXBjSnNGemlhMWRDbW00?= =?utf-8?B?ZFVaUUhnNjFwd2tUdmpHVWxha21UUGVpTCtFcmdVQlRkdlBTT1Z3K1JLNXdR?= =?utf-8?B?Q20rS2NSZE55dXZST0hYMm44SnFzbDFZVE5sbzJyTjh0NXVOTkZ6dXFCSEhr?= =?utf-8?B?Nmd4bkNrMk5qM29KMExZaGEySmdZWDdlRlAyWWpEaUY2NmxIN3RKN1hSSUVY?= =?utf-8?B?N25IdVdQandrbzB2ak81K01sN1VhUWNkWlVlS1FPSmtyNjBjQ3V0NlhtWklF?= =?utf-8?B?UXgrcTRIQUVnMXB0cnZ1czRZSEhCWnZsRzVtL2drSHR3V0FxcmYyT21FM0la?= =?utf-8?B?Y2pVcVFGUXdQTmYvT0F5VFhRbUxpMDV6UkVxbWwrTmZLdmhlaktwZm1XdGMr?= =?utf-8?B?cm1jS1FtTEdRQ2U1b0ZVcW5ONlp2OWxJVmZVWkYzYlFOOU5lbnVya1Z2NXU3?= =?utf-8?B?QWk5MnUwSHM3L0ZVK0IwYTQxMXNQN3RsN29reWNmN29FN1NDaVVxSFFWTXRn?= =?utf-8?B?Z0FybGJXQXhPWElhbVBvM003WGNrbWtZTVE3ZFU2dHlYOVpmODBCbjk3dkUy?= =?utf-8?B?dVRTTkl4V2ZYYm91MTFKUzdvRmI3d1dSVEh5dUlsQUk2RFJ2dllha0U2QWNs?= =?utf-8?B?dHZiREVMV2tWMitGR3Vvb3hpaUZQcUtTN1lmdXptWDV0ZkhtamJPSnJ3Q0RM?= =?utf-8?B?UWMxM0pOclpMZGd3eE1Jcm5yU1k0SzJMcm5aYXNCUlZnc1BTZCs5S05EaCt5?= =?utf-8?B?cW90eFMyWjRPa0I4WUxFS1o3OXdjUnV2Qk82OXR5ZVdiR3I0RVV6UlpzRHhE?= =?utf-8?B?S1dSU3FhdnJxS2NkSjdyNlVYL2pES1A5b2VPZmxCMm56cWZDbHliRXI0QU1w?= =?utf-8?B?MzA3SVBGR1hQbHZFTzhqNE9uZmNpdmF4cFMwSk02c29aSE1LeC9lREs5eFpl?= =?utf-8?B?OTBFb0pOWnNkUzNYT1F4d2ZNMmhFaVNmNXZyZ2Ztc3ZrR2xSYTNSeGhtSytX?= =?utf-8?B?cmoxOVJPdXVFb3dsa2xSQmh4RDdSdFdxaERkSzdTUjcrcnplMVJxeURMaTBl?= =?utf-8?B?YVpmQmhoa2xnZHVENVQwcjRKN3dFL2FIbUZLVEszcHFibkNITEIzVVY5U3VI?= =?utf-8?B?R2lwRFZMdWh6b3BidFdUZz09?= X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1137;6:hORWsUT1eKGUzANq1IHJ4isxcM+6uBpsdTV37E031vCls9AZcIJcCpOECQbzbS+k4FZCm6UqC97kMJYSUvWFOgu6oEFw/u3YalpbUvWeiOg47KsjPZejiTT21lO0Cymp0ikI6etcxpxhQqIV5T6RpGnvESrrAXOcJL4d5Fgt4yF1duMTSVFkOSUdCub4rqOFaB+ncarzozc3H52Ofu1XgUDrKySGTOg1Tvb0tp1Jj19dTfTl2rA1PoBapnvrmliEhjrLypq0rf8145QIHP58rPpA3cnSfSK6uNWd57r71Bte2FOP+pollGK6pzutSWNd+te9Yt4gLi6ygfVCqqmdLhljo1UeKibspQbq5HswwlOnbuPEZtVi4Fe3oTzXPpdEdiMLkoDns0FVSOYZysY4R1y2rpdyMJDzivLkMdR30oL5M/+JlYMVHxeUwOqw4PVOfPjs3SwMNBjPLKxVou1X5UJaSPr7Q/ZFYs0g352QjyB5JjeWkiuoiSZXL7zT/qD1js/KmOMKfRikwEwOJFxLVXat9pv9nKQmhavRseB+o1M= X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1137;5:18uD20jBytNcBnwiNCgg2V0PsBaIaxgc9K8ZujI7VGi699m8npKAAcMU7s/4Cpo/WvPc75G3pKIkU4QAQ4pBZSVbBQZLKvHE1Judz4Z/2OuM3ehSH5ekmSraUYgMfW5RdW3kTONsZH2ieNxcRB1uvs8ycB/qWx/12yGER/5IPIyfq9khBpqTwzh73X1vO0pDMvlO/YuucW5kn0vgoD7i9QokGCIfA/WRB/C6Ieo5PhabM7xki/Ud25jEr/vMizWRjTXvzEupTKnXUsBpkvVduIXlNUea5YqgnbIirzdiRynBLegMGiMzx2Sbq8cPEJipPS1n7jfOcl9t69vsrudlcv4hJ3V1I87Y5u4lWYy47oKH6Xl+ld3ngiSCE1Tm2cPUuhfkUxrZY8jOSE1RoiL1k9JDTY5QMMHvdBgM7TyYNH7+afO+iYngDHkB2C+qoA0UR/XlS3Hlh4rLznbdq7One5qjVjvHNQ4L2wMkzQuKGrDDQLpGNzVGvMSz5S0bA9YC;24:JfZMyV8dI8swtd3dlD156cKE1b/NO0lGX15EwNJRTOSWS8QkQesVezhKnCyJ5y11hAGDPmDQ/3JbKuq5pXj3q5ZX3Uyl/CnmtEFJ/i0ODsw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1137;7:cqkZ4mU3Zv100CNMZ9WE70MHmT+SXmuenuCPP3ThqmUF2EeukY1woRVBF+8TnDTeExueKRkawOByZpUZXMnlcgC3jKoGJ8bFI3ZAlXct3B2ghxHmsR91HJa/UQmpqcHP5roW9Mqe+mDB6QpJtQeh02iL23GqXD/fliyNjAFXypJ8fXNYu+XGIxwRloTY6z2A6Hi3+OCwQ+e63dut7CtTat7fEadEYtoO+uNddHEaFQS3BVWx/ALsZC0G5LMWP7vrWf8intNXakljgHdyk0KwAyefXvlwt3MBagDQq5dQf2nkrxshaDbjKf0db6urIyFoxORfvLNsDYIvClU6FpFr1w==;20:r6yDNp9NUHQoPvcwbQAgrsG6oUr8fzLinx/sLWskpY2qYFo/0uPtC+q1pFX+yBxebyrXIPl6JEccZZTwlM+aqInNGND6rOgz1TaZYrgmOSh/MFI4Y5AbRR1QFVmh7fizao0MrSgCLRL7D2nvNNiAO1rdVMJ+Ke4KnhaUbqcxyqf8kDzBzPrGaKYuifdEsscryMxeVyJLGg5PQmAt9l+Qgk1VMgdBpumix/a2V9lzZ2ZcMvWoQEPoskrBgNS6MqtR X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2017 19:14:58.4193 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1137 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The cr3 register entry can contain the SME encryption bit that indicates the PGD is encrypted. The encryption bit should not be used when creating a virtual address for the PGD table. Create a new function, read_cr3_pa(), that will extract the physical address from the cr3 register. This function is then used where a virtual address of the PGD needs to be created/used from the cr3 register. Signed-off-by: Tom Lendacky --- arch/x86/include/asm/special_insns.h | 9 +++++++++ arch/x86/kernel/head64.c | 2 +- arch/x86/mm/fault.c | 10 +++++----- arch/x86/mm/ioremap.c | 2 +- arch/x86/platform/olpc/olpc-xo1-pm.c | 2 +- arch/x86/power/hibernate_64.c | 2 +- arch/x86/xen/mmu_pv.c | 6 +++--- 7 files changed, 21 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h index 12af3e3..d8e8ace 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -234,6 +234,15 @@ static inline void clwb(volatile void *__p) #define nop() asm volatile ("nop") +static inline unsigned long native_read_cr3_pa(void) +{ + return (native_read_cr3() & PHYSICAL_PAGE_MASK); +} + +static inline unsigned long read_cr3_pa(void) +{ + return (read_cr3() & PHYSICAL_PAGE_MASK); +} #endif /* __KERNEL__ */ diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 43b7002..dc03624 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -55,7 +55,7 @@ int __init early_make_pgtable(unsigned long address) pmdval_t pmd, *pmd_p; /* Invalid address or early pgt is done ? */ - if (physaddr >= MAXMEM || read_cr3() != __pa_nodebug(early_level4_pgt)) + if (physaddr >= MAXMEM || read_cr3_pa() != __pa_nodebug(early_level4_pgt)) return -1; again: diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 8ad91a0..2a1fa10c 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -346,7 +346,7 @@ static noinline int vmalloc_fault(unsigned long address) * Do _not_ use "current" here. We might be inside * an interrupt in the middle of a task switch.. */ - pgd_paddr = read_cr3(); + pgd_paddr = read_cr3_pa(); pmd_k = vmalloc_sync_one(__va(pgd_paddr), address); if (!pmd_k) return -1; @@ -388,7 +388,7 @@ static bool low_pfn(unsigned long pfn) static void dump_pagetable(unsigned long address) { - pgd_t *base = __va(read_cr3()); + pgd_t *base = __va(read_cr3_pa()); pgd_t *pgd = &base[pgd_index(address)]; p4d_t *p4d; pud_t *pud; @@ -451,7 +451,7 @@ static noinline int vmalloc_fault(unsigned long address) * happen within a race in page table update. In the later * case just flush: */ - pgd = (pgd_t *)__va(read_cr3()) + pgd_index(address); + pgd = (pgd_t *)__va(read_cr3_pa()) + pgd_index(address); pgd_ref = pgd_offset_k(address); if (pgd_none(*pgd_ref)) return -1; @@ -555,7 +555,7 @@ static int bad_address(void *p) static void dump_pagetable(unsigned long address) { - pgd_t *base = __va(read_cr3() & PHYSICAL_PAGE_MASK); + pgd_t *base = __va(read_cr3_pa()); pgd_t *pgd = base + pgd_index(address); p4d_t *p4d; pud_t *pud; @@ -700,7 +700,7 @@ static int is_f00f_bug(struct pt_regs *regs, unsigned long address) pgd_t *pgd; pte_t *pte; - pgd = __va(read_cr3() & PHYSICAL_PAGE_MASK); + pgd = __va(read_cr3_pa()); pgd += pgd_index(address); pte = lookup_address_in_pgd(pgd, address, &level); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 2a0fa89..e6305dd 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -427,7 +427,7 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) static inline pmd_t * __init early_ioremap_pmd(unsigned long addr) { /* Don't assume we're using swapper_pg_dir at this point */ - pgd_t *base = __va(read_cr3()); + pgd_t *base = __va(read_cr3_pa()); pgd_t *pgd = &base[pgd_index(addr)]; p4d_t *p4d = p4d_offset(pgd, addr); pud_t *pud = pud_offset(p4d, addr); diff --git a/arch/x86/platform/olpc/olpc-xo1-pm.c b/arch/x86/platform/olpc/olpc-xo1-pm.c index c5350fd..0668aaf 100644 --- a/arch/x86/platform/olpc/olpc-xo1-pm.c +++ b/arch/x86/platform/olpc/olpc-xo1-pm.c @@ -77,7 +77,7 @@ static int xo1_power_state_enter(suspend_state_t pm_state) asmlinkage __visible int xo1_do_sleep(u8 sleep_state) { - void *pgd_addr = __va(read_cr3()); + void *pgd_addr = __va(read_cr3_pa()); /* Program wakeup mask (using dword access to CS5536_PM1_EN) */ outl(wakeup_mask << 16, acpi_base + CS5536_PM1_STS); diff --git a/arch/x86/power/hibernate_64.c b/arch/x86/power/hibernate_64.c index a6e21fe..0a7650d 100644 --- a/arch/x86/power/hibernate_64.c +++ b/arch/x86/power/hibernate_64.c @@ -150,7 +150,7 @@ static int relocate_restore_code(void) memcpy((void *)relocated_restore_code, &core_restore_code, PAGE_SIZE); /* Make the page containing the relocated code executable */ - pgd = (pgd_t *)__va(read_cr3()) + pgd_index(relocated_restore_code); + pgd = (pgd_t *)__va(read_cr3_pa()) + pgd_index(relocated_restore_code); p4d = p4d_offset(pgd, relocated_restore_code); if (p4d_large(*p4d)) { set_p4d(p4d, __p4d(p4d_val(*p4d) & ~_PAGE_NX)); diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c index 1f386d7..2dc5243 100644 --- a/arch/x86/xen/mmu_pv.c +++ b/arch/x86/xen/mmu_pv.c @@ -2022,7 +2022,7 @@ static phys_addr_t __init xen_early_virt_to_phys(unsigned long vaddr) pmd_t pmd; pte_t pte; - pa = read_cr3(); + pa = read_cr3_pa(); pgd = native_make_pgd(xen_read_phys_ulong(pa + pgd_index(vaddr) * sizeof(pgd))); if (!pgd_present(pgd)) @@ -2102,7 +2102,7 @@ void __init xen_relocate_p2m(void) pt_phys = pmd_phys + PFN_PHYS(n_pmd); p2m_pfn = PFN_DOWN(pt_phys) + n_pt; - pgd = __va(read_cr3()); + pgd = __va(read_cr3_pa()); new_p2m = (unsigned long *)(2 * PGDIR_SIZE); idx_p4d = 0; save_pud = n_pud; @@ -2209,7 +2209,7 @@ static void __init xen_write_cr3_init(unsigned long cr3) { unsigned long pfn = PFN_DOWN(__pa(swapper_pg_dir)); - BUG_ON(read_cr3() != __pa(initial_page_table)); + BUG_ON(read_cr3_pa() != __pa(initial_page_table)); BUG_ON(cr3 != __pa(swapper_pg_dir)); /*