From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756157AbcHVWxB (ORCPT ); Mon, 22 Aug 2016 18:53:01 -0400 Received: from mail-sn1nam02on0089.outbound.protection.outlook.com ([104.47.36.89]:60719 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754398AbcHVWww (ORCPT ); Mon, 22 Aug 2016 18:52:52 -0400 X-Greylist: delayed 805 seconds by postgrey-1.27 at vger.kernel.org; Mon, 22 Aug 2016 18:52:51 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Thomas.Lendacky@amd.com; From: Tom Lendacky Subject: [RFC PATCH v2 11/20] mm: Access BOOT related data in the clear To: , , , , , , , , CC: Radim =?utf-8?b?S3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , "Konrad Rzeszutek Wilk" , Andrey Ryabinin , Ingo Molnar , Borislav Petkov , "Andy Lutomirski" , "H. Peter Anvin" , Paolo Bonzini , Alexander Potapenko , "Thomas Gleixner" , Dmitry Vyukov Date: Mon, 22 Aug 2016 17:37:38 -0500 Message-ID: <20160822223738.29880.6909.stgit@tlendack-t1.amdoffice.net> In-Reply-To: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> References: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: SN1PR15CA0005.namprd15.prod.outlook.com (10.163.200.15) To MWHPR12MB1149.namprd12.prod.outlook.com (10.169.204.13) X-MS-Office365-Filtering-Correlation-Id: 32d3817d-a75a-4d6d-e8df-08d3cadcee8f X-Microsoft-Exchange-Diagnostics: 1;MWHPR12MB1149;2:6WMy9EJJh+q14AjEUaL7VLIJGWRLXCgyiklPJlEcuP7cvvgaTV5rLZCUJBo107St0wf7iYmSzNyN/i+vD5MT130O6P3H1ZtTsxWRP3pfSwgOOhG6A81CMXOlwAKt4PuXqqr9OAjWCTYq6GWgEBFfkthxAWAfonVWicswXIoososZmKKpI4F/ad1IPNiHfGLw;3:r3YpC+djJasF07rFu/ItURivrtikDM6etq6q638aJOnP3/0ScPN7nslH3Kutev+zg5uljxQLxA+zhcC9eCiZkaKClUnKjmQZ3UNc/FMYTF2en385KnKllI6IAF+kEZ1/;25:9oOCLQNdT4/Wf6nsNQkdbq5W8iDbHiZBs4+yZhxLWht/tk0YXnyOrAYtUHd7u70OIBGdWdXR4TdpO3EC25lz1JLioFZ9MagqLcJxqNOTEITh3y24DWaBetzbOuLAw+8ljxfBb8o0wbvq1jJVFdayI2CAZWcZhbpgWIUZb0f4krDbDETQzcNcOIHPIdw/+bqNorQ1puTLfhT1TEHpgPMQQQSlzLGnJ2qTcq2f9vPIxqqQtuiu5XBW0zThbfSN4sUQOpBcpW/oc17K1T4/qXAkd3+JzyVNtTVFpO7K92FwDWTT1bCTiq7sE3tayi9PdfiqUMvgd8HBZCodIHGFyZTtxTHxjPTQx3pxQIDsj/zMCTCC8lXCkt6AbrrrXT+1LrGHm7w/P4TnSmhxKsNEZKHyQ6dOAOtVln6IHG/F3IHKnrY= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:MWHPR12MB1149; X-Microsoft-Exchange-Diagnostics: 1;MWHPR12MB1149;31:OdXLQdgXxrgpbKREqdO7FOG6jfq8k9w+F05WnRJ8NOtFlPtlc9+8L5MkuKM+/HrVLtEX+jYK91EmD+ht7arY3Fbwr8swMqiC8E/Y3gEA7jLhnNp6xuAciJxlDsI0pbtiFuJtOG86QnFaRnL0rYrR6yGblkAcvMbbyiEprAh4zyE93SH7xTSeRowe27YgepZkA4O9ynLYBkoPVYX3XSdCQ3e9BYdcoINuSLiEaQnUb34=;20:3M8dkyRRNcArBpmHKZq5/40xg81qdfZKWWM7NQnXdQ2NnJXixRabw3baKQ6BaqhyZN8FLwCn397KjaZHNjSzVMtoRVVCq2DG0hspjd/c8TESTEd/SVaqS6KWfvopm9aj65EAMpDa//RStHA7ZwmwAY1OWj2ZQzyQjk2pXuRP8BXF9Enoju7sf9v4+E7XAy1S3gNMgrYHAVlMz4/s4puELpAcBFxqr64qmKW0iwgBlUMD98VOPCrWX/QSTuOM+nZXujk+zIA26IfDZ7IP7mTh1Y2FXL4LidAZ2ocNsqTbNZIPwKyk5DL4O3Gpc50LL00JeWSC2s8G1jwyKWhtu4aj9heiTGTzH9D61vMYVsG6GbAfVTFp+dQAYVXUF6SVE41xe9xXWPA5+XJWd1k33KQmS7Wp1VHqMJemZGtw3HmaWlYcX55uaoQ+kkmu74TU4n4k8BD+nfFwAPaq1ggwxR/linh9VQnJnheMXhzNQTmEeCe73slKvrmkIq4a3YmLM1BG X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026);SRVR:MWHPR12MB1149;BCL:0;PCL:0;RULEID:;SRVR:MWHPR12MB1149; X-Microsoft-Exchange-Diagnostics: 1;MWHPR12MB1149;4:r0C54+VZocaRrFmcAa8gEOksTmRCht/F1KrOYlm3Lv06bHLdVPQerMn6Jlwhqt0NPVFGkkGC/T1qssHaMEjLlrxAoRHXRybUNpOGyVdpCot6qxoEFYYmLOinJjAtcTHVqH+UzGwpROiRryOd2DnenT/yEln928egd0xyrfEzzEDK4bXZzKq0J12DBM8BmbGM34FXHPylEyvIzXrXXa805788k7fmjoWmlUy5nW/JSWqbTSias+utV9aUEdqUaATTT7wnCP49kk3fBG6gTZb9F2PEPUlYG8xBIBBtLTnmjC5gNye/35yOloG3aT5GZIFVCTUXDYtyZ5/RQAKs7ILrL/T7Qqgsw6Ac5d48+iO2K7rl9FepR7f0KElSS2wAwnob8RvpOxOcdyrNiV8IbkV4EQiYRY4F756uFSGGp3SXBrkKk1BVPdBIMGn3zHCGuDpT X-Forefront-PRVS: 00429279BA X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(189002)(68736007)(7736002)(1076002)(305945005)(586003)(6116002)(3846002)(42186005)(97746001)(53416004)(33646002)(4326007)(81166006)(81156014)(2906002)(230700001)(5660300001)(8676002)(77096005)(2950100001)(7846002)(9686002)(19580395003)(23676002)(101416001)(19580405001)(103116003)(97736004)(5001770100001)(189998001)(69596002)(105586002)(92566002)(83506001)(47776003)(106356001)(229853001)(66066001)(86362001)(7416002)(575784001)(2201001)(54356999)(76176999)(50986999)(4001350100001)(50466002)(71626007)(217873001);DIR:OUT;SFP:1101;SCL:1;SRVR:MWHPR12MB1149;H:tlendack-t1.amdoffice.net;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtNV0hQUjEyTUIxMTQ5OzIzOlpCa3RSMXM1OEs1dWlpWHZVZ0hxSnZySWpU?= =?utf-8?B?ZUN6TG8xalZJODk1Um84eFE3elVmVjJtYkUyT3ZvV3pGUGhFU1Z2QUZ1LzJP?= =?utf-8?B?SUl5RVo0Ykh1dWczdHE3SU5kQUZBV0hvVUhXd1BJSFFKZFdKb3h5UWE5K3d2?= =?utf-8?B?eWN6WXJSbHl4UTlLMTU2NXZ3VE9xREdvWHprVXducVlaWWR5ZHgrczBPMUF2?= =?utf-8?B?TE8wMGhrNGY0cENyeVB2WHQzdHRWNnNTSE5uYXl0WkJOS2t6Z3cwSnQ1c0xk?= =?utf-8?B?VmhOcTFGYjlMMEV6QVl3VXhXejZ1empHUHhlTDYxM1BDcVR6d0oxUWlkQzg2?= =?utf-8?B?VTVJN0J3eUx3dGdYaEdiNUp1c0NkdnBVVDBCQVFxVzdyWmdGcFZLNDl4TDFQ?= =?utf-8?B?cXdMTnNJZElEeWxPTkRid1lTTVJKdkNRQi9rRUFsdWlSbVNtYUwvRHkraUZU?= =?utf-8?B?Z093ZURHOUdFbGh3VDhnMkJrYldkMGwzc2ZKSUF4NVlxbkgzK2M2M1RUYVln?= =?utf-8?B?NGo5YVRFZWh5YncxRUZ3V0wvSS9KVTRJRTUwUzFQYldVTGw4ZWVpc3lnUWRx?= =?utf-8?B?dEVCWXQyMHlzMkljbC8rM1ova3ptVWJwTlRVQ28wR09tdFp3Qk1CQzFlZTdF?= =?utf-8?B?SzVNQlhVY2JKNlV0cFpka1JpSWxyV20yM3N4ZS8yTytycHhCM2ZCYnY0NUlD?= =?utf-8?B?ZXArV01zcUhMVlpXSHovTi9nWkpOcVpSY3NNWlNmRG0yWW1jOU1Fc0FSNWll?= =?utf-8?B?cy9CbTRsUUZ3SmhVQlFyK3YzdnFzeDBzeC96UmY4bTFtQ2E5eXhoZXlCazFI?= =?utf-8?B?LzJIQXo4aGV6RzR4dVRTTTFzSUVBVGh4enVwOHptbUtnMjRYOHpxL2IwbmU1?= =?utf-8?B?OUNwaDlLcjlvT1ZwZytUNEdnN1p0ZG9YcVVvaDY3NHc3cTFsTXhxODViZzBz?= =?utf-8?B?Z3VyNkNHeU0rVVVmcnM0b1QwMWJ1bGZHaWN0WXpPS0lDWGFYbW9vcklJck1M?= =?utf-8?B?UzJjY3RtQnVIWU5jQVVvZG5IR0NzQzBWYXBGSzVyOEFvYi9zQ1Y4STVVa29G?= =?utf-8?B?WW8vUmt3azhpVFZKczAycTF4OU1seExHamtRLzNHcEV0cTBUK1lXNS9vOUJK?= =?utf-8?B?cVlIS1dKRm53WWp4Y0ZxRDI2R0RtYU1rVmN6bE1yaDloeHJrdGJrdTdXc05B?= =?utf-8?B?M2hKUHl3Q3NmYUdFYkVENkg2MHdZOGxuSFFUOW03UytBVjVtRHc4aEVtUTdx?= =?utf-8?B?c1A0V1FpV3Z6M3FoMFhWakNkZkluVkh1Tnh1SHZ3Z0p3OG5Od1BsclQxb0d4?= =?utf-8?B?a2NsZ2hMbUl6TkxDVHRJNnlYeWZqSU1Deld2RCtNSVFlVjhJK011SFpodGZ0?= =?utf-8?B?VGVHTG5waDRKNGNKRVd5a05qdTdjelNRM2dWdDZZMDJEckZCRVdkdDNOajB2?= =?utf-8?B?SG5DeGE2NTN5ekZRSWF2Sy9lcEJEazNvdXhQYmhWSjNITUJ4ZldCbmVzSmtv?= =?utf-8?B?UWZCUTNVa1JoYmxidTJSUXJ4ZUsvKzRVUDR2aDFMR3ZiSXloVDdDc3NyMWpI?= =?utf-8?B?RHZuUVlVNmF3bDAzNk9zRkNOaGtzcWEyTEtYQ0U0bFpKQzlPVU9xdjBhRjgz?= =?utf-8?B?QnNoZmhMdERlOElRTDV3L1JsYXFZdTdLa1djZjY3NThZQk5GczU2cU01Rzdt?= =?utf-8?B?aGJTVjd0TFRYNmZRQXpNVEc0b3JhNlJ3QnQ3Zjk1N05YaU5RdnpXQ1F3ei9j?= =?utf-8?B?enlvRVlxQ3RBa3NESUswaXl5d2FMSVc5Nk5WOE1pSkJDQVhwSDVpQ2NFWGJ1?= =?utf-8?Q?5fAbpL6oQhSv7?= X-Microsoft-Exchange-Diagnostics: 1;MWHPR12MB1149;6:Piv/gi8Y5ufRj//lf/NYwt+vWQpNTUPdv5yoP1mMFDq2ynsXSPKqfDLltp263rqdMmnKMtR0mPud2rIE6ES2F5Vr1MXp8mJja6QSAtKFPUQlMjXKQ3Vzm5sKguIEGWawb7cLiOv32aRXP1OPY1nEyjyy1L+KtewdCwOwCLoC8LlLLNpUMwAy3qINeVyop0jg9bW24g4G/TtWYSaYbtHbnOYKyhqgBdHetcMPOWpCTYGJjrZi4BHHqZYuLoy+RppIetKxvxBpvSK1hbjxiomubQ9x1Aobyb58VaEHnw1vXuP8dZ/Uygt16xym2ZbZ2b5GOBI5yGosOGBydLpbjBDNwQ==;5:O7iEH5irs9oQv2yWUSjf2mnhAV1v1ttx+du0rY3jeQci9FkAseM6oESNVGeBrtuHSlIYDeQwFDeqQnjLZdh3CbyjiQbzM+JuJnKfhemKWK8ZnXs9EQHKz/ZXkg0TgLvY9U5UOMeApRCtQp7leghBpw==;24:/SGG3Z+ZC0vw4ZPfhjfJQ6Dn4s2VhVkQwMB55D1SKqzV47i0EhUZtym43WLq4SAKyZsrajC8+Hwe7MIcHApd0zZ/8vXMkAneKaeT+DBEPdw=;7:H9XCkWNbEv7gc6NlYA5z0QtpE+UenGdKPRtqtBuApkvAyoBi9t5CbCf1dnRyEltEXddamkhqcSbhJ4T6eXuv9XbeH1RO5ztoWcubBnbkrdxDlUpo4jd8z/QtbgMuBVA1bpacUeMAq8Cg37KhQ0J318rznb18RLot5CNwQi5LRoL/71ekmXIvaUmrNvXmjY53CY1kA48DYd/BUkeJxiz32Hf3GHWx/he6tuYUOTpzWOz3KKgtdx6xL7jst9RciTNm SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;MWHPR12MB1149;20:wqlp7HyxHMr2qOcNGRconlreh7tNcqFWx5XVanKGKbbvlH/P41yeSkhtTFzLZyQvW4GrJNQKcNJINFVkZYKHIE9UOf5EwOm1mXjbW6BWpavgR9rOT/qU/Rp0DpZ9H2BsxBpfjbHHYXDATVKbX/bPOF13IOgrVXFCkWsO0bjSA8BJugoqgiWHB6F86P82yuX1hluQbWNiFbld3uxte3GpT9szUtq4fYKs/n1nz+EnUy8hAcnyS7Zkox0K2Ao1V89l X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Aug 2016 22:37:41.6372 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1149 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org BOOT data (such as EFI related data) is not encyrpted when the system is booted and needs to be accessed as non-encrypted. Add support to the early_memremap API to identify the type of data being accessed so that the proper encryption attribute can be applied. Currently, two types of data are defined, KERNEL_DATA and BOOT_DATA. Signed-off-by: Tom Lendacky --- arch/arm64/kernel/acpi.c | 2 +- arch/ia64/include/asm/early_ioremap.h | 2 +- arch/x86/kernel/devicetree.c | 6 ++++-- arch/x86/kernel/e820.c | 2 +- arch/x86/kernel/setup.c | 9 +++++--- arch/x86/mm/ioremap.c | 19 +++++++++++++++++ arch/x86/platform/efi/efi.c | 15 +++++++------- arch/x86/platform/efi/efi_64.c | 13 +++++++++--- arch/x86/platform/efi/quirks.c | 4 ++-- arch/x86/xen/mmu.c | 9 +++++--- arch/x86/xen/setup.c | 6 ++++-- drivers/acpi/tables.c | 2 +- drivers/firmware/efi/arm-init.c | 13 +++++++----- drivers/firmware/efi/efi.c | 7 ++++-- drivers/firmware/efi/esrt.c | 4 ++-- drivers/firmware/efi/fake_mem.c | 3 ++- drivers/firmware/efi/memattr.c | 2 +- include/asm-generic/early_ioremap.h | 15 +++++++++++--- mm/early_ioremap.c | 36 +++++++++++++++++++++++++-------- 19 files changed, 117 insertions(+), 52 deletions(-) diff --git a/arch/arm64/kernel/acpi.c b/arch/arm64/kernel/acpi.c index 3e4f1a4..33fdedd 100644 --- a/arch/arm64/kernel/acpi.c +++ b/arch/arm64/kernel/acpi.c @@ -98,7 +98,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size) if (!size) return NULL; - return early_memremap(phys, size); + return early_memremap(phys, size, BOOT_DATA); } void __init __acpi_unmap_table(char *map, unsigned long size) diff --git a/arch/ia64/include/asm/early_ioremap.h b/arch/ia64/include/asm/early_ioremap.h index eec9e1d..bc8c210 100644 --- a/arch/ia64/include/asm/early_ioremap.h +++ b/arch/ia64/include/asm/early_ioremap.h @@ -2,7 +2,7 @@ #define _ASM_IA64_EARLY_IOREMAP_H extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); -#define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) +#define early_memremap(phys_addr, size, owner) early_ioremap(phys_addr, size) extern void early_iounmap (volatile void __iomem *addr, unsigned long size); #define early_memunmap(addr, size) early_iounmap(addr, size) diff --git a/arch/x86/kernel/devicetree.c b/arch/x86/kernel/devicetree.c index 3fe45f8..556e986 100644 --- a/arch/x86/kernel/devicetree.c +++ b/arch/x86/kernel/devicetree.c @@ -276,11 +276,13 @@ static void __init x86_flattree_get_config(void) map_len = max(PAGE_SIZE - (initial_dtb & ~PAGE_MASK), (u64)128); - initial_boot_params = dt = early_memremap(initial_dtb, map_len); + initial_boot_params = dt = early_memremap(initial_dtb, map_len, + BOOT_DATA); size = of_get_flat_dt_size(); if (map_len < size) { early_memunmap(dt, map_len); - initial_boot_params = dt = early_memremap(initial_dtb, size); + initial_boot_params = dt = early_memremap(initial_dtb, size, + BOOT_DATA); map_len = size; } diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c index 621b501..71b237f 100644 --- a/arch/x86/kernel/e820.c +++ b/arch/x86/kernel/e820.c @@ -661,7 +661,7 @@ void __init parse_e820_ext(u64 phys_addr, u32 data_len) struct e820entry *extmap; struct setup_data *sdata; - sdata = early_memremap(phys_addr, data_len); + sdata = early_memremap(phys_addr, data_len, BOOT_DATA); entries = sdata->len / sizeof(struct e820entry); extmap = (struct e820entry *)(sdata->data); __append_e820_map(extmap, entries); diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 1fdaa11..cec8a63 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -344,7 +344,8 @@ static void __init relocate_initrd(void) printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n", relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1); - copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size); + copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size, + BOOT_DATA); printk(KERN_INFO "Move RAMDISK from [mem %#010llx-%#010llx] to" " [mem %#010llx-%#010llx]\n", @@ -426,7 +427,7 @@ static void __init parse_setup_data(void) while (pa_data) { u32 data_len, data_type; - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); data_len = data->len + sizeof(struct setup_data); data_type = data->type; pa_next = data->next; @@ -459,7 +460,7 @@ static void __init e820_reserve_setup_data(void) return; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); e820_update_range(pa_data, sizeof(*data)+data->len, E820_RAM, E820_RESERVED_KERN); pa_data = data->next; @@ -479,7 +480,7 @@ static void __init memblock_x86_reserve_range_setup_data(void) pa_data = boot_params.hdr.setup_data; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); memblock_reserve(pa_data, sizeof(*data) + data->len); pa_data = data->next; early_memunmap(data, sizeof(*data)); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 031db21..e3bdc5a 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -419,6 +419,25 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK)); } +/* + * Architecure override of __weak function to adjust the protection attributes + * used when remapping memory. + */ +pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + /* + * If memory encryption is enabled and BOOT_DATA is being mapped + * then remove the encryption bit. + */ + if (_PAGE_ENC && (owner == BOOT_DATA)) + prot = __pgprot(pgprot_val(prot) & ~_PAGE_ENC); + + return prot; +} + /* Remap memory with encryption */ void __init *early_memremap_enc(resource_size_t phys_addr, unsigned long size) diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c index 1fbb408..2c7e6b0 100644 --- a/arch/x86/platform/efi/efi.c +++ b/arch/x86/platform/efi/efi.c @@ -239,12 +239,13 @@ static int __init efi_systab_init(void *phys) u64 tmp = 0; if (efi_setup) { - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), + BOOT_DATA); if (!data) return -ENOMEM; } systab64 = early_memremap((unsigned long)phys, - sizeof(*systab64)); + sizeof(*systab64), BOOT_DATA); if (systab64 == NULL) { pr_err("Couldn't map the system table!\n"); if (data) @@ -293,7 +294,7 @@ static int __init efi_systab_init(void *phys) efi_system_table_32_t *systab32; systab32 = early_memremap((unsigned long)phys, - sizeof(*systab32)); + sizeof(*systab32), BOOT_DATA); if (systab32 == NULL) { pr_err("Couldn't map the system table!\n"); return -ENOMEM; @@ -338,7 +339,7 @@ static int __init efi_runtime_init32(void) efi_runtime_services_32_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_32_t)); + sizeof(efi_runtime_services_32_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -362,7 +363,7 @@ static int __init efi_runtime_init64(void) efi_runtime_services_64_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_64_t)); + sizeof(efi_runtime_services_64_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -425,7 +426,7 @@ static int __init efi_memmap_init(void) size = efi.memmap.nr_map * efi.memmap.desc_size; addr = (unsigned long)efi.memmap.phys_map; - efi.memmap.map = early_memremap(addr, size); + efi.memmap.map = early_memremap(addr, size, BOOT_DATA); if (efi.memmap.map == NULL) { pr_err("Could not map the memory map!\n"); return -ENOMEM; @@ -471,7 +472,7 @@ void __init efi_init(void) /* * Show what we know for posterity */ - c16 = tmp = early_memremap(efi.systab->fw_vendor, 2); + c16 = tmp = early_memremap(efi.systab->fw_vendor, 2, BOOT_DATA); if (c16) { for (i = 0; i < sizeof(vendor) - 1 && *c16; ++i) vendor[i] = *c16++; diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 677e29e..0871ea4 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -222,7 +222,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) if (efi_enabled(EFI_OLD_MEMMAP)) return 0; - efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd); + /* + * Since the PGD is encrypted, set the encryption mask so that when + * this value is loaded into cr3 the PGD will be decrypted during + * the pagetable walk. + */ + efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd); pgd = efi_pgd; /* @@ -261,7 +266,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) pfn = md->phys_addr >> PAGE_SHIFT; npages = md->num_pages; - if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map 1:1 memory\n"); return 1; } @@ -278,7 +284,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) text = __pa(_text); pfn = text >> PAGE_SHIFT; - if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map kernel text 1:1\n"); return 1; } diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c index 89d1146..606bf551 100644 --- a/arch/x86/platform/efi/quirks.c +++ b/arch/x86/platform/efi/quirks.c @@ -311,7 +311,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) if (!efi_enabled(EFI_64BIT)) return 0; - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), BOOT_DATA); if (!data) { ret = -ENOMEM; goto out; @@ -322,7 +322,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) sz = sizeof(efi_config_table_64_t); - p = tablep = early_memremap(tables, nr_tables * sz); + p = tablep = early_memremap(tables, nr_tables * sz, BOOT_DATA); if (!p) { pr_err("Could not map Configuration table!\n"); ret = -ENOMEM; diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c index 7d5afdb..00db54a 100644 --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -2020,7 +2020,7 @@ static unsigned long __init xen_read_phys_ulong(phys_addr_t addr) unsigned long *vaddr; unsigned long val; - vaddr = early_memremap_ro(addr, sizeof(val)); + vaddr = early_memremap_ro(addr, sizeof(val), KERNEL_DATA); val = *vaddr; early_memunmap(vaddr, sizeof(val)); return val; @@ -2114,15 +2114,16 @@ void __init xen_relocate_p2m(void) pgd = __va(read_cr3()); new_p2m = (unsigned long *)(2 * PGDIR_SIZE); for (idx_pud = 0; idx_pud < n_pud; idx_pud++) { - pud = early_memremap(pud_phys, PAGE_SIZE); + pud = early_memremap(pud_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pud); for (idx_pmd = 0; idx_pmd < min(n_pmd, PTRS_PER_PUD); idx_pmd++) { - pmd = early_memremap(pmd_phys, PAGE_SIZE); + pmd = early_memremap(pmd_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pmd); for (idx_pt = 0; idx_pt < min(n_pt, PTRS_PER_PMD); idx_pt++) { - pt = early_memremap(pt_phys, PAGE_SIZE); + pt = early_memremap(pt_phys, PAGE_SIZE, + KERNEL_DATA); clear_page(pt); for (idx_pte = 0; idx_pte < min(n_pte, PTRS_PER_PTE); diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c index 1764252..a8e2724 100644 --- a/arch/x86/xen/setup.c +++ b/arch/x86/xen/setup.c @@ -686,8 +686,10 @@ static void __init xen_phys_memcpy(phys_addr_t dest, phys_addr_t src, if (src_len > (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off) src_len = (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off; len = min(dest_len, src_len); - to = early_memremap(dest - dest_off, dest_len + dest_off); - from = early_memremap(src - src_off, src_len + src_off); + to = early_memremap(dest - dest_off, dest_len + dest_off, + KERNEL_DATA); + from = early_memremap(src - src_off, src_len + src_off, + KERNEL_DATA); memcpy(to, from, len); early_memunmap(to, dest_len + dest_off); early_memunmap(from, src_len + src_off); diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c index 9f0ad6e..06b75a2 100644 --- a/drivers/acpi/tables.c +++ b/drivers/acpi/tables.c @@ -583,7 +583,7 @@ void __init acpi_table_upgrade(void) if (clen > MAP_CHUNK_SIZE - slop) clen = MAP_CHUNK_SIZE - slop; dest_p = early_memremap(dest_addr & PAGE_MASK, - clen + slop); + clen + slop, BOOT_DATA); memcpy(dest_p + slop, src_p, clen); early_memunmap(dest_p, clen + slop); src_p += clen; diff --git a/drivers/firmware/efi/arm-init.c b/drivers/firmware/efi/arm-init.c index c49d50e..0a3fd48 100644 --- a/drivers/firmware/efi/arm-init.c +++ b/drivers/firmware/efi/arm-init.c @@ -67,7 +67,8 @@ static void __init init_screen_info(void) struct screen_info *si; if (screen_info_table != EFI_INVALID_TABLE_ADDR) { - si = early_memremap_ro(screen_info_table, sizeof(*si)); + si = early_memremap_ro(screen_info_table, sizeof(*si), + BOOT_DATA); if (!si) { pr_err("Could not map screen_info config table\n"); return; @@ -94,7 +95,7 @@ static int __init uefi_init(void) int i, retval; efi.systab = early_memremap_ro(efi_system_table, - sizeof(efi_system_table_t)); + sizeof(efi_system_table_t), BOOT_DATA); if (efi.systab == NULL) { pr_warn("Unable to map EFI system table.\n"); return -ENOMEM; @@ -121,7 +122,8 @@ static int __init uefi_init(void) /* Show what we know for posterity */ c16 = early_memremap_ro(efi_to_phys(efi.systab->fw_vendor), - sizeof(vendor) * sizeof(efi_char16_t)); + sizeof(vendor) * sizeof(efi_char16_t), + BOOT_DATA); if (c16) { for (i = 0; i < (int) sizeof(vendor) - 1 && *c16; ++i) vendor[i] = c16[i]; @@ -135,7 +137,7 @@ static int __init uefi_init(void) table_size = sizeof(efi_config_table_64_t) * efi.systab->nr_tables; config_tables = early_memremap_ro(efi_to_phys(efi.systab->tables), - table_size); + table_size, BOOT_DATA); if (config_tables == NULL) { pr_warn("Unable to map EFI config table array.\n"); retval = -ENOMEM; @@ -226,7 +228,8 @@ void __init efi_init(void) efi_system_table = params.system_table; efi.memmap.phys_map = params.mmap; - efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size); + efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size, + BOOT_DATA); if (efi.memmap.map == NULL) { /* * If we are booting via UEFI, the UEFI memory map is the only diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index 5a2631a..f9286c6 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -386,7 +386,7 @@ int __init efi_mem_desc_lookup(u64 phys_addr, efi_memory_desc_t *out_md) * So just always get our own virtual map on the CPU. * */ - md = early_memremap(p, sizeof (*md)); + md = early_memremap(p, sizeof (*md), BOOT_DATA); if (!md) { pr_err_once("early_memremap(%pa, %zu) failed.\n", &p, sizeof (*md)); @@ -501,7 +501,8 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz, if (efi.properties_table != EFI_INVALID_TABLE_ADDR) { efi_properties_table_t *tbl; - tbl = early_memremap(efi.properties_table, sizeof(*tbl)); + tbl = early_memremap(efi.properties_table, sizeof(*tbl), + BOOT_DATA); if (tbl == NULL) { pr_err("Could not map Properties table!\n"); return -ENOMEM; @@ -531,7 +532,7 @@ int __init efi_config_init(efi_config_table_type_t *arch_tables) * Let's see what config tables the firmware passed to us. */ config_tables = early_memremap(efi.systab->tables, - efi.systab->nr_tables * sz); + efi.systab->nr_tables * sz, BOOT_DATA); if (config_tables == NULL) { pr_err("Could not map Configuration table!\n"); return -ENOMEM; diff --git a/drivers/firmware/efi/esrt.c b/drivers/firmware/efi/esrt.c index 75feb3f..10ee547 100644 --- a/drivers/firmware/efi/esrt.c +++ b/drivers/firmware/efi/esrt.c @@ -273,7 +273,7 @@ void __init efi_esrt_init(void) return; } - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); @@ -323,7 +323,7 @@ void __init efi_esrt_init(void) /* remap it with our (plausible) new pages */ early_memunmap(va, size); size += entries_size; - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); diff --git a/drivers/firmware/efi/fake_mem.c b/drivers/firmware/efi/fake_mem.c index 48430ab..8e87388 100644 --- a/drivers/firmware/efi/fake_mem.c +++ b/drivers/firmware/efi/fake_mem.c @@ -101,7 +101,8 @@ void __init efi_fake_memmap(void) /* create new EFI memmap */ new_memmap = early_memremap(new_memmap_phy, - efi.memmap.desc_size * new_nr_map); + efi.memmap.desc_size * new_nr_map, + BOOT_DATA); if (!new_memmap) { memblock_free(new_memmap_phy, efi.memmap.desc_size * new_nr_map); return; diff --git a/drivers/firmware/efi/memattr.c b/drivers/firmware/efi/memattr.c index 236004b..f351c2a 100644 --- a/drivers/firmware/efi/memattr.c +++ b/drivers/firmware/efi/memattr.c @@ -28,7 +28,7 @@ int __init efi_memattr_init(void) if (efi.mem_attr_table == EFI_INVALID_TABLE_ADDR) return 0; - tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl)); + tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl), BOOT_DATA); if (!tbl) { pr_err("Failed to map EFI Memory Attributes table @ 0x%lx\n", efi.mem_attr_table); diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h index 2edef8d..61de27a 100644 --- a/include/asm-generic/early_ioremap.h +++ b/include/asm-generic/early_ioremap.h @@ -3,6 +3,11 @@ #include +enum memremap_owner { + KERNEL_DATA = 0, + BOOT_DATA, +}; + /* * early_ioremap() and early_iounmap() are for temporary early boot-time * mappings, before the real ioremap() is functional. @@ -10,9 +15,13 @@ extern void __iomem *early_ioremap(resource_size_t phys_addr, unsigned long size); extern void *early_memremap(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); extern void *early_memremap_ro(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); +/* + * When supplying the protection value assume the caller knows the + * situation, so the memremap_owner data is not required. + */ extern void *early_memremap_prot(resource_size_t phys_addr, unsigned long size, unsigned long prot_val); extern void early_iounmap(void __iomem *addr, unsigned long size); @@ -41,7 +50,7 @@ extern void early_ioremap_reset(void); * Early copy from unmapped memory to kernel mapped memory. */ extern void copy_from_early_mem(void *dest, phys_addr_t src, - unsigned long size); + unsigned long size, enum memremap_owner owner); #else static inline void early_ioremap_init(void) { } diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c index d71b98b..ad40720 100644 --- a/mm/early_ioremap.c +++ b/mm/early_ioremap.c @@ -34,6 +34,14 @@ void __init __weak early_ioremap_shutdown(void) { } +pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + return prot; +} + void __init early_ioremap_reset(void) { early_ioremap_shutdown(); @@ -213,16 +221,23 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, - FIXMAP_PAGE_NORMAL); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_NORMAL); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #ifdef FIXMAP_PAGE_RO void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_RO); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #endif @@ -236,7 +251,8 @@ early_memremap_prot(resource_size_t phys_addr, unsigned long size, #define MAX_MAP_CHUNK (NR_FIX_BTMAPS << PAGE_SHIFT) -void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) +void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size, + enum memremap_owner owner) { unsigned long slop, clen; char *p; @@ -246,7 +262,7 @@ void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) clen = size; if (clen > MAX_MAP_CHUNK - slop) clen = MAX_MAP_CHUNK - slop; - p = early_memremap(src & PAGE_MASK, clen + slop); + p = early_memremap(src & PAGE_MASK, clen + slop, owner); memcpy(dest, p + slop, clen); early_memunmap(p, clen + slop); dest += clen; @@ -265,12 +281,14 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Lendacky Subject: [RFC PATCH v2 11/20] mm: Access BOOT related data in the clear Date: Mon, 22 Aug 2016 17:37:38 -0500 Message-ID: <20160822223738.29880.6909.stgit@tlendack-t1.amdoffice.net> References: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> Sender: owner-linux-mm@kvack.org To: linux-arch@vger.kernel.org, linux-efi@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: Radim =?utf-8?b?S3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , Konrad Rzeszutek Wilk , Andrey Ryabinin , Ingo Molnar , Borislav Petkov , Andy Lutomirski , "H. Peter Anvin" , Paolo Bonzini , Alexander Potapenko , Thomas Gleixner , Dmitry Vyukov List-Id: linux-efi@vger.kernel.org BOOT data (such as EFI related data) is not encyrpted when the system is booted and needs to be accessed as non-encrypted. Add support to the early_memremap API to identify the type of data being accessed so that the proper encryption attribute can be applied. Currently, two types of data are defined, KERNEL_DATA and BOOT_DATA. Signed-off-by: Tom Lendacky --- arch/arm64/kernel/acpi.c | 2 +- arch/ia64/include/asm/early_ioremap.h | 2 +- arch/x86/kernel/devicetree.c | 6 ++++-- arch/x86/kernel/e820.c | 2 +- arch/x86/kernel/setup.c | 9 +++++--- arch/x86/mm/ioremap.c | 19 +++++++++++++++++ arch/x86/platform/efi/efi.c | 15 +++++++------- arch/x86/platform/efi/efi_64.c | 13 +++++++++--- arch/x86/platform/efi/quirks.c | 4 ++-- arch/x86/xen/mmu.c | 9 +++++--- arch/x86/xen/setup.c | 6 ++++-- drivers/acpi/tables.c | 2 +- drivers/firmware/efi/arm-init.c | 13 +++++++----- drivers/firmware/efi/efi.c | 7 ++++-- drivers/firmware/efi/esrt.c | 4 ++-- drivers/firmware/efi/fake_mem.c | 3 ++- drivers/firmware/efi/memattr.c | 2 +- include/asm-generic/early_ioremap.h | 15 +++++++++++--- mm/early_ioremap.c | 36 +++++++++++++++++++++++++-------- 19 files changed, 117 insertions(+), 52 deletions(-) diff --git a/arch/arm64/kernel/acpi.c b/arch/arm64/kernel/acpi.c index 3e4f1a4..33fdedd 100644 --- a/arch/arm64/kernel/acpi.c +++ b/arch/arm64/kernel/acpi.c @@ -98,7 +98,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size) if (!size) return NULL; - return early_memremap(phys, size); + return early_memremap(phys, size, BOOT_DATA); } void __init __acpi_unmap_table(char *map, unsigned long size) diff --git a/arch/ia64/include/asm/early_ioremap.h b/arch/ia64/include/asm/early_ioremap.h index eec9e1d..bc8c210 100644 --- a/arch/ia64/include/asm/early_ioremap.h +++ b/arch/ia64/include/asm/early_ioremap.h @@ -2,7 +2,7 @@ #define _ASM_IA64_EARLY_IOREMAP_H extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); -#define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) +#define early_memremap(phys_addr, size, owner) early_ioremap(phys_addr, size) extern void early_iounmap (volatile void __iomem *addr, unsigned long size); #define early_memunmap(addr, size) early_iounmap(addr, size) diff --git a/arch/x86/kernel/devicetree.c b/arch/x86/kernel/devicetree.c index 3fe45f8..556e986 100644 --- a/arch/x86/kernel/devicetree.c +++ b/arch/x86/kernel/devicetree.c @@ -276,11 +276,13 @@ static void __init x86_flattree_get_config(void) map_len = max(PAGE_SIZE - (initial_dtb & ~PAGE_MASK), (u64)128); - initial_boot_params = dt = early_memremap(initial_dtb, map_len); + initial_boot_params = dt = early_memremap(initial_dtb, map_len, + BOOT_DATA); size = of_get_flat_dt_size(); if (map_len < size) { early_memunmap(dt, map_len); - initial_boot_params = dt = early_memremap(initial_dtb, size); + initial_boot_params = dt = early_memremap(initial_dtb, size, + BOOT_DATA); map_len = size; } diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c index 621b501..71b237f 100644 --- a/arch/x86/kernel/e820.c +++ b/arch/x86/kernel/e820.c @@ -661,7 +661,7 @@ void __init parse_e820_ext(u64 phys_addr, u32 data_len) struct e820entry *extmap; struct setup_data *sdata; - sdata = early_memremap(phys_addr, data_len); + sdata = early_memremap(phys_addr, data_len, BOOT_DATA); entries = sdata->len / sizeof(struct e820entry); extmap = (struct e820entry *)(sdata->data); __append_e820_map(extmap, entries); diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 1fdaa11..cec8a63 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -344,7 +344,8 @@ static void __init relocate_initrd(void) printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n", relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1); - copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size); + copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size, + BOOT_DATA); printk(KERN_INFO "Move RAMDISK from [mem %#010llx-%#010llx] to" " [mem %#010llx-%#010llx]\n", @@ -426,7 +427,7 @@ static void __init parse_setup_data(void) while (pa_data) { u32 data_len, data_type; - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); data_len = data->len + sizeof(struct setup_data); data_type = data->type; pa_next = data->next; @@ -459,7 +460,7 @@ static void __init e820_reserve_setup_data(void) return; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); e820_update_range(pa_data, sizeof(*data)+data->len, E820_RAM, E820_RESERVED_KERN); pa_data = data->next; @@ -479,7 +480,7 @@ static void __init memblock_x86_reserve_range_setup_data(void) pa_data = boot_params.hdr.setup_data; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); memblock_reserve(pa_data, sizeof(*data) + data->len); pa_data = data->next; early_memunmap(data, sizeof(*data)); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 031db21..e3bdc5a 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -419,6 +419,25 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK)); } +/* + * Architecure override of __weak function to adjust the protection attributes + * used when remapping memory. + */ +pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + /* + * If memory encryption is enabled and BOOT_DATA is being mapped + * then remove the encryption bit. + */ + if (_PAGE_ENC && (owner == BOOT_DATA)) + prot = __pgprot(pgprot_val(prot) & ~_PAGE_ENC); + + return prot; +} + /* Remap memory with encryption */ void __init *early_memremap_enc(resource_size_t phys_addr, unsigned long size) diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c index 1fbb408..2c7e6b0 100644 --- a/arch/x86/platform/efi/efi.c +++ b/arch/x86/platform/efi/efi.c @@ -239,12 +239,13 @@ static int __init efi_systab_init(void *phys) u64 tmp = 0; if (efi_setup) { - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), + BOOT_DATA); if (!data) return -ENOMEM; } systab64 = early_memremap((unsigned long)phys, - sizeof(*systab64)); + sizeof(*systab64), BOOT_DATA); if (systab64 == NULL) { pr_err("Couldn't map the system table!\n"); if (data) @@ -293,7 +294,7 @@ static int __init efi_systab_init(void *phys) efi_system_table_32_t *systab32; systab32 = early_memremap((unsigned long)phys, - sizeof(*systab32)); + sizeof(*systab32), BOOT_DATA); if (systab32 == NULL) { pr_err("Couldn't map the system table!\n"); return -ENOMEM; @@ -338,7 +339,7 @@ static int __init efi_runtime_init32(void) efi_runtime_services_32_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_32_t)); + sizeof(efi_runtime_services_32_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -362,7 +363,7 @@ static int __init efi_runtime_init64(void) efi_runtime_services_64_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_64_t)); + sizeof(efi_runtime_services_64_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -425,7 +426,7 @@ static int __init efi_memmap_init(void) size = efi.memmap.nr_map * efi.memmap.desc_size; addr = (unsigned long)efi.memmap.phys_map; - efi.memmap.map = early_memremap(addr, size); + efi.memmap.map = early_memremap(addr, size, BOOT_DATA); if (efi.memmap.map == NULL) { pr_err("Could not map the memory map!\n"); return -ENOMEM; @@ -471,7 +472,7 @@ void __init efi_init(void) /* * Show what we know for posterity */ - c16 = tmp = early_memremap(efi.systab->fw_vendor, 2); + c16 = tmp = early_memremap(efi.systab->fw_vendor, 2, BOOT_DATA); if (c16) { for (i = 0; i < sizeof(vendor) - 1 && *c16; ++i) vendor[i] = *c16++; diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 677e29e..0871ea4 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -222,7 +222,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) if (efi_enabled(EFI_OLD_MEMMAP)) return 0; - efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd); + /* + * Since the PGD is encrypted, set the encryption mask so that when + * this value is loaded into cr3 the PGD will be decrypted during + * the pagetable walk. + */ + efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd); pgd = efi_pgd; /* @@ -261,7 +266,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) pfn = md->phys_addr >> PAGE_SHIFT; npages = md->num_pages; - if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map 1:1 memory\n"); return 1; } @@ -278,7 +284,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) text = __pa(_text); pfn = text >> PAGE_SHIFT; - if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map kernel text 1:1\n"); return 1; } diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c index 89d1146..606bf551 100644 --- a/arch/x86/platform/efi/quirks.c +++ b/arch/x86/platform/efi/quirks.c @@ -311,7 +311,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) if (!efi_enabled(EFI_64BIT)) return 0; - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), BOOT_DATA); if (!data) { ret = -ENOMEM; goto out; @@ -322,7 +322,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) sz = sizeof(efi_config_table_64_t); - p = tablep = early_memremap(tables, nr_tables * sz); + p = tablep = early_memremap(tables, nr_tables * sz, BOOT_DATA); if (!p) { pr_err("Could not map Configuration table!\n"); ret = -ENOMEM; diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c index 7d5afdb..00db54a 100644 --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -2020,7 +2020,7 @@ static unsigned long __init xen_read_phys_ulong(phys_addr_t addr) unsigned long *vaddr; unsigned long val; - vaddr = early_memremap_ro(addr, sizeof(val)); + vaddr = early_memremap_ro(addr, sizeof(val), KERNEL_DATA); val = *vaddr; early_memunmap(vaddr, sizeof(val)); return val; @@ -2114,15 +2114,16 @@ void __init xen_relocate_p2m(void) pgd = __va(read_cr3()); new_p2m = (unsigned long *)(2 * PGDIR_SIZE); for (idx_pud = 0; idx_pud < n_pud; idx_pud++) { - pud = early_memremap(pud_phys, PAGE_SIZE); + pud = early_memremap(pud_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pud); for (idx_pmd = 0; idx_pmd < min(n_pmd, PTRS_PER_PUD); idx_pmd++) { - pmd = early_memremap(pmd_phys, PAGE_SIZE); + pmd = early_memremap(pmd_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pmd); for (idx_pt = 0; idx_pt < min(n_pt, PTRS_PER_PMD); idx_pt++) { - pt = early_memremap(pt_phys, PAGE_SIZE); + pt = early_memremap(pt_phys, PAGE_SIZE, + KERNEL_DATA); clear_page(pt); for (idx_pte = 0; idx_pte < min(n_pte, PTRS_PER_PTE); diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c index 1764252..a8e2724 100644 --- a/arch/x86/xen/setup.c +++ b/arch/x86/xen/setup.c @@ -686,8 +686,10 @@ static void __init xen_phys_memcpy(phys_addr_t dest, phys_addr_t src, if (src_len > (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off) src_len = (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off; len = min(dest_len, src_len); - to = early_memremap(dest - dest_off, dest_len + dest_off); - from = early_memremap(src - src_off, src_len + src_off); + to = early_memremap(dest - dest_off, dest_len + dest_off, + KERNEL_DATA); + from = early_memremap(src - src_off, src_len + src_off, + KERNEL_DATA); memcpy(to, from, len); early_memunmap(to, dest_len + dest_off); early_memunmap(from, src_len + src_off); diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c index 9f0ad6e..06b75a2 100644 --- a/drivers/acpi/tables.c +++ b/drivers/acpi/tables.c @@ -583,7 +583,7 @@ void __init acpi_table_upgrade(void) if (clen > MAP_CHUNK_SIZE - slop) clen = MAP_CHUNK_SIZE - slop; dest_p = early_memremap(dest_addr & PAGE_MASK, - clen + slop); + clen + slop, BOOT_DATA); memcpy(dest_p + slop, src_p, clen); early_memunmap(dest_p, clen + slop); src_p += clen; diff --git a/drivers/firmware/efi/arm-init.c b/drivers/firmware/efi/arm-init.c index c49d50e..0a3fd48 100644 --- a/drivers/firmware/efi/arm-init.c +++ b/drivers/firmware/efi/arm-init.c @@ -67,7 +67,8 @@ static void __init init_screen_info(void) struct screen_info *si; if (screen_info_table != EFI_INVALID_TABLE_ADDR) { - si = early_memremap_ro(screen_info_table, sizeof(*si)); + si = early_memremap_ro(screen_info_table, sizeof(*si), + BOOT_DATA); if (!si) { pr_err("Could not map screen_info config table\n"); return; @@ -94,7 +95,7 @@ static int __init uefi_init(void) int i, retval; efi.systab = early_memremap_ro(efi_system_table, - sizeof(efi_system_table_t)); + sizeof(efi_system_table_t), BOOT_DATA); if (efi.systab == NULL) { pr_warn("Unable to map EFI system table.\n"); return -ENOMEM; @@ -121,7 +122,8 @@ static int __init uefi_init(void) /* Show what we know for posterity */ c16 = early_memremap_ro(efi_to_phys(efi.systab->fw_vendor), - sizeof(vendor) * sizeof(efi_char16_t)); + sizeof(vendor) * sizeof(efi_char16_t), + BOOT_DATA); if (c16) { for (i = 0; i < (int) sizeof(vendor) - 1 && *c16; ++i) vendor[i] = c16[i]; @@ -135,7 +137,7 @@ static int __init uefi_init(void) table_size = sizeof(efi_config_table_64_t) * efi.systab->nr_tables; config_tables = early_memremap_ro(efi_to_phys(efi.systab->tables), - table_size); + table_size, BOOT_DATA); if (config_tables == NULL) { pr_warn("Unable to map EFI config table array.\n"); retval = -ENOMEM; @@ -226,7 +228,8 @@ void __init efi_init(void) efi_system_table = params.system_table; efi.memmap.phys_map = params.mmap; - efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size); + efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size, + BOOT_DATA); if (efi.memmap.map == NULL) { /* * If we are booting via UEFI, the UEFI memory map is the only diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index 5a2631a..f9286c6 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -386,7 +386,7 @@ int __init efi_mem_desc_lookup(u64 phys_addr, efi_memory_desc_t *out_md) * So just always get our own virtual map on the CPU. * */ - md = early_memremap(p, sizeof (*md)); + md = early_memremap(p, sizeof (*md), BOOT_DATA); if (!md) { pr_err_once("early_memremap(%pa, %zu) failed.\n", &p, sizeof (*md)); @@ -501,7 +501,8 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz, if (efi.properties_table != EFI_INVALID_TABLE_ADDR) { efi_properties_table_t *tbl; - tbl = early_memremap(efi.properties_table, sizeof(*tbl)); + tbl = early_memremap(efi.properties_table, sizeof(*tbl), + BOOT_DATA); if (tbl == NULL) { pr_err("Could not map Properties table!\n"); return -ENOMEM; @@ -531,7 +532,7 @@ int __init efi_config_init(efi_config_table_type_t *arch_tables) * Let's see what config tables the firmware passed to us. */ config_tables = early_memremap(efi.systab->tables, - efi.systab->nr_tables * sz); + efi.systab->nr_tables * sz, BOOT_DATA); if (config_tables == NULL) { pr_err("Could not map Configuration table!\n"); return -ENOMEM; diff --git a/drivers/firmware/efi/esrt.c b/drivers/firmware/efi/esrt.c index 75feb3f..10ee547 100644 --- a/drivers/firmware/efi/esrt.c +++ b/drivers/firmware/efi/esrt.c @@ -273,7 +273,7 @@ void __init efi_esrt_init(void) return; } - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); @@ -323,7 +323,7 @@ void __init efi_esrt_init(void) /* remap it with our (plausible) new pages */ early_memunmap(va, size); size += entries_size; - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); diff --git a/drivers/firmware/efi/fake_mem.c b/drivers/firmware/efi/fake_mem.c index 48430ab..8e87388 100644 --- a/drivers/firmware/efi/fake_mem.c +++ b/drivers/firmware/efi/fake_mem.c @@ -101,7 +101,8 @@ void __init efi_fake_memmap(void) /* create new EFI memmap */ new_memmap = early_memremap(new_memmap_phy, - efi.memmap.desc_size * new_nr_map); + efi.memmap.desc_size * new_nr_map, + BOOT_DATA); if (!new_memmap) { memblock_free(new_memmap_phy, efi.memmap.desc_size * new_nr_map); return; diff --git a/drivers/firmware/efi/memattr.c b/drivers/firmware/efi/memattr.c index 236004b..f351c2a 100644 --- a/drivers/firmware/efi/memattr.c +++ b/drivers/firmware/efi/memattr.c @@ -28,7 +28,7 @@ int __init efi_memattr_init(void) if (efi.mem_attr_table == EFI_INVALID_TABLE_ADDR) return 0; - tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl)); + tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl), BOOT_DATA); if (!tbl) { pr_err("Failed to map EFI Memory Attributes table @ 0x%lx\n", efi.mem_attr_table); diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h index 2edef8d..61de27a 100644 --- a/include/asm-generic/early_ioremap.h +++ b/include/asm-generic/early_ioremap.h @@ -3,6 +3,11 @@ #include +enum memremap_owner { + KERNEL_DATA = 0, + BOOT_DATA, +}; + /* * early_ioremap() and early_iounmap() are for temporary early boot-time * mappings, before the real ioremap() is functional. @@ -10,9 +15,13 @@ extern void __iomem *early_ioremap(resource_size_t phys_addr, unsigned long size); extern void *early_memremap(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); extern void *early_memremap_ro(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); +/* + * When supplying the protection value assume the caller knows the + * situation, so the memremap_owner data is not required. + */ extern void *early_memremap_prot(resource_size_t phys_addr, unsigned long size, unsigned long prot_val); extern void early_iounmap(void __iomem *addr, unsigned long size); @@ -41,7 +50,7 @@ extern void early_ioremap_reset(void); * Early copy from unmapped memory to kernel mapped memory. */ extern void copy_from_early_mem(void *dest, phys_addr_t src, - unsigned long size); + unsigned long size, enum memremap_owner owner); #else static inline void early_ioremap_init(void) { } diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c index d71b98b..ad40720 100644 --- a/mm/early_ioremap.c +++ b/mm/early_ioremap.c @@ -34,6 +34,14 @@ void __init __weak early_ioremap_shutdown(void) { } +pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + return prot; +} + void __init early_ioremap_reset(void) { early_ioremap_shutdown(); @@ -213,16 +221,23 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, - FIXMAP_PAGE_NORMAL); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_NORMAL); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #ifdef FIXMAP_PAGE_RO void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_RO); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #endif @@ -236,7 +251,8 @@ early_memremap_prot(resource_size_t phys_addr, unsigned long size, #define MAX_MAP_CHUNK (NR_FIX_BTMAPS << PAGE_SHIFT) -void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) +void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size, + enum memremap_owner owner) { unsigned long slop, clen; char *p; @@ -246,7 +262,7 @@ void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) clen = size; if (clen > MAX_MAP_CHUNK - slop) clen = MAX_MAP_CHUNK - slop; - p = early_memremap(src & PAGE_MASK, clen + slop); + p = early_memremap(src & PAGE_MASK, clen + slop, owner); memcpy(dest, p + slop, clen); early_memunmap(p, clen + slop); dest += clen; @@ -265,12 +281,14 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-sn1nam02on0089.outbound.protection.outlook.com ([104.47.36.89]:60719 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754398AbcHVWww (ORCPT ); Mon, 22 Aug 2016 18:52:52 -0400 From: Tom Lendacky Subject: [RFC PATCH v2 11/20] mm: Access BOOT related data in the clear Date: Mon, 22 Aug 2016 17:37:38 -0500 Message-ID: <20160822223738.29880.6909.stgit@tlendack-t1.amdoffice.net> In-Reply-To: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> References: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-arch-owner@vger.kernel.org List-ID: To: linux-arch@vger.kernel.org, linux-efi@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: Radim =?utf-8?b?S3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , Konrad Rzeszutek Wilk , Andrey Ryabinin , Ingo Molnar , Borislav Petkov , Andy Lutomirski , "H. Peter Anvin" , Paolo Bonzini , Alexander Potapenko , Thomas Gleixner , Dmitry Vyukov Message-ID: <20160822223738.mHsaIGf0vqL2EZwAnTZsKs9O1kjWm6SsLvHtd5Fr7F8@z> BOOT data (such as EFI related data) is not encyrpted when the system is booted and needs to be accessed as non-encrypted. Add support to the early_memremap API to identify the type of data being accessed so that the proper encryption attribute can be applied. Currently, two types of data are defined, KERNEL_DATA and BOOT_DATA. Signed-off-by: Tom Lendacky --- arch/arm64/kernel/acpi.c | 2 +- arch/ia64/include/asm/early_ioremap.h | 2 +- arch/x86/kernel/devicetree.c | 6 ++++-- arch/x86/kernel/e820.c | 2 +- arch/x86/kernel/setup.c | 9 +++++--- arch/x86/mm/ioremap.c | 19 +++++++++++++++++ arch/x86/platform/efi/efi.c | 15 +++++++------- arch/x86/platform/efi/efi_64.c | 13 +++++++++--- arch/x86/platform/efi/quirks.c | 4 ++-- arch/x86/xen/mmu.c | 9 +++++--- arch/x86/xen/setup.c | 6 ++++-- drivers/acpi/tables.c | 2 +- drivers/firmware/efi/arm-init.c | 13 +++++++----- drivers/firmware/efi/efi.c | 7 ++++-- drivers/firmware/efi/esrt.c | 4 ++-- drivers/firmware/efi/fake_mem.c | 3 ++- drivers/firmware/efi/memattr.c | 2 +- include/asm-generic/early_ioremap.h | 15 +++++++++++--- mm/early_ioremap.c | 36 +++++++++++++++++++++++++-------- 19 files changed, 117 insertions(+), 52 deletions(-) diff --git a/arch/arm64/kernel/acpi.c b/arch/arm64/kernel/acpi.c index 3e4f1a4..33fdedd 100644 --- a/arch/arm64/kernel/acpi.c +++ b/arch/arm64/kernel/acpi.c @@ -98,7 +98,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size) if (!size) return NULL; - return early_memremap(phys, size); + return early_memremap(phys, size, BOOT_DATA); } void __init __acpi_unmap_table(char *map, unsigned long size) diff --git a/arch/ia64/include/asm/early_ioremap.h b/arch/ia64/include/asm/early_ioremap.h index eec9e1d..bc8c210 100644 --- a/arch/ia64/include/asm/early_ioremap.h +++ b/arch/ia64/include/asm/early_ioremap.h @@ -2,7 +2,7 @@ #define _ASM_IA64_EARLY_IOREMAP_H extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); -#define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) +#define early_memremap(phys_addr, size, owner) early_ioremap(phys_addr, size) extern void early_iounmap (volatile void __iomem *addr, unsigned long size); #define early_memunmap(addr, size) early_iounmap(addr, size) diff --git a/arch/x86/kernel/devicetree.c b/arch/x86/kernel/devicetree.c index 3fe45f8..556e986 100644 --- a/arch/x86/kernel/devicetree.c +++ b/arch/x86/kernel/devicetree.c @@ -276,11 +276,13 @@ static void __init x86_flattree_get_config(void) map_len = max(PAGE_SIZE - (initial_dtb & ~PAGE_MASK), (u64)128); - initial_boot_params = dt = early_memremap(initial_dtb, map_len); + initial_boot_params = dt = early_memremap(initial_dtb, map_len, + BOOT_DATA); size = of_get_flat_dt_size(); if (map_len < size) { early_memunmap(dt, map_len); - initial_boot_params = dt = early_memremap(initial_dtb, size); + initial_boot_params = dt = early_memremap(initial_dtb, size, + BOOT_DATA); map_len = size; } diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c index 621b501..71b237f 100644 --- a/arch/x86/kernel/e820.c +++ b/arch/x86/kernel/e820.c @@ -661,7 +661,7 @@ void __init parse_e820_ext(u64 phys_addr, u32 data_len) struct e820entry *extmap; struct setup_data *sdata; - sdata = early_memremap(phys_addr, data_len); + sdata = early_memremap(phys_addr, data_len, BOOT_DATA); entries = sdata->len / sizeof(struct e820entry); extmap = (struct e820entry *)(sdata->data); __append_e820_map(extmap, entries); diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 1fdaa11..cec8a63 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -344,7 +344,8 @@ static void __init relocate_initrd(void) printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n", relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1); - copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size); + copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size, + BOOT_DATA); printk(KERN_INFO "Move RAMDISK from [mem %#010llx-%#010llx] to" " [mem %#010llx-%#010llx]\n", @@ -426,7 +427,7 @@ static void __init parse_setup_data(void) while (pa_data) { u32 data_len, data_type; - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); data_len = data->len + sizeof(struct setup_data); data_type = data->type; pa_next = data->next; @@ -459,7 +460,7 @@ static void __init e820_reserve_setup_data(void) return; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); e820_update_range(pa_data, sizeof(*data)+data->len, E820_RAM, E820_RESERVED_KERN); pa_data = data->next; @@ -479,7 +480,7 @@ static void __init memblock_x86_reserve_range_setup_data(void) pa_data = boot_params.hdr.setup_data; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); memblock_reserve(pa_data, sizeof(*data) + data->len); pa_data = data->next; early_memunmap(data, sizeof(*data)); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 031db21..e3bdc5a 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -419,6 +419,25 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK)); } +/* + * Architecure override of __weak function to adjust the protection attributes + * used when remapping memory. + */ +pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + /* + * If memory encryption is enabled and BOOT_DATA is being mapped + * then remove the encryption bit. + */ + if (_PAGE_ENC && (owner == BOOT_DATA)) + prot = __pgprot(pgprot_val(prot) & ~_PAGE_ENC); + + return prot; +} + /* Remap memory with encryption */ void __init *early_memremap_enc(resource_size_t phys_addr, unsigned long size) diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c index 1fbb408..2c7e6b0 100644 --- a/arch/x86/platform/efi/efi.c +++ b/arch/x86/platform/efi/efi.c @@ -239,12 +239,13 @@ static int __init efi_systab_init(void *phys) u64 tmp = 0; if (efi_setup) { - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), + BOOT_DATA); if (!data) return -ENOMEM; } systab64 = early_memremap((unsigned long)phys, - sizeof(*systab64)); + sizeof(*systab64), BOOT_DATA); if (systab64 == NULL) { pr_err("Couldn't map the system table!\n"); if (data) @@ -293,7 +294,7 @@ static int __init efi_systab_init(void *phys) efi_system_table_32_t *systab32; systab32 = early_memremap((unsigned long)phys, - sizeof(*systab32)); + sizeof(*systab32), BOOT_DATA); if (systab32 == NULL) { pr_err("Couldn't map the system table!\n"); return -ENOMEM; @@ -338,7 +339,7 @@ static int __init efi_runtime_init32(void) efi_runtime_services_32_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_32_t)); + sizeof(efi_runtime_services_32_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -362,7 +363,7 @@ static int __init efi_runtime_init64(void) efi_runtime_services_64_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_64_t)); + sizeof(efi_runtime_services_64_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -425,7 +426,7 @@ static int __init efi_memmap_init(void) size = efi.memmap.nr_map * efi.memmap.desc_size; addr = (unsigned long)efi.memmap.phys_map; - efi.memmap.map = early_memremap(addr, size); + efi.memmap.map = early_memremap(addr, size, BOOT_DATA); if (efi.memmap.map == NULL) { pr_err("Could not map the memory map!\n"); return -ENOMEM; @@ -471,7 +472,7 @@ void __init efi_init(void) /* * Show what we know for posterity */ - c16 = tmp = early_memremap(efi.systab->fw_vendor, 2); + c16 = tmp = early_memremap(efi.systab->fw_vendor, 2, BOOT_DATA); if (c16) { for (i = 0; i < sizeof(vendor) - 1 && *c16; ++i) vendor[i] = *c16++; diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 677e29e..0871ea4 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -222,7 +222,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) if (efi_enabled(EFI_OLD_MEMMAP)) return 0; - efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd); + /* + * Since the PGD is encrypted, set the encryption mask so that when + * this value is loaded into cr3 the PGD will be decrypted during + * the pagetable walk. + */ + efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd); pgd = efi_pgd; /* @@ -261,7 +266,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) pfn = md->phys_addr >> PAGE_SHIFT; npages = md->num_pages; - if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map 1:1 memory\n"); return 1; } @@ -278,7 +284,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) text = __pa(_text); pfn = text >> PAGE_SHIFT; - if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map kernel text 1:1\n"); return 1; } diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c index 89d1146..606bf551 100644 --- a/arch/x86/platform/efi/quirks.c +++ b/arch/x86/platform/efi/quirks.c @@ -311,7 +311,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) if (!efi_enabled(EFI_64BIT)) return 0; - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), BOOT_DATA); if (!data) { ret = -ENOMEM; goto out; @@ -322,7 +322,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) sz = sizeof(efi_config_table_64_t); - p = tablep = early_memremap(tables, nr_tables * sz); + p = tablep = early_memremap(tables, nr_tables * sz, BOOT_DATA); if (!p) { pr_err("Could not map Configuration table!\n"); ret = -ENOMEM; diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c index 7d5afdb..00db54a 100644 --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -2020,7 +2020,7 @@ static unsigned long __init xen_read_phys_ulong(phys_addr_t addr) unsigned long *vaddr; unsigned long val; - vaddr = early_memremap_ro(addr, sizeof(val)); + vaddr = early_memremap_ro(addr, sizeof(val), KERNEL_DATA); val = *vaddr; early_memunmap(vaddr, sizeof(val)); return val; @@ -2114,15 +2114,16 @@ void __init xen_relocate_p2m(void) pgd = __va(read_cr3()); new_p2m = (unsigned long *)(2 * PGDIR_SIZE); for (idx_pud = 0; idx_pud < n_pud; idx_pud++) { - pud = early_memremap(pud_phys, PAGE_SIZE); + pud = early_memremap(pud_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pud); for (idx_pmd = 0; idx_pmd < min(n_pmd, PTRS_PER_PUD); idx_pmd++) { - pmd = early_memremap(pmd_phys, PAGE_SIZE); + pmd = early_memremap(pmd_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pmd); for (idx_pt = 0; idx_pt < min(n_pt, PTRS_PER_PMD); idx_pt++) { - pt = early_memremap(pt_phys, PAGE_SIZE); + pt = early_memremap(pt_phys, PAGE_SIZE, + KERNEL_DATA); clear_page(pt); for (idx_pte = 0; idx_pte < min(n_pte, PTRS_PER_PTE); diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c index 1764252..a8e2724 100644 --- a/arch/x86/xen/setup.c +++ b/arch/x86/xen/setup.c @@ -686,8 +686,10 @@ static void __init xen_phys_memcpy(phys_addr_t dest, phys_addr_t src, if (src_len > (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off) src_len = (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off; len = min(dest_len, src_len); - to = early_memremap(dest - dest_off, dest_len + dest_off); - from = early_memremap(src - src_off, src_len + src_off); + to = early_memremap(dest - dest_off, dest_len + dest_off, + KERNEL_DATA); + from = early_memremap(src - src_off, src_len + src_off, + KERNEL_DATA); memcpy(to, from, len); early_memunmap(to, dest_len + dest_off); early_memunmap(from, src_len + src_off); diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c index 9f0ad6e..06b75a2 100644 --- a/drivers/acpi/tables.c +++ b/drivers/acpi/tables.c @@ -583,7 +583,7 @@ void __init acpi_table_upgrade(void) if (clen > MAP_CHUNK_SIZE - slop) clen = MAP_CHUNK_SIZE - slop; dest_p = early_memremap(dest_addr & PAGE_MASK, - clen + slop); + clen + slop, BOOT_DATA); memcpy(dest_p + slop, src_p, clen); early_memunmap(dest_p, clen + slop); src_p += clen; diff --git a/drivers/firmware/efi/arm-init.c b/drivers/firmware/efi/arm-init.c index c49d50e..0a3fd48 100644 --- a/drivers/firmware/efi/arm-init.c +++ b/drivers/firmware/efi/arm-init.c @@ -67,7 +67,8 @@ static void __init init_screen_info(void) struct screen_info *si; if (screen_info_table != EFI_INVALID_TABLE_ADDR) { - si = early_memremap_ro(screen_info_table, sizeof(*si)); + si = early_memremap_ro(screen_info_table, sizeof(*si), + BOOT_DATA); if (!si) { pr_err("Could not map screen_info config table\n"); return; @@ -94,7 +95,7 @@ static int __init uefi_init(void) int i, retval; efi.systab = early_memremap_ro(efi_system_table, - sizeof(efi_system_table_t)); + sizeof(efi_system_table_t), BOOT_DATA); if (efi.systab == NULL) { pr_warn("Unable to map EFI system table.\n"); return -ENOMEM; @@ -121,7 +122,8 @@ static int __init uefi_init(void) /* Show what we know for posterity */ c16 = early_memremap_ro(efi_to_phys(efi.systab->fw_vendor), - sizeof(vendor) * sizeof(efi_char16_t)); + sizeof(vendor) * sizeof(efi_char16_t), + BOOT_DATA); if (c16) { for (i = 0; i < (int) sizeof(vendor) - 1 && *c16; ++i) vendor[i] = c16[i]; @@ -135,7 +137,7 @@ static int __init uefi_init(void) table_size = sizeof(efi_config_table_64_t) * efi.systab->nr_tables; config_tables = early_memremap_ro(efi_to_phys(efi.systab->tables), - table_size); + table_size, BOOT_DATA); if (config_tables == NULL) { pr_warn("Unable to map EFI config table array.\n"); retval = -ENOMEM; @@ -226,7 +228,8 @@ void __init efi_init(void) efi_system_table = params.system_table; efi.memmap.phys_map = params.mmap; - efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size); + efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size, + BOOT_DATA); if (efi.memmap.map == NULL) { /* * If we are booting via UEFI, the UEFI memory map is the only diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index 5a2631a..f9286c6 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -386,7 +386,7 @@ int __init efi_mem_desc_lookup(u64 phys_addr, efi_memory_desc_t *out_md) * So just always get our own virtual map on the CPU. * */ - md = early_memremap(p, sizeof (*md)); + md = early_memremap(p, sizeof (*md), BOOT_DATA); if (!md) { pr_err_once("early_memremap(%pa, %zu) failed.\n", &p, sizeof (*md)); @@ -501,7 +501,8 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz, if (efi.properties_table != EFI_INVALID_TABLE_ADDR) { efi_properties_table_t *tbl; - tbl = early_memremap(efi.properties_table, sizeof(*tbl)); + tbl = early_memremap(efi.properties_table, sizeof(*tbl), + BOOT_DATA); if (tbl == NULL) { pr_err("Could not map Properties table!\n"); return -ENOMEM; @@ -531,7 +532,7 @@ int __init efi_config_init(efi_config_table_type_t *arch_tables) * Let's see what config tables the firmware passed to us. */ config_tables = early_memremap(efi.systab->tables, - efi.systab->nr_tables * sz); + efi.systab->nr_tables * sz, BOOT_DATA); if (config_tables == NULL) { pr_err("Could not map Configuration table!\n"); return -ENOMEM; diff --git a/drivers/firmware/efi/esrt.c b/drivers/firmware/efi/esrt.c index 75feb3f..10ee547 100644 --- a/drivers/firmware/efi/esrt.c +++ b/drivers/firmware/efi/esrt.c @@ -273,7 +273,7 @@ void __init efi_esrt_init(void) return; } - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); @@ -323,7 +323,7 @@ void __init efi_esrt_init(void) /* remap it with our (plausible) new pages */ early_memunmap(va, size); size += entries_size; - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); diff --git a/drivers/firmware/efi/fake_mem.c b/drivers/firmware/efi/fake_mem.c index 48430ab..8e87388 100644 --- a/drivers/firmware/efi/fake_mem.c +++ b/drivers/firmware/efi/fake_mem.c @@ -101,7 +101,8 @@ void __init efi_fake_memmap(void) /* create new EFI memmap */ new_memmap = early_memremap(new_memmap_phy, - efi.memmap.desc_size * new_nr_map); + efi.memmap.desc_size * new_nr_map, + BOOT_DATA); if (!new_memmap) { memblock_free(new_memmap_phy, efi.memmap.desc_size * new_nr_map); return; diff --git a/drivers/firmware/efi/memattr.c b/drivers/firmware/efi/memattr.c index 236004b..f351c2a 100644 --- a/drivers/firmware/efi/memattr.c +++ b/drivers/firmware/efi/memattr.c @@ -28,7 +28,7 @@ int __init efi_memattr_init(void) if (efi.mem_attr_table == EFI_INVALID_TABLE_ADDR) return 0; - tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl)); + tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl), BOOT_DATA); if (!tbl) { pr_err("Failed to map EFI Memory Attributes table @ 0x%lx\n", efi.mem_attr_table); diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h index 2edef8d..61de27a 100644 --- a/include/asm-generic/early_ioremap.h +++ b/include/asm-generic/early_ioremap.h @@ -3,6 +3,11 @@ #include +enum memremap_owner { + KERNEL_DATA = 0, + BOOT_DATA, +}; + /* * early_ioremap() and early_iounmap() are for temporary early boot-time * mappings, before the real ioremap() is functional. @@ -10,9 +15,13 @@ extern void __iomem *early_ioremap(resource_size_t phys_addr, unsigned long size); extern void *early_memremap(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); extern void *early_memremap_ro(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); +/* + * When supplying the protection value assume the caller knows the + * situation, so the memremap_owner data is not required. + */ extern void *early_memremap_prot(resource_size_t phys_addr, unsigned long size, unsigned long prot_val); extern void early_iounmap(void __iomem *addr, unsigned long size); @@ -41,7 +50,7 @@ extern void early_ioremap_reset(void); * Early copy from unmapped memory to kernel mapped memory. */ extern void copy_from_early_mem(void *dest, phys_addr_t src, - unsigned long size); + unsigned long size, enum memremap_owner owner); #else static inline void early_ioremap_init(void) { } diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c index d71b98b..ad40720 100644 --- a/mm/early_ioremap.c +++ b/mm/early_ioremap.c @@ -34,6 +34,14 @@ void __init __weak early_ioremap_shutdown(void) { } +pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + return prot; +} + void __init early_ioremap_reset(void) { early_ioremap_shutdown(); @@ -213,16 +221,23 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, - FIXMAP_PAGE_NORMAL); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_NORMAL); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #ifdef FIXMAP_PAGE_RO void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_RO); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #endif @@ -236,7 +251,8 @@ early_memremap_prot(resource_size_t phys_addr, unsigned long size, #define MAX_MAP_CHUNK (NR_FIX_BTMAPS << PAGE_SHIFT) -void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) +void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size, + enum memremap_owner owner) { unsigned long slop, clen; char *p; @@ -246,7 +262,7 @@ void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) clen = size; if (clen > MAX_MAP_CHUNK - slop) clen = MAX_MAP_CHUNK - slop; - p = early_memremap(src & PAGE_MASK, clen + slop); + p = early_memremap(src & PAGE_MASK, clen + slop, owner); memcpy(dest, p + slop, clen); early_memunmap(p, clen + slop); dest += clen; @@ -265,12 +281,14 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Lendacky Subject: [RFC PATCH v2 11/20] mm: Access BOOT related data in the clear Date: Mon, 22 Aug 2016 17:37:38 -0500 Message-ID: <20160822223738.29880.6909.stgit@tlendack-t1.amdoffice.net> References: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Cc: Radim =?utf-8?b?S3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , "Konrad Rzeszutek Wilk" , Andrey Ryabinin , Ingo Molnar , Borislav Petkov , "Andy Lutomirski" , "H. Peter Anvin" , Paolo Bonzini , Alexander Potapenko , "Thomas Gleixner" , Dmitry Vyukov To: , , , , , , , , Return-path: In-Reply-To: <20160822223529.29880.50884.stgit@tlendack-t1.amdoffice.net> Sender: owner-linux-mm@kvack.org List-Id: kvm.vger.kernel.org BOOT data (such as EFI related data) is not encyrpted when the system is booted and needs to be accessed as non-encrypted. Add support to the early_memremap API to identify the type of data being accessed so that the proper encryption attribute can be applied. Currently, two types of data are defined, KERNEL_DATA and BOOT_DATA. Signed-off-by: Tom Lendacky --- arch/arm64/kernel/acpi.c | 2 +- arch/ia64/include/asm/early_ioremap.h | 2 +- arch/x86/kernel/devicetree.c | 6 ++++-- arch/x86/kernel/e820.c | 2 +- arch/x86/kernel/setup.c | 9 +++++--- arch/x86/mm/ioremap.c | 19 +++++++++++++++++ arch/x86/platform/efi/efi.c | 15 +++++++------- arch/x86/platform/efi/efi_64.c | 13 +++++++++--- arch/x86/platform/efi/quirks.c | 4 ++-- arch/x86/xen/mmu.c | 9 +++++--- arch/x86/xen/setup.c | 6 ++++-- drivers/acpi/tables.c | 2 +- drivers/firmware/efi/arm-init.c | 13 +++++++----- drivers/firmware/efi/efi.c | 7 ++++-- drivers/firmware/efi/esrt.c | 4 ++-- drivers/firmware/efi/fake_mem.c | 3 ++- drivers/firmware/efi/memattr.c | 2 +- include/asm-generic/early_ioremap.h | 15 +++++++++++--- mm/early_ioremap.c | 36 +++++++++++++++++++++++++-------- 19 files changed, 117 insertions(+), 52 deletions(-) diff --git a/arch/arm64/kernel/acpi.c b/arch/arm64/kernel/acpi.c index 3e4f1a4..33fdedd 100644 --- a/arch/arm64/kernel/acpi.c +++ b/arch/arm64/kernel/acpi.c @@ -98,7 +98,7 @@ char *__init __acpi_map_table(unsigned long phys, unsigned long size) if (!size) return NULL; - return early_memremap(phys, size); + return early_memremap(phys, size, BOOT_DATA); } void __init __acpi_unmap_table(char *map, unsigned long size) diff --git a/arch/ia64/include/asm/early_ioremap.h b/arch/ia64/include/asm/early_ioremap.h index eec9e1d..bc8c210 100644 --- a/arch/ia64/include/asm/early_ioremap.h +++ b/arch/ia64/include/asm/early_ioremap.h @@ -2,7 +2,7 @@ #define _ASM_IA64_EARLY_IOREMAP_H extern void __iomem * early_ioremap (unsigned long phys_addr, unsigned long size); -#define early_memremap(phys_addr, size) early_ioremap(phys_addr, size) +#define early_memremap(phys_addr, size, owner) early_ioremap(phys_addr, size) extern void early_iounmap (volatile void __iomem *addr, unsigned long size); #define early_memunmap(addr, size) early_iounmap(addr, size) diff --git a/arch/x86/kernel/devicetree.c b/arch/x86/kernel/devicetree.c index 3fe45f8..556e986 100644 --- a/arch/x86/kernel/devicetree.c +++ b/arch/x86/kernel/devicetree.c @@ -276,11 +276,13 @@ static void __init x86_flattree_get_config(void) map_len = max(PAGE_SIZE - (initial_dtb & ~PAGE_MASK), (u64)128); - initial_boot_params = dt = early_memremap(initial_dtb, map_len); + initial_boot_params = dt = early_memremap(initial_dtb, map_len, + BOOT_DATA); size = of_get_flat_dt_size(); if (map_len < size) { early_memunmap(dt, map_len); - initial_boot_params = dt = early_memremap(initial_dtb, size); + initial_boot_params = dt = early_memremap(initial_dtb, size, + BOOT_DATA); map_len = size; } diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c index 621b501..71b237f 100644 --- a/arch/x86/kernel/e820.c +++ b/arch/x86/kernel/e820.c @@ -661,7 +661,7 @@ void __init parse_e820_ext(u64 phys_addr, u32 data_len) struct e820entry *extmap; struct setup_data *sdata; - sdata = early_memremap(phys_addr, data_len); + sdata = early_memremap(phys_addr, data_len, BOOT_DATA); entries = sdata->len / sizeof(struct e820entry); extmap = (struct e820entry *)(sdata->data); __append_e820_map(extmap, entries); diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 1fdaa11..cec8a63 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -344,7 +344,8 @@ static void __init relocate_initrd(void) printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n", relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1); - copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size); + copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size, + BOOT_DATA); printk(KERN_INFO "Move RAMDISK from [mem %#010llx-%#010llx] to" " [mem %#010llx-%#010llx]\n", @@ -426,7 +427,7 @@ static void __init parse_setup_data(void) while (pa_data) { u32 data_len, data_type; - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); data_len = data->len + sizeof(struct setup_data); data_type = data->type; pa_next = data->next; @@ -459,7 +460,7 @@ static void __init e820_reserve_setup_data(void) return; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); e820_update_range(pa_data, sizeof(*data)+data->len, E820_RAM, E820_RESERVED_KERN); pa_data = data->next; @@ -479,7 +480,7 @@ static void __init memblock_x86_reserve_range_setup_data(void) pa_data = boot_params.hdr.setup_data; while (pa_data) { - data = early_memremap(pa_data, sizeof(*data)); + data = early_memremap(pa_data, sizeof(*data), BOOT_DATA); memblock_reserve(pa_data, sizeof(*data) + data->len); pa_data = data->next; early_memunmap(data, sizeof(*data)); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 031db21..e3bdc5a 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -419,6 +419,25 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK)); } +/* + * Architecure override of __weak function to adjust the protection attributes + * used when remapping memory. + */ +pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + /* + * If memory encryption is enabled and BOOT_DATA is being mapped + * then remove the encryption bit. + */ + if (_PAGE_ENC && (owner == BOOT_DATA)) + prot = __pgprot(pgprot_val(prot) & ~_PAGE_ENC); + + return prot; +} + /* Remap memory with encryption */ void __init *early_memremap_enc(resource_size_t phys_addr, unsigned long size) diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c index 1fbb408..2c7e6b0 100644 --- a/arch/x86/platform/efi/efi.c +++ b/arch/x86/platform/efi/efi.c @@ -239,12 +239,13 @@ static int __init efi_systab_init(void *phys) u64 tmp = 0; if (efi_setup) { - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), + BOOT_DATA); if (!data) return -ENOMEM; } systab64 = early_memremap((unsigned long)phys, - sizeof(*systab64)); + sizeof(*systab64), BOOT_DATA); if (systab64 == NULL) { pr_err("Couldn't map the system table!\n"); if (data) @@ -293,7 +294,7 @@ static int __init efi_systab_init(void *phys) efi_system_table_32_t *systab32; systab32 = early_memremap((unsigned long)phys, - sizeof(*systab32)); + sizeof(*systab32), BOOT_DATA); if (systab32 == NULL) { pr_err("Couldn't map the system table!\n"); return -ENOMEM; @@ -338,7 +339,7 @@ static int __init efi_runtime_init32(void) efi_runtime_services_32_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_32_t)); + sizeof(efi_runtime_services_32_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -362,7 +363,7 @@ static int __init efi_runtime_init64(void) efi_runtime_services_64_t *runtime; runtime = early_memremap((unsigned long)efi.systab->runtime, - sizeof(efi_runtime_services_64_t)); + sizeof(efi_runtime_services_64_t), BOOT_DATA); if (!runtime) { pr_err("Could not map the runtime service table!\n"); return -ENOMEM; @@ -425,7 +426,7 @@ static int __init efi_memmap_init(void) size = efi.memmap.nr_map * efi.memmap.desc_size; addr = (unsigned long)efi.memmap.phys_map; - efi.memmap.map = early_memremap(addr, size); + efi.memmap.map = early_memremap(addr, size, BOOT_DATA); if (efi.memmap.map == NULL) { pr_err("Could not map the memory map!\n"); return -ENOMEM; @@ -471,7 +472,7 @@ void __init efi_init(void) /* * Show what we know for posterity */ - c16 = tmp = early_memremap(efi.systab->fw_vendor, 2); + c16 = tmp = early_memremap(efi.systab->fw_vendor, 2, BOOT_DATA); if (c16) { for (i = 0; i < sizeof(vendor) - 1 && *c16; ++i) vendor[i] = *c16++; diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 677e29e..0871ea4 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -222,7 +222,12 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) if (efi_enabled(EFI_OLD_MEMMAP)) return 0; - efi_scratch.efi_pgt = (pgd_t *)__pa(efi_pgd); + /* + * Since the PGD is encrypted, set the encryption mask so that when + * this value is loaded into cr3 the PGD will be decrypted during + * the pagetable walk. + */ + efi_scratch.efi_pgt = (pgd_t *)__sme_pa(efi_pgd); pgd = efi_pgd; /* @@ -261,7 +266,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) pfn = md->phys_addr >> PAGE_SHIFT; npages = md->num_pages; - if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map 1:1 memory\n"); return 1; } @@ -278,7 +284,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) text = __pa(_text); pfn = text >> PAGE_SHIFT; - if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, _PAGE_RW)) { + if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, + _PAGE_RW | _PAGE_ENC)) { pr_err("Failed to map kernel text 1:1\n"); return 1; } diff --git a/arch/x86/platform/efi/quirks.c b/arch/x86/platform/efi/quirks.c index 89d1146..606bf551 100644 --- a/arch/x86/platform/efi/quirks.c +++ b/arch/x86/platform/efi/quirks.c @@ -311,7 +311,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) if (!efi_enabled(EFI_64BIT)) return 0; - data = early_memremap(efi_setup, sizeof(*data)); + data = early_memremap(efi_setup, sizeof(*data), BOOT_DATA); if (!data) { ret = -ENOMEM; goto out; @@ -322,7 +322,7 @@ int __init efi_reuse_config(u64 tables, int nr_tables) sz = sizeof(efi_config_table_64_t); - p = tablep = early_memremap(tables, nr_tables * sz); + p = tablep = early_memremap(tables, nr_tables * sz, BOOT_DATA); if (!p) { pr_err("Could not map Configuration table!\n"); ret = -ENOMEM; diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c index 7d5afdb..00db54a 100644 --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -2020,7 +2020,7 @@ static unsigned long __init xen_read_phys_ulong(phys_addr_t addr) unsigned long *vaddr; unsigned long val; - vaddr = early_memremap_ro(addr, sizeof(val)); + vaddr = early_memremap_ro(addr, sizeof(val), KERNEL_DATA); val = *vaddr; early_memunmap(vaddr, sizeof(val)); return val; @@ -2114,15 +2114,16 @@ void __init xen_relocate_p2m(void) pgd = __va(read_cr3()); new_p2m = (unsigned long *)(2 * PGDIR_SIZE); for (idx_pud = 0; idx_pud < n_pud; idx_pud++) { - pud = early_memremap(pud_phys, PAGE_SIZE); + pud = early_memremap(pud_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pud); for (idx_pmd = 0; idx_pmd < min(n_pmd, PTRS_PER_PUD); idx_pmd++) { - pmd = early_memremap(pmd_phys, PAGE_SIZE); + pmd = early_memremap(pmd_phys, PAGE_SIZE, KERNEL_DATA); clear_page(pmd); for (idx_pt = 0; idx_pt < min(n_pt, PTRS_PER_PMD); idx_pt++) { - pt = early_memremap(pt_phys, PAGE_SIZE); + pt = early_memremap(pt_phys, PAGE_SIZE, + KERNEL_DATA); clear_page(pt); for (idx_pte = 0; idx_pte < min(n_pte, PTRS_PER_PTE); diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c index 1764252..a8e2724 100644 --- a/arch/x86/xen/setup.c +++ b/arch/x86/xen/setup.c @@ -686,8 +686,10 @@ static void __init xen_phys_memcpy(phys_addr_t dest, phys_addr_t src, if (src_len > (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off) src_len = (NR_FIX_BTMAPS << PAGE_SHIFT) - src_off; len = min(dest_len, src_len); - to = early_memremap(dest - dest_off, dest_len + dest_off); - from = early_memremap(src - src_off, src_len + src_off); + to = early_memremap(dest - dest_off, dest_len + dest_off, + KERNEL_DATA); + from = early_memremap(src - src_off, src_len + src_off, + KERNEL_DATA); memcpy(to, from, len); early_memunmap(to, dest_len + dest_off); early_memunmap(from, src_len + src_off); diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c index 9f0ad6e..06b75a2 100644 --- a/drivers/acpi/tables.c +++ b/drivers/acpi/tables.c @@ -583,7 +583,7 @@ void __init acpi_table_upgrade(void) if (clen > MAP_CHUNK_SIZE - slop) clen = MAP_CHUNK_SIZE - slop; dest_p = early_memremap(dest_addr & PAGE_MASK, - clen + slop); + clen + slop, BOOT_DATA); memcpy(dest_p + slop, src_p, clen); early_memunmap(dest_p, clen + slop); src_p += clen; diff --git a/drivers/firmware/efi/arm-init.c b/drivers/firmware/efi/arm-init.c index c49d50e..0a3fd48 100644 --- a/drivers/firmware/efi/arm-init.c +++ b/drivers/firmware/efi/arm-init.c @@ -67,7 +67,8 @@ static void __init init_screen_info(void) struct screen_info *si; if (screen_info_table != EFI_INVALID_TABLE_ADDR) { - si = early_memremap_ro(screen_info_table, sizeof(*si)); + si = early_memremap_ro(screen_info_table, sizeof(*si), + BOOT_DATA); if (!si) { pr_err("Could not map screen_info config table\n"); return; @@ -94,7 +95,7 @@ static int __init uefi_init(void) int i, retval; efi.systab = early_memremap_ro(efi_system_table, - sizeof(efi_system_table_t)); + sizeof(efi_system_table_t), BOOT_DATA); if (efi.systab == NULL) { pr_warn("Unable to map EFI system table.\n"); return -ENOMEM; @@ -121,7 +122,8 @@ static int __init uefi_init(void) /* Show what we know for posterity */ c16 = early_memremap_ro(efi_to_phys(efi.systab->fw_vendor), - sizeof(vendor) * sizeof(efi_char16_t)); + sizeof(vendor) * sizeof(efi_char16_t), + BOOT_DATA); if (c16) { for (i = 0; i < (int) sizeof(vendor) - 1 && *c16; ++i) vendor[i] = c16[i]; @@ -135,7 +137,7 @@ static int __init uefi_init(void) table_size = sizeof(efi_config_table_64_t) * efi.systab->nr_tables; config_tables = early_memremap_ro(efi_to_phys(efi.systab->tables), - table_size); + table_size, BOOT_DATA); if (config_tables == NULL) { pr_warn("Unable to map EFI config table array.\n"); retval = -ENOMEM; @@ -226,7 +228,8 @@ void __init efi_init(void) efi_system_table = params.system_table; efi.memmap.phys_map = params.mmap; - efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size); + efi.memmap.map = early_memremap_ro(params.mmap, params.mmap_size, + BOOT_DATA); if (efi.memmap.map == NULL) { /* * If we are booting via UEFI, the UEFI memory map is the only diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index 5a2631a..f9286c6 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -386,7 +386,7 @@ int __init efi_mem_desc_lookup(u64 phys_addr, efi_memory_desc_t *out_md) * So just always get our own virtual map on the CPU. * */ - md = early_memremap(p, sizeof (*md)); + md = early_memremap(p, sizeof (*md), BOOT_DATA); if (!md) { pr_err_once("early_memremap(%pa, %zu) failed.\n", &p, sizeof (*md)); @@ -501,7 +501,8 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz, if (efi.properties_table != EFI_INVALID_TABLE_ADDR) { efi_properties_table_t *tbl; - tbl = early_memremap(efi.properties_table, sizeof(*tbl)); + tbl = early_memremap(efi.properties_table, sizeof(*tbl), + BOOT_DATA); if (tbl == NULL) { pr_err("Could not map Properties table!\n"); return -ENOMEM; @@ -531,7 +532,7 @@ int __init efi_config_init(efi_config_table_type_t *arch_tables) * Let's see what config tables the firmware passed to us. */ config_tables = early_memremap(efi.systab->tables, - efi.systab->nr_tables * sz); + efi.systab->nr_tables * sz, BOOT_DATA); if (config_tables == NULL) { pr_err("Could not map Configuration table!\n"); return -ENOMEM; diff --git a/drivers/firmware/efi/esrt.c b/drivers/firmware/efi/esrt.c index 75feb3f..10ee547 100644 --- a/drivers/firmware/efi/esrt.c +++ b/drivers/firmware/efi/esrt.c @@ -273,7 +273,7 @@ void __init efi_esrt_init(void) return; } - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); @@ -323,7 +323,7 @@ void __init efi_esrt_init(void) /* remap it with our (plausible) new pages */ early_memunmap(va, size); size += entries_size; - va = early_memremap(efi.esrt, size); + va = early_memremap(efi.esrt, size, BOOT_DATA); if (!va) { pr_err("early_memremap(%p, %zu) failed.\n", (void *)efi.esrt, size); diff --git a/drivers/firmware/efi/fake_mem.c b/drivers/firmware/efi/fake_mem.c index 48430ab..8e87388 100644 --- a/drivers/firmware/efi/fake_mem.c +++ b/drivers/firmware/efi/fake_mem.c @@ -101,7 +101,8 @@ void __init efi_fake_memmap(void) /* create new EFI memmap */ new_memmap = early_memremap(new_memmap_phy, - efi.memmap.desc_size * new_nr_map); + efi.memmap.desc_size * new_nr_map, + BOOT_DATA); if (!new_memmap) { memblock_free(new_memmap_phy, efi.memmap.desc_size * new_nr_map); return; diff --git a/drivers/firmware/efi/memattr.c b/drivers/firmware/efi/memattr.c index 236004b..f351c2a 100644 --- a/drivers/firmware/efi/memattr.c +++ b/drivers/firmware/efi/memattr.c @@ -28,7 +28,7 @@ int __init efi_memattr_init(void) if (efi.mem_attr_table == EFI_INVALID_TABLE_ADDR) return 0; - tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl)); + tbl = early_memremap(efi.mem_attr_table, sizeof(*tbl), BOOT_DATA); if (!tbl) { pr_err("Failed to map EFI Memory Attributes table @ 0x%lx\n", efi.mem_attr_table); diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h index 2edef8d..61de27a 100644 --- a/include/asm-generic/early_ioremap.h +++ b/include/asm-generic/early_ioremap.h @@ -3,6 +3,11 @@ #include +enum memremap_owner { + KERNEL_DATA = 0, + BOOT_DATA, +}; + /* * early_ioremap() and early_iounmap() are for temporary early boot-time * mappings, before the real ioremap() is functional. @@ -10,9 +15,13 @@ extern void __iomem *early_ioremap(resource_size_t phys_addr, unsigned long size); extern void *early_memremap(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); extern void *early_memremap_ro(resource_size_t phys_addr, - unsigned long size); + unsigned long size, enum memremap_owner); +/* + * When supplying the protection value assume the caller knows the + * situation, so the memremap_owner data is not required. + */ extern void *early_memremap_prot(resource_size_t phys_addr, unsigned long size, unsigned long prot_val); extern void early_iounmap(void __iomem *addr, unsigned long size); @@ -41,7 +50,7 @@ extern void early_ioremap_reset(void); * Early copy from unmapped memory to kernel mapped memory. */ extern void copy_from_early_mem(void *dest, phys_addr_t src, - unsigned long size); + unsigned long size, enum memremap_owner owner); #else static inline void early_ioremap_init(void) { } diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c index d71b98b..ad40720 100644 --- a/mm/early_ioremap.c +++ b/mm/early_ioremap.c @@ -34,6 +34,14 @@ void __init __weak early_ioremap_shutdown(void) { } +pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr, + unsigned long size, + enum memremap_owner owner, + pgprot_t prot) +{ + return prot; +} + void __init early_ioremap_reset(void) { early_ioremap_shutdown(); @@ -213,16 +221,23 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, - FIXMAP_PAGE_NORMAL); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_NORMAL); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #ifdef FIXMAP_PAGE_RO void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { - return (__force void *)__early_ioremap(phys_addr, size, FIXMAP_PAGE_RO); + pgprot_t prot = early_memremap_pgprot_adjust(phys_addr, size, owner, + FIXMAP_PAGE_RO); + + return (__force void *)__early_ioremap(phys_addr, size, prot); } #endif @@ -236,7 +251,8 @@ early_memremap_prot(resource_size_t phys_addr, unsigned long size, #define MAX_MAP_CHUNK (NR_FIX_BTMAPS << PAGE_SHIFT) -void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) +void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size, + enum memremap_owner owner) { unsigned long slop, clen; char *p; @@ -246,7 +262,7 @@ void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size) clen = size; if (clen > MAX_MAP_CHUNK - slop) clen = MAX_MAP_CHUNK - slop; - p = early_memremap(src & PAGE_MASK, clen + slop); + p = early_memremap(src & PAGE_MASK, clen + slop, owner); memcpy(dest, p + slop, clen); early_memunmap(p, clen + slop); dest += clen; @@ -265,12 +281,14 @@ early_ioremap(resource_size_t phys_addr, unsigned long size) /* Remap memory */ void __init * -early_memremap(resource_size_t phys_addr, unsigned long size) +early_memremap(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } void __init * -early_memremap_ro(resource_size_t phys_addr, unsigned long size) +early_memremap_ro(resource_size_t phys_addr, unsigned long size, + enum memremap_owner owner) { return (void *)phys_addr; } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org