From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753567AbcDZW5q (ORCPT ); Tue, 26 Apr 2016 18:57:46 -0400 Received: from mail-bl2on0086.outbound.protection.outlook.com ([65.55.169.86]:2896 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753227AbcDZW5l (ORCPT ); Tue, 26 Apr 2016 18:57:41 -0400 Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=amd.com; From: Tom Lendacky Subject: [RFC PATCH v1 09/18] x86: Insure that memory areas are encrypted when possible To: , , , , , , , , CC: Radim =?utf-8?b?S3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , "Konrad Rzeszutek Wilk" , Paolo Bonzini , "Ingo Molnar" , Borislav Petkov , "H. Peter Anvin" , Andrey Ryabinin , "Alexander Potapenko" , Thomas Gleixner , "Dmitry Vyukov" Date: Tue, 26 Apr 2016 17:57:29 -0500 Message-ID: <20160426225729.13567.577.stgit@tlendack-t1.amdoffice.net> In-Reply-To: <20160426225553.13567.19459.stgit@tlendack-t1.amdoffice.net> References: <20160426225553.13567.19459.stgit@tlendack-t1.amdoffice.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: SN1PR15CA0010.namprd15.prod.outlook.com (10.163.200.20) To DM3PR1201MB1118.namprd12.prod.outlook.com (10.164.198.18) X-MS-Office365-Filtering-Correlation-Id: a3f1fc85-3deb-4349-3862-08d36e262819 X-Microsoft-Exchange-Diagnostics: 1;DM3PR1201MB1118;2:f1qrtokU/Pa+FD+DMVyPdUkgyd1X7nyjYMjTSVj+39fubXxoqUjnAZLfXAoz8xTWYYswld1Rch8yG0yl5f/2inusLMmChr3NQCBPZvxMjZGJZh7mg7/6G0MZFdVZoFK/YkNNQhFmQf/qpRpnfEqlvl0w+tmjyLhYjpri/84nQ/cKyho071wuj76fmfmGtxUO;3:aG6RRGPE0zrJkgOgrmXsDBFaVgMCg1IPd4asvkE6CcknK42pvocSLQfp7FjzWRzonJSqtLte3lNHpOJep/Ue8G+M1UJj2t0fP6YLrcVYFctYtqeCAgyg2fl4VwWUtOfT;25:saoVZB67n4g2jDOUJtwc+CZ83GTDzl1AFNxvEWTcF62e4+8atl+xSWbDgnnAJ7O0HhVs9YOZYD0Iauqa/Vt6oSzbl59pCGqrWSQCJsJcib5Hq3llFAJynCBPBk6TGuA7nYgtTfoFE0BwjAw6P6RXSwxvuFLih2jyBu5PG1vpsGxAWG8YghO5VF9EmcKJTCVrV2PGkWL6cEAzpZShSrrFTy4NyOIDVbstDYNFwt79LT3r+98B2dtKSweurQfjzOncDygxKqp8hzx81Iva5tuCgCCWPwjTzvqffxYXvIsZL33mRZagutv283LcaRUlT623EgQ7yV1lKtI+jfBJFW23Bg== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM3PR1201MB1118; X-Microsoft-Exchange-Diagnostics: 1;DM3PR1201MB1118;20:AIaTO8C5jIeFSsopWFyGszQIWS+ETzFYcDQbm2XRxgJ26OrzxuKh3OZZWFCmIUpfK+QpbknWrdnxXYBO3ishWKWa9Ucqh/gX8VS5neC88lQNaP/DgAX0b0/Kx9GVxNKoOgFIBl8LbqYRf2nFIpjPw1G1+xpFCUjAygKePk8j4pSDutECwSVIH894zCs7zSJv6YDmrkOyVWo0BLzCNtzEIBJ/Wy3NhaCpBwHIXksLtE6s3QT2Q3YOH44Jg83nOO9/cm1WvcvpywqVC3hcqtgdzg/J9AkPw47XOXemUx8/na2fgwelDzSgobydgmi2LI+y+IDK+sbVJ0LyRZKOG4Fkh5yx5hd59hQpgwkFiQncACaVbwOWDE+ETHp8/gpm9j/l5FLMlxOhhcAefL3fafpY61+1QHpO6/G3aVpK8U1ZhYXE7jLMlMl5qqgiUp05NhGbMjnZ+gPPiwPPsiJy1cWaRe470zEGvfFC9i5jZeqGMNlbu8csUdB0MhF0DQ0vppsv X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(187975280028177); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(9101521072)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026);SRVR:DM3PR1201MB1118;BCL:0;PCL:0;RULEID:;SRVR:DM3PR1201MB1118; X-Microsoft-Exchange-Diagnostics: 1;DM3PR1201MB1118;4:fru8gu4BthR3gUItlrJYakOJJn/Owo26Ratkjm5LSBMxyBhzJocYkjI4ch8e3CooBpEoLQyoimfBD41kS5XJoxMd9zeplIkHncf/ZDrxtrGXcxJbGzdHW/o+LWbriaG4+bEKKigTFexHlS3w2MkGskdZNP2pX5tltGbLtNnXoa1SSW0NY1CqKBtpBUyAMeZMq++CFvJXRhu1RNRkLTiJjG7ko8kKQd22DsoGGFNGk9krge/1qNVwTkbHZWD2/2RXNYdkj7VD9FUpYOdpDX093CYgTS3BsfyTjheYtMrIYCSW1XB0g4q/ZNelEQvcVF6G4Z5qVSUFlf8T/IhChEFhFbKJb7wAID+iDjx+n8yf+wVDWXaU1Iy34vCvj8JuywrJyQiHaJy1JtVNjY/2XRiGz+ZAn9eKbGWfH+5tZrxOQuq/bAraEEicw3bT7kljnILV+DLRytCeCSkj4bK2nSNu3g== X-Forefront-PRVS: 0924C6A0D5 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(229853001)(5001770100001)(4326007)(92566002)(1096002)(77096005)(23676002)(2906002)(2201001)(5004730100002)(4001350100001)(6116002)(53416004)(81166005)(2950100001)(586003)(33646002)(189998001)(1076002)(86362001)(50986999)(103116003)(42186005)(66066001)(19580395003)(19580405001)(47776003)(9686002)(50466002)(97746001)(54356999)(5008740100001)(230700001)(76176999)(71626007)(217873001);DIR:OUT;SFP:1101;SCL:1;SRVR:DM3PR1201MB1118;H:tlendack-t1.amdoffice.net;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTNQUjEyMDFNQjExMTg7MjM6alprM0pOb3h3M21vNmk0SS9GczFHK29v?= =?utf-8?B?ZlNMU0QzdnVyS3NDOG8zZXFvRVFLT3NhZ01jU1Nzck4xYVdTbW5PSGlYbzBP?= =?utf-8?B?ZitpdnNvUkZzcVJtTXhsZ2l5SlEySWZtVThDeTZZeXR1d0F1VWdZSkY1SG44?= =?utf-8?B?eUFXS3NNa0lyRVNuNWtLeUFqazA4R0xXdkw0WVdEbUpiWU5BK3ZhOWhPTWVO?= =?utf-8?B?d1A0QVhnTmd2cmtFRlJja1hpKzh1UmxJNXJadGdDM3V0OU1pMS9CRDBpTjAr?= =?utf-8?B?STFxVFQyZUg2dm1GdGkxWVVFR0lEQ0MrYStFYTdZNVRjYUJHdStFQ1RiT3dW?= =?utf-8?B?MXRsMDJLZkUvbzJ4NlBQQTdHbDBqcWxkUXBINXFHeDRML1ZrbytJMmp1TW9I?= =?utf-8?B?cWhCMjEwaEp1anNYWklWRUVBUGJTRktJR3ZnNWVwc1lYL2VGM2NNcFhvK0Fk?= =?utf-8?B?NkErWmJHYmlmdTIvUFV5dU43bGNKcytDaDNybW9kUERmNXRabmFjdEJiWU5R?= =?utf-8?B?QXp3Z3NsQUQybVI5Z1B1Qk04OEVhOUN4V210SkRjSURFUE1XVW5Xdm55aHlO?= =?utf-8?B?OWpQelp0MzZaaW01VmhpVUtvNjUyRldWMG1pOXkrdnBsdUxNdWFjbTljTWE0?= =?utf-8?B?M2ZjV0JJTTlIUXpKOUpYRHhxbTh3RGFDemxueEVlNXRZQ3pHT2NVSG1PVmVU?= =?utf-8?B?bmtHdlpONC9LMW01R2FMZ0wxbFcxWmFEZm9sMEtJWmlTVkgxOE5tbWNrZHZx?= =?utf-8?B?YmtVN0lrRHBkSHBacytvSVpIQWJCZTN0eUZTTFJlZkplWEtobThORDYzVTdV?= =?utf-8?B?dzZLUzQyQ2hUREkyY2ZHYUpzV0RpYk10VXh1WGQ0MG16WEZobXFXbzhEaDlw?= =?utf-8?B?VnVtUlY3N2ZvMWNSVFFaOUNtRml4Q3ZVYUpLMk5ua0prYjk5a0dtcTFNQVJq?= =?utf-8?B?UFd5clBYd05lc2YxdmQrdmxseFlnQktIdldDTGhlV1NzbFNqOHN1QVIxcVhD?= =?utf-8?B?S3Q4ZTJJWmlSUGR0dzkyQ0w3ZVBYQWMwV1A3NE8zaVJ2Unl3MGtDMWpKRXk5?= =?utf-8?B?ZHFLOGFzZTlvMTFMd1djNDB4Wm5RQTB5QzR5eHAxUUNCclBxTWJnU2JVaWdx?= =?utf-8?B?cTZoNWJUbVlHdi84UzZNMHpETVJZSGExeHBDM3ZuTnNPYVBOUUZoS2llNlpr?= =?utf-8?B?bWVBY09BckRzRUx6WFMvWlV2bUt4amZGNUNNaGNqUTczOUdTelFHZ0xtTHlt?= =?utf-8?B?UUFQRDkwOGthSzV1dXY0Zm5sZGRFRFZUaUcwcDVEQjk4SGRuQjJkdjEzaGtw?= =?utf-8?B?WlY5VmUvZC9pZjlnb3BVb0pBczZydTNYVTlycGlJWGk0ZmFxUExRZGFybHVm?= =?utf-8?B?aHVCMFJSby9WNEFuODZoNzBQTTJYb3cxTGxpVTNwNVE9PQ==?= X-Microsoft-Exchange-Diagnostics: 1;DM3PR1201MB1118;5:Ny3dKGaqoT9A6cAErz1kqvhUtr+oHGr91BNKqR/V8TwcBUcEn5r1qel9HslTUyi7KtvKwwyWGN1Z2jr2dOlc6GEdIzmZPls+yOsBNZcdFmsii1CNUscyw6mEHH1AO9Y66Pnw6Rlme8wKyj8AXrUEOQ==;24:A32j/2D0IkadErroD+hkQoVtjHzsYFnM6bp1DUzP2wI8S5UQWWAmdGDvM7qYIAUUdosLwcLKgdVBywdSJwtvOO0y3nNGihDfoa12bRjItIc=;7:nGEsbtnKMx7Dm7q3e8sOr9cj+gjbP52sF25PE5UxUJhsUeutncxE9eftAjWstj8w/NcHVb+1McXvDXw00LaPoP1ma1xKCFIRVyPKXOB8kB0rYIOj7LqIQmzd/e2/IjZm1+Q59PxEctlVdgj4dUdLMkqmh/Bs/XmODqXuM7fMK/o=;20:e/Cu/wdK0cr4UbD0IJLoHe0POl1JW7N1NVUh6mS4rWqmaBOL+i5CAmQAgu7+A3RWNvQYeTew/dwQz74i+8migxYjX456Yxp1JhPOFNqSqfXTCxTGJ2NqezqVakV8NQBibDODah4Flaj37OhJESxPUfMZrZ6PPwxT8vOu/iIO/qC//i2zREwGO9Cljuy8SWhqK1Ik3zOkHVrSEvjVt8+4+k+VHQAXw3saV3cv0wnzl1RKWp/eVDDK75dIwL5XnBC9 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2016 22:57:33.9456 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR1201MB1118 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Encrypt memory areas in place when possible (e.g. zero page, etc.) so that special handling isn't needed afterwards. Signed-off-by: Tom Lendacky --- arch/x86/kernel/head64.c | 90 +++++++++++++++++++++++++++++++++++++++++++--- arch/x86/kernel/setup.c | 8 ++++ 2 files changed, 93 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 3516f9f..ac3a2bf 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -47,12 +47,12 @@ static void __init reset_early_page_tables(void) } /* Create a new PMD entry */ -int __init early_make_pgtable(unsigned long address) +static int __init __early_make_pgtable(unsigned long address, pmdval_t pmd) { unsigned long physaddr = address - __PAGE_OFFSET; pgdval_t pgd, *pgd_p; pudval_t pud, *pud_p; - pmdval_t pmd, *pmd_p; + pmdval_t *pmd_p; /* Invalid address or early pgt is done ? */ if (physaddr >= MAXMEM || read_cr3() != __sme_pa_nodebug(early_level4_pgt)) @@ -94,12 +94,92 @@ again: memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD); *pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base + _KERNPG_TABLE; } - pmd = (physaddr & PMD_MASK) + early_pmd_flags; pmd_p[pmd_index(address)] = pmd; return 0; } +int __init early_make_pgtable(unsigned long address) +{ + unsigned long physaddr = address - __PAGE_OFFSET; + pmdval_t pmd; + + pmd = (physaddr & PMD_MASK) + early_pmd_flags; + + return __early_make_pgtable(address, pmd); +} + +static void __init create_unencrypted_mapping(void *address, unsigned long size) +{ + unsigned long physaddr = (unsigned long)address - __PAGE_OFFSET; + pmdval_t pmd_flags, pmd; + + if (!sme_me_mask) + return; + + /* Clear the encryption mask from the early_pmd_flags */ + pmd_flags = early_pmd_flags & ~sme_me_mask; + + do { + pmd = (physaddr & PMD_MASK) + pmd_flags; + __early_make_pgtable((unsigned long)address, pmd); + + address += PMD_SIZE; + physaddr += PMD_SIZE; + size = (size < PMD_SIZE) ? 0 : size - PMD_SIZE; + } while (size); +} + +static void __init __clear_mapping(unsigned long address) +{ + unsigned long physaddr = address - __PAGE_OFFSET; + pgdval_t pgd, *pgd_p; + pudval_t pud, *pud_p; + pmdval_t *pmd_p; + + /* Invalid address or early pgt is done ? */ + if (physaddr >= MAXMEM || + read_cr3() != __sme_pa_nodebug(early_level4_pgt)) + return; + + pgd_p = &early_level4_pgt[pgd_index(address)].pgd; + pgd = *pgd_p; + + if (!pgd) + return; + + /* + * The use of __START_KERNEL_map rather than __PAGE_OFFSET here matches + * __early_make_pgtable where the entry was created. + */ + pud_p = (pudval_t *)((pgd & PTE_PFN_MASK) + __START_KERNEL_map - phys_base); + pud_p += pud_index(address); + pud = *pud_p; + + if (!pud) + return; + + pmd_p = (pmdval_t *)((pud & PTE_PFN_MASK) + __START_KERNEL_map - phys_base); + pmd_p[pmd_index(address)] = 0; +} + +static void __init clear_mapping(void *address, unsigned long size) +{ + do { + __clear_mapping((unsigned long)address); + + address += PMD_SIZE; + size = (size < PMD_SIZE) ? 0 : size - PMD_SIZE; + } while (size); +} + +static void __init sme_memcpy(void *dst, void *src, unsigned long size) +{ + create_unencrypted_mapping(src, size); + memcpy(dst, src, size); + clear_mapping(src, size); +} + /* Don't add a printk in there. printk relies on the PDA which is not initialized yet. */ static void __init clear_bss(void) @@ -122,12 +202,12 @@ static void __init copy_bootdata(char *real_mode_data) char * command_line; unsigned long cmd_line_ptr; - memcpy(&boot_params, real_mode_data, sizeof boot_params); + sme_memcpy(&boot_params, real_mode_data, sizeof boot_params); sanitize_boot_params(&boot_params); cmd_line_ptr = get_cmd_line_ptr(); if (cmd_line_ptr) { command_line = __va(cmd_line_ptr); - memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE); + sme_memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE); } } diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 2367ae0..1d29cf9 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -113,6 +113,7 @@ #include #include #include +#include /* * max_low_pfn_mapped: highest direct mapped pfn under 4GB @@ -375,6 +376,13 @@ static void __init reserve_initrd(void) !ramdisk_image || !ramdisk_size) return; /* No initrd provided by bootloader */ + /* + * This memory is marked encrypted by the kernel but the ramdisk + * was loaded in the clear by the bootloader, so make sure that + * the ramdisk image is encrypted. + */ + sme_early_mem_enc(ramdisk_image, ramdisk_end - ramdisk_image); + initrd_start = 0; mapped_size = memblock_mem_size(max_pfn_mapped);