From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756846AbcK2SBR (ORCPT ); Tue, 29 Nov 2016 13:01:17 -0500 Received: from mail-dm3nam03on0081.outbound.protection.outlook.com ([104.47.41.81]:54880 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754392AbcK2SBI (ORCPT ); Tue, 29 Nov 2016 13:01:08 -0500 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Thomas.Lendacky@amd.com; Subject: Re: [RFC PATCH v3 15/20] x86: Check for memory encryption on the APs To: Borislav Petkov References: <20161110003426.3280.2999.stgit@tlendack-t1.amdoffice.net> <20161110003740.3280.57300.stgit@tlendack-t1.amdoffice.net> <20161122192526.vg63jjhwsbjwex7i@pd.tnic> CC: , , , , , , , , , Rik van Riel , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , Konrad Rzeszutek Wilk , Paolo Bonzini , Larry Woodman , Ingo Molnar , Andy Lutomirski , "H. Peter Anvin" , Andrey Ryabinin , Alexander Potapenko , Thomas Gleixner , Dmitry Vyukov From: Tom Lendacky Message-ID: Date: Tue, 29 Nov 2016 12:00:56 -0600 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161122192526.vg63jjhwsbjwex7i@pd.tnic> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: MWHPR07CA0002.namprd07.prod.outlook.com (10.172.94.12) To CY4PR12MB1143.namprd12.prod.outlook.com (10.168.164.135) X-MS-Office365-Filtering-Correlation-Id: 3ebc42ff-d6c5-41f0-6b04-08d41881aeb8 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:CY4PR12MB1143; X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1143;3:UBmeNGpaeNdiJ+JNzKpEQ258SqwiJIHRhcqK3eACS3/w5KnQOBUowgVMOEOtUX03njZBhjiVyiKUFVrkDu+Zti+/yvgZ9WBM2ex8vb+xpQDHvkBG6Gean29okeEtImpr8L+o8xjm9ni4Hx9107H9PlPgbvaB3AVAGxcq6QrR6CKLJnVjpJDzQvD8OeJX8Ob5mNjTbgRwBCWmBdoKShMoOlDlEcCTWwMKnoSdP2KPl+riK3u5DgCYsuht+mj8g3y1i2ihrmB8hYFBY9EMrki+XA==;25:I6yPMuDBUZnwawoDOlh9B8cdxVoABHXCVEqYbTkcrcp6QwbHpSxoIeWF+OQ3A1s5gGBPdjC012bp6vGmFKpnjZnFKQSZrQdOScgTEC8ML3aH+PNNfJALdZyjoyZFplgJ45F2O5ifp8Nq5r6+vHgLkBGijqy0g/792B8NpuYwyjUp9YHAdNTMdli6mOantxLEtLR3B92gF+CPiv4Ctjxx9m5oe4yN9/dltTfW1HuzUC059+VoABZG+h0YxgiR/zOvRkgIRbarxqopaAYkA+faNArbuyorQEGJH810oos/vl7OLMYvMQBgKZ8XPBjgQ49SxIgVZisdFSA0p9Tjc/VyBIehZmhv+vpxLDnjKG/3ctTeaEAQlyQfJdI0KQLADOlbLyhOMZ6liIb+XXYvAkk8JvYCSY1m7LqCHo19x1OUNcz3V1gnVi0kcgwssjuIVMjHDkQHh++SCE8Bw9vvdyrslg== X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1143;31:1PxNoW8tcgJe1fXEZCijN7pexSaaMa66yRO2TjXqu1gq2eQUuPV5mbSDcsW5PicsEOXGYPQNeLJVSlisKmM8kfD6GyE7+vRlNJ25ITJpRGqULGG4CMesQdrWJzebCMYEs3fiIYnQQd1z4EueU047krClUtiilpJ5zs9Fn2iv9NHVS/z8fOORLX1woYQghoBgKzlUkXzAwFdJ3p9G9ZSy2Zfx8JtnEcqEimaF/x/xJ/+NiVPoA8l85jvPJZjx/Yx8EHLt7qU4mrb6pYCpIXMye0jxlYhRTjII+cqgzcKZ5gA=;20:aBPwRJkmYDgG0+l9OeCXTY71gsqoyf4jZFhbQGGwt+ceBIaHjwINwNeWleOxJdZN9N9lDPPAXmwSJRZqZptcBIE9d7MBcKRuGbh78EqJaGBPaEkACXBE1D+BqrZkoQJNl4+kZWhX2ttErFfWTxj1PKnB1XQjnAPBjFa3of7oIog0GF29v4NjfMmEpvINdoFj93f4PRr+vXRm1E8TW1U5lDA8DO1qH5g5LoLCbT2ChYeSgjo0Drm5Cm+bI//scndynM8KMG/XfTKYXgMpv8PSygvNpT8+oQS1vZxmyhTnUWlei92yweZDB1eGU1qrMzJRCPEO1sygxBw32Nfn5EkUoQ8VENZRPdaKk7H+EJl6EG3Y5rxzpcXWGdCXgDhb2W2ljMJ/dgiDyd/3QE1HwRKGvG+EoDEpoGxtyKaWFOrvlkZxVeIOqCVqeVGvNI2rMXXzNZW+uF0ZbCzFD8s99VHJCa8XdLsy0T/rUuDWak3QArFqeflu2e2jLzMPGxBY/jRs X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6060326)(6040361)(6045199)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026)(6041248)(6061324)(20161123564025)(20161123562025)(20161123560025)(20161123555025)(6072148);SRVR:CY4PR12MB1143;BCL:0;PCL:0;RULEID:;SRVR:CY4PR12MB1143; X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1143;4:959m753TKPan6Cy/menDHXqd8UMPvlCBVPeJKR8rG1WCgTbjWHIxxfQ+dyHaCyFREP0bCR1rChEGt5zm1GjQ4viWuSpnAdbAOj3Z7k2Y0nkKHJC29fwE+PvOOc8mv2w1mExZpEn12W8L/+OStmR5aj6GlbFVFNbbXL0hkSDM0knWyftl8iBjj/twwAh/xJIYfMu1lGNlfs+y0XSnLXfwvSwuOyAFdqW3+7RQvQhZzeYu4yP4pJsfBpnRdknDJ/6+kvfqWhrB3qicO5MFzr9t1P+UK8VO+gPY5caA4AavfVfDUED7BkMlbSwjuj6+Gfm4JRyFOCjxUKY9PvTxyE5z+ZN4LOVpKDMmEvyUVLvs1t7EVw2mEyMHKFd+JmkdqV75MChzoA4b7hB9BHTR5cV983A8Oeb9DtjoOwgf0k5jzw1qYjGHfHIBZnLTNHtbMdw8ZCudAoqIdqzvW3TJgoGfGqDEcQtxvJuaOyardYe/1T00Lgi+z+1KuoMeNTuRPDeJXazSbRlJkzg1MMWO+Hz78GDPQmMtO7Js3RDITaMhFoD7OBol0i6rXkvAlYAxo1k0CgIwL2cis7cFhB/oEdPEyvNmjbThsz81Oa9MhJ0QED5wutu/vKeuG3NIEePiO6kf7N3MpeQsvwFDxdUEDDeep3vAZVrVSNNQD7B5xLDeUlcA2mHsZIpx4gswLu5IBSDL X-Forefront-PRVS: 01415BB535 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(189002)(377454003)(24454002)(199003)(6486002)(86362001)(110136003)(47776003)(2906002)(5660300001)(6916009)(68736007)(6666003)(2950100002)(65826007)(81166006)(81156014)(36756003)(8676002)(4326007)(31696002)(66066001)(230700001)(65956001)(83506001)(65806001)(76176999)(7416002)(54356999)(7736002)(50986999)(31686004)(42186005)(189998001)(101416001)(77096006)(39410400001)(733004)(38730400001)(39450400002)(4001350100001)(39400400001)(7846002)(305945005)(106356001)(39380400001)(105586002)(229853002)(64126003)(50466002)(97736004)(23676002)(92566002)(33646002)(3846002)(6116002)(217873001);DIR:OUT;SFP:1101;SCL:1;SRVR:CY4PR12MB1143;H:[10.236.64.222];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTRQUjEyTUIxMTQzOzIzOktWYmkzZ01raEMxNDhUNjFCcy9oMnE4ZUhi?= =?utf-8?B?V1hqalNRd0ZjdjAva1lScTM0QTRVd1FOUnJHMkQ2VHZnWDNveXBsQ0dNT0VZ?= =?utf-8?B?L2g3Y3VWTGp2clVJV3pmT3h2ZnNPYXNSaDhJTjhQcDNaZnJEaFZ0QXQ2d1Ey?= =?utf-8?B?STd0dVBNdXE0U1lVVDR6RmhkbDY3T2ZrbjV0dFNTUndPSWpWNWZyYUtJajI4?= =?utf-8?B?TEJmN1ZLeWdkTmMySFJodFVuU2hDZG9VaUN1aXVESzVvQkY1b0poYlkvOHR6?= =?utf-8?B?ZnZoNmJFczJwMjFBWUU4aHFjWmdpQjBTVjF3U1JIdGUrZFlJNGZ1VkFmR2NP?= =?utf-8?B?dCs0eU4yOGFDbzJuS1o3SitCTnR0V3FFRTVZQjJtTy90RFZ5TFAwclQyR3Vh?= =?utf-8?B?TjZ1c2lyMFNLa1c2bnFBalZDVTdpVkIwc0JuajFOcGgwSmVKSDZDekc0Q3Mw?= =?utf-8?B?SENOTFF3SmVIRkgzT2RuNlVISUlWTHI2RVc3UTlyVWY1Z3NhWEJBVEZYWGE5?= =?utf-8?B?Y0M2NFl2V0dXd05raHdZSG0xQlEyKyt4cEwzQjRDV1A3T1JzSk9GUGVWdFpK?= =?utf-8?B?SDIybzNNVDc0QmJveGprakw3dzcvWFRhTTUxM1BMZzVBZmM3RUJHYVd5cXQ4?= =?utf-8?B?R3AvSUIvdEpIdGp2WStIRTZqYi80L2dsdnIvYjFMZDNUTHFLZ0Nnd3IzcVI3?= =?utf-8?B?eWpjTFVyYnJLZXYxc3VVNDdzZFZOcmE5M0tBZXBIMm1XSE9tVTcwSXNaMjdT?= =?utf-8?B?aU9zTmpvOG81ZmpVSnlzd0NYS1FYTkJSVzR6NHVIdGhwNG1WYVdIVVdWSm1V?= =?utf-8?B?WldVeXp2R0d1R01rRVFyMXlsN0k5Q29pblpaWDNUVGo5ZHliV0JPaGxJaWtI?= =?utf-8?B?L1gxRVpiOGFDdlgraG1mcmdXL0ZwVDljdlVSK1FLVEI2MkJKb25jSXNzNXNS?= =?utf-8?B?NHVzck5ZWTVEWUt2MEE0WEdjQlMxRzRwTFlvSG9BV3orUVJjOEs3aU5paHZF?= =?utf-8?B?TG4zOWlzb01NN3g2NzRObjFJS3ZDc2ZoSzVWbEp5SGRSU1JKSTRGRlBnUDJx?= =?utf-8?B?TlMwVk81ck1nWW12ZzZDR3RrOHFvRFBmUGM4eXBjdXg0U3FMNGpycDFDWDF2?= =?utf-8?B?cG55dkFpNmZ0ZkZrUjZadmw3QzAwY0xuOTYwWmVpY245Q3ZqVXNHcnJtcldw?= =?utf-8?B?bW9pSjFIZ0h6QVBuaDlPVEdqM0lSZGxnWmV5VFB2L3hjTkRoZzRLbGQvR0NJ?= =?utf-8?B?VVpxRmNkcldjc1cxYTNkVmwrR01Ia1AyREY5OEpPUjZHT3BnMWVTT24xcU5r?= =?utf-8?B?K0drLzFVV1pmNlB4a1d1UktzWThwREd4QU52aEpBRjhSYlhtTElsQ1p2a1Ba?= =?utf-8?B?Zm9vVjBaMnl3aFR4emhPb2NDaVVWYTVmUUcrYndiR2VFVEZRZHNRWFdMY282?= =?utf-8?B?eWE1UlZwMFhVcGhVb2daeWRKMzhFZXEraFpZc3k5VDFtK01QTUpuaFIzckgv?= =?utf-8?B?aWw3MkF1TVVnMS9SaDRVazBzOVd6UVVPaDR2Zi9iNDNOMWJXeE5MY21OblJG?= =?utf-8?B?YXF5T2s2RHdxMHV6RUdWQ0FvM1BUUkxlNDdaaVJvQzR4czN1ck9zemNUTWpJ?= =?utf-8?B?NEFsREtEVmU0WUlNbVZmNjd0YzBLOVRkTHJUcG5TakJJNWpGYnYrTkUwOGll?= =?utf-8?B?VlREV05ycU1rZktTeWREQ2J3eFAvRjJTTHNmTzhUbmZycS9LREtyc0IvUXdp?= =?utf-8?B?UjhkUkFxNlJtNWFxZ2pMbEFRWFFzWlBLY21iZXBLNTA5Sk9hRjNHUERGdmZV?= =?utf-8?B?UTFRUlZHT3FibzZaUXhEd0tVS2RDWlk1T1RhMVJ0L0F5UGtQdGs4YWdvVHJo?= =?utf-8?B?V0xMUmlHVVZaam5BdlVOdENrUHpRNy9zdzg3STNlT1VhWHk2VmFrRlArTWVt?= =?utf-8?B?UDdWVTN5UzZUYmhNTUQwL25pdTFaMW9NcE8wVHh3SWZQelFCaFFLK3d3UUZX?= =?utf-8?Q?/jYsT2?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1143;6:dqDSYYqwR53MWZTu0HVVjpoKSuOpWy5vWaayjoF2LFRjUajZGQbXc3BNLdOg7mzcKJITRA6kR3Vf18Y0WRNykPLWNyBtijcVK23d04Y6Qk7Vb7LbBq67g4FWcnSrP65W5foHtd8hbhkrDHWzyQYuuphy8YmarBH/bon8MDiwb9TkWmk19xnOOd6dubNXcD5NqQgybs3qxn6MrIBvRaiKD8FSBKRXvg/B6boh5Voj4QQHqMFkN7+ByXO8GpadlknrHR/b5EYVE+SDaI/Z0wqCDZis1hvszqTerU/7t0tcA1BvUNmmqe5GDmwJq+/lc8cl1t4QpWGjWCLJ6V4bTMotdhg0zINJ20+RvGKhXNJcSAztbjlUnv+q7ti9JFKuLZiubET6zGhVlFsTjz3sJTcUw6dghwIqqWSF9CutCUEB9pTnhK9geGU82V/9T6y6CkVoSG0ewCoZXeLFQQ1Ycf/1BKjxog/q2HiM64+WGmi1exY=;5:6rAEyfr3ZtH6tO6gJDro1ZiUZbe1jIiZs47V5oVN74jr/M/Swgbm90B06kHY/Idu7/azDZL9TOT2kdVVCBzmi0eEFcOnEOBHsGJL3JiyasN4M48X2/wX7RXIxogwGmE7eUrLCRvhNzUxf6H3TJXsFlSYX+Gw2N4MNLwZsdpDAjc=;24:h4DG4YiQJgIbp7xFffvqaN1UwIZ6U2M+ORSwituu+lIQ6iiAO+AhwvD8hyygquWndybDOLakNsRnZJgaOAFJnEQgV5hXgHfpTUt7VtAQZSQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1143;7:2QNzHj/OKPQQh+K1cr0bmuKbnZObOwREeOu5AhakwAj7k+3Rqu+RUnDNutrOghPrsJBO9x3/XgAzMX43mVEXRuj+CjuE4sWI9smoLmCRjBPcHTnKEbl/JqF1xbuA/TIisw3IWD8R+Dg57SV7jrE+7rwEgejkhjfLdb8LU3y3RqGX0l6tseP4y614D1UqDZ6t8HP1YhDDXuAccbhYw0KJvcYy5EO78wI5liHyZrm2hfIukE9d/YeQW0ELhgMf2Sg1QcN1D7MretyXDouE6ZWFSMiCRcIaKFRSY6DNWQuMXIMw7m2SfrzsHoIiB03iVW/6ZB5cMxhjxbgODsbj1SxQ9pr9NAPmz7GADwBYT1v4pnfSETE5+0gBNU5+MgtTtoyQGxZ4S3q2kyqLzhXgxkQ/Z+RLF1JuqNd1LrNjqXx+nE3grOWR8xDR1+4CfK5fiWRfGjzhOKoXrCFL8+nkSB8QjQ==;20:EVq2MK9CtM85mxE0JzgouospHHNREesC+qPdJn7/RNFY5hjmGcHIc0O7MeMMGdPYv10k0/RvsiUCNzVddRbtUFGYyn6Gnoy63ljVHdXM+Dq+QZthyqYlZQ2HMp0wO6eGQhGPY4wKm5MgtLisI1bOTNaQyV6GdvyKmnVqSe/F1H3FwyTvlVrzZTgCM6EdQyzM42+gbQMwfLNv8nRToDpROXm46LCZryiyG6vcVxXKhHCTigdP6Ad+hDV/8cvCqDJI X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Nov 2016 18:01:00.5353 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1143 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/22/2016 1:25 PM, Borislav Petkov wrote: > On Wed, Nov 09, 2016 at 06:37:40PM -0600, Tom Lendacky wrote: >> Add support to check if memory encryption is active in the kernel and that >> it has been enabled on the AP. If memory encryption is active in the kernel >> but has not been enabled on the AP then do not allow the AP to continue >> start up. >> >> Signed-off-by: Tom Lendacky >> --- >> arch/x86/include/asm/realmode.h | 12 ++++++++++++ >> arch/x86/realmode/init.c | 4 ++++ >> arch/x86/realmode/rm/trampoline_64.S | 19 +++++++++++++++++++ >> 3 files changed, 35 insertions(+) >> >> diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h >> index 230e190..850dbe0 100644 >> --- a/arch/x86/include/asm/realmode.h >> +++ b/arch/x86/include/asm/realmode.h >> @@ -1,6 +1,15 @@ >> #ifndef _ARCH_X86_REALMODE_H >> #define _ARCH_X86_REALMODE_H >> >> +/* >> + * Flag bit definitions for use with the flags field of the trampoline header >> + * when configured for X86_64 > > Let's use kernel nomenclature: "... of the trampoline header in the > CONFIG_X86_64 variant." Ok. > >> + */ >> +#define TH_FLAGS_SME_ENABLE_BIT 0 >> +#define TH_FLAGS_SME_ENABLE BIT_ULL(TH_FLAGS_SME_ENABLE_BIT) > > BIT() is the proper one for u32 flags variable. Yup, not sure why I used BIT_ULL... will fix. > >> + >> +#ifndef __ASSEMBLY__ >> + >> #include >> #include >> >> @@ -38,6 +47,7 @@ struct trampoline_header { >> u64 start; >> u64 efer; >> u32 cr4; >> + u32 flags; >> #endif >> }; >> >> @@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void) >> void set_real_mode_mem(phys_addr_t mem, size_t size); >> void reserve_real_mode(void); >> >> +#endif /* __ASSEMBLY__ */ >> + >> #endif /* _ARCH_X86_REALMODE_H */ >> diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c >> index 44ed32a..a8e7ebe 100644 >> --- a/arch/x86/realmode/init.c >> +++ b/arch/x86/realmode/init.c >> @@ -101,6 +101,10 @@ static void __init setup_real_mode(void) >> trampoline_cr4_features = &trampoline_header->cr4; >> *trampoline_cr4_features = mmu_cr4_features; >> >> + trampoline_header->flags = 0; >> + if (sme_me_mask) >> + trampoline_header->flags |= TH_FLAGS_SME_ENABLE; >> + >> trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd); >> trampoline_pgd[0] = trampoline_pgd_entry.pgd; >> trampoline_pgd[511] = init_level4_pgt[511].pgd; >> diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S >> index dac7b20..94e29f4 100644 >> --- a/arch/x86/realmode/rm/trampoline_64.S >> +++ b/arch/x86/realmode/rm/trampoline_64.S >> @@ -30,6 +30,7 @@ >> #include >> #include >> #include >> +#include >> #include "realmode.h" >> >> .text >> @@ -92,6 +93,23 @@ ENTRY(startup_32) >> movl %edx, %fs >> movl %edx, %gs >> >> + /* Check for memory encryption support */ >> + bt $TH_FLAGS_SME_ENABLE_BIT, pa_tr_flags >> + jnc .Ldone >> + movl $MSR_K8_SYSCFG, %ecx >> + rdmsr >> + bt $MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax >> + jc .Ldone >> + >> + /* >> + * Memory encryption is enabled but the MSR has not been set on this >> + * CPU so we can't continue > > Can this ever happen? > > I mean, we set TH_FLAGS_SME_ENABLE when sme_me_mask is set and this > would have happened only if the BSP has MSR_K8_SYSCFG[23] set. > > How is it possible that that bit won't be set on some of the APs but set > on the BSP? > > I'd assume the BIOS is doing a consistent setting everywhere... > It can happen if BIOS doesn't do something right. So this is just a safety thing. I thought I had changed this to set the bit if it wasn't set, which is safe to do. I'm not sure what happened to that change (based on your previous comment actually!)... I'll add that back so that the AP is useful. Thanks, Tom From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Lendacky Subject: Re: [RFC PATCH v3 15/20] x86: Check for memory encryption on the APs Date: Tue, 29 Nov 2016 12:00:56 -0600 Message-ID: References: <20161110003426.3280.2999.stgit@tlendack-t1.amdoffice.net> <20161110003740.3280.57300.stgit@tlendack-t1.amdoffice.net> <20161122192526.vg63jjhwsbjwex7i@pd.tnic> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20161122192526.vg63jjhwsbjwex7i-fF5Pk5pvG8Y@public.gmane.org> Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Borislav Petkov Cc: linux-arch-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, x86-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kasan-dev-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Rik van Riel , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , Konrad Rzeszutek Wilk , Paolo Bonzini , Larry Woodman , Ingo Molnar , Andy Lutomirski , "H. Peter Anvin" , Andrey Ryabinin , Alexander List-Id: linux-efi@vger.kernel.org On 11/22/2016 1:25 PM, Borislav Petkov wrote: > On Wed, Nov 09, 2016 at 06:37:40PM -0600, Tom Lendacky wrote: >> Add support to check if memory encryption is active in the kernel and that >> it has been enabled on the AP. If memory encryption is active in the kernel >> but has not been enabled on the AP then do not allow the AP to continue >> start up. >> >> Signed-off-by: Tom Lendacky >> --- >> arch/x86/include/asm/realmode.h | 12 ++++++++++++ >> arch/x86/realmode/init.c | 4 ++++ >> arch/x86/realmode/rm/trampoline_64.S | 19 +++++++++++++++++++ >> 3 files changed, 35 insertions(+) >> >> diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h >> index 230e190..850dbe0 100644 >> --- a/arch/x86/include/asm/realmode.h >> +++ b/arch/x86/include/asm/realmode.h >> @@ -1,6 +1,15 @@ >> #ifndef _ARCH_X86_REALMODE_H >> #define _ARCH_X86_REALMODE_H >> >> +/* >> + * Flag bit definitions for use with the flags field of the trampoline header >> + * when configured for X86_64 > > Let's use kernel nomenclature: "... of the trampoline header in the > CONFIG_X86_64 variant." Ok. > >> + */ >> +#define TH_FLAGS_SME_ENABLE_BIT 0 >> +#define TH_FLAGS_SME_ENABLE BIT_ULL(TH_FLAGS_SME_ENABLE_BIT) > > BIT() is the proper one for u32 flags variable. Yup, not sure why I used BIT_ULL... will fix. > >> + >> +#ifndef __ASSEMBLY__ >> + >> #include >> #include >> >> @@ -38,6 +47,7 @@ struct trampoline_header { >> u64 start; >> u64 efer; >> u32 cr4; >> + u32 flags; >> #endif >> }; >> >> @@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void) >> void set_real_mode_mem(phys_addr_t mem, size_t size); >> void reserve_real_mode(void); >> >> +#endif /* __ASSEMBLY__ */ >> + >> #endif /* _ARCH_X86_REALMODE_H */ >> diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c >> index 44ed32a..a8e7ebe 100644 >> --- a/arch/x86/realmode/init.c >> +++ b/arch/x86/realmode/init.c >> @@ -101,6 +101,10 @@ static void __init setup_real_mode(void) >> trampoline_cr4_features = &trampoline_header->cr4; >> *trampoline_cr4_features = mmu_cr4_features; >> >> + trampoline_header->flags = 0; >> + if (sme_me_mask) >> + trampoline_header->flags |= TH_FLAGS_SME_ENABLE; >> + >> trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd); >> trampoline_pgd[0] = trampoline_pgd_entry.pgd; >> trampoline_pgd[511] = init_level4_pgt[511].pgd; >> diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S >> index dac7b20..94e29f4 100644 >> --- a/arch/x86/realmode/rm/trampoline_64.S >> +++ b/arch/x86/realmode/rm/trampoline_64.S >> @@ -30,6 +30,7 @@ >> #include >> #include >> #include >> +#include >> #include "realmode.h" >> >> .text >> @@ -92,6 +93,23 @@ ENTRY(startup_32) >> movl %edx, %fs >> movl %edx, %gs >> >> + /* Check for memory encryption support */ >> + bt $TH_FLAGS_SME_ENABLE_BIT, pa_tr_flags >> + jnc .Ldone >> + movl $MSR_K8_SYSCFG, %ecx >> + rdmsr >> + bt $MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax >> + jc .Ldone >> + >> + /* >> + * Memory encryption is enabled but the MSR has not been set on this >> + * CPU so we can't continue > > Can this ever happen? > > I mean, we set TH_FLAGS_SME_ENABLE when sme_me_mask is set and this > would have happened only if the BSP has MSR_K8_SYSCFG[23] set. > > How is it possible that that bit won't be set on some of the APs but set > on the BSP? > > I'd assume the BIOS is doing a consistent setting everywhere... > It can happen if BIOS doesn't do something right. So this is just a safety thing. I thought I had changed this to set the bit if it wasn't set, which is safe to do. I'm not sure what happened to that change (based on your previous comment actually!)... I'll add that back so that the AP is useful. Thanks, Tom From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f70.google.com (mail-pg0-f70.google.com [74.125.83.70]) by kanga.kvack.org (Postfix) with ESMTP id 9C3126B025E for ; Tue, 29 Nov 2016 13:01:08 -0500 (EST) Received: by mail-pg0-f70.google.com with SMTP id x23so449042774pgx.6 for ; Tue, 29 Nov 2016 10:01:08 -0800 (PST) Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0058.outbound.protection.outlook.com. [104.47.40.58]) by mx.google.com with ESMTPS id 60si32299490ple.301.2016.11.29.10.01.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 29 Nov 2016 10:01:07 -0800 (PST) Subject: Re: [RFC PATCH v3 15/20] x86: Check for memory encryption on the APs References: <20161110003426.3280.2999.stgit@tlendack-t1.amdoffice.net> <20161110003740.3280.57300.stgit@tlendack-t1.amdoffice.net> <20161122192526.vg63jjhwsbjwex7i@pd.tnic> From: Tom Lendacky Message-ID: Date: Tue, 29 Nov 2016 12:00:56 -0600 MIME-Version: 1.0 In-Reply-To: <20161122192526.vg63jjhwsbjwex7i@pd.tnic> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Borislav Petkov Cc: linux-arch@vger.kernel.org, linux-efi@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, iommu@lists.linux-foundation.org, Rik van Riel , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , Konrad Rzeszutek Wilk , Paolo Bonzini , Larry Woodman , Ingo Molnar , Andy Lutomirski , "H. Peter Anvin" , Andrey Ryabinin , Alexander Potapenko , Thomas Gleixner , Dmitry Vyukov On 11/22/2016 1:25 PM, Borislav Petkov wrote: > On Wed, Nov 09, 2016 at 06:37:40PM -0600, Tom Lendacky wrote: >> Add support to check if memory encryption is active in the kernel and that >> it has been enabled on the AP. If memory encryption is active in the kernel >> but has not been enabled on the AP then do not allow the AP to continue >> start up. >> >> Signed-off-by: Tom Lendacky >> --- >> arch/x86/include/asm/realmode.h | 12 ++++++++++++ >> arch/x86/realmode/init.c | 4 ++++ >> arch/x86/realmode/rm/trampoline_64.S | 19 +++++++++++++++++++ >> 3 files changed, 35 insertions(+) >> >> diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h >> index 230e190..850dbe0 100644 >> --- a/arch/x86/include/asm/realmode.h >> +++ b/arch/x86/include/asm/realmode.h >> @@ -1,6 +1,15 @@ >> #ifndef _ARCH_X86_REALMODE_H >> #define _ARCH_X86_REALMODE_H >> >> +/* >> + * Flag bit definitions for use with the flags field of the trampoline header >> + * when configured for X86_64 > > Let's use kernel nomenclature: "... of the trampoline header in the > CONFIG_X86_64 variant." Ok. > >> + */ >> +#define TH_FLAGS_SME_ENABLE_BIT 0 >> +#define TH_FLAGS_SME_ENABLE BIT_ULL(TH_FLAGS_SME_ENABLE_BIT) > > BIT() is the proper one for u32 flags variable. Yup, not sure why I used BIT_ULL... will fix. > >> + >> +#ifndef __ASSEMBLY__ >> + >> #include >> #include >> >> @@ -38,6 +47,7 @@ struct trampoline_header { >> u64 start; >> u64 efer; >> u32 cr4; >> + u32 flags; >> #endif >> }; >> >> @@ -69,4 +79,6 @@ static inline size_t real_mode_size_needed(void) >> void set_real_mode_mem(phys_addr_t mem, size_t size); >> void reserve_real_mode(void); >> >> +#endif /* __ASSEMBLY__ */ >> + >> #endif /* _ARCH_X86_REALMODE_H */ >> diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c >> index 44ed32a..a8e7ebe 100644 >> --- a/arch/x86/realmode/init.c >> +++ b/arch/x86/realmode/init.c >> @@ -101,6 +101,10 @@ static void __init setup_real_mode(void) >> trampoline_cr4_features = &trampoline_header->cr4; >> *trampoline_cr4_features = mmu_cr4_features; >> >> + trampoline_header->flags = 0; >> + if (sme_me_mask) >> + trampoline_header->flags |= TH_FLAGS_SME_ENABLE; >> + >> trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd); >> trampoline_pgd[0] = trampoline_pgd_entry.pgd; >> trampoline_pgd[511] = init_level4_pgt[511].pgd; >> diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S >> index dac7b20..94e29f4 100644 >> --- a/arch/x86/realmode/rm/trampoline_64.S >> +++ b/arch/x86/realmode/rm/trampoline_64.S >> @@ -30,6 +30,7 @@ >> #include >> #include >> #include >> +#include >> #include "realmode.h" >> >> .text >> @@ -92,6 +93,23 @@ ENTRY(startup_32) >> movl %edx, %fs >> movl %edx, %gs >> >> + /* Check for memory encryption support */ >> + bt $TH_FLAGS_SME_ENABLE_BIT, pa_tr_flags >> + jnc .Ldone >> + movl $MSR_K8_SYSCFG, %ecx >> + rdmsr >> + bt $MSR_K8_SYSCFG_MEM_ENCRYPT_BIT, %eax >> + jc .Ldone >> + >> + /* >> + * Memory encryption is enabled but the MSR has not been set on this >> + * CPU so we can't continue > > Can this ever happen? > > I mean, we set TH_FLAGS_SME_ENABLE when sme_me_mask is set and this > would have happened only if the BSP has MSR_K8_SYSCFG[23] set. > > How is it possible that that bit won't be set on some of the APs but set > on the BSP? > > I'd assume the BIOS is doing a consistent setting everywhere... > It can happen if BIOS doesn't do something right. So this is just a safety thing. I thought I had changed this to set the bit if it wasn't set, which is safe to do. I'm not sure what happened to that change (based on your previous comment actually!)... I'll add that back so that the AP is useful. Thanks, Tom -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org