From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754289AbdCBQvd (ORCPT ); Thu, 2 Mar 2017 11:51:33 -0500 Received: from mail-dm3nam03on0050.outbound.protection.outlook.com ([104.47.41.50]:1516 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750787AbdCBQvW (ORCPT ); Thu, 2 Mar 2017 11:51:22 -0500 Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=amd.com; Subject: [RFC PATCH v2 14/32] x86: mm: Provide support to use memblock when spliting large pages From: Brijesh Singh To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Date: Thu, 2 Mar 2017 10:15:15 -0500 Message-ID: <148846771545.2349.9373586041426414252.stgit@brijesh-build-machine> In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> References: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: MWHPR10CA0003.namprd10.prod.outlook.com (10.172.48.13) To CY4PR12MB1607.namprd12.prod.outlook.com (10.172.72.16) X-MS-Office365-Filtering-Correlation-Id: e63e9228-9ea7-4dc2-d6d8-08d4617ef279 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(48565401081);SRVR:CY4PR12MB1607; X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1607;3:Co3z9vQo61C23f4Aj+cgEn9+qu1gmfKYGQ17mG3U5Qn+TvtqC6jplRGEMbmaYmaaovoPE/dJQGrj+s8rJVKflDpdAMDmwQx8Subc5pK0Dr8y/Hhr5bscck2oAdFe6IKqUDcH6tHYC/cwzBxfxyxn5sJLTyVcj/82IkyQM/rwBufvTmBa4TZEkEeZ6JRj+YeVk9hh3RviL0YE8Q+mhtL4tJOd/Wrvn/+4hCKCkLnrm5YW2Er5MvwGTOWaKWWGO/tCAb8HjR5/w6SloAwFbPQDmAxPuNm2GjwdLyI8dNYfL6U=;25:nix/FMrA2Bc9KuIOhShngdxly/5ZrMvQFmI7ezSTtnMk8buiey8GAgq1XN6K/gikpXQSxXfDMDV+YihSIwgVgqev3zmUj+mhQcFeBcZ+yxSdwrKJWg03MITc85YM7g5awbP3phlaRf8R0sr9TxD3mQ7z0wu3yWLv3my6SlOla/DVkK2ozHMZaggODTl6UnVhPNxB3uWZ6zhIMRAv/KhgO00pjAexLf3l8qLltSSOX3843y9mTZ2Srud0s/pfqxjdFeoz3v204w0uSvNcF+uGrDIWz8AQVwN8oqtTMlY4+imKvVnYCLXKlPD6zoyK9Ix8gObc4jAmwPqlC9Q5WCBG+WLQiVosI751+BBQebk+Jd0DZ3kuPAT7WziY96y1hpqzk/280nYqHmhfP8XJ90w09jpYI/CRUxoYaYu3HyjliBq3WCk2bWetkgr6l6UwDUBn X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1607;31:WvQkbSpJB7CcfSdOSeCHj5szfrXl5POTkjsyFY7YLIeUFbyuDEmHNqTuhQlwcl0981pxlvRU56odoQQq8yPo5xYEg+iSF+rNvjrRchIaJm9FBQi0JvG8ftsBUFGZdoHk5Vve1bBhO4vpDs1L6TCiyv2w4gwWbOs+xzDJn94cGnc3QmQemwxqyvjPY0xoDexSZ/Cx2rQUHW2E1pKD1ckVirhbA7jYWEJ3V7AB/r/+6HIApFWSMAboUtrrY/TMSUgg;20:9crmA/mIQ4UfLgxftoGCk34cWEhWNfjNDEVHmCS6WprkQiCemIUoqk9TrRefKQfSvd72juUhFeScHvhBTxAPYyJZmMCi+LnFTuqqbJNDRfbQgLzmUbDN6eN1tkFsxrN78mDGnvbx/bZGe7+PPFesoifKGjgZycy4bcqYHsoYk7RPOvVJ1NhH5BzxREX/mA2wD94YEvHzajQn6cV/K5kX9dLTpLNyPFN9N6vBXLZzyx1zqM4gwMuaTnZduCd63Mo86n+czPcz0LhYTm0VZSrZ98vhIoJtX7Oqn0ieg4pV4/Wl5KMVugt6jSIrRQGs+ClSgMTzjyN6riC+McxYOqUwowEEtpc8J/Ijxqckk4p4m2z/Fv59Rnm5NMi44dBn6rpvojxOvJ80oVJ0UFR8gbrDkGZEtUgZXgPwMcAjldF0euTvD9Z6DKTo0vUG4VNs5Oc6J+l0lt6dVzgMF2NACCthnDh454n5TW1IrU70sGDt9waDn/9AjRU3xI954SRb93o4 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026)(6041248)(20161123562025)(20161123558025)(20161123564025)(20161123560025)(20161123555025)(6072148);SRVR:CY4PR12MB1607;BCL:0;PCL:0;RULEID:;SRVR:CY4PR12MB1607; X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1607;4:RmoFkxn/AnXHylgctffrynMrBk7oJXSmk4GhUT44z2CejmbPZia2fUJIGNgs/29kuR9cQe0kh1LNTEDrk19lHQtFsF3GYaxvpPEyfYhkwxWre+lSSk+mffpwk4ep8dogCq9BFTSt5gH4xYCrlsab9KhgVXy7BwcUyIf3sVZLNfP0BtRBxcqbxwIHOA2r17JNkPAxXvHJm9XgZrZznF7CUDL2Wz1ZslIYJxIRoSPyhiecgRXBk9EASZAO0omqtpEOqXif9uF4Q0fRzDbuCkxbeQcbPjRZANX72z2KoMpH/KobWaamcLidp/PIpUMmR5z37twKVqiPaqfGIJRgv8B6qWmdT35dJbEITDFkgdZLPvq8SdiNYIuCBlT9ac8k3gHcmjJH11F/ghAcXzPb1hxC1EXGCH9QFDLsXQCNy5gX8ukLGFycZlrGRmacLOirac1xofEiQkikaS1A7JgA4Ed1v52MtcNPmCIC5T6b5SobaRNxrhKGUxU7uHqz9OGxaH35sf/PXXrzssJiXV8//1pputQjhyNN1cxJfmPMuWvApOhed8mOq0WZuEmOMvkaHL6udNbdrlbrE57GA9CSb0u6//MQCU5CxPV+aHI+zOyxYzDSWJmVg6Thitl+YWWnBa+2Q6EB0PrzNk0oW/jWkvjwWY9Jcynxp4RxhjI2Fap3o6E= X-Forefront-PRVS: 023495660C X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(6049001)(7916002)(39410400002)(39840400002)(39850400002)(39860400002)(39450400003)(33716001)(53936002)(7736002)(38730400002)(6486002)(77096006)(25786008)(90366009)(189998001)(1191002)(305945005)(50986999)(76176999)(54356999)(33646002)(2950100002)(5660300001)(6666003)(83506001)(9686003)(6116002)(3846002)(92566002)(2201001)(230700001)(7406005)(86362001)(7366002)(7416002)(50466002)(2906002)(23676002)(42186005)(103116003)(47776003)(81166006)(66066001)(8676002)(921003)(1121003)(2101003)(83996005)(84006005)(217873001);DIR:OUT;SFP:1101;SCL:1;SRVR:CY4PR12MB1607;H:[127.0.1.1];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTRQUjEyTUIxNjA3OzIzOlY4bFQ1Z2ZnY0dhRWxZSWYwZisrdVF4Tis5?= =?utf-8?B?WU1BeDBCV3NkclhTbW5yVUhaMjZqQUR3Qm9VMCtEVExoc3Q3emttVFF1bFMz?= =?utf-8?B?MUdMSDNIQ3Y2N3RGaWxPMG9jcHhsR3lEMjlJek5zSGVPR2xEVWZUd2FGMEdR?= =?utf-8?B?eHhtb21hQWp4WXI1UXVzUGVhM0krNlQ5em12K2RrZ1kzenlWbFMzeTVPMzFp?= =?utf-8?B?ek5qUjg4NGE3TFg3dDFzL2JPQzhsanV2NFNvWWV4VkdRTi80OW1xZUJhQlFa?= =?utf-8?B?ZHY3Rjl5Y3ZJT0wzOUhyYzEwVEREenNyaUNvVjRrRk5nemZhdWVQWmZoc2M2?= =?utf-8?B?OWhyN0hzT1l2ZUdudlVpcnFPcGJub2FQZ1Z3TmhWcEpucUc3RUIrLzZBYzBI?= =?utf-8?B?UjZUcXErUStadytpSTYxMGR0UGJORTRWZGFoNFExRTcrRFYrQmgvRTREOFZR?= =?utf-8?B?dGg2cVlIcm5aRVVaMUhDVXh4aHZPa0xSRUtMeXVjVUdVMXQ5bUtnbVowR1NY?= =?utf-8?B?V2ZzQ3dFcy9vVzJOMjUrbXdFWDV3RktkakJFeFVRSlBacDdxM3pKc0N6ejJK?= =?utf-8?B?dUhicmU3alBYUStwTDVINmhBdHJZNmM0aHlybkFMenhudkJoVThWS3h5aVVS?= =?utf-8?B?UVdsMlc3MzNxS1ZXKzlzN1lkSVhKSWYvQlhaNStrYi82OEZxaExZeDFlZXk4?= =?utf-8?B?dUxvVEQzWG1HMGNZOHB6dmUvTmFERnlPYXdmcm1peHJST2gwWXhhYjlXakFl?= =?utf-8?B?NmNEeUJ3czE5alNlTkFiYUpHWjV6SkJjSUdYejYzbXo3SWN5ZFMyVlpLcS9F?= =?utf-8?B?Yk5DbFpZdldZaURKN3BqczhERzZPWFR6STN1a1ZNZTcwczhpU21XeEc1d3Fm?= =?utf-8?B?a2kzQnR4bno5WUZXRTFEWlcrRGJjaHFjZEdsNkF4TlZMNmVndWoyakY3eW5I?= =?utf-8?B?R0x3aGVicWRUQ0V0bzVQUCt3VXo3NXU1bk5VOHIrd1VWOGV2R200d01YWEh0?= =?utf-8?B?OHZ3UDNObERPVUMzMXozSmJOMmQwaHZsWnExUThPVnFFMkVzaml0U292N0RZ?= =?utf-8?B?UVlrVys1Q05FRHhROE5sMGNxdlI2SzFYVytBczVjS2lBd1JFRHo3ZkJSMEJW?= =?utf-8?B?QzgyeDY4OWFlS1grM1A3YjZhZlpHMnlhSDlVdHJxZWtiZWVOSWZwWjBVUk45?= =?utf-8?B?Qkl2R3Jzb0pPRFZPOS9mZFNTZHlqYnZkdnl6Z0dLaEEzd2hOK3h5cE9sekVq?= =?utf-8?B?ZnVOS2RuT2dIVjVQdC9MWksySG9LZzE5NGtWeStSclo4cER1RG15cEllVG8y?= =?utf-8?B?Y0VBejRHY3hmYXdyanVrZ0VvbkVtYVAvcVZXRVRBTmxHWWkyb2FCMmIvc3dY?= =?utf-8?B?STNrc3JjV2VRTjJtWExHSTNaeEN6UWRmcGJQbHhsRjJFOHlxWnhHUHBtL2E3?= =?utf-8?B?L29TRnZBT1NpVERIblVjcHBEUTJ4MFBLOVppemZoUUdYL2R6ZWlxcE1Uclpp?= =?utf-8?B?NFEvTS9VU084cEZTTnhjWDhFcDdVVElYeEVZMFpzcGw5dWNFMHlKQ0ltUW1i?= =?utf-8?B?clFyZ1JxRitKS2JGZXc5b21oc0JLL3FZc25TTzFKeWYyTzN4NnB6NUxZalpr?= =?utf-8?B?QXlMeDk4c2pjZklsekNFcERzNUJsY1ByeHZndzAyMWFvV2pnZTY4cHN0OG8w?= =?utf-8?Q?R1VYyI3Mr7/oEtCn9656oHbWyCTO5C0brR14jKP?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1607;6:JK5qlfsau02S6vBJ0G0deLYDkd4CZ1iF2WgPMEh+o/GCC3J9CxSDGRvTl5DglrPl1xWVOAXdQ8G4PexAp09VeZG4CA+FsGDOmVeNgn3CFdKdzmbMvm6QLJykJUMn/ZbpOiziaLq/iCUP9U3HHVy1zeDA16tG9KFP00WmNrQA1vlO+m/47FwUaE/kMEbylIB2Cf4W1mIkGV7HALAc+79VHtEHylbYUA/dN7GGg6ohVU57ky9zyrM3jKMh38B4z8rcvuSFzv5ZWm1+rrUr+wDu/9E42R4otmmwUfjUgu4kRgP9JH9adsIa2Y+Pa9l3XIqArcZY+Ptimk6r5DzKGdHP92NyIxgqbgwOmRgK02UrL2QTLcPxBTlOK135YkhoyrBEft6txhmvDk8EUqLP3p8/dg==;5:Ial4jwScix+nbeWLlK0dNjja3cb1GBDKEbSFFmwaICgQIlchepi+05cohPhfos0L1e2nKMAOVRbbd5J9xhIWSqxtC/DJ0J5sTUHVIig5NTdE31uWxtiyJHcBR296TdxGnJbSC8sV0+pz6UeJRbWfIg==;24:bwrzfI9cyPG9Df7DVwRNfm3ed7LUr8MhTpRb+8RKctQnnIxhHvy0GSBAH+FN3ljBnvfMbdJ+izaeAPb+2bJr9Wzl3GWZyDEYRwgXW8KCLs8= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1607;7:HF7nAEHSYchmqM5GUYB4uZtvBLg2U166Z9JXjsI/Pf0PvOR8tvslfn6LxkrfSXXCzPnXmAuy/2+9FdfG0r76rquQPVy2oiT/1sO2Se9V9ZJ5zD3idIvUNjcUfURh8e0QT3Q1MohTnjnjhpAhf/AXult/0FRV3DQx17bBF8VCg5ZIz2Oa3WGnioXdOrsbOgRAd9CXqLFItbOfGEss9llB6uMuoJLqEPxRTXKQRQAy81DF2hcy6bF+4/VA49hW/uNTGUEGHfIIGjRtE3tyDw709PJged4NRz7Cgg0yFrnrFZWKAZRHQfoGSisJ29hKVr6HA701+S74UUCvG96JVZwqdQ==;20:B6SIP8XpQHUKtRR3XwTyz+DYAJ4kWcUoWwXGNt2xPcS+Xj4QlJQ71IZOQ6wcAf4GR6EXiA9T20LWgfKQ5MAB5lmnTok7Vi6c9ijqtlK+JpMbY4RG2Ke/cbl01w7m2Rfpto0tlxkDoNsdxanMk6jOVbBKN+PuFxYgUW9gaQlUNzPMP8CHhUPv4D9uG+mdKJ7iYlfNLc7Fmfbk/g1NkzTDImr7ch5MdyOvm6Oa2Jd+MFpc6J5K8dKEaXwO0foVYNrF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Mar 2017 15:15:18.6364 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1607 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If kernel_maps_pages_in_pgd is called early in boot process to change the memory attributes then it fails to allocate memory when spliting large pages. The patch extends the cpa_data to provide the support to use memblock_alloc when slab allocator is not available. The feature will be used in Secure Encrypted Virtualization (SEV) mode, where we may need to change the memory region attributes in early boot process. Signed-off-by: Brijesh Singh --- arch/x86/mm/pageattr.c | 51 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 46cc89d..9e4ab3b 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -37,6 +38,7 @@ struct cpa_data { int flags; unsigned long pfn; unsigned force_split : 1; + unsigned force_memblock :1; int curpage; struct page **pages; }; @@ -627,9 +629,8 @@ try_preserve_large_page(pte_t *kpte, unsigned long address, static int __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, - struct page *base) + pte_t *pbase, unsigned long new_pfn) { - pte_t *pbase = (pte_t *)page_address(base); unsigned long ref_pfn, pfn, pfninc = 1; unsigned int i, level; pte_t *tmp; @@ -646,7 +647,7 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 1; } - paravirt_alloc_pte(&init_mm, page_to_pfn(base)); + paravirt_alloc_pte(&init_mm, new_pfn); switch (level) { case PG_LEVEL_2M: @@ -707,7 +708,8 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, * pagetable protections, the actual ptes set above control the * primary protection behavior: */ - __set_pmd_pte(kpte, address, mk_pte(base, __pgprot(_KERNPG_TABLE))); + __set_pmd_pte(kpte, address, + native_make_pte((new_pfn << PAGE_SHIFT) + _KERNPG_TABLE)); /* * Intel Atom errata AAH41 workaround. @@ -723,21 +725,50 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 0; } +static pte_t *try_alloc_pte(struct cpa_data *cpa, unsigned long *pfn) +{ + unsigned long phys; + struct page *base; + + if (cpa->force_memblock) { + phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + if (!phys) + return NULL; + *pfn = phys >> PAGE_SHIFT; + return (pte_t *)__va(phys); + } + + base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + if (!base) + return NULL; + *pfn = page_to_pfn(base); + return (pte_t *)page_address(base); +} + +static void try_free_pte(struct cpa_data *cpa, pte_t *pte) +{ + if (cpa->force_memblock) + memblock_free(__pa(pte), PAGE_SIZE); + else + __free_page((struct page *)pte); +} + static int split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address) { - struct page *base; + pte_t *new_pte; + unsigned long new_pfn; if (!debug_pagealloc_enabled()) spin_unlock(&cpa_lock); - base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + new_pte = try_alloc_pte(cpa, &new_pfn); if (!debug_pagealloc_enabled()) spin_lock(&cpa_lock); - if (!base) + if (!new_pte) return -ENOMEM; - if (__split_large_page(cpa, kpte, address, base)) - __free_page(base); + if (__split_large_page(cpa, kpte, address, new_pte, new_pfn)) + try_free_pte(cpa, new_pte); return 0; } @@ -2035,6 +2066,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, unsigned numpages, unsigned long page_flags) { int retval = -EINVAL; + int use_memblock = !slab_is_available(); struct cpa_data cpa = { .vaddr = &address, @@ -2044,6 +2076,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, .mask_set = __pgprot(0), .mask_clr = __pgprot(0), .flags = 0, + .force_memblock = use_memblock, }; if (!(__supported_pte_mask & _PAGE_NX)) From mboxrd@z Thu Jan 1 00:00:00 1970 From: Brijesh Singh Subject: [RFC PATCH v2 14/32] x86: mm: Provide support to use memblock when spliting large pages Date: Thu, 2 Mar 2017 10:15:15 -0500 Message-ID: <148846771545.2349.9373586041426414252.stgit@brijesh-build-machine> References: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> Sender: owner-linux-mm@kvack.org To: simon.guinot@sequanux.org, linux-efi@vger.kernel.org, brijesh.singh@amd.com, kvm@vger.kernel.org, rkrcmar@redhat.com, matt@codeblueprint.co.uk, linux-pci@vger.kernel.org, linus.walleij@linaro.org, gary.hook@amd.com, linux-mm@kvack.org, paul.gortmaker@windriver.com, hpa@zytor.com, cl@linux.com, dan.j.williams@intel.com, aarcange@redhat.com, sfr@canb.auug.org.au, andriy.shevchenko@linux.intel.com, herbert@gondor.apana.org.au, bhe@redhat.com, xemul@parallels.com, joro@8bytes.org, x86@kernel.org, peterz@infradead.org, piotr.luc@intel.com, mingo@redhat.com, msalter@redhat.com, ross.zwisler@linux.intel.com, bp@suse.de, dyoung@redhat.com, thomas.lendacky@amd.com, jroedel@suse.de, keescook@chromium.org, arnd@arndb.de, toshi.kani@hpe.com, mathieu.desnoyers@efficios.com, luto@kernel.org, devel@linuxdriverproj List-Id: linux-efi@vger.kernel.org If kernel_maps_pages_in_pgd is called early in boot process to change the memory attributes then it fails to allocate memory when spliting large pages. The patch extends the cpa_data to provide the support to use memblock_alloc when slab allocator is not available. The feature will be used in Secure Encrypted Virtualization (SEV) mode, where we may need to change the memory region attributes in early boot process. Signed-off-by: Brijesh Singh --- arch/x86/mm/pageattr.c | 51 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 46cc89d..9e4ab3b 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -37,6 +38,7 @@ struct cpa_data { int flags; unsigned long pfn; unsigned force_split : 1; + unsigned force_memblock :1; int curpage; struct page **pages; }; @@ -627,9 +629,8 @@ try_preserve_large_page(pte_t *kpte, unsigned long address, static int __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, - struct page *base) + pte_t *pbase, unsigned long new_pfn) { - pte_t *pbase = (pte_t *)page_address(base); unsigned long ref_pfn, pfn, pfninc = 1; unsigned int i, level; pte_t *tmp; @@ -646,7 +647,7 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 1; } - paravirt_alloc_pte(&init_mm, page_to_pfn(base)); + paravirt_alloc_pte(&init_mm, new_pfn); switch (level) { case PG_LEVEL_2M: @@ -707,7 +708,8 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, * pagetable protections, the actual ptes set above control the * primary protection behavior: */ - __set_pmd_pte(kpte, address, mk_pte(base, __pgprot(_KERNPG_TABLE))); + __set_pmd_pte(kpte, address, + native_make_pte((new_pfn << PAGE_SHIFT) + _KERNPG_TABLE)); /* * Intel Atom errata AAH41 workaround. @@ -723,21 +725,50 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 0; } +static pte_t *try_alloc_pte(struct cpa_data *cpa, unsigned long *pfn) +{ + unsigned long phys; + struct page *base; + + if (cpa->force_memblock) { + phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + if (!phys) + return NULL; + *pfn = phys >> PAGE_SHIFT; + return (pte_t *)__va(phys); + } + + base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + if (!base) + return NULL; + *pfn = page_to_pfn(base); + return (pte_t *)page_address(base); +} + +static void try_free_pte(struct cpa_data *cpa, pte_t *pte) +{ + if (cpa->force_memblock) + memblock_free(__pa(pte), PAGE_SIZE); + else + __free_page((struct page *)pte); +} + static int split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address) { - struct page *base; + pte_t *new_pte; + unsigned long new_pfn; if (!debug_pagealloc_enabled()) spin_unlock(&cpa_lock); - base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + new_pte = try_alloc_pte(cpa, &new_pfn); if (!debug_pagealloc_enabled()) spin_lock(&cpa_lock); - if (!base) + if (!new_pte) return -ENOMEM; - if (__split_large_page(cpa, kpte, address, base)) - __free_page(base); + if (__split_large_page(cpa, kpte, address, new_pte, new_pfn)) + try_free_pte(cpa, new_pte); return 0; } @@ -2035,6 +2066,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, unsigned numpages, unsigned long page_flags) { int retval = -EINVAL; + int use_memblock = !slab_is_available(); struct cpa_data cpa = { .vaddr = &address, @@ -2044,6 +2076,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, .mask_set = __pgprot(0), .mask_clr = __pgprot(0), .flags = 0, + .force_memblock = use_memblock, }; if (!(__supported_pte_mask & _PAGE_NX)) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Brijesh Singh Subject: [RFC PATCH v2 14/32] x86: mm: Provide support to use memblock when spliting large pages Date: Thu, 2 Mar 2017 10:15:15 -0500 Message-ID: <148846771545.2349.9373586041426414252.stgit@brijesh-build-machine> References: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> Sender: owner-linux-mm@kvack.org List-Id: kvm.vger.kernel.org If kernel_maps_pages_in_pgd is called early in boot process to change the memory attributes then it fails to allocate memory when spliting large pages. The patch extends the cpa_data to provide the support to use memblock_alloc when slab allocator is not available. The feature will be used in Secure Encrypted Virtualization (SEV) mode, where we may need to change the memory region attributes in early boot process. Signed-off-by: Brijesh Singh --- arch/x86/mm/pageattr.c | 51 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 46cc89d..9e4ab3b 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -37,6 +38,7 @@ struct cpa_data { int flags; unsigned long pfn; unsigned force_split : 1; + unsigned force_memblock :1; int curpage; struct page **pages; }; @@ -627,9 +629,8 @@ try_preserve_large_page(pte_t *kpte, unsigned long address, static int __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, - struct page *base) + pte_t *pbase, unsigned long new_pfn) { - pte_t *pbase = (pte_t *)page_address(base); unsigned long ref_pfn, pfn, pfninc = 1; unsigned int i, level; pte_t *tmp; @@ -646,7 +647,7 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 1; } - paravirt_alloc_pte(&init_mm, page_to_pfn(base)); + paravirt_alloc_pte(&init_mm, new_pfn); switch (level) { case PG_LEVEL_2M: @@ -707,7 +708,8 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, * pagetable protections, the actual ptes set above control the * primary protection behavior: */ - __set_pmd_pte(kpte, address, mk_pte(base, __pgprot(_KERNPG_TABLE))); + __set_pmd_pte(kpte, address, + native_make_pte((new_pfn << PAGE_SHIFT) + _KERNPG_TABLE)); /* * Intel Atom errata AAH41 workaround. @@ -723,21 +725,50 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 0; } +static pte_t *try_alloc_pte(struct cpa_data *cpa, unsigned long *pfn) +{ + unsigned long phys; + struct page *base; + + if (cpa->force_memblock) { + phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + if (!phys) + return NULL; + *pfn = phys >> PAGE_SHIFT; + return (pte_t *)__va(phys); + } + + base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + if (!base) + return NULL; + *pfn = page_to_pfn(base); + return (pte_t *)page_address(base); +} + +static void try_free_pte(struct cpa_data *cpa, pte_t *pte) +{ + if (cpa->force_memblock) + memblock_free(__pa(pte), PAGE_SIZE); + else + __free_page((struct page *)pte); +} + static int split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address) { - struct page *base; + pte_t *new_pte; + unsigned long new_pfn; if (!debug_pagealloc_enabled()) spin_unlock(&cpa_lock); - base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + new_pte = try_alloc_pte(cpa, &new_pfn); if (!debug_pagealloc_enabled()) spin_lock(&cpa_lock); - if (!base) + if (!new_pte) return -ENOMEM; - if (__split_large_page(cpa, kpte, address, base)) - __free_page(base); + if (__split_large_page(cpa, kpte, address, new_pte, new_pfn)) + try_free_pte(cpa, new_pte); return 0; } @@ -2035,6 +2066,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, unsigned numpages, unsigned long page_flags) { int retval = -EINVAL; + int use_memblock = !slab_is_available(); struct cpa_data cpa = { .vaddr = &address, @@ -2044,6 +2076,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, .mask_set = __pgprot(0), .mask_clr = __pgprot(0), .flags = 0, + .force_memblock = use_memblock, }; if (!(__supported_pte_mask & _PAGE_NX)) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f200.google.com (mail-pf0-f200.google.com [209.85.192.200]) by kanga.kvack.org (Postfix) with ESMTP id 326896B03A2 for ; Thu, 2 Mar 2017 10:15:25 -0500 (EST) Received: by mail-pf0-f200.google.com with SMTP id e66so12970376pfe.5 for ; Thu, 02 Mar 2017 07:15:25 -0800 (PST) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0055.outbound.protection.outlook.com. [104.47.41.55]) by mx.google.com with ESMTPS id r9si3180180pfe.12.2017.03.02.07.15.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 02 Mar 2017 07:15:24 -0800 (PST) Subject: [RFC PATCH v2 14/32] x86: mm: Provide support to use memblock when spliting large pages From: Brijesh Singh Date: Thu, 2 Mar 2017 10:15:15 -0500 Message-ID: <148846771545.2349.9373586041426414252.stgit@brijesh-build-machine> In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> References: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: simon.guinot@sequanux.org, linux-efi@vger.kernel.org, brijesh.singh@amd.com, kvm@vger.kernel.org, rkrcmar@redhat.com, matt@codeblueprint.co.uk, linux-pci@vger.kernel.org, linus.walleij@linaro.org, gary.hook@amd.com, linux-mm@kvack.org, paul.gortmaker@windriver.com, hpa@zytor.com, cl@linux.com, dan.j.williams@intel.com, aarcange@redhat.com, sfr@canb.auug.org.au, andriy.shevchenko@linux.intel.com, herbert@gondor.apana.org.au, bhe@redhat.com, xemul@parallels.com, joro@8bytes.org, x86@kernel.org, peterz@infradead.org, piotr.luc@intel.com, mingo@redhat.com, msalter@redhat.com, ross.zwisler@linux.intel.com, bp@suse.de, dyoung@redhat.com, thomas.lendacky@amd.com, jroedel@suse.de, keescook@chromium.org, arnd@arndb.de, toshi.kani@hpe.com, mathieu.desnoyers@efficios.com, luto@kernel.org, devel@linuxdriverproject.org, bhelgaas@google.com, tglx@linutronix.de, mchehab@kernel.org, iamjoonsoo.kim@lge.com, labbott@fedoraproject.org, tony.luck@intel.com, alexandre.bounine@idt.com, kuleshovmail@gmail.com, linux-kernel@vger.kernel.org, mcgrof@kernel.org, mst@redhat.com, linux-crypto@vger.kernel.org, tj@kernel.org, pbonzini@redhat.com, akpm@linux-foundation.org, davem@davemloft.net If kernel_maps_pages_in_pgd is called early in boot process to change the memory attributes then it fails to allocate memory when spliting large pages. The patch extends the cpa_data to provide the support to use memblock_alloc when slab allocator is not available. The feature will be used in Secure Encrypted Virtualization (SEV) mode, where we may need to change the memory region attributes in early boot process. Signed-off-by: Brijesh Singh --- arch/x86/mm/pageattr.c | 51 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 46cc89d..9e4ab3b 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -37,6 +38,7 @@ struct cpa_data { int flags; unsigned long pfn; unsigned force_split : 1; + unsigned force_memblock :1; int curpage; struct page **pages; }; @@ -627,9 +629,8 @@ try_preserve_large_page(pte_t *kpte, unsigned long address, static int __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, - struct page *base) + pte_t *pbase, unsigned long new_pfn) { - pte_t *pbase = (pte_t *)page_address(base); unsigned long ref_pfn, pfn, pfninc = 1; unsigned int i, level; pte_t *tmp; @@ -646,7 +647,7 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 1; } - paravirt_alloc_pte(&init_mm, page_to_pfn(base)); + paravirt_alloc_pte(&init_mm, new_pfn); switch (level) { case PG_LEVEL_2M: @@ -707,7 +708,8 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, * pagetable protections, the actual ptes set above control the * primary protection behavior: */ - __set_pmd_pte(kpte, address, mk_pte(base, __pgprot(_KERNPG_TABLE))); + __set_pmd_pte(kpte, address, + native_make_pte((new_pfn << PAGE_SHIFT) + _KERNPG_TABLE)); /* * Intel Atom errata AAH41 workaround. @@ -723,21 +725,50 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 0; } +static pte_t *try_alloc_pte(struct cpa_data *cpa, unsigned long *pfn) +{ + unsigned long phys; + struct page *base; + + if (cpa->force_memblock) { + phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + if (!phys) + return NULL; + *pfn = phys >> PAGE_SHIFT; + return (pte_t *)__va(phys); + } + + base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + if (!base) + return NULL; + *pfn = page_to_pfn(base); + return (pte_t *)page_address(base); +} + +static void try_free_pte(struct cpa_data *cpa, pte_t *pte) +{ + if (cpa->force_memblock) + memblock_free(__pa(pte), PAGE_SIZE); + else + __free_page((struct page *)pte); +} + static int split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address) { - struct page *base; + pte_t *new_pte; + unsigned long new_pfn; if (!debug_pagealloc_enabled()) spin_unlock(&cpa_lock); - base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + new_pte = try_alloc_pte(cpa, &new_pfn); if (!debug_pagealloc_enabled()) spin_lock(&cpa_lock); - if (!base) + if (!new_pte) return -ENOMEM; - if (__split_large_page(cpa, kpte, address, base)) - __free_page(base); + if (__split_large_page(cpa, kpte, address, new_pte, new_pfn)) + try_free_pte(cpa, new_pte); return 0; } @@ -2035,6 +2066,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, unsigned numpages, unsigned long page_flags) { int retval = -EINVAL; + int use_memblock = !slab_is_available(); struct cpa_data cpa = { .vaddr = &address, @@ -2044,6 +2076,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, .mask_set = __pgprot(0), .mask_clr = __pgprot(0), .flags = 0, + .force_memblock = use_memblock, }; if (!(__supported_pte_mask & _PAGE_NX)) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org