From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B77FC433F5 for ; Fri, 8 Apr 2022 19:21:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD9C76B0072; Fri, 8 Apr 2022 15:21:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C88DC6B0073; Fri, 8 Apr 2022 15:21:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B29AD6B0074; Fri, 8 Apr 2022 15:21:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id A4E5E6B0072 for ; Fri, 8 Apr 2022 15:21:19 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 74E8A258E2 for ; Fri, 8 Apr 2022 19:21:19 +0000 (UTC) X-FDA: 79334680278.18.4DB527D Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf06.hostedemail.com (Postfix) with ESMTP id 7DEF1180009 for ; Fri, 8 Apr 2022 19:21:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649445678; x=1680981678; h=message-id:date:mime-version:to:cc:references:from: subject:in-reply-to:content-transfer-encoding; bh=an1kgKT916kkjGguwaSRlVUu2JyLvI48Hqx3SubWWec=; b=JaQwxSWdhJLJMvsri5LYocW3Q9L53hfr+GHbsUFUvIoEv14NMp10iBQu oyzsRLrGVTeYhwhdqz5C64LRqkiH9ZWJiBbtWDom4rX3wR/mIUQ0pg+vR CQkzceSl+9+ULUDomM0LpPdhCKauej/STBfMoRtuU9eo3SRkQ1EBSsfnl UkNt3WQrDsiVuwbTbwD+r8lJa0HlIaUzrdb4UgrVH2U4uYTwCtx15P034 xvHMAGZ1Bdfjixavghb59z6A+qszMkZLEL7vwIzU/YIxA1MtG4SQDrRYf WsFZt8urCsBZpI4MhRQAjrsgxL9t6FTrGDdNH0Z+X1IWd0zfUZ16efPUm A==; X-IronPort-AV: E=McAfee;i="6400,9594,10311"; a="261662094" X-IronPort-AV: E=Sophos;i="5.90,245,1643702400"; d="scan'208";a="261662094" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2022 12:21:16 -0700 X-IronPort-AV: E=Sophos;i="5.90,245,1643702400"; d="scan'208";a="525495656" Received: from tsungtae-mobl.amr.corp.intel.com (HELO [10.134.43.198]) ([10.134.43.198]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2022 12:21:15 -0700 Message-ID: <0e366406-9a3a-0bf3-e073-272279f6abf2@intel.com> Date: Fri, 8 Apr 2022 12:21:19 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Content-Language: en-US To: "Kirill A. Shutemov" , Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel , Ard Biesheuvel Cc: Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Varad Gautam , Dario Faggioli , Brijesh Singh , Mike Rapoport , David Hildenbrand , x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org References: <20220405234343.74045-1-kirill.shutemov@linux.intel.com> <20220405234343.74045-7-kirill.shutemov@linux.intel.com> From: Dave Hansen Subject: Re: [PATCHv4 6/8] x86/mm: Provide helpers for unaccepted memory In-Reply-To: <20220405234343.74045-7-kirill.shutemov@linux.intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JaQwxSWd; spf=none (imf06.hostedemail.com: domain of dave.hansen@intel.com has no SPF policy when checking 134.134.136.65) smtp.mailfrom=dave.hansen@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7DEF1180009 X-Stat-Signature: omhf68eu74s3rxmuyj7s6sjfx7r9sdnw X-HE-Tag: 1649445678-772536 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 4/5/22 16:43, Kirill A. Shutemov wrote: > +void accept_memory(phys_addr_t start, phys_addr_t end) > +{ > + unsigned long *unaccepted_memory; > + unsigned long flags; > + unsigned int rs, re; > + > + if (!boot_params.unaccepted_memory) > + return; > + > + unaccepted_memory = __va(boot_params.unaccepted_memory); > + rs = start / PMD_SIZE; > + > + spin_lock_irqsave(&unaccepted_memory_lock, flags); > + for_each_set_bitrange_from(rs, re, unaccepted_memory, > + DIV_ROUND_UP(end, PMD_SIZE)) { > + /* Platform-specific memory-acceptance call goes here */ > + panic("Cannot accept memory"); > + bitmap_clear(unaccepted_memory, rs, re - rs); > + } > + spin_unlock_irqrestore(&unaccepted_memory_lock, flags); > +} Just to reiterate: this is a global spinlock. It's disabling interrupts. "Platform-specific memory-acceptance call" will soon become: tdx_accept_memory(rs * PMD_SIZE, re * PMD_SIZE); which is a page-by-page __tdx_module_call(): > + for (i = 0; i < (end - start) / PAGE_SIZE; i++) { > + if (__tdx_module_call(TDACCEPTPAGE, start + i * PAGE_SIZE, > + 0, 0, 0, NULL)) { > + error("Cannot accept memory: page accept failed\n"); > + } > + } Each __tdx_module_call() involves a privilege transition that also surely includes things like changing CR3. It can't be cheap. It also is presumably touching the memory and probably flushing it out of the CPU caches. It's also unbounded: spin_lock_irqsave(&unaccepted_memory_lock, flags); for (i = 0; i < (end - start) / PAGE_SIZE; i++) // thousands? tens-of-thousands of cycles?? spin_lock_irqsave(&unaccepted_memory_lock, flags); How far apart can end and start be? It's at *least* 2MB in the page allocator, which is on the order of a millisecond. Are we sure there aren't any callers that want to do this at a gigabyte granularity? That would hold the global lock and disable interrupts on the order of a second. Do we want to bound the time that the lock can be held? Or, should we just let the lockup detectors tell us that we're being naughty?