From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13025C433E0 for ; Thu, 25 Mar 2021 14:47:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C90C061A10 for ; Thu, 25 Mar 2021 14:47:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229963AbhCYOqy (ORCPT ); Thu, 25 Mar 2021 10:46:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22006 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230516AbhCYOqf (ORCPT ); Thu, 25 Mar 2021 10:46:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616683591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dchsKoS2hatPZ8CWZWHmKwa7mr1Z7EkRjZQTq43V0Rg=; b=H/VcpvQfsUQo2XVCp0OeyfVIKD6cREYmnC4iGRl34ruwhgaCM/M3kRT4035y5rF1YKhdht HQFfbTO37QZ8rLaOlPQwg6v0CFREaYpzOUFfD3La5oTfrR0G2A9HiwXzo6OEMfTi6xttEW QGIuaAWAiAFSW2jEshdu0NcS9hFzzqc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-100-G6YmT2LVNjKee9LulUPAsQ-1; Thu, 25 Mar 2021 10:46:27 -0400 X-MC-Unique: G6YmT2LVNjKee9LulUPAsQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E028580006E; Thu, 25 Mar 2021 14:46:25 +0000 (UTC) Received: from [10.36.115.72] (ovpn-115-72.ams2.redhat.com [10.36.115.72]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE92260C2E; Thu, 25 Mar 2021 14:46:23 +0000 (UTC) To: Michal Hocko Cc: Oscar Salvador , Andrew Morton , Anshuman Khandual , Vlastimil Babka , Pavel Tatashin , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <9591a0b8-c000-2f61-67a6-4402678fe50b@redhat.com> <31110e58-c99a-8dee-6f6e-98f456b77759@redhat.com> <062bc5d7-a83c-1c1a-7b77-9f043643f4fa@redhat.com> From: David Hildenbrand Organization: Red Hat GmbH Subject: Re: [PATCH v5 1/5] mm,memory_hotplug: Allocate memmap from the added memory range Message-ID: <31c3e6f7-f631-7b00-2c33-518b0f24a75f@redhat.com> Date: Thu, 25 Mar 2021 15:46:22 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 25.03.21 15:34, Michal Hocko wrote: > On Thu 25-03-21 15:09:35, David Hildenbrand wrote: >> On 25.03.21 15:08, Michal Hocko wrote: >>> On Thu 25-03-21 13:40:45, David Hildenbrand wrote: >>>> On 25.03.21 13:35, Michal Hocko wrote: >>>>> On Thu 25-03-21 12:08:43, David Hildenbrand wrote: >>>>>> On 25.03.21 11:55, Oscar Salvador wrote: >>> [...] >>>>>>> - When moving the initialization/accounting to hot-add/hot-remove, >>>>>>> the section containing the vmemmap pages will remain offline. >>>>>>> It might get onlined once the pages get online in online_pages(), >>>>>>> or not if vmemmap pages span a whole section. >>>>>>> I remember (but maybe David rmemeber better) that that was a problem >>>>>>> wrt. pfn_to_online_page() and hybernation/kdump. >>>>>>> So, if that is really a problem, we would have to care of ot setting >>>>>>> the section to the right state. >>>>>> >>>>>> Good memory. Indeed, hibernation/kdump won't save the state of the vmemmap, >>>>>> because the memory is marked as offline and, thus, logically without any >>>>>> valuable content. > > ^^^^ THIS > >>>>> >>>>> Could you point me to the respective hibernation code please? I always >>>>> get lost in that area. Anyway, we do have the same problem even if the >>>>> whole accounting is handled during {on,off}lining, no? >>>> >>>> kernel/power/snapshot.c:saveable_page(). >>> >>> Thanks! So this is as I've suspected. The very same problem is present >>> if the memory block is marked offline. So we need a solution here >>> anyway. One way to go would be to consider these vmemmap pages always >>> online. pfn_to_online_page would have to special case them but we would >>> need to identify them first. I used to have PageVmemmap or something >>> like that in my early attempt to do this. >>> >>> That being said this is not an argument for one or the other aproach. >>> Both need fixing. >> >> Can you elaborate? What is the issue there? What needs fixing? > > offline section containing vmemmap will be lost during hibernation cycle > IIU the above correctly. > Can tell me how that is a problem with Oscars current patch? I only see this being a problem with what you propose - most probably I am missing something important here. Offline memory sections don't have a valid memmap (assumption: garbage). On hibernation, the whole offline memory block won't be saved, including the vmemmap content that resides on the block. This includes the vmemmap of the vmemmap pages, which is itself. When restoring, the whole memory block will contain garbage, including the whole vmemmap - which is marked to be offline and to contain garbage. Oscars patch: Works as expected. Onlining the memory block will properly intiialize the vmemmap (including the vmemmap of the vmemmap), then mark the vmemmap to be valid (by marking the sections to be online). Your porposal: Does not work as expected. Once we online the memory block we don't initialize the vmemmap of the vmemmap pages. There is garbage, which gets exposed to the system as soon as we mark the vmemmap to be valid (by marking the sections to be online). -- Thanks, David / dhildenb