From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2B3BC35242 for ; Mon, 17 Feb 2020 04:49:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 49EE120718 for ; Mon, 17 Feb 2020 04:49:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IZnpQRfF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 49EE120718 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B057A6B0003; Sun, 16 Feb 2020 23:49:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB8036B0005; Sun, 16 Feb 2020 23:49:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CB546B0007; Sun, 16 Feb 2020 23:49:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0095.hostedemail.com [216.40.44.95]) by kanga.kvack.org (Postfix) with ESMTP id 829596B0003 for ; Sun, 16 Feb 2020 23:49:09 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0EA6C181AEF00 for ; Mon, 17 Feb 2020 04:49:09 +0000 (UTC) X-FDA: 76498389618.08.joke76_89f9d06a5485e X-HE-Tag: joke76_89f9d06a5485e X-Filterd-Recvd-Size: 9025 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Mon, 17 Feb 2020 04:49:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581914947; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1xWaLqtgVGHFWT+uUYHDGnG/a9tVbhVxPV9uwqoYYUo=; b=IZnpQRfFygLZcXAoAH1qXxpoNhl0bRk9xMLu3QC4/qRi+/q9/BvnoA0FNlEJQQebGJqaEL uebZVf/pLcGpuCa0o+fnuN2urLC7AKKtSOKB9ff2ByjBuDVTScAQqwBxSQB2UsgsCr8Pc8 QXcTPu5UOou0TI9pOsbJRZuVn3jzOZI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-370-9mphpKtIMqafazVlfw83Rw-1; Sun, 16 Feb 2020 23:48:58 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 784D0100550E; Mon, 17 Feb 2020 04:48:56 +0000 (UTC) Received: from localhost (ovpn-12-200.pek2.redhat.com [10.72.12.200]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 71B245C10D; Mon, 17 Feb 2020 04:48:52 +0000 (UTC) Date: Mon, 17 Feb 2020 12:48:50 +0800 From: Baoquan He To: david@redhat.com, kkabe@vega.pgw.jp Cc: bugzilla-daemon@bugzilla.kernel.org, akpm@linux-foundation.org, richardw.yang@linux.intel.com, mhocko@kernel.org, n-horiguchi@ah.jp.nec.com, linux-mm@kvack.org Subject: Re: [Bug 206401] kernel panic on Hyper-V after 5 minutes duetomemory hot-add Message-ID: <20200217044850.GD4816@MiWiFi-R3L-srv> References: <20200213081941.GA19207@MiWiFi-R3L-srv> <200214232629.M0108877@vega.pgw.jp> MIME-Version: 1.0 In-Reply-To: <200214232629.M0108877@vega.pgw.jp> User-Agent: Mutt/1.10.1 (2018-07-13) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: 9mphpKtIMqafazVlfw83Rw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 02/14/20 at 11:26pm, kkabe@vega.pgw.jp wrote: > bhe@redhat.com sed in <20200213081941.GA19207@MiWiFi-R3L-srv> >=20 > >> On 02/13/20 at 01:22pm, kabe@vega.pgw.jp wrote: > >> > bhe@redhat.com sed in <20200212073123.GG8965@MiWiFi-R3L-srv> > >> >=20 > >> > >> On 02/11/20 at 04:41pm, Andrew Morton wrote: > >> > >> > On Tue, 11 Feb 2020 07:07:41 +0800 Wei Yang wrote: > >> > >> >=20 > >> > >> > > On Mon, Feb 10, 2020 at 02:15:51PM +0800, Baoquan He wrote: > >> > >> > > >On 02/10/20 at 02:09pm, Baoquan He wrote: > >> > >> > > >> On 02/09/20 at 09:56pm, Andrew Morton wrote: > >> > >> > > >> > On Mon, 10 Feb 2020 13:40:27 +0800 Baoquan He wrote: > >> > >> > > >> >=20 > >> > >> > > >> > > Hi Andrew, > >> > >> > > >> > >=20 > >> > >> > > >> > > On 02/09/20 at 09:32pm, Andrew Morton wrote: > >> > >> > > >> > > > On Tue, 04 Feb 2020 11:25:48 +0000 bugzilla-daemon@b= ugzilla.kernel.org wrote: > >> > >> > > >> > > >=20 > >> > >> > > >> > > > > https://bugzilla.kernel.org/show_bug.cgi?id=3D2064= 01 > >> > >> > > >> > > > >=20 > >> > >> > > >> > > >=20 > >> > >> > > >> > > > An oops during mem hotadd. Could someone please tak= e a look when > >> > >> > > >> > > > convenient? > >> > >> > > >> > >=20 > >> > >> > > >> > > This has been addressed by Wei Yang's patch, please ch= eck it here: > >> > >> > > >> > >=20 > >> > >> > > >> > > http://lkml.kernel.org/r/20200209104826.3385-7-bhe@red= hat.com > >> > >> > > >> > >=20 > >> > >> > > >> >=20 > >> > >> > > >> > hm, OK, thanks. It's unfortunate that a 5.5 fix is buri= ed in a > >> > >> > > >> > six-patch series which is still in progress! Can we ple= ase merge that > >> > >> > > >> > as a standalone fix with a cc:stable, Fixes:, etc? > >> > >> > > > > >> > >> > > >Maybe can add Fixes tag as follow when merge: > >> > >> > > > > >> > >> > > >Fixes: ba72b4c8cf60 ("mm/sparsemem: support sub-section hotp= lug") > >> > >> > > > > >> > >> >=20 > >> > >> > The reporter (cc'ed here) is still seeing issues: > >> > >> > https://bugzilla.kernel.org/show_bug.cgi?id=3D206401 > >> > >> >=20 > >> > >> > Could we please continue this investigation via emailed reply-t= o-all, > >> > >> > rather than via the bugzilla interface? > >> > >>=20 > >> > >> Yes, people prefer mailing list to discuss issues. > >> > >>=20 > >> > >> Hi T.Kabe,=20 > >> > >>=20 > >> > >> Could you provide the call trace again after below patch is appli= ed? > >> > >> The comment #9 in bugzilla is not very clear to me. > >> > >>=20 > >> > >> mm/sparsemem: pfn_to_page is not valid yet on SPARSEMEM > >> > >> http://lkml.kernel.org/r/20200209104826.3385-7-bhe@redhat.com > >> > >>=20 > >> > >> And, as you said, applying above patch, and do not call > >> > >> __free_pages_core() in generic_online_page() will work. I doubt i= t, > >> > >> because without __free_pages_core(), your added pages are not add= ed > >> > >> into buddy for managing. I think we should make clear this proble= m > >> > >> firstly, in order not to introduce new problem by improper work a= round, > >> > >> then check next. > >> > >>=20 > >> > >> Thanks > >> > >> Baoquan > >> >=20 > >> > Got it, I restarted off fresh from kernel-5.6-rc1, > >> > applied patch > >> > >> http://lkml.kernel.org/r/20200209104826.3385-7-bhe@redhat.com > >> > and got the following panic. > >> >=20 > >> > Diag printk's for add_memory() et al is not there, but I guess > >> > memory hot-add request from hypervisor is returning "success",=20 > >> > corrupting something else and bombing out later. > >> >=20 > >> >=20 > >> > [ 24.289967] Not activating Mandatory Access Control as /sbin/tomo= yo-init does not exist. > >> > [ 302.263730] hv_balloon: Max. dynamic memory size: 1048576 MB > >> > [ 635.216014] BUG: unable to handle page fault for address: d13ff00= 0 > >> > [ 635.216058] #PF: supervisor write access in kernel mode > >> > [ 635.216076] #PF: error_code(0x0002) - not-present page > >> > [ 635.216106] *pde =3D 00000000 > >>=20 > >> Thanks for the info. What ARCH is your system? Could you attach your > >> kernel config and paste the output of executing 'readelf /proc/kcore'? >=20 > Arch is i386(i586), non-PAE. Sorry, I roughly went through code, didn't get clue. Not sure if David have idea about it. By the way, may I know why you would like to run i386 guest on Hyper-V? Found people are talking about the 32bit kernel supporting in upstream, below is Linus's point of view. https://lore.kernel.org/linux-fsdevel/CAHk-=3DwiGbz3oRvAVFtN-whW-d2F-STKsP1= MZT4m_VeycAr1_VQ@mail.gmail.com/ >=20 > I'll attach the "readelf -a /proc/kcore", dmesg and .config . > The stack trace is different this time also; > it seems to have slightly difference panic trace every time=20 > after handle_mm_fault(). >=20 > I've temporary added pr_info() before and after add_memory() in hv_baloon= .ko, > so it says it's taining the kernel. > add_memory() itself is returning 0 (success). >=20 >=20 > >> The pmd entry is not filled, I want to check which address range the k= ernel > >> is acessing, and please attach the log of dmesg. Probably it's hot add= ed > >> page area, I guess, since this time the preceding trace is different > >> with comment #9. > >>=20 > >> > [ 635.216139] Oops: 0002 [#1] SMP > >> > [ 635.216171] CPU: 0 PID: 470 Comm: systemd-udevd Not tainted 5.6.0= -rc1.el8.i586 #1 > >> > [ 635.216199] Hardware name: Microsoft Corporation Virtual Machine/= Virtual Machine, BIOS 090006 05/23/2012 > >> > [ 635.216233] EIP: wp_page_copy+0x8e/0x750 > >> > [ 635.216253] Code: 03 00 00 8b 45 d0 85 c0 0f 84 46 05 00 00 e8 d9= 85 e5 ff 89 45 bc 89 f8 e8 cf 85 e5 ff 8b 55 bc 8d 78 04 8b 0a 83 e7 fc 89= d6 <89> 08 8b 8a fc 0f 00 00 89 88 fc 0f 00 00 89 c1 29 f9 89 55 bc 29 > >> > [ 635.216293] EAX: d13ff000 EBX: c3743f28 ECX: 00000000 EDX: c10c90= 00 > >> > [ 635.216314] ESI: c10c9000 EDI: d13ff004 EBP: c3743eec ESP: c3743e= a8 > >> > [ 635.216336] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: = 00210282 > >> > [ 635.216368] CR0: 80050033 CR2: d13ff000 CR3: 03add000 CR4: 003406= d0 > >> > [ 635.216389] Call Trace: > >> > [ 635.216407] ? reuse_swap_page+0x83/0x390 > >> > [ 635.216425] do_wp_page+0x87/0x6e0 > >> > [ 635.216438] ? __do_sys_fstat64+0x4a/0x60 > >> > [ 635.216453] handle_mm_fault+0x808/0xe30 > >> > [ 635.216468] do_page_fault+0x19f/0x4d0 > >> > [ 635.216484] ? do_kern_addr_fault+0x80/0x80 > >> > [ 635.216500] common_exception_read_cr2+0x15a/0x15f > >> > [ 635.216521] EIP: 0xb7b28104