From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B38D9C4360D for ; Wed, 25 Sep 2019 15:18:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 502812054F for ; Wed, 25 Sep 2019 15:18:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=lca.pw header.i=@lca.pw header.b="dvvzP1gw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 502812054F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lca.pw Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E38446B0007; Wed, 25 Sep 2019 11:18:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE62A6B0008; Wed, 25 Sep 2019 11:18:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD6216B0275; Wed, 25 Sep 2019 11:18:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0131.hostedemail.com [216.40.44.131]) by kanga.kvack.org (Postfix) with ESMTP id A57D56B0007 for ; Wed, 25 Sep 2019 11:18:53 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 3990C180AD802 for ; Wed, 25 Sep 2019 15:18:53 +0000 (UTC) X-FDA: 75973800546.18.mark71_6a9b1b4b6285e X-HE-Tag: mark71_6a9b1b4b6285e X-Filterd-Recvd-Size: 17772 Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Wed, 25 Sep 2019 15:18:52 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id m15so7031231qtq.2 for ; Wed, 25 Sep 2019 08:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lca.pw; s=google; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=ghAxxiYIqFLDFa+1mBl8boCylk/lh1uTWDDIQYvrkHY=; b=dvvzP1gwHT3G+DFKG9KecVFPHNJ7Fh/0IGBOK35cbnb03U3PwJ52GnpLdTuli7d0fs QNNu0ZTScR+a9D5Az78wSIC7DwTymMMyeiqgrrTn0OmYOt9W8YS0NGJLM8MN0KlJKUE6 F9TOFquYdIgHlll2bIRXDLGRnbc5pB46/WC1bk9GGjmmmaHaKvNI3+3o1Z1P+OcJVGqE uPn2gRz/h5qCgZ1JtHegt+9pFXwHxsE1HJw4CJCk2TeAl2EhN/iaN5U+l2uc2dHjsuuk be9n8qo1VASTsU89OhS5SJ2oSf3lBRNClX/G7g62ZXyw9EaOHlaoTF1d/NQ946NnSven G/vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=ghAxxiYIqFLDFa+1mBl8boCylk/lh1uTWDDIQYvrkHY=; b=c/g/04Hj29KP0DGu2eKo4XW1SferVnigcbNMuq+0RgBTxPLo+R5Aa1ZlTqQvUgQWgq DoV3L2n1JSM4ogEv8TD7j9OihPBnneLHv9FQ/MWkAsVImJbAlYXm1IG8EjQgHlFHin5V uOin1AoFOy4xBW+rJx2j+ZVN8OjEQoq0Ghs3XSHg3UQ6go+OzoM7vX17pUM42D9deH+a GwwaU0upt71/gnZwOpimP8Wa6u+5PyEMKaxYhvfJ5beO0YS4i0Z8mnbd/idL6CGK+OGo YNi7m+Xuf25dFZqJNZpr1LqVm9AaBwiF6gsp0d+mt92AKe8YzQ+uGr4bcoCVmfAFHNq/ 4gQw== X-Gm-Message-State: APjAAAVFNzTk6qt0RECWwUPlIBUg6I94T92sSceZpipy7nfaFh1YhxVY trAR9lCQ1MoJCs+T9TUElJ0RRA== X-Google-Smtp-Source: APXvYqzXrPkILsLkpLfMz8EbdENfPe9ZQ3t8JKL8zalv3Xh3qfXOCdQnszAjeZYj1aN4AlPMk/kz1Q== X-Received: by 2002:a05:6214:1354:: with SMTP id b20mr7949547qvw.43.1569424731492; Wed, 25 Sep 2019 08:18:51 -0700 (PDT) Received: from dhcp-41-57.bos.redhat.com (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id k46sm4508510qtc.96.2019.09.25.08.18.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Sep 2019 08:18:50 -0700 (PDT) Message-ID: <1569424727.5576.221.camel@lca.pw> Subject: Re: [PATCH] mm/slub: fix a deadlock in shuffle_freelist() From: Qian Cai To: Peter Zijlstra Cc: akpm@linux-foundation.org, bigeasy@linutronix.de, tglx@linutronix.de, thgarnie@google.com, tytso@mit.edu, cl@linux.com, penberg@kernel.org, rientjes@google.com, mingo@redhat.com, will@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, keescook@chromium.org Date: Wed, 25 Sep 2019 11:18:47 -0400 In-Reply-To: <20190925093153.GC4553@hirez.programming.kicks-ass.net> References: <1568392064-3052-1-git-send-email-cai@lca.pw> <20190925093153.GC4553@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6 (3.22.6-10.el7) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 2019-09-25 at 11:31 +0200, Peter Zijlstra wrote: > On Fri, Sep 13, 2019 at 12:27:44PM -0400, Qian Cai wrote: > > The commit b7d5dc21072c ("random: add a spinlock_t to struct > > batched_entropy") insists on acquiring "batched_entropy_u32.lock" in > > get_random_u32() which introduced the lock chain, > >=20 > > "&rq->lock --> batched_entropy_u32.lock" > >=20 > > even after crng init. As the result, it could result in deadlock belo= w. > > Fix it by using get_random_bytes() in shuffle_freelist() which does n= ot > > need to take on the batched_entropy locks. > >=20 > > WARNING: possible circular locking dependency detected > > 5.3.0-rc7-mm1+ #3 Tainted: G L > > ------------------------------------------------------ > > make/7937 is trying to acquire lock: > > ffff900012f225f8 (random_write_wait.lock){....}, at: > > __wake_up_common_lock+0xa8/0x11c > >=20 > > but task is already holding lock: > > ffff0096b9429c00 (batched_entropy_u32.lock){-.-.}, at: > > get_random_u32+0x6c/0x1dc > >=20 > > which lock already depends on the new lock. > >=20 > > the existing dependency chain (in reverse order) is: > >=20 > > -> #3 (batched_entropy_u32.lock){-.-.}: > > lock_acquire+0x31c/0x360 > > _raw_spin_lock_irqsave+0x7c/0x9c > > get_random_u32+0x6c/0x1dc > > new_slab+0x234/0x6c0 > > ___slab_alloc+0x3c8/0x650 > > kmem_cache_alloc+0x4b0/0x590 > > __debug_object_init+0x778/0x8b4 > > debug_object_init+0x40/0x50 > > debug_init+0x30/0x29c > > hrtimer_init+0x30/0x50 > > init_dl_task_timer+0x24/0x44 > > __sched_fork+0xc0/0x168 > > init_idle+0x78/0x26c > > fork_idle+0x12c/0x178 > > idle_threads_init+0x108/0x178 > > smp_init+0x20/0x1bc > > kernel_init_freeable+0x198/0x26c > > kernel_init+0x18/0x334 > > ret_from_fork+0x10/0x18 > >=20 > > -> #2 (&rq->lock){-.-.}: >=20 > This relation is silly.. >=20 > I suspect the below 'works'... Unfortunately, the relation is still there, copy_process()->rt_mutex_init_task()->"&p->pi_lock" [24438.676716][=C2=A0=C2=A0=C2=A0=C2=A0T2] -> #2 (&rq->lock){-.-.}: [24438.676727][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__lock_acquire+0x5b4/0xbf0 [24438.676736][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0lock_acquire+0x130/0x360 [24438.676754][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_raw_spin_lock+0x54/0x80 [24438.676771][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0task_fork_fair+0x60/0x190 [24438.676788][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0sched_fork+0x128/0x270 [24438.676806][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0copy_process+0x7a4/0x1bf0 [24438.676823][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_do_fork+0xac/0xac0 [24438.676841][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kernel_thread+0x70/0xa0 [24438.676858][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0rest_init+0x4c/0x42c [24438.676884][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0start_kernel+0x778/0x7c0 [24438.676902][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0start_here_common+0x1c/0x334 Whole thing, [24438.675704][=C2=A0=C2=A0=C2=A0=C2=A0T2] WARNING: possible circular loc= king dependency detected [24438.675714][=C2=A0=C2=A0=C2=A0=C2=A0T2] 5.3.0-next-20190924 #2 Not tai= nted [24438.675722][=C2=A0=C2=A0=C2=A0=C2=A0T2] ------------------------------= ------------------------ [24438.675731][=C2=A0=C2=A0=C2=A0=C2=A0T2] kthreadd/2 is trying to acquir= e lock: [24438.675740][=C2=A0=C2=A0=C2=A0=C2=A0T2] c0000000010a7450 (random_write= _wait.lock){..-.}, at: __wake_up_common_lock+0x88/0x110 [24438.675768][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.675768][=C2=A0=C2=A0=C2=A0=C2=A0T2] but task is already holding lo= ck: [24438.675778][=C2=A0=C2=A0=C2=A0=C2=A0T2] c000001ffd2f06e0 (batched_entr= opy_u64.lock){-...}, at: get_random_u64+0x60/0x100 [24438.675803][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.675803][=C2=A0=C2=A0=C2=A0=C2=A0T2] which lock already depends on = the new lock. [24438.675803][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.675816][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.675816][=C2=A0=C2=A0=C2=A0=C2=A0T2] the existing dependency chain = (in reverse order) is: [24438.675836][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.675836][=C2=A0=C2=A0=C2=A0=C2=A0T2] -> #4 (batched_entropy_u64.loc= k){-...}: [24438.675860][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__lock_acquire+0x5b4/0xbf0 [24438.675878][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0lock_acquire+0x130/0x360 [24438.675906][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_raw_spin_lock_irqsave+0x70/0xa0 [24438.675923][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0get_random_u64+0x60/0x100 [24438.675944][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0add_to_free_area_random+0x164/0x1b0 [24438.675962][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0free_one_page+0xb24/0xcf0 [24438.675980][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__free_pages_ok+0x448/0xbf0 [24438.675999][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0deferred_init_maxorder+0x404/0x4a4 [24438.676018][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0deferred_grow_zone+0x158/0x1f0 [24438.676035][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0get_page_from_freelist+0x1dc8/0x1e10 [24438.676063][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__alloc_pages_nodemask+0x1d8/0x1940 [24438.676083][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0allocate_slab+0x130/0x2740 [24438.676091][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0new_slab+0xa8/0xe0 [24438.676101][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kmem_cache_open+0x254/0x660 [24438.676119][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__kmem_cache_create+0x44/0x2a0 [24438.676136][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0create_boot_cache+0xcc/0x110 [24438.676154][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kmem_cache_init+0x90/0x1f0 [24438.676173][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0start_kernel+0x3b8/0x7c0 [24438.676191][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0start_here_common+0x1c/0x334 [24438.676208][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.676208][=C2=A0=C2=A0=C2=A0=C2=A0T2] -> #3 (&(&zone->lock)->rlock){= -.-.}: [24438.676221][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__lock_acquire+0x5b4/0xbf0 [24438.676247][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0lock_acquire+0x130/0x360 [24438.676264][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_raw_spin_lock+0x54/0x80 [24438.676282][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0rmqueue_bulk.constprop.23+0x64/0xf20 [24438.676300][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0get_page_from_freelist+0x718/0x1e10 [24438.676319][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__alloc_pages_nodemask+0x1d8/0x1940 [24438.676339][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0alloc_page_interleave+0x34/0x170 [24438.676356][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0allocate_slab+0xd1c/0x2740 [24438.676374][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0new_slab+0xa8/0xe0 [24438.676391][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0___slab_alloc+0x580/0xef0 [24438.676408][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__slab_alloc+0x64/0xd0 [24438.676426][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kmem_cache_alloc+0x5c4/0x6c0 [24438.676444][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0fill_pool+0x280/0x540 [24438.676461][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__debug_object_init+0x60/0x6b0 [24438.676479][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0hrtimer_init+0x5c/0x310 [24438.676497][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0init_dl_task_timer+0x34/0x60 [24438.676516][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__sched_fork+0x8c/0x110 [24438.676535][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0init_idle+0xb4/0x3c0 [24438.676553][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0idle_thread_get+0x78/0x120 [24438.676572][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0bringup_cpu+0x30/0x230 [24438.676590][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0cpuhp_invoke_callback+0x190/0x1580 [24438.676618][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0do_cpu_up+0x248/0x460 [24438.676636][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0smp_init+0x118/0x1c0 [24438.676662][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kernel_init_freeable+0x3f8/0x8dc [24438.676681][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kernel_init+0x2c/0x154 [24438.676699][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0ret_from_kernel_thread+0x5c/0x74 [24438.676716][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.676716][=C2=A0=C2=A0=C2=A0=C2=A0T2] -> #2 (&rq->lock){-.-.}: [24438.676727][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__lock_acquire+0x5b4/0xbf0 [24438.676736][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0lock_acquire+0x130/0x360 [24438.676754][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_raw_spin_lock+0x54/0x80 [24438.676771][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0task_fork_fair+0x60/0x190 [24438.676788][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0sched_fork+0x128/0x270 [24438.676806][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0copy_process+0x7a4/0x1bf0 [24438.676823][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_do_fork+0xac/0xac0 [24438.676841][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kernel_thread+0x70/0xa0 [24438.676858][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0rest_init+0x4c/0x42c [24438.676884][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0start_kernel+0x778/0x7c0 [24438.676902][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0start_here_common+0x1c/0x334 [24438.676910][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.676910][=C2=A0=C2=A0=C2=A0=C2=A0T2] -> #1 (&p->pi_lock){-.-.}: [24438.676921][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__lock_acquire+0x5b4/0xbf0 [24438.676929][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0lock_acquire+0x130/0x360 [24438.676947][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_raw_spin_lock_irqsave+0x70/0xa0 [24438.676973][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0try_to_wake_up+0x70/0x1600 [24438.676991][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0pollwake+0x88/0xc0 [24438.677009][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__wake_up_common+0xec/0x280 [24438.677026][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__wake_up_common_lock+0xac/0x110 [24438.677044][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0account.constprop.8+0x284/0x430 [24438.677061][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0extract_entropy.constprop.7+0xd4/0x330 [24438.677080][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_xfer_secondary_pool+0x104/0x3e0 [24438.677097][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0push_to_pool+0x58/0x310 [24438.677116][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0process_one_work+0x300/0x8e0 [24438.677133][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0worker_thread+0x78/0x530 [24438.677151][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kthread+0x1a8/0x1b0 [24438.677180][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0ret_from_kernel_thread+0x5c/0x74 [24438.677245][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.677245][=C2=A0=C2=A0=C2=A0=C2=A0T2] -> #0 (random_write_wait.lock)= {..-.}: [24438.677329][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0check_prev_add+0x100/0x11b0 [24438.677377][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0validate_chain+0x868/0x1530 [24438.677446][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__lock_acquire+0x5b4/0xbf0 [24438.677516][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0lock_acquire+0x130/0x360 [24438.677563][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_raw_spin_lock_irqsave+0x70/0xa0 [24438.677618][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0__wake_up_common_lock+0x88/0x110 [24438.677678][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0account.constprop.8+0x284/0x430 [24438.677743][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0extract_entropy.constprop.7+0xd4/0x330 [24438.677802][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0crng_reseed+0x68/0x490 [24438.677867][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_extract_crng+0x104/0x110 [24438.677914][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0crng_reseed+0x284/0x490 [24438.677983][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_extract_crng+0x104/0x110 [24438.678032][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0get_random_u64+0xdc/0x100 [24438.678101][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0copy_process+0x2d8/0x1bf0 [24438.678148][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0_do_fork+0xac/0xac0 [24438.678208][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kernel_thread+0x70/0xa0 [24438.678246][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0kthreadd+0x270/0x330 [24438.678301][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0ret_from_kernel_thread+0x5c/0x74 [24438.678342][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.678342][=C2=A0=C2=A0=C2=A0=C2=A0T2] other info that might help us = debug this: [24438.678342][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.678459][=C2=A0=C2=A0=C2=A0=C2=A0T2] Chain exists of: [24438.678459][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0random_write_= wait.lock --> &(&zone->lock)->rlock --> batched_entropy_u64.lock [24438.678459][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.678636][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0Possible unsafe loc= king scenario: [24438.678636][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.678692][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0CPU0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0CPU1 [24438.678754][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0----=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0---- [24438.678814][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0lock(batched_= entropy_u64.lock); [24438.678878][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0lock(&(&zone->lock)- >rlock); [24438.678951][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0lock(batched_entropy_u64.l ock); [24438.679038][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0=C2=A0lock(random_w= rite_wait.lock); [24438.679098][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.679098][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0*** DEADLOCK *** [24438.679098][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.679174][=C2=A0=C2=A0=C2=A0=C2=A0T2] 1 lock held by kthreadd/2: [24438.679230][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0=C2=A0#0: c000001ffd2f06e= 0 (batched_entropy_u64.lock){-...}, at: get_random_u64+0x60/0x100 [24438.679341][=C2=A0=C2=A0=C2=A0=C2=A0T2]=C2=A0 [24438.679341][=C2=A0=C2=A0=C2=A0=C2=A0T2] stack backtrace: [24438.679413][=C2=A0=C2=A0=C2=A0=C2=A0T2] CPU: 13 PID: 2 Comm: kthreadd = Not tainted 5.3.0-next- 20190924 #2 [24438.679485][=C2=A0=C2=A0=C2=A0=C2=A0T2] Call Trace: [24438.679507][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84efe0] [c000000000= 91a574] dump_stack+0xe8/0x164 (unreliable) [24438.679618][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f030] [c000000000= 1cc9b8] print_circular_bug+0x3a8/0x420 [24438.679701][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f0e0] [c000000000= 1ccc90] check_noncircular+0x260/0x320 [24438.679769][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f1e0] [c000000000= 1ce7e0] check_prev_add+0x100/0x11b0 [24438.679868][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f2c0] [c000000000= 1d00f8] validate_chain+0x868/0x1530 [24438.679950][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f3f0] [c000000000= 1d3064] __lock_acquire+0x5b4/0xbf0 [24438.680059][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f4f0] [c000000000= 1d3ed0] lock_acquire+0x130/0x360 [24438.680122][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f5d0] [c000000000= 947d70] _raw_spin_lock_irqsave+0x70/0xa0 [24438.680207][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f610] [c000000000= 1a9488] __wake_up_common_lock+0x88/0x110 [24438.680298][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f690] [c000000000= 6f11a4] account.constprop.8+0x284/0x430 [24438.680399][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f750] [c000000000= 6f1554] extract_entropy.constprop.7+0xd4/0x330 [24438.680495][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f7d0] [c000000000= 6f1818] crng_reseed+0x68/0x490 [24438.680590][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f910] [c000000000= 6f4094] _extract_crng+0x104/0x110 [24438.680662][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84f950] [c000000000= 6f1a34] crng_reseed+0x284/0x490 [24438.680751][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84fa90] [c000000000= 6f4094] _extract_crng+0x104/0x110 [24438.680828][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84fad0] [c000000000= 6f4c0c] get_random_u64+0xdc/0x100 [24438.680931][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84fb10] [c000000000= 106988] copy_process+0x2d8/0x1bf0 [24438.681007][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84fc30] [c000000000= 10861c] _do_fork+0xac/0xac0 [24438.681074][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84fd10] [c000000000= 1090d0] kernel_thread+0x70/0xa0 [24438.681170][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84fd80] [c000000000= 1518f0] kthreadd+0x270/0x330 [24438.681257][=C2=A0=C2=A0=C2=A0=C2=A0T2] [c00000002c84fe20] [c000000000= 00b748] ret_from_kernel_thread+0x5c/0x74 >=20 > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > index 63900ca029e0..ec1d72f18b34 100644 > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -6027,10 +6027,11 @@ void init_idle(struct task_struct *idle, int cp= u) > struct rq *rq =3D cpu_rq(cpu); > unsigned long flags; > =20 > + __sched_fork(0, idle); > + > raw_spin_lock_irqsave(&idle->pi_lock, flags); > raw_spin_lock(&rq->lock); > =20 > - __sched_fork(0, idle); > idle->state =3D TASK_RUNNING; > idle->se.exec_start =3D sched_clock(); > idle->flags |=3D PF_IDLE;