From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_ADSP_ALL, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 664F9C63697 for ; Sat, 28 Nov 2020 14:09:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C856924684 for ; Sat, 28 Nov 2020 14:09:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=natalenko.name header.i=@natalenko.name header.b="sqowTDwT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C856924684 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=natalenko.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3128D6B0068; Sat, 28 Nov 2020 09:09:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C27F6B006C; Sat, 28 Nov 2020 09:09:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B0E56B006E; Sat, 28 Nov 2020 09:09:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id 0485B6B0068 for ; Sat, 28 Nov 2020 09:09:29 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C07E4181AEF31 for ; Sat, 28 Nov 2020 14:09:28 +0000 (UTC) X-FDA: 77534009616.24.frame54_4e1237127391 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id A93FE1A4A0 for ; Sat, 28 Nov 2020 14:09:28 +0000 (UTC) X-HE-Tag: frame54_4e1237127391 X-Filterd-Recvd-Size: 4999 Received: from vulcan.natalenko.name (vulcan.natalenko.name [104.207.131.136]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Sat, 28 Nov 2020 14:09:27 +0000 (UTC) Received: from localhost (home.natalenko.name [151.237.229.131]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by vulcan.natalenko.name (Postfix) with ESMTPSA id 53C758B184D; Sat, 28 Nov 2020 15:09:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=natalenko.name; s=dkim-20170712; t=1606572564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=80LrCKLmbaN4QFAd+vAXkfzcVSLmNqUxi0hyFB1oaKE=; b=sqowTDwTj1W5XbWiqI0623pVVQO38XYLOBRIRNl+gdmzR8jFUrNYeWQWVm54Q+/eAwhGnl 5a0Yx0jJ4a5lPrhepGYlGvSmITmWYokq661Wy5yPYSHTmuHS1Pbxqdgy3pS/x9C0eGZFMm pe8RxWxCp5ZqWNgwXgE5NFEcuQ+H7fY= Date: Sat, 28 Nov 2020 15:09:24 +0100 From: Oleksandr Natalenko To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andrew Morton , Sebastian Andrzej Siewior , Steven Rostedt , Mike Galbraith , Thomas Gleixner , linux-rt-users@vger.kernel.org Subject: Re: scheduling while atomic in z3fold Message-ID: <20201128140924.iyqr2h52z2olt6zb@spock.localdomain> References: <20201128140523.ovmqon5fjetvpby4@spock.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20201128140523.ovmqon5fjetvpby4@spock.localdomain> Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Nov 28, 2020 at 03:05:24PM +0100, Oleksandr Natalenko wrote: > Hi. >=20 > While running v5.10-rc5-rt11 I bumped into the following: >=20 > ``` > BUG: scheduling while atomic: git/18695/0x00000002 > Preemption disabled at: > [] z3fold_zpool_malloc+0x463/0x6e0 > =E2=80=A6 > Call Trace: > dump_stack+0x6d/0x88 > __schedule_bug.cold+0x88/0x96 > __schedule+0x69e/0x8c0 > preempt_schedule_lock+0x51/0x150 > rt_spin_lock_slowlock_locked+0x117/0x2c0 > rt_spin_lock_slowlock+0x58/0x80 > rt_spin_lock+0x2a/0x40 > z3fold_zpool_malloc+0x4c1/0x6e0 > zswap_frontswap_store+0x39c/0x980 > __frontswap_store+0x6e/0xf0 > swap_writepage+0x39/0x70 > shmem_writepage+0x31b/0x490 > pageout+0xf4/0x350 > shrink_page_list+0xa28/0xcc0 > shrink_inactive_list+0x300/0x690 > shrink_lruvec+0x59a/0x770 > shrink_node+0x2d6/0x8d0 > do_try_to_free_pages+0xda/0x530 > try_to_free_pages+0xff/0x260 > __alloc_pages_slowpath.constprop.0+0x3d5/0x1230 > __alloc_pages_nodemask+0x2f6/0x350 > allocate_slab+0x3da/0x660 > ___slab_alloc+0x4ff/0x760 > __slab_alloc.constprop.0+0x7a/0x100 > kmem_cache_alloc+0x27b/0x2c0 > __d_alloc+0x22/0x230 > d_alloc_parallel+0x67/0x5e0 > __lookup_slow+0x5c/0x150 > path_lookupat+0x2ea/0x4d0 > filename_lookup+0xbf/0x210 > vfs_statx.constprop.0+0x4d/0x110 > __do_sys_newlstat+0x3d/0x80 > do_syscall_64+0x33/0x40 > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > ``` >=20 > The preemption seems to be disabled here: >=20 > ``` > $ scripts/faddr2line mm/z3fold.o z3fold_zpool_malloc+0x463 > z3fold_zpool_malloc+0x463/0x6e0: > add_to_unbuddied at mm/z3fold.c:645 > (inlined by) z3fold_alloc at mm/z3fold.c:1195 > (inlined by) z3fold_zpool_malloc at mm/z3fold.c:1737 > ``` >=20 > The call to the rt_spin_lock() seems to be here: >=20 > ``` > $ scripts/faddr2line mm/z3fold.o z3fold_zpool_malloc+0x4c1 > z3fold_zpool_malloc+0x4c1/0x6e0: > add_to_unbuddied at mm/z3fold.c:649 > (inlined by) z3fold_alloc at mm/z3fold.c:1195 > (inlined by) z3fold_zpool_malloc at mm/z3fold.c:1737 > ``` >=20 > Or, in source code: >=20 > ``` > 639 /* Add to the appropriate unbuddied list */ > 640 static inline void add_to_unbuddied(struct z3fold_pool *pool, > 641 struct z3fold_header *zhdr) > 642 { > 643 if (zhdr->first_chunks =3D=3D 0 || zhdr->last_chunks =3D=3D 0 = || > 644 zhdr->middle_chunks =3D=3D 0) { > 645 struct list_head *unbuddied =3D get_cpu_ptr(pool->unbuddie= d); > 646 > 647 int freechunks =3D num_free_chunks(zhdr); > 648 spin_lock(&pool->lock); > 649 list_add(&zhdr->buddy, &unbuddied[freechunks]); > 650 spin_unlock(&pool->lock); > 651 zhdr->cpu =3D smp_processor_id(); > 652 put_cpu_ptr(pool->unbuddied); > 653 } > 654 } > ``` >=20 > Shouldn't the list manipulation be protected with > local_lock+this_cpu_ptr instead of get_cpu_ptr+spin_lock? >=20 > Thanks. >=20 > --=20 > Oleksandr Natalenko (post-factum) Forgot to Cc linux-rt-users@, sorry. --=20 Oleksandr Natalenko (post-factum)