From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 098BEC4361B for ; Mon, 7 Dec 2020 01:06:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5843C22CF6 for ; Mon, 7 Dec 2020 01:06:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5843C22CF6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 51F3B8D0003; Sun, 6 Dec 2020 20:06:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D0AE8D0001; Sun, 6 Dec 2020 20:06:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BFD98D0003; Sun, 6 Dec 2020 20:06:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 25A9A8D0001 for ; Sun, 6 Dec 2020 20:06:10 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CC7193628 for ; Mon, 7 Dec 2020 01:06:09 +0000 (UTC) X-FDA: 77564694858.18.pin72_480c080273da Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id A5872100ED3C2 for ; Mon, 7 Dec 2020 01:06:09 +0000 (UTC) X-HE-Tag: pin72_480c080273da X-Filterd-Recvd-Size: 9159 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Mon, 7 Dec 2020 01:06:08 +0000 (UTC) Received: by mail-lf1-f51.google.com with SMTP id z21so15622499lfe.12 for ; Sun, 06 Dec 2020 17:06:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=zFByNs1/8aZ5IpU6WuheQ0sr8SNLN084Q9h5NXJ0lQQ=; b=d6WJNYzOqqsAvcU1AHzqtt2gV8dydoUZp9jEqVK/wu13SyQpuiUoQeDZdjf3wRQrpA b9EPc2a8AloG4P4FnhfDBV/U3RSa3FDnt5gySQYQcOXF6gLBt3cxksEeP9Xkvrq6+gKS 8jIzLVvYnUdYEQWGjLY57jaoMsOavrBc9qniA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=zFByNs1/8aZ5IpU6WuheQ0sr8SNLN084Q9h5NXJ0lQQ=; b=GolNP8Y/tdqoY2xFG0k3gUlMH/ijo/WJqNplPmmdQB7P1ZG27VBDht0jA+cJ14p+gY uqDvIum4Fng432Isf9btqBi5dah6AgMVDG+rV7HH31OuBjwtV442Ljj11HW7p9oS0ERv WexglZhbmef7iBQdW1K5hPKOPJqcKI1SYQkZ7eUxYLdQhC/tyZtPKG0YeIdJg9Lcpqg5 pvdIcTr3UFsigawtfuna7l3qcmb5A5qvTCzHZoJ3cLDlSS8ro6dDEj1PluyB9Ow+RJqP bwPT3eMWS/bf4lpII2Er2zZ6piUZrlVpsuq3EJP+Go/fT2hm5nYeaT0irWDj57jqqS+W 7Rug== X-Gm-Message-State: AOAM530Jmg6JJ0QDNJk6HvIqmVsZzejBQYeHKnpWUhGln+dKERRAm4tT XL6Vmi0TBlryrq7pDeN2PAG61w== X-Google-Smtp-Source: ABdhPJyPs2ASFWAvhhuYLL/nafkzchiDwvN7uxYxldNghW3g4vUGwn4VN3b0EUuPSu6xT0l7z9SRtQ== X-Received: by 2002:ac2:4212:: with SMTP id y18mr3691636lfh.141.1607303167458; Sun, 06 Dec 2020 17:06:07 -0800 (PST) Received: from [192.168.10.101] (c83-250-124-157.bredband.comhem.se. [83.250.124.157]) by smtp.gmail.com with ESMTPSA id j27sm486237lfm.178.2020.12.06.17.06.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 06 Dec 2020 17:06:06 -0800 (PST) Subject: Re: scheduling while atomic in z3fold To: Mike Galbraith , Sebastian Andrzej Siewior Cc: Oleksandr Natalenko , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Steven Rostedt , Thomas Gleixner , linux-rt-users@vger.kernel.org References: <90c4857c53b657147bfb71a281ece9839b0373c2.camel@gmx.de> <20201130132014.mlvxeyiub3fpwyw7@linutronix.de> <856b5cc2a3d4eb673743b52956bf1e60dcdf87a1.camel@gmx.de> <20201130145229.mhbkrfuvyctniaxi@linutronix.de> <05121515e73891ceb9e5caf64b6111fc8ff43fab.camel@gmx.de> <20201130160327.ov32m4rapk4h432a@linutronix.de> <20201202220826.5chy56mbgvrwmg3d@linutronix.de> <64ab382309c41ca5c7a601fc3efbb6d2a6e68602.camel@gmx.de> <20201203133934.37aotbdjnd36lrxv@linutronix.de> <10d5088861ba219f3f7cd657fc95b0bedc19010a.camel@gmx.de> From: Vitaly Wool Message-ID: Date: Mon, 7 Dec 2020 02:05:59 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <10d5088861ba219f3f7cd657fc95b0bedc19010a.camel@gmx.de> Content-Type: multipart/alternative; boundary="------------3125DE5B8011B2852B3744DA" Content-Language: en-US X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a multi-part message in MIME format. --------------3125DE5B8011B2852B3744DA Content-Type: text/plain; charset=iso-8859-15; format=flowed Content-Transfer-Encoding: 7bit On 2020-12-06 10:18, Mike Galbraith wrote: > On Thu, 2020-12-03 at 14:39 +0100, Sebastian Andrzej Siewior wrote: >> On 2020-12-03 09:18:21 [+0100], Mike Galbraith wrote: >>> On Thu, 2020-12-03 at 03:16 +0100, Mike Galbraith wrote: >>>> On Wed, 2020-12-02 at 23:08 +0100, Sebastian Andrzej Siewior wrote: >>>> Looks like... d8f117abb380 z3fold: fix use-after-free when freeing >>>> handles ...wasn't completely effective... >>> The top two hunks seem to have rendered the thing RT tolerant. >> Yes, it appears to. I have no idea if this is a proper fix or not. >> Without your write lock, after a few attempts, KASAN says: | BUG: >> KASAN: use-after-free in __pv_queued_spin_lock_slowpath+0x293/0x770 | >> Write of size 2 at addr ffff88800e0e10aa by task kworker/u16:3/237 > Things that make ya go hmmm... I started poking at it thinking ok, > given write_lock() fixes it, bad juju must be happening under another > read_lock(), so I poisoned the structure about to be freed by > __release_z3fold_page() under lock, and opened a delay window for bad > juju to materialize, but it didn't, so I just poisoned instead, and > sprinkled landmines all over the place. My landmines are not > triggering but the detonator is materializing inside the structure > containing the lock we explode on. Somebody blasts a recycled > z3fold_buddy_slots into ram we were working on? Could you please try the following patch in your setup: diff --git a/mm/z3fold.c b/mm/z3fold.c index 18feaa0bc537..efe9a012643d 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -544,12 +544,17 @@ static void __release_z3fold_page(struct z3fold_header *zhdr, bool locked) break; } } - if (!is_free) + if (!is_free) { set_bit(HANDLES_ORPHANED, &zhdr->slots->pool); - read_unlock(&zhdr->slots->lock); - - if (is_free) + read_unlock(&zhdr->slots->lock); + } else { + zhdr->slots->slot[0] = + zhdr->slots->slot[1] = + zhdr->slots->slot[2] = + zhdr->slots->slot[3] = 0xdeadbeef; + read_unlock(&zhdr->slots->lock); kmem_cache_free(pool->c_handle, zhdr->slots); + } if (locked) z3fold_page_unlock(zhdr); --------------3125DE5B8011B2852B3744DA Content-Type: text/html; charset=iso-8859-15 Content-Transfer-Encoding: 7bit 
On 2020-12-06 10:18, Mike Galbraith wrote:

On Thu, 2020-12-03 at 14:39 +0100, Sebastian Andrzej Siewior wrote:

On 2020-12-03 09:18:21 [+0100], Mike Galbraith wrote:

On Thu, 2020-12-03 at 03:16 +0100, Mike Galbraith wrote:

On Wed, 2020-12-02 at 23:08 +0100, Sebastian Andrzej Siewior wrote:
Looks like...

d8f117abb380 z3fold: fix use-after-free when freeing handles

...wasn't completely effective...


The top two hunks seem to have rendered the thing RT tolerant.


Yes, it appears to. I have no idea if this is a proper fix or not.
Without your write lock, after a few attempts, KASAN says:

| BUG: KASAN: use-after-free in __pv_queued_spin_lock_slowpath+0x293/0x770
| Write of size 2 at addr ffff88800e0e10aa by task kworker/u16:3/237


Things that make ya go hmmm...

I started poking at it thinking ok, given write_lock() fixes it, bad
juju must be happening under another read_lock(), so I poisoned the
structure about to be freed by __release_z3fold_page() under lock, and
opened a delay window for bad juju to materialize, but it didn't, so I
just poisoned instead, and sprinkled landmines all over the place.

My landmines are not triggering but the detonator is materializing
inside the structure containing the lock we explode on.  Somebody
blasts a recycled z3fold_buddy_slots into ram we were working on?
Could you please try the following patch in your setup:
diff --git a/mm/z3fold.c b/mm/z3fold.c
index 18feaa0bc537..efe9a012643d 100644
--- a/mm/z3fold.c
+++ b/mm/z3fold.c
@@ -544,12 +544,17 @@ static void __release_z3fold_page(struct z3fold_header *zhdr, bool locked)
 			break;
 		}
 	}
-	if (!is_free)
+	if (!is_free) {
 		set_bit(HANDLES_ORPHANED, &zhdr->slots->pool);
-	read_unlock(&zhdr->slots->lock);
-
-	if (is_free)
+		read_unlock(&zhdr->slots->lock);
+	} else {
+		zhdr->slots->slot[0] =
+			zhdr->slots->slot[1] =
+			zhdr->slots->slot[2] =
+			zhdr->slots->slot[3] = 0xdeadbeef;
+		read_unlock(&zhdr->slots->lock);
 		kmem_cache_free(pool->c_handle, zhdr->slots);
+	}
 
 	if (locked)
 		z3fold_page_unlock(zhdr);
--------------3125DE5B8011B2852B3744DA--