From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8088C43457 for ; Thu, 15 Oct 2020 02:44:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 361752222E for ; Thu, 15 Oct 2020 02:44:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="WXWAc3ci" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 361752222E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 621076B0062; Wed, 14 Oct 2020 22:44:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A9E3900002; Wed, 14 Oct 2020 22:44:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4721A6B006E; Wed, 14 Oct 2020 22:44:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id 180956B0062 for ; Wed, 14 Oct 2020 22:44:37 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4F2961EF1 for ; Thu, 15 Oct 2020 02:44:36 +0000 (UTC) X-FDA: 77372616552.16.twist81_3c0dbd927210 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 29C3D1020813E for ; Thu, 15 Oct 2020 02:44:36 +0000 (UTC) X-HE-Tag: twist81_3c0dbd927210 X-Filterd-Recvd-Size: 5158 Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Thu, 15 Oct 2020 02:44:35 +0000 (UTC) Received: by mail-lj1-f195.google.com with SMTP id h20so1530568lji.9 for ; Wed, 14 Oct 2020 19:44:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ID2miLyRzca1e47IO9+77v5sqIUwWiNgIYab8O8WsPw=; b=WXWAc3ciMucmkyvNWAoGrn4REmOCOUzeaaHQMmYOa1MWzBMnoa+yAQgkwNPhcw/vv9 yb9KyKHZzvM3KsB71XIkWDKcpf+2jun/9jMfkxRGmXRW10JzITI7XsAuNVnRu3JW2O2b 0Q6/cvSRWnt/FFcRqvGqAAag9lu0+Q8mHQ2ZM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ID2miLyRzca1e47IO9+77v5sqIUwWiNgIYab8O8WsPw=; b=FVJQrDbWr61jEn7W9bIUKr7olz7VeHKiIvUVO633lbLkY+G4RuGbbSmDp00jfBUKSd RXwBX1c/rvZfisdiHdMv8d3kSPb6VhbulwcwATO9KNjlLM9Wl7Z47OZKg7bsjkfWJnoX FJ3X8AlQI0w1yoUY1iXp2iviwkB6ChNHXEQRug16+1dggAiE5gGJK/+RdUsUx/GO/3J9 dXf5dfw8cwqHl8x027KVAh6WDiXpZylFpNqvh1F0nop2TXvLpoZ28Sp0ulWfDGiKdA9E DvBXn/MKg/xpi1IbUPgmu09VcvV4oxFoL6wrvg8QFdBZWUsqhwsSl4m2b33gPc7iec94 Od1g== X-Gm-Message-State: AOAM5331EbjEhRJHOep8nuhu4Wkun8PXdWH6J2k+Q/Y9K8EqTG6SBTrw +Y/TxlZ7+Hmxmt6FkDM0h/ZjMHNkEIcg4w== X-Google-Smtp-Source: ABdhPJwUM0tygejtPHtxtpRos3jE2v66maM75AUMe7m6CxidjmyeRXdQVztq3BtV51r7eXlODJ+13w== X-Received: by 2002:a2e:958f:: with SMTP id w15mr407150ljh.449.1602729873550; Wed, 14 Oct 2020 19:44:33 -0700 (PDT) Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com. [209.85.167.52]) by smtp.gmail.com with ESMTPSA id x19sm452077lff.189.2020.10.14.19.44.32 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 14 Oct 2020 19:44:32 -0700 (PDT) Received: by mail-lf1-f52.google.com with SMTP id l2so1803169lfk.0 for ; Wed, 14 Oct 2020 19:44:32 -0700 (PDT) X-Received: by 2002:ac2:5f48:: with SMTP id 8mr333222lfz.344.1602729871973; Wed, 14 Oct 2020 19:44:31 -0700 (PDT) MIME-Version: 1.0 References: <4794a3fa3742a5e84fb0f934944204b55730829b.camel@lca.pw> In-Reply-To: <4794a3fa3742a5e84fb0f934944204b55730829b.camel@lca.pw> From: Linus Torvalds Date: Wed, 14 Oct 2020 19:44:16 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 0/4] Some more lock_page work.. To: Qian Cai Cc: Hugh Dickins , Matthew Wilcox , "Kirill A . Shutemov" , Linux-MM , Andrew Morton , linux-fsdevel , Amir Goldstein , Vivek Goyal Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Oct 14, 2020 at 6:48 PM Qian Cai wrote: > > While on this topic, I just want to bring up a bug report that we are chasing an > issue that a process is stuck in the loop of wait_on_page_bit_common() for more > than 10 minutes before I gave up. Judging by call trace, that looks like a deadlock rather than a missed wakeup. The trace isn't reliable, but I find it suspicious that the call trace just before the fault contains that "iov_iter_copy_from_user_atomic()". IOW, I think you're in fuse_fill_write_pages(), which has allocated the page, locked it, and then it takes a page fault. And the page fault waits on a page that is locked. This is a classic deadlock. The *intent* is that iov_iter_copy_from_user_atomic() returns zero, and you retry without the page lock held. HOWEVER. That's not what fuse actually does. Fuse will do multiple pages, and it will unlock only the _last_ page. It keeps the other pages locked, and puts them in an array: ap->pages[ap->num_pages] = page; And after the iov_iter_copy_from_user_atomic() fails, it does that "unlock" and repeat. But while the _last_ page was unlocked, the *previous* pages are still locked in that array. Deadlock. I really don't think this has anything at all to do with page locking, and everything to do with fuse_fill_write_pages() having a deadlock if the source of data is a mmap of one of the pages it is trying to write to (just with an offset, so that it's not the last page). See a similar code sequence in generic_perform_write(), but notice how that code only has *one* page that it locks, and never holds an array of pages around over that iov_iter_fault_in_readable() thing. Linus