linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: john.p.donnelly@oracle.com
To: chenguanyou <chenguanyou9338@gmail.com>,
	longman@redhat.com, gregkh@linuxfoundation.org
Cc: dave@stgolabs.net, hdanton@sina.com,
	linux-kernel@vger.kernel.org, mazhenhua@xiaomi.com,
	mingo@redhat.com, peterz@infradead.org, quic_aiquny@quicinc.com,
	will@kernel.org, John Donnelly <john.p.donnelly@oracle.com>,
	"Peter Zijlstra (Intel)" <peterz@infradead.org>,
	sashal@kernel.org
Subject: Re: [PATCH v5] locking/rwsem: Make handoff bit handling more consistent
Date: Mon, 11 Apr 2022 13:26:22 -0500	[thread overview]
Message-ID: <3f02975c-1a9d-be20-32cf-f1d8e3dfafcc@oracle.com> (raw)
In-Reply-To: <20220214154741.12399-1-chenguanyou@xiaomi.com>

On 2/14/22 9:47 AM, chenguanyou wrote:
> Hi Waiman, Greg,
> This patch has been merged in branch linux-5.16.y.
> Can we take it to the linux-5.10.y LTS version?
> 
> thanks,

Hi,

As a FYI:

We have observed that following lockup with this commit added to 5.15.LTS:

d257cc8cb8d5 - locking/rwsem: Make handoff bit handling more consistent 
(4 months ago) <Waiman Long>

The "fio" test suit fails with LVM devices composed of four NVME devices 
with these observed lockup, panic.



ext4:

PID: 3682   TASK: ffff8f489ae34bc0  CPU: 2   COMMAND: "dio/dm-0"
  #0 [fffffe0000083e50] crash_nmi_callback at ffffffff828772b3
  #1 [fffffe0000083e58] nmi_handle at ffffffff82840778
  #2 [fffffe0000083ea0] default_do_nmi at ffffffff8337a1e2
  #3 [fffffe0000083ec8] exc_nmi at ffffffff8337a48d
  #4 [fffffe0000083ef0] end_repeat_nmi at ffffffff8340153b
     [exception RIP: _raw_spin_lock_irq+23]
     RIP: ffffffff8338b2e7  RSP: ffff9c4409b47c78  RFLAGS: 00000046
     RAX: 0000000000000000  RBX: ffff8f489ae34bc0  RCX: 0000000000000000
     RDX: 0000000000000001  RSI: 0000000000000000  RDI: ffff8f47f7b90104
     RBP: ffff9c4409b47d20   R8: 0000000000000000   R9: 0000000000000000
     R10: 0000000000000000  R11: 0000000000000000  R12: ffff8f47f7b90104
     R13: ffff9c4409b47cb0  R14: ffff8f47f7b900f0  R15: 0000000000000000
     ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
     <NMI exception stack> ---
  #5 [ffff9c4409b47c78] _raw_spin_lock_irq at ffffffff8338b2e7
  #6 [ffff9c4409b47c78] rwsem_down_write_slowpath at ffffffff82925be9
  #7 [ffff9c4409b47d28] ext4_map_blocks at ffffffffc11c26dc [ext4]
  #8 [ffff9c4409b47d98] ext4_convert_unwritten_extents at
ffffffffc11ad9e0 [ext4]
  #9 [ffff9c4409b47df0] ext4_dio_write_end_io at ffffffffc11b22aa [ext4]

xfs:

PID: 3719   TASK: ffff9f81d2d74bc0  CPU: 37  COMMAND: "dio/dm-0"
  #0 [fffffe0000894e50] crash_nmi_callback at ffffffffad6772b3
  #1 [fffffe0000894e58] nmi_handle at ffffffffad640778
  #2 [fffffe0000894ea0] default_do_nmi at ffffffffae17a1e2
  #3 [fffffe0000894ec8] exc_nmi at ffffffffae17a48d
  #4 [fffffe0000894ef0] end_repeat_nmi at ffffffffae20153b
     [exception RIP: _raw_spin_lock_irq+23]
     RIP: ffffffffae18b2e7  RSP: ffffbb7ec9637c48  RFLAGS: 00000046
     RAX: 0000000000000000  RBX: ffff9f81d2d74bc0  RCX: 0000000000000000
     RDX: 0000000000000001  RSI: 0000000000000000  RDI: ffff9f81c04a918c
     RBP: ffffbb7ec9637ce8   R8: 0000000000000000   R9: 0000000000000000
     R10: 0000000000000000  R11: 0000000000000000  R12: ffff9f81c04a918c
     R13: ffffbb7ec9637c80  R14: ffff9f81c04a9178  R15: 0000000000000000
     ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
     <NMI exception stack> ---
  #5 [ffffbb7ec9637c48] _raw_spin_lock_irq at ffffffffae18b2e7
  #6 [ffffbb7ec9637c48] rwsem_down_write_slowpath at ffffffffad725be9
  #7 [ffffbb7ec9637cf0] xfs_trans_alloc_inode at ffffffffc074f2bd [xfs]
  #8 [ffffbb7ec9637d50] xfs_iomap_write_unwritten at ffffffffc073ad15
[xfs]
  #9 [ffffbb7ec9637dd0] xfs_dio_write_end_io at ffffffffc072db62 [xfs]


I have reached out to Waiman and he suggested this for our next test pass:


1ee326196c6658 locking/rwsem: Always try to wake waiters in out_nolock path

-- 

Thank you.

  parent reply	other threads:[~2022-04-11 18:27 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-16  1:29 Waiman Long
2021-11-16  2:52 ` Aiqun(Maria) Yu
2021-11-16  9:14   ` Peter Zijlstra
2021-11-16  9:24     ` Peter Zijlstra
2021-11-16 14:52       ` Waiman Long
2021-11-17 13:36 ` Peter Zijlstra
2021-11-23  8:53 ` [tip: locking/urgent] " tip-bot2 for Waiman Long
2022-02-14 15:47 ` Re:[PATCH v5] " chenguanyou
2022-02-14 16:01   ` [PATCH " Greg KH
2022-04-11 18:26   ` john.p.donnelly [this message]
2022-04-11 18:40     ` Waiman Long
2022-04-11 21:03       ` john.p.donnelly
2022-04-11 21:07         ` Waiman Long
2022-04-12 16:28           ` john.p.donnelly
2022-04-12 17:04             ` Waiman Long
2022-04-14 10:48               ` Greg KH
2022-04-14 15:18                 ` Waiman Long
2022-04-14 15:42                   ` Greg KH
2022-04-14 15:44                     ` Waiman Long
2022-04-20 13:55             ` john.p.donnelly
2022-04-26 20:21               ` Waiman Long
2022-04-26 21:22                 ` john.p.donnelly
2022-02-14 16:22 ` chenguanyou
2022-02-15  7:41   ` [PATCH " Greg KH
2022-02-16 16:30     ` Waiman Long
2022-02-17 15:41       ` chenguanyou
2022-03-14  8:07         ` [PATCH " Greg KH
2022-03-22  2:49           ` chenguanyou
2022-03-24 12:51             ` [PATCH " Greg KH
2022-07-19  0:27 ` Doug Anderson
     [not found]   ` <20220719104104.1634-1-hdanton@sina.com>
2022-07-19 15:30     ` Doug Anderson
     [not found]       ` <20220722115510.2101-1-hdanton@sina.com>
2022-07-22 14:02         ` Doug Anderson
     [not found]           ` <20220723001713.2156-1-hdanton@sina.com>
2022-08-05 17:14             ` Doug Anderson
2022-08-05 19:02               ` Waiman Long
2022-08-05 19:16                 ` Doug Anderson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3f02975c-1a9d-be20-32cf-f1d8e3dfafcc@oracle.com \
    --to=john.p.donnelly@oracle.com \
    --cc=chenguanyou9338@gmail.com \
    --cc=dave@stgolabs.net \
    --cc=gregkh@linuxfoundation.org \
    --cc=hdanton@sina.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=longman@redhat.com \
    --cc=mazhenhua@xiaomi.com \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=quic_aiquny@quicinc.com \
    --cc=sashal@kernel.org \
    --cc=will@kernel.org \
    --subject='Re: [PATCH v5] locking/rwsem: Make handoff bit handling more consistent' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).