linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Byungchul Park <byungchul.park@lge.com>
To: torvalds@linux-foundation.org
Cc: damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org,
	adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org,
	mingo@redhat.com, linux-kernel@vger.kernel.org,
	peterz@infradead.org, will@kernel.org, tglx@linutronix.de,
	rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org,
	daniel.vetter@ffwll.ch, duyuyang@gmail.com,
	johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu,
	willy@infradead.org, david@fromorbit.com, amir73il@gmail.com,
	gregkh@linuxfoundation.org, kernel-team@lge.com,
	linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org,
	minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com,
	sj@kernel.org, jglisse@redhat.com, dennis@kernel.org,
	cl@linux.com, penberg@kernel.org, rientjes@google.com,
	vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org,
	paolo.valente@linaro.org, josef@toxicpanda.com,
	linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk,
	jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com,
	hch@infradead.org, djwong@kernel.org,
	dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com,
	melissa.srw@gmail.com, hamohammed.sa@gmail.com,
	42.hyeyoo@gmail.com, chris.p.wilson@intel.com,
	gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com
Subject: Re: [PATCH RFC v7 00/23] DEPT(Dependency Tracker)
Date: Thu, 19 Jan 2023 09:58:27 +0900	[thread overview]
Message-ID: <1674089907-31690-1-git-send-email-byungchul.park@lge.com> (raw)
In-Reply-To: <CAHk-=whpkWbdeZE1zask8YPzVYevJU2xOXqOposBujxZsa2-tQ@mail.gmail.com>

Torvalds wrote:
> On Sun, Jan 8, 2023 at 7:33 PM Byungchul Park <byungchul.park@lge.com> wrote:
>>
>> I've been developing a tool for detecting deadlock possibilities by
>> tracking wait/event rather than lock(?) acquisition order to try to
>> cover all synchonization machanisms. It's done on v6.2-rc2.
> 
> Ugh. I hate how this adds random patterns like

I undertand what you mean.. But all the synchronization primitives
should let DEPT know the beginning and the end of each. However, I will
remove the 'if' statement that looks ugly from the next spin, and place
the pattern to a better place if possible.

> 	if (timeout == MAX_SCHEDULE_TIMEOUT)
> 		sdt_might_sleep_strong(NULL);
> 	else
> 		sdt_might_sleep_strong_timeout(NULL);
> 	...
> 	sdt_might_sleep_finish();
> 
> to various places, it seems so very odd and unmaintainable.
> 
> I also recall this giving a fair amount of false positives, are they all fixed?

Yes. Of course I removed all the false positives we found.

> Anyway, I'd really like the lockdep people to comment and be involved.
> We did have a fairly recent case of "lockdep doesn't track page lock
> dependencies because it fundamentally cannot" issue, so DEPT might fix
> those kinds of missing dependency analysis. See

Sure. That's exactly what DEPT works for e.g. PG_locked.

> 	https://lore.kernel.org/lkml/00000000000060d41f05f139aa44@google.com/

I will reproduce it and share the result.

> for some context to that one, but at teh same time I would *really*
> want the lockdep people more involved and acking this work.
> 
> Maybe I missed the email where you reported on things DEPT has found
> (and on the lack of false positives)?

Maybe you didn't miss. It's still too hard to make a decision between:

	Aggressive detection with false alarms that need to be fixed by
	manual classification as Lockdep did, focusing on potential
	possibility more.

	versus

	Conservative detection with few false alarms, which requires us
	to test much longer to get result we expect, focusing on actual
	happening.

> 
> 	Linus

	Byungchul

  parent reply	other threads:[~2023-01-19  0:58 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-09  3:33 [PATCH RFC v7 00/23] DEPT(Dependency Tracker) Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 01/23] llist: Move llist_{head,node} definition to types.h Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 02/23] dept: Implement Dept(Dependency Tracker) Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 03/23] dept: Add single event dependency tracker APIs Byungchul Park
2023-01-18 13:01   ` Thomas Gleixner
2023-01-09  3:33 ` [PATCH RFC v7 04/23] dept: Add lock " Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 05/23] dept: Tie to Lockdep and IRQ tracing Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 06/23] dept: Add proc knobs to show stats and dependency graph Byungchul Park
2023-01-18 12:56   ` Thomas Gleixner
2023-01-09  3:33 ` [PATCH RFC v7 07/23] dept: Apply sdt_might_sleep_strong() to wait_for_completion()/complete() Byungchul Park
2023-01-18 12:59   ` Thomas Gleixner
2023-01-09  3:33 ` [PATCH RFC v7 08/23] dept: Apply sdt_might_sleep_strong() to PG_{locked,writeback} wait Byungchul Park
2023-01-09  9:10   ` Sergey Shtylyov
2023-01-09  3:33 ` [PATCH RFC v7 09/23] dept: Apply sdt_might_sleep_weak() to swait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 10/23] dept: Apply sdt_might_sleep_weak() to waitqueue wait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 11/23] dept: Apply sdt_might_sleep_weak() to hashed-waitqueue wait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 12/23] dept: Distinguish each syscall context from another Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 13/23] dept: Distinguish each work " Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 14/23] dept: Add a mechanism to refill the internal memory pools on running out Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 15/23] locking/lockdep, cpu/hotplus: Use a weaker annotation in AP thread Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 16/23] dept: Apply sdt_might_sleep_strong() to dma fence wait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 17/23] dept: Track timeout waits separately with a new Kconfig Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 18/23] dept: Apply timeout consideration to wait_for_completion()/complete() Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 19/23] dept: Apply timeout consideration to swait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 20/23] dept: Apply timeout consideration to waitqueue wait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 21/23] dept: Apply timeout consideration to hashed-waitqueue wait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 22/23] dept: Apply timeout consideration to dma fence wait Byungchul Park
2023-01-09  3:33 ` [PATCH RFC v7 23/23] dept: Record the latest one out of consecutive waits of the same class Byungchul Park
2023-01-16 18:00 ` [PATCH RFC v7 00/23] DEPT(Dependency Tracker) Linus Torvalds
2023-01-17 18:18   ` Boqun Feng
2023-01-17 18:40     ` Waiman Long
2023-01-18 12:55     ` Thomas Gleixner
2023-01-19  9:05       ` Byungchul Park
2023-01-19  6:23     ` Byungchul Park
2023-01-19  7:06       ` Byungchul Park
2023-01-19 13:33       ` Matthew Wilcox
2023-01-19 19:25         ` Boqun Feng
2023-01-20  1:51           ` Byungchul Park
2023-01-20  2:23             ` Boqun Feng
2023-01-20  3:07               ` Boqun Feng
2023-01-20  3:26                 ` Boqun Feng
2023-01-21  3:28                 ` Byungchul Park
2023-01-21  3:44                   ` Boqun Feng
2023-01-21  4:01                     ` Boqun Feng
2023-01-21  4:47                     ` Byungchul Park
2023-01-19  0:58   ` Byungchul Park [this message]
2023-01-21  2:40     ` Byungchul Park

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1674089907-31690-1-git-send-email-byungchul.park@lge.com \
    --to=byungchul.park@lge.com \
    --cc=42.hyeyoo@gmail.com \
    --cc=adilger.kernel@dilger.ca \
    --cc=akpm@linux-foundation.org \
    --cc=amir73il@gmail.com \
    --cc=chris.p.wilson@intel.com \
    --cc=cl@linux.com \
    --cc=damien.lemoal@opensource.wdc.com \
    --cc=dan.j.williams@intel.com \
    --cc=daniel.vetter@ffwll.ch \
    --cc=david@fromorbit.com \
    --cc=dennis@kernel.org \
    --cc=djwong@kernel.org \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=duyuyang@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=gwan-gyeong.mun@intel.com \
    --cc=hamohammed.sa@gmail.com \
    --cc=hannes@cmpxchg.org \
    --cc=hch@infradead.org \
    --cc=jack@suse.cz \
    --cc=jglisse@redhat.com \
    --cc=jlayton@kernel.org \
    --cc=joel@joelfernandes.org \
    --cc=johannes.berg@intel.com \
    --cc=josef@toxicpanda.com \
    --cc=kernel-team@lge.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-ide@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=max.byungchul.park@gmail.com \
    --cc=melissa.srw@gmail.com \
    --cc=mhocko@kernel.org \
    --cc=minchan@kernel.org \
    --cc=mingo@redhat.com \
    --cc=ngupta@vflare.org \
    --cc=paolo.valente@linaro.org \
    --cc=penberg@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rientjes@google.com \
    --cc=rodrigosiqueiramelo@gmail.com \
    --cc=rostedt@goodmis.org \
    --cc=sashal@kernel.org \
    --cc=sj@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=tytso@mit.edu \
    --cc=vbabka@suse.cz \
    --cc=vdavydov.dev@gmail.com \
    --cc=viro@zeniv.linux.org.uk \
    --cc=will@kernel.org \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).