Re: [PATCH] kmemleak: don't use __GFP_NOFAIL

[Michal Hocko <mhocko@suse.com>]

On Thu 31-05-18 06:51:22, Chunyu Hu wrote:
> 
> 
> ----- Original Message -----
> > From: "Michal Hocko" <mhocko@suse.com>
> > To: "Chunyu Hu" <chuhu@redhat.com>
> > Cc: "Tetsuo Handa" <penguin-kernel@i-love.sakura.ne.jp>, malat@debian.org, dvyukov@google.com, linux-mm@kvack.org,
> > "catalin marinas" <catalin.marinas@arm.com>, "Akinobu Mita" <akinobu.mita@gmail.com>
> > Sent: Wednesday, May 30, 2018 8:38:26 PM
> > Subject: Re: [PATCH] kmemleak: don't use __GFP_NOFAIL
> > 
> > On Wed 30-05-18 07:42:59, Chunyu Hu wrote:
> > > 
> > > ----- Original Message -----
> > > > From: "Michal Hocko" <mhocko@suse.com>
> > > > To: "Chunyu Hu" <chuhu@redhat.com>
> > > > Cc: "Tetsuo Handa" <penguin-kernel@i-love.sakura.ne.jp>,
> > > > malat@debian.org, dvyukov@google.com, linux-mm@kvack.org,
> > > > "catalin marinas" <catalin.marinas@arm.com>
> > > > Sent: Wednesday, May 30, 2018 6:46:37 PM
> > > > Subject: Re: [PATCH] kmemleak: don't use __GFP_NOFAIL
> > > > 
> > > > On Wed 30-05-18 05:35:37, Chunyu Hu wrote:
> > > > [...]
> > > > > I'm trying to reuse the make_it_fail field in task for fault injection.
> > > > > As
> > > > > adding
> > > > > an extra memory alloc flag is not thought so good,  I think adding task
> > > > > flag
> > > > > is either?
> > > > 
> > > > Yeah, task flag will be reduced to KMEMLEAK enabled configurations
> > > > without an additional maint. overhead. Anyway, you should really think
> > > > about how to guarantee trackability for atomic allocation requests. You
> > > > cannot simply assume that GFP_NOWAIT will succeed. I guess you really
> > > 
> > > Sure. While I'm using task->make_it_fail, I'm still in the direction of
> > > making kmemleak avoid fault inject with task flag instead of page alloc
> > > flag.
> > > 
> > > > want to have a pre-populated pool of objects for those requests. The
> > > > obvious question is how to balance such a pool. It ain't easy to track
> > > > memory by allocating more memory...
> > > 
> > > This solution is going to make kmemleak trace really nofail. We can think
> > > later.
> > > 
> > > while I'm thinking about if fault inject can be disabled via flag in task.
> > > 
> > > Actually, I'm doing something like below, the disable_fault_inject() is
> > > just setting a flag in task->make_it_fail. But this will depend on if
> > > fault injection accept a change like this. CCing Akinobu
> > 
> > You still seem to be missing my point I am afraid (or I am ;). So say
> > that you want to track a GFP_NOWAIT allocation request. So create_object
> > will get called with that gfp mask and no matter what you try here your
> > tracking object will be allocated in a weak allocation context as well
> > and disable kmemleak. So it only takes a more heavy memory pressure and
> > the tracing is gone...
> 
> Michal,
> 
> Thank you for the good suggestion. You mean GFP_NOWAIT still can make create_object
> fail and as a result kmemleak disable itself. So it's not so useful, just like
> the current __GFP_NOFAIL usage in create_object. 
> 
> In the first thread, we discussed this. and that time you suggested we have 
> fault injection disabled when kmemleak is working and suggested per task way.
> so my head has been stuck in that point. While now you gave a better suggestion
> that why not we pre allocate a urgent pool for kmemleak objects. After thinking
> for a while, I got  your point, it's a good way for improving kmemleak to make
> it can tolerate light allocation failure. And catalin mentioned that we have
> one option that use the early_log array as urgent pool, which has the similar
> ideology.
> 
> Basing on your suggestions, I tried to draft this, what does it look to you? 
> another strong alloc mask and an extra thread for fill the pool, which containts
> 1M objects in a frequency of 100 ms. If first kmem_cache_alloc failed, then
> get a object from the pool. 

I am not really familiar with kmemleak code base to judge the
implementation. Could you be more specific about the highlevel design
please? Who is the producer and how does it sync with consumers?
-- 
Michal Hocko
SUSE Labs