How to detect slab memory leak

How to detect slab memory leak

[Naruto Nguyen]

Hi everyone,

When using slabtop to display the cache size growing, I see that
kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour.
Is there any way to detect which kernel components cause this slab
increase?

Thanks a lot,
Brs,
Naruto

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to detect slab memory leak

[Mulyadi Santosa]

[-- Attachment #1.1: Type: text/plain, Size: 855 bytes --]

On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <narutonguyen2018@gmail.com>
wrote:

> Hi everyone,
>
> When using slabtop to display the cache size growing, I see that
> kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour.
> Is there any way to detect which kernel components cause this slab
> increase?
>
> Thanks a lot,
> Brs,
> Naruto
>
>
Hello Naruto

I think you can do it by instrumenting call to kmalloc(). These days, eBPF
is the hot stuff to try for this matter. See if it fits yours too.

regards,

Mulyadi


> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

[-- Attachment #1.2: Type: text/html, Size: 1827 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to detect slab memory leak

[Harsh Jain]

You can try kernel config option "CONFIG_DEBUG_KMEMLEAK".


https://www.kernel.org/doc/html/v4.10/dev-tools/kmemleak.html

On Tue, Dec 18, 2018 at 1:14 PM Mulyadi Santosa
<mulyadi.santosa@gmail.com> wrote:
>
>
>
> On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <narutonguyen2018@gmail.com> wrote:
>>
>> Hi everyone,
>>
>> When using slabtop to display the cache size growing, I see that
>> kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour.
>> Is there any way to detect which kernel components cause this slab
>> increase?
>>
>> Thanks a lot,
>> Brs,
>> Naruto
>>
>
> Hello Naruto
>
> I think you can do it by instrumenting call to kmalloc(). These days, eBPF is the hot stuff to try for this matter. See if it fits yours too.
>
> regards,
>
> Mulyadi
>
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies@kernelnewbies.org
>> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
>
> --
> regards,
>
> Mulyadi Santosa
> Freelance Linux trainer and consultant
>
> blog: the-hydra.blogspot.com
> training: mulyaditraining.blogspot.com
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to detect slab memory leak

[Naruto Nguyen]

Hi Harsh and Mulyadi,

Thanks a lot  for your reply.

Just another question, in the slabtop output, I see that we have
kmalloc-64 and kmalloc-96 cache, when these caches increase their
size, is it calculated in the "cache" of the "free" command or in
"used". Is it reclaimable? and count in SReclaimable or SUnreclaim of
cat /proc/meminfo? I know some cache like dentry or inode cache will
be calculated in "cache" of free command output and SReclaimable of
cat /proc/meminfo and can be freed by "echo 3 >
/proc/sys/vm/drop_caches" but not sure if it the same situation for
kmalloc-64 and kmalloc-96 cache.

Thanks again,
Brs,
Bao

On Tue, 18 Dec 2018 at 15:31, Harsh Jain <harshjain.prof@gmail.com> wrote:
>
> You can try kernel config option "CONFIG_DEBUG_KMEMLEAK".
>
>
> https://www.kernel.org/doc/html/v4.10/dev-tools/kmemleak.html
>
> On Tue, Dec 18, 2018 at 1:14 PM Mulyadi Santosa
> <mulyadi.santosa@gmail.com> wrote:
> >
> >
> >
> > On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <narutonguyen2018@gmail.com> wrote:
> >>
> >> Hi everyone,
> >>
> >> When using slabtop to display the cache size growing, I see that
> >> kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour.
> >> Is there any way to detect which kernel components cause this slab
> >> increase?
> >>
> >> Thanks a lot,
> >> Brs,
> >> Naruto
> >>
> >
> > Hello Naruto
> >
> > I think you can do it by instrumenting call to kmalloc(). These days, eBPF is the hot stuff to try for this matter. See if it fits yours too.
> >
> > regards,
> >
> > Mulyadi
> >
> >>
> >> _______________________________________________
> >> Kernelnewbies mailing list
> >> Kernelnewbies@kernelnewbies.org
> >> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >
> >
> >
> > --
> > regards,
> >
> > Mulyadi Santosa
> > Freelance Linux trainer and consultant
> >
> > blog: the-hydra.blogspot.com
> > training: mulyaditraining.blogspot.com
> > _______________________________________________
> > Kernelnewbies mailing list
> > Kernelnewbies@kernelnewbies.org
> > https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to detect slab memory leak

[Mulyadi Santosa]

[-- Attachment #1.1: Type: text/plain, Size: 3015 bytes --]

On Wed, Dec 19, 2018 at 6:58 PM Naruto Nguyen <narutonguyen2018@gmail.com>
wrote:

> Hi Harsh and Mulyadi,
>
> Thanks a lot  for your reply.
>
> Just another question, in the slabtop output, I see that we have
> kmalloc-64 and kmalloc-96 cache, when these caches increase their
> size, is it calculated in the "cache" of the "free" command or in
> "used".


If the cache is used for page cache related operations, then AFAIK it is
calculated in cache. If not, say for mmap non file backed operation, then
it is calculated in overall used.


> Is it reclaimable?


AFAIK yes, as long as it is not locked after being allocated.


> and count in SReclaimable or SUnreclaim of
> cat /proc/meminfo? I know some cache like dentry or inode cache will
> be calculated in "cache" of free command output and SReclaimable of
> cat /proc/meminfo and can be freed by "echo 3 >
> /proc/sys/vm/drop_caches" but not sure if it the same situation for
> kmalloc-64 and kmalloc-96 cache.
>
> Thanks again,
> Brs,
> Bao
>
> On Tue, 18 Dec 2018 at 15:31, Harsh Jain <harshjain.prof@gmail.com> wrote:
> >
> > You can try kernel config option "CONFIG_DEBUG_KMEMLEAK".
> >
> >
> > https://www.kernel.org/doc/html/v4.10/dev-tools/kmemleak.html
> >
> > On Tue, Dec 18, 2018 at 1:14 PM Mulyadi Santosa
> > <mulyadi.santosa@gmail.com> wrote:
> > >
> > >
> > >
> > > On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <
> narutonguyen2018@gmail.com> wrote:
> > >>
> > >> Hi everyone,
> > >>
> > >> When using slabtop to display the cache size growing, I see that
> > >> kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour.
> > >> Is there any way to detect which kernel components cause this slab
> > >> increase?
> > >>
> > >> Thanks a lot,
> > >> Brs,
> > >> Naruto
> > >>
> > >
> > > Hello Naruto
> > >
> > > I think you can do it by instrumenting call to kmalloc(). These days,
> eBPF is the hot stuff to try for this matter. See if it fits yours too.
> > >
> > > regards,
> > >
> > > Mulyadi
> > >
> > >>
> > >> _______________________________________________
> > >> Kernelnewbies mailing list
> > >> Kernelnewbies@kernelnewbies.org
> > >> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> > >
> > >
> > >
> > > --
> > > regards,
> > >
> > > Mulyadi Santosa
> > > Freelance Linux trainer and consultant
> > >
> > > blog: the-hydra.blogspot.com
> > > training: mulyaditraining.blogspot.com
> > > _______________________________________________
> > > Kernelnewbies mailing list
> > > Kernelnewbies@kernelnewbies.org
> > > https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

[-- Attachment #1.2: Type: text/html, Size: 5941 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies