[alsa-devel] change kmalloc into vmalloc for large memory allocations

[alsa-devel] change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi 


I find there is some drivers use kmalloc to allocate large 
Memorys during module_init,  some can be changed to use vmalloc
To save some low mem, I add log in kernel to track ,
And list them here:

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/usb/gadget/f_mass_storage.c?h=master#n2724

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_ftp.c?h=master#n603
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_h323_main.c?h=master#n1849
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_irc.c?h=master#n247
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_sane.c?h=master#n195

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/input/evdev.c?h=master#n403


they allocate large memory from 10k~64K , 
this will use lots of low mem, instead if we use 
vmalloc for these drivers , it will be good for some devices
like smart phone, we often encounter some errors like kmalloc failed 
because there is not enough low mem , especially when the device has physical 
memory less than 1GB .


could this module change the memory allocation into vmalloc ?

I was thinking that if we can introduce a helper function in kmalloc.h like this :

Kmalloc(size, flags)
{
	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
		return vmalloc(size);
	Else
		return real_kmalloc(size);
}

Kfree(ptr)
{
	If (is_vmalloc_addr(ptr))
		Vfree(ptr);
	Else
		Kfree(ptr);
}


But we need add some flags to ensure always use kmalloc for
Some special use (dma etc..)

How do you think of it ?

Thanks



_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: change kmalloc into vmalloc for large memory allocations

[Huang Shijie]

于 2014年02月28日 16:15, Wang, Yalin 写道:
> could this module change the memory allocation into vmalloc ?
the buffer allocated by vmalloc can not be used for the DMA.

I think that's why drivers use the kmalloc.

thanks
Huang Shijie


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi  


Yeah,  
Dma buffer must be allocated by kmalloc,

But the modules I list should can all be changed to use
vmalloc, because the buffer is only used by software,
Not by any hardware .


Thanks

-----Original Message-----
From: Huang Shijie [mailto:b32955@freescale.com] 
Sent: Friday, February 28, 2014 4:55 PM
To: Wang, Yalin
Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'tiwai@suse.de'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
Subject: Re: change kmalloc into vmalloc for large memory allocations

于 2014年02月28日 16:15, Wang, Yalin 写道:
> could this module change the memory allocation into vmalloc ?
the buffer allocated by vmalloc can not be used for the DMA.

I think that's why drivers use the kmalloc.

thanks
Huang Shijie

_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: change kmalloc into vmalloc for large memory allocations

[Steven Rostedt]

On Fri, 28 Feb 2014 16:15:23 +0800
"Wang, Yalin" <Yalin.Wang@sonymobile.com> wrote:

> Kfree(ptr)
> {
> 	If (is_vmalloc_addr(ptr))
> 		Vfree(ptr);
> 	Else
> 		Kfree(ptr);
> }
> 
> 
> But we need add some flags to ensure always use kmalloc for
> Some special use (dma etc..)
> 
> How do you think of it ?

But vmalloc also takes up tlb entries. The more vmalloc space you have,
the more tlb entries that will need to be used, which will have a
performance affect on the entire system.

I would be against making kmalloc() secretly doing a vmalloc. If it is
better for a driver to use a vmalloc on large items, than change the
driver. Don't change a core function.

-- Steve

Re: change kmalloc into vmalloc for large memory allocations

[Takashi Iwai]

At Fri, 28 Feb 2014 16:15:23 +0800,
Wang, Yalin wrote:
> 
> Hi 
> 
> 
> I find there is some drivers use kmalloc to allocate large 
> Memorys during module_init,  some can be changed to use vmalloc
> To save some low mem, I add log in kernel to track ,
> And list them here:
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/usb/gadget/f_mass_storage.c?h=master#n2724
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772

At least the ASoC runtime case doesn't use the allocated memory as
buffer, and they are allocated only once per device, thus it shouldn't
be the problem you stated.  If it really consumes so much memory, we
need to rethink, instead of allocating an array but allocate each
object, for example.


thanks,

Takashi


> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_ftp.c?h=master#n603
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_h323_main.c?h=master#n1849
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_irc.c?h=master#n247
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_sane.c?h=master#n195
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/input/evdev.c?h=master#n403
> 
> 
> they allocate large memory from 10k~64K , 
> this will use lots of low mem, instead if we use 
> vmalloc for these drivers , it will be good for some devices
> like smart phone, we often encounter some errors like kmalloc failed 
> because there is not enough low mem , especially when the device has physical 
> memory less than 1GB .
> 
> 
> could this module change the memory allocation into vmalloc ?
> 
> I was thinking that if we can introduce a helper function in kmalloc.h like this :
> 
> Kmalloc(size, flags)
> {
> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
> 		return vmalloc(size);
> 	Else
> 		return real_kmalloc(size);
> }
> 
> Kfree(ptr)
> {
> 	If (is_vmalloc_addr(ptr))
> 		Vfree(ptr);
> 	Else
> 		Kfree(ptr);
> }
> 
> 
> But we need add some flags to ensure always use kmalloc for
> Some special use (dma etc..)
> 
> How do you think of it ?
> 
> Thanks
> 
> 
> 

Re: change kmalloc into vmalloc for large memory allocations

['gregkh@linuxfoundation.org']

On Fri, Feb 28, 2014 at 05:20:08PM +0800, Wang, Yalin wrote:
> Hi  
> 
> 
> Yeah,  
> Dma buffer must be allocated by kmalloc,
> 
> But the modules I list should can all be changed to use
> vmalloc, because the buffer is only used by software,
> Not by any hardware .

Are you sure about that?  The USB gadget driver needs DMA memory from
what I can tell, have you tried your change out on a system that does
not allow the USB controller to access non-DMA memory?

And I agree with Steve, just fix the individual drivers, don't do a
"hidden" change of where the memory is allocated from, that's not a good
idea and will cause problems later.

greg k-h

Re: change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi  Takashi,

For this one:
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772

in my kernel log,
the allocation size 62496,
about 62KB,
I think need to  change to vmalloc .
How do you think of it ?

Logs:
<4>[   11.469789] ------------[ cut here ]------------
<4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
<4>[   11.469811] size=62496 flags=80d0
<4>[   11.469815] Modules linked in:
<4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
<4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
<4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
<4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
<4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
<4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
<4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
<4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
<4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
<4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
<4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
<4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
<4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
<4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
<4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---

BRs/ Yalin
-----Original Message-----
From: Takashi Iwai [mailto:tiwai@suse.de] 
Sent: Friday, February 28, 2014 10:20 PM
To: Wang, Yalin
Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
Subject: Re: change kmalloc into vmalloc for large memory allocations

At Fri, 28 Feb 2014 16:15:23 +0800,
Wang, Yalin wrote:
> 
> Hi
> 
> 
> I find there is some drivers use kmalloc to allocate large Memorys 
> during module_init,  some can be changed to use vmalloc To save some 
> low mem, I add log in kernel to track , And list them here:
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> rs/usb/gadget/f_mass_storage.c?h=master#n2724
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound
> /soc/soc-core.c?h=master#n3772

At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.


thanks,

Takashi


> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_ftp.c?h=master#n603
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_h323_main.c?h=master#n1849
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_irc.c?h=master#n247
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_sane.c?h=master#n195
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> rs/input/evdev.c?h=master#n403
> 
> 
> they allocate large memory from 10k~64K , this will use lots of low 
> mem, instead if we use vmalloc for these drivers , it will be good for 
> some devices like smart phone, we often encounter some errors like 
> kmalloc failed because there is not enough low mem , especially when 
> the device has physical memory less than 1GB .
> 
> 
> could this module change the memory allocation into vmalloc ?
> 
> I was thinking that if we can introduce a helper function in kmalloc.h like this :
> 
> Kmalloc(size, flags)
> {
> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
> 		return vmalloc(size);
> 	Else
> 		return real_kmalloc(size);
> }
> 
> Kfree(ptr)
> {
> 	If (is_vmalloc_addr(ptr))
> 		Vfree(ptr);
> 	Else
> 		Kfree(ptr);
> }
> 
> 
> But we need add some flags to ensure always use kmalloc for Some 
> special use (dma etc..)
> 
> How do you think of it ?
> 
> Thanks
> 
> 
> 

RE: change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi  greg,

I am sorry,
I make a mistake ,
You are right ,
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/usb/gadget/f_mass_storage.c?h=master#n2724

this one should not changed to use vmalloc,
the buffer will be used by DMA，

others should be safe to change:

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_ftp.c?h=master#n603
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_h323_main.c?h=master#n1849
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_irc.c?h=master#n247
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_sane.c?h=master#n195

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/input/evdev.c?h=master#n403


Thanks 
-----Original Message-----
From: 'gregkh@linuxfoundation.org' [mailto:gregkh@linuxfoundation.org] 
Sent: Saturday, March 01, 2014 12:33 AM
To: Wang, Yalin
Cc: 'Huang Shijie'; 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'tiwai@suse.de'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
Subject: Re: change kmalloc into vmalloc for large memory allocations

On Fri, Feb 28, 2014 at 05:20:08PM +0800, Wang, Yalin wrote:
> Hi
> 
> 
> Yeah,
> Dma buffer must be allocated by kmalloc,
> 
> But the modules I list should can all be changed to use vmalloc, 
> because the buffer is only used by software, Not by any hardware .

Are you sure about that?  The USB gadget driver needs DMA memory from what I can tell, have you tried your change out on a system that does not allow the USB controller to access non-DMA memory?

And I agree with Steve, just fix the individual drivers, don't do a "hidden" change of where the memory is allocated from, that's not a good idea and will cause problems later.

greg k-h

Re: change kmalloc into vmalloc for large memory allocations

['gregkh@linuxfoundation.org']

On Mon, Mar 03, 2014 at 10:51:23AM +0800, Wang, Yalin wrote:
> Hi  greg,
> 
> I am sorry,
> I make a mistake ,
> You are right ,
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/usb/gadget/f_mass_storage.c?h=master#n2724
> 
> this one should not changed to use vmalloc,
> the buffer will be used by DMA，

Which is why a wrapper function will never work.

> others should be safe to change:
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_ftp.c?h=master#n603
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_h323_main.c?h=master#n1849
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_irc.c?h=master#n247
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_sane.c?h=master#n195
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/input/evdev.c?h=master#n403

Then send individual patches for these and see what happens.

greg k-h
_______________________________________________
Alsa-devel mailing list
Alsa-devel@alsa-project.org
http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

RE: change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi  greg,

For 
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/input/evdev.c?h=master#n403

there have been a patch for kmalloc failed ,

for 
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772

I have not change it , need some more code to change in devm_kzalloc ..

I make a patch for netfilter part :

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_ftp.c?h=master#n603
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_h323_main.c?h=master#n1849
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_irc.c?h=master#n247
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_sane.c?h=master#n195

seems work well ,
these module will allocate 64KB large memory,
is it possible to be merged ?

Thanks 

-- >8 --
diff --git a/net/netfilter/nf_conntrack_ftp.c b/net/netfilter/nf_conntrack_ftp.c
index b8a0924..0e92b0d 100644
--- a/net/netfilter/nf_conntrack_ftp.c
+++ b/net/netfilter/nf_conntrack_ftp.c
@@ -14,7 +14,7 @@
 #include <linux/moduleparam.h>
 #include <linux/netfilter.h>
 #include <linux/ip.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 #include <linux/ipv6.h>
 #include <linux/ctype.h>
 #include <linux/inet.h>
@@ -593,14 +593,14 @@ static void nf_conntrack_ftp_fini(void)
 		}
 	}
 
-	kfree(ftp_buffer);
+	vfree(ftp_buffer);
 }
 
 static int __init nf_conntrack_ftp_init(void)
 {
 	int i, j = -1, ret = 0;
 
-	ftp_buffer = kmalloc(65536, GFP_KERNEL);
+	ftp_buffer = vmalloc(65536, GFP_KERNEL);
 	if (!ftp_buffer)
 		return -ENOMEM;
 
diff --git a/net/netfilter/nf_conntrack_h323_main.c b/net/netfilter/nf_conntrack_h323_main.c
index 70866d1..49ae092 100644
--- a/net/netfilter/nf_conntrack_h323_main.c
+++ b/net/netfilter/nf_conntrack_h323_main.c
@@ -18,7 +18,7 @@
 #include <linux/inet.h>
 #include <linux/in.h>
 #include <linux/ip.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 #include <linux/udp.h>
 #include <linux/tcp.h>
 #include <linux/skbuff.h>
@@ -1837,7 +1837,7 @@ static void __exit nf_conntrack_h323_fini(void)
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_q931[1]);
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_q931[0]);
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_h245);
-	kfree(h323_buffer);
+	vfree(h323_buffer);
 	pr_debug("nf_ct_h323: fini\n");
 }
 
@@ -1846,7 +1846,7 @@ static int __init nf_conntrack_h323_init(void)
 {
 	int ret;
 
-	h323_buffer = kmalloc(65536, GFP_KERNEL);
+	h323_buffer = vmalloc(65536, GFP_KERNEL);
 	if (!h323_buffer)
 		return -ENOMEM;
 	ret = nf_conntrack_helper_register(&nf_conntrack_helper_h245);
@@ -1876,7 +1876,7 @@ err3:
 err2:
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_h245);
 err1:
-	kfree(h323_buffer);
+	vfree(h323_buffer);
 	return ret;
 }
 
diff --git a/net/netfilter/nf_conntrack_irc.c b/net/netfilter/nf_conntrack_irc.c
index 0fd2976..b57df10 100644
--- a/net/netfilter/nf_conntrack_irc.c
+++ b/net/netfilter/nf_conntrack_irc.c
@@ -16,7 +16,7 @@
 #include <linux/ip.h>
 #include <linux/tcp.h>
 #include <linux/netfilter.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 
 #include <net/netfilter/nf_conntrack.h>
 #include <net/netfilter/nf_conntrack_expect.h>
@@ -244,7 +244,7 @@ static int __init nf_conntrack_irc_init(void)
 	irc_exp_policy.max_expected = max_dcc_channels;
 	irc_exp_policy.timeout = dcc_timeout;
 
-	irc_buffer = kmalloc(65536, GFP_KERNEL);
+	irc_buffer = vmalloc(65536);
 	if (!irc_buffer)
 		return -ENOMEM;
 
@@ -285,7 +285,7 @@ static void nf_conntrack_irc_fini(void)
 
 	for (i = 0; i < ports_c; i++)
 		nf_conntrack_helper_unregister(&irc[i]);
-	kfree(irc_buffer);
+	vfree(irc_buffer);
 }
 
 module_init(nf_conntrack_irc_init);
diff --git a/net/netfilter/nf_conntrack_sane.c b/net/netfilter/nf_conntrack_sane.c
index 4a2134f..a4c8bf3 100644
--- a/net/netfilter/nf_conntrack_sane.c
+++ b/net/netfilter/nf_conntrack_sane.c
@@ -20,7 +20,7 @@
 #include <linux/module.h>
 #include <linux/moduleparam.h>
 #include <linux/netfilter.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 #include <linux/in.h>
 #include <linux/tcp.h>
 #include <net/netfilter/nf_conntrack.h>
@@ -185,14 +185,14 @@ static void nf_conntrack_sane_fini(void)
 		}
 	}
 
-	kfree(sane_buffer);
+	vfree(sane_buffer);
 }
 
 static int __init nf_conntrack_sane_init(void)
 {
 	int i, j = -1, ret = 0;
 
-	sane_buffer = kmalloc(65536, GFP_KERNEL);
+	sane_buffer = vmalloc(65536);
 	if (!sane_buffer)
 		return -ENOMEM;



-----Original Message-----
From: 'gregkh@linuxfoundation.org' [mailto:gregkh@linuxfoundation.org] 
Sent: Monday, March 03, 2014 11:08 AM
To: Wang, Yalin
Cc: 'Huang Shijie'; 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'tiwai@suse.de'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
Subject: Re: change kmalloc into vmalloc for large memory allocations

On Mon, Mar 03, 2014 at 10:51:23AM +0800, Wang, Yalin wrote:
> Hi  greg,
> 
> I am sorry,
> I make a mistake ,
> You are right ,
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> rs/usb/gadget/f_mass_storage.c?h=master#n2724
> 
> this one should not changed to use vmalloc, the buffer will be used by 
> DMA，

Which is why a wrapper function will never work.

> others should be safe to change:
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound
> /soc/soc-core.c?h=master#n3772
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_ftp.c?h=master#n603
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_h323_main.c?h=master#n1849
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_irc.c?h=master#n247
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> etfilter/nf_conntrack_sane.c?h=master#n195
> 
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> rs/input/evdev.c?h=master#n403

Then send individual patches for these and see what happens.

greg k-h
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: change kmalloc into vmalloc for large memory allocations

[Takashi Iwai]

[Dropped unrelated subsystem MLs, and corrected Liam's address]

At Mon, 3 Mar 2014 09:55:15 +0800,
Wang, Yalin wrote:
> 
> Hi  Takashi,
> 
> For this one:
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772
> 
> in my kernel log,
> the allocation size 62496,
> about 62KB,

This looks way too much than expected.  I thought each struct
snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an arary
of snd_soc_pcm_runtime, so this might be due to some invalid values
there.

> I think need to  change to vmalloc .
> How do you think of it ?

This looks rather like a bug somewhere around it.

Could you check the values there, sizeof(struct snd_soc_pcm_runtime),
card->num_links and card->num_aux_devs?

And, with which hardware is the issue examined?


thanks,

Takashi

> 
> Logs:
> <4>[   11.469789] ------------[ cut here ]------------
> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
> <4>[   11.469811] size=62496 flags=80d0
> <4>[   11.469815] Modules linked in:
> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
> 
> BRs/ Yalin
> -----Original Message-----
> From: Takashi Iwai [mailto:tiwai@suse.de] 
> Sent: Friday, February 28, 2014 10:20 PM
> To: Wang, Yalin
> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
> Subject: Re: change kmalloc into vmalloc for large memory allocations
> 
> At Fri, 28 Feb 2014 16:15:23 +0800,
> Wang, Yalin wrote:
> > 
> > Hi
> > 
> > 
> > I find there is some drivers use kmalloc to allocate large Memorys 
> > during module_init,  some can be changed to use vmalloc To save some 
> > low mem, I add log in kernel to track , And list them here:
> > 
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> > rs/usb/gadget/f_mass_storage.c?h=master#n2724
> > 
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound
> > /soc/soc-core.c?h=master#n3772
> 
> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
> 
> 
> thanks,
> 
> Takashi
> 
> 
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > etfilter/nf_conntrack_ftp.c?h=master#n603
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > etfilter/nf_conntrack_h323_main.c?h=master#n1849
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > etfilter/nf_conntrack_irc.c?h=master#n247
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > etfilter/nf_conntrack_sane.c?h=master#n195
> > 
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> > rs/input/evdev.c?h=master#n403
> > 
> > 
> > they allocate large memory from 10k~64K , this will use lots of low 
> > mem, instead if we use vmalloc for these drivers , it will be good for 
> > some devices like smart phone, we often encounter some errors like 
> > kmalloc failed because there is not enough low mem , especially when 
> > the device has physical memory less than 1GB .
> > 
> > 
> > could this module change the memory allocation into vmalloc ?
> > 
> > I was thinking that if we can introduce a helper function in kmalloc.h like this :
> > 
> > Kmalloc(size, flags)
> > {
> > 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
> > 		return vmalloc(size);
> > 	Else
> > 		return real_kmalloc(size);
> > }
> > 
> > Kfree(ptr)
> > {
> > 	If (is_vmalloc_addr(ptr))
> > 		Vfree(ptr);
> > 	Else
> > 		Kfree(ptr);
> > }
> > 
> > 
> > But we need add some flags to ensure always use kmalloc for Some 
> > special use (dma etc..)
> > 
> > How do you think of it ?
> > 
> > Thanks
> > 
> > 
> > 
> 

Re: change kmalloc into vmalloc for large memory allocations

[Takashi Iwai]

At Mon, 03 Mar 2014 10:19:58 +0100,
Takashi Iwai wrote:
> 
> [Dropped unrelated subsystem MLs, and corrected Liam's address]

Grrr, even Mark's address was obsoleted.  Here corrected.


Takashi

> 
> At Mon, 3 Mar 2014 09:55:15 +0800,
> Wang, Yalin wrote:
> > 
> > Hi  Takashi,
> > 
> > For this one:
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772
> > 
> > in my kernel log,
> > the allocation size 62496,
> > about 62KB,
> 
> This looks way too much than expected.  I thought each struct
> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an arary
> of snd_soc_pcm_runtime, so this might be due to some invalid values
> there.
> 
> > I think need to  change to vmalloc .
> > How do you think of it ?
> 
> This looks rather like a bug somewhere around it.
> 
> Could you check the values there, sizeof(struct snd_soc_pcm_runtime),
> card->num_links and card->num_aux_devs?
> 
> And, with which hardware is the issue examined?
> 
> 
> thanks,
> 
> Takashi
> 
> > 
> > Logs:
> > <4>[   11.469789] ------------[ cut here ]------------
> > <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
> > <4>[   11.469811] size=62496 flags=80d0
> > <4>[   11.469815] Modules linked in:
> > <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
> > <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
> > <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
> > <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
> > <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
> > <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
> > <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
> > <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
> > <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
> > <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
> > <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
> > <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
> > <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
> > <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
> > <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
> > 
> > BRs/ Yalin
> > -----Original Message-----
> > From: Takashi Iwai [mailto:tiwai@suse.de] 
> > Sent: Friday, February 28, 2014 10:20 PM
> > To: Wang, Yalin
> > Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
> > Subject: Re: change kmalloc into vmalloc for large memory allocations
> > 
> > At Fri, 28 Feb 2014 16:15:23 +0800,
> > Wang, Yalin wrote:
> > > 
> > > Hi
> > > 
> > > 
> > > I find there is some drivers use kmalloc to allocate large Memorys 
> > > during module_init,  some can be changed to use vmalloc To save some 
> > > low mem, I add log in kernel to track , And list them here:
> > > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> > > rs/usb/gadget/f_mass_storage.c?h=master#n2724
> > > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound
> > > /soc/soc-core.c?h=master#n3772
> > 
> > At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
> > 
> > 
> > thanks,
> > 
> > Takashi
> > 
> > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > > etfilter/nf_conntrack_ftp.c?h=master#n603
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > > etfilter/nf_conntrack_h323_main.c?h=master#n1849
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > > etfilter/nf_conntrack_irc.c?h=master#n247
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/n
> > > etfilter/nf_conntrack_sane.c?h=master#n195
> > > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drive
> > > rs/input/evdev.c?h=master#n403
> > > 
> > > 
> > > they allocate large memory from 10k~64K , this will use lots of low 
> > > mem, instead if we use vmalloc for these drivers , it will be good for 
> > > some devices like smart phone, we often encounter some errors like 
> > > kmalloc failed because there is not enough low mem , especially when 
> > > the device has physical memory less than 1GB .
> > > 
> > > 
> > > could this module change the memory allocation into vmalloc ?
> > > 
> > > I was thinking that if we can introduce a helper function in kmalloc.h like this :
> > > 
> > > Kmalloc(size, flags)
> > > {
> > > 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
> > > 		return vmalloc(size);
> > > 	Else
> > > 		return real_kmalloc(size);
> > > }
> > > 
> > > Kfree(ptr)
> > > {
> > > 	If (is_vmalloc_addr(ptr))
> > > 		Vfree(ptr);
> > > 	Else
> > > 		Kfree(ptr);
> > > }
> > > 
> > > 
> > > But we need add some flags to ensure always use kmalloc for Some 
> > > special use (dma etc..)
> > > 
> > > How do you think of it ?
> > > 
> > > Thanks
> > > 
> > > 
> > > 
> > 

RE: change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi 


I add logs in kernel , the result is :

532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0

card->num_links=42 , is it correct ?

our hardware is qcom msm8x26 platform .

Thanks 

-----Original Message-----
From: Takashi Iwai [mailto:tiwai@suse.de] 
Sent: Monday, March 03, 2014 5:23 PM
To: Wang, Yalin
Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz'; 'alsa-devel@alsa-project.org'; Liam Girdwood
Subject: Re: change kmalloc into vmalloc for large memory allocations

At Mon, 03 Mar 2014 10:19:58 +0100,
Takashi Iwai wrote:
> 
> [Dropped unrelated subsystem MLs, and corrected Liam's address]

Grrr, even Mark's address was obsoleted.  Here corrected.


Takashi

> 
> At Mon, 3 Mar 2014 09:55:15 +0800,
> Wang, Yalin wrote:
> > 
> > Hi  Takashi,
> > 
> > For this one:
> > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sou
> > nd/soc/soc-core.c?h=master#n3772
> > 
> > in my kernel log,
> > the allocation size 62496,
> > about 62KB,
> 
> This looks way too much than expected.  I thought each struct 
> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an arary 
> of snd_soc_pcm_runtime, so this might be due to some invalid values 
> there.
> 
> > I think need to  change to vmalloc .
> > How do you think of it ?
> 
> This looks rather like a bug somewhere around it.
> 
> Could you check the values there, sizeof(struct snd_soc_pcm_runtime),
> card->num_links and card->num_aux_devs?
> 
> And, with which hardware is the issue examined?
> 
> 
> thanks,
> 
> Takashi
> 
> > 
> > Logs:
> > <4>[   11.469789] ------------[ cut here ]------------
> > <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
> > <4>[   11.469811] size=62496 flags=80d0
> > <4>[   11.469815] Modules linked in:
> > <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
> > <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
> > <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
> > <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
> > <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
> > <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
> > <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
> > <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
> > <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
> > <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
> > <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
> > <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
> > <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
> > <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
> > <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
> > 
> > BRs/ Yalin
> > -----Original Message-----
> > From: Takashi Iwai [mailto:tiwai@suse.de]
> > Sent: Friday, February 28, 2014 10:20 PM
> > To: Wang, Yalin
> > Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
> > Subject: Re: change kmalloc into vmalloc for large memory 
> > allocations
> > 
> > At Fri, 28 Feb 2014 16:15:23 +0800,
> > Wang, Yalin wrote:
> > > 
> > > Hi
> > > 
> > > 
> > > I find there is some drivers use kmalloc to allocate large Memorys 
> > > during module_init,  some can be changed to use vmalloc To save 
> > > some low mem, I add log in kernel to track , And list them here:
> > > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
> > > rive
> > > rs/usb/gadget/f_mass_storage.c?h=master#n2724
> > > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
> > > ound
> > > /soc/soc-core.c?h=master#n3772
> > 
> > At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
> > 
> > 
> > thanks,
> > 
> > Takashi
> > 
> > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
> > > et/n
> > > etfilter/nf_conntrack_ftp.c?h=master#n603
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
> > > et/n
> > > etfilter/nf_conntrack_h323_main.c?h=master#n1849
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
> > > et/n
> > > etfilter/nf_conntrack_irc.c?h=master#n247
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
> > > et/n
> > > etfilter/nf_conntrack_sane.c?h=master#n195
> > > 
> > > https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
> > > rive
> > > rs/input/evdev.c?h=master#n403
> > > 
> > > 
> > > they allocate large memory from 10k~64K , this will use lots of 
> > > low mem, instead if we use vmalloc for these drivers , it will be 
> > > good for some devices like smart phone, we often encounter some 
> > > errors like kmalloc failed because there is not enough low mem , 
> > > especially when the device has physical memory less than 1GB .
> > > 
> > > 
> > > could this module change the memory allocation into vmalloc ?
> > > 
> > > I was thinking that if we can introduce a helper function in kmalloc.h like this :
> > > 
> > > Kmalloc(size, flags)
> > > {
> > > 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
> > > 		return vmalloc(size);
> > > 	Else
> > > 		return real_kmalloc(size);
> > > }
> > > 
> > > Kfree(ptr)
> > > {
> > > 	If (is_vmalloc_addr(ptr))
> > > 		Vfree(ptr);
> > > 	Else
> > > 		Kfree(ptr);
> > > }
> > > 
> > > 
> > > But we need add some flags to ensure always use kmalloc for Some 
> > > special use (dma etc..)
> > > 
> > > How do you think of it ?
> > > 
> > > Thanks
> > > 
> > > 
> > > 
> > 

Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

[Lars-Peter Clausen]

On 03/03/2014 12:13 PM, Wang, Yalin wrote:
> Hi
>
>
> I add logs in kernel , the result is :
>
> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
>
> card->num_links=42 , is it correct ?

Very unlikely. Make sure that it is properly initialized in your machine driver.

>
> our hardware is qcom msm8x26 platform .

This one doesn't seem to be in upstream.

>
> Thanks
>
> -----Original Message-----
> From: Takashi Iwai [mailto:tiwai@suse.de]
> Sent: Monday, March 03, 2014 5:23 PM
> To: Wang, Yalin
> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz'; 'alsa-devel@alsa-project.org'; Liam Girdwood
> Subject: Re: change kmalloc into vmalloc for large memory allocations
>
> At Mon, 03 Mar 2014 10:19:58 +0100,
> Takashi Iwai wrote:
>>
>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
>
> Grrr, even Mark's address was obsoleted.  Here corrected.
>
>
> Takashi
>
>>
>> At Mon, 3 Mar 2014 09:55:15 +0800,
>> Wang, Yalin wrote:
>>>
>>> Hi  Takashi,
>>>
>>> For this one:
>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sou
>>> nd/soc/soc-core.c?h=master#n3772
>>>
>>> in my kernel log,
>>> the allocation size 62496,
>>> about 62KB,
>>
>> This looks way too much than expected.  I thought each struct
>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an arary
>> of snd_soc_pcm_runtime, so this might be due to some invalid values
>> there.
>>
>>> I think need to  change to vmalloc .
>>> How do you think of it ?
>>
>> This looks rather like a bug somewhere around it.
>>
>> Could you check the values there, sizeof(struct snd_soc_pcm_runtime),
>> card->num_links and card->num_aux_devs?
>>
>> And, with which hardware is the issue examined?
>>
>>
>> thanks,
>>
>> Takashi
>>
>>>
>>> Logs:
>>> <4>[   11.469789] ------------[ cut here ]------------
>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
>>> <4>[   11.469811] size=62496 flags=80d0
>>> <4>[   11.469815] Modules linked in:
>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
>>>
>>> BRs/ Yalin
>>> -----Original Message-----
>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>> Sent: Friday, February 28, 2014 10:20 PM
>>> To: Wang, Yalin
>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
>>> Subject: Re: change kmalloc into vmalloc for large memory
>>> allocations
>>>
>>> At Fri, 28 Feb 2014 16:15:23 +0800,
>>> Wang, Yalin wrote:
>>>>
>>>> Hi
>>>>
>>>>
>>>> I find there is some drivers use kmalloc to allocate large Memorys
>>>> during module_init,  some can be changed to use vmalloc To save
>>>> some low mem, I add log in kernel to track , And list them here:
>>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>> rive
>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
>>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
>>>> ound
>>>> /soc/soc-core.c?h=master#n3772
>>>
>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
>>>
>>>
>>> thanks,
>>>
>>> Takashi
>>>
>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_irc.c?h=master#n247
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_sane.c?h=master#n195
>>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>> rive
>>>> rs/input/evdev.c?h=master#n403
>>>>
>>>>
>>>> they allocate large memory from 10k~64K , this will use lots of
>>>> low mem, instead if we use vmalloc for these drivers , it will be
>>>> good for some devices like smart phone, we often encounter some
>>>> errors like kmalloc failed because there is not enough low mem ,
>>>> especially when the device has physical memory less than 1GB .
>>>>
>>>>
>>>> could this module change the memory allocation into vmalloc ?
>>>>
>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
>>>>
>>>> Kmalloc(size, flags)
>>>> {
>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
>>>> 		return vmalloc(size);
>>>> 	Else
>>>> 		return real_kmalloc(size);
>>>> }
>>>>
>>>> Kfree(ptr)
>>>> {
>>>> 	If (is_vmalloc_addr(ptr))
>>>> 		Vfree(ptr);
>>>> 	Else
>>>> 		Kfree(ptr);
>>>> }
>>>>
>>>>
>>>> But we need add some flags to ensure always use kmalloc for Some
>>>> special use (dma etc..)
>>>>
>>>> How do you think of it ?
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>

RE: [alsa-devel] change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi  

I am curious what number range is correct here for card->num_links ?


Thanks 

-----Original Message-----
From: Lars-Peter Clausen [mailto:lars@metafoo.de] 
Sent: Monday, March 03, 2014 7:35 PM
To: Wang, Yalin
Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 'linux-kernel@vger.kernel.org'; Liam Girdwood
Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

On 03/03/2014 12:13 PM, Wang, Yalin wrote:
> Hi
>
>
> I add logs in kernel , the result is :
>
> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
>
> card->num_links=42 , is it correct ?

Very unlikely. Make sure that it is properly initialized in your machine driver.

>
> our hardware is qcom msm8x26 platform .

This one doesn't seem to be in upstream.

>
> Thanks
>
> -----Original Message-----
> From: Takashi Iwai [mailto:tiwai@suse.de]
> Sent: Monday, March 03, 2014 5:23 PM
> To: Wang, Yalin
> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz'; 
> 'alsa-devel@alsa-project.org'; Liam Girdwood
> Subject: Re: change kmalloc into vmalloc for large memory allocations
>
> At Mon, 03 Mar 2014 10:19:58 +0100,
> Takashi Iwai wrote:
>>
>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
>
> Grrr, even Mark's address was obsoleted.  Here corrected.
>
>
> Takashi
>
>>
>> At Mon, 3 Mar 2014 09:55:15 +0800,
>> Wang, Yalin wrote:
>>>
>>> Hi  Takashi,
>>>
>>> For this one:
>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sou
>>> nd/soc/soc-core.c?h=master#n3772
>>>
>>> in my kernel log,
>>> the allocation size 62496,
>>> about 62KB,
>>
>> This looks way too much than expected.  I thought each struct 
>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an arary 
>> of snd_soc_pcm_runtime, so this might be due to some invalid values 
>> there.
>>
>>> I think need to  change to vmalloc .
>>> How do you think of it ?
>>
>> This looks rather like a bug somewhere around it.
>>
>> Could you check the values there, sizeof(struct snd_soc_pcm_runtime),
>> card->num_links and card->num_aux_devs?
>>
>> And, with which hardware is the issue examined?
>>
>>
>> thanks,
>>
>> Takashi
>>
>>>
>>> Logs:
>>> <4>[   11.469789] ------------[ cut here ]------------
>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
>>> <4>[   11.469811] size=62496 flags=80d0
>>> <4>[   11.469815] Modules linked in:
>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
>>>
>>> BRs/ Yalin
>>> -----Original Message-----
>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>> Sent: Friday, February 28, 2014 10:20 PM
>>> To: Wang, Yalin
>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
>>> Subject: Re: change kmalloc into vmalloc for large memory 
>>> allocations
>>>
>>> At Fri, 28 Feb 2014 16:15:23 +0800,
>>> Wang, Yalin wrote:
>>>>
>>>> Hi
>>>>
>>>>
>>>> I find there is some drivers use kmalloc to allocate large Memorys 
>>>> during module_init,  some can be changed to use vmalloc To save 
>>>> some low mem, I add log in kernel to track , And list them here:
>>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>> rive
>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
>>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
>>>> ound
>>>> /soc/soc-core.c?h=master#n3772
>>>
>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
>>>
>>>
>>> thanks,
>>>
>>> Takashi
>>>
>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_irc.c?h=master#n247
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>> et/n
>>>> etfilter/nf_conntrack_sane.c?h=master#n195
>>>>
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>> rive
>>>> rs/input/evdev.c?h=master#n403
>>>>
>>>>
>>>> they allocate large memory from 10k~64K , this will use lots of low 
>>>> mem, instead if we use vmalloc for these drivers , it will be good 
>>>> for some devices like smart phone, we often encounter some errors 
>>>> like kmalloc failed because there is not enough low mem , 
>>>> especially when the device has physical memory less than 1GB .
>>>>
>>>>
>>>> could this module change the memory allocation into vmalloc ?
>>>>
>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
>>>>
>>>> Kmalloc(size, flags)
>>>> {
>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
>>>> 		return vmalloc(size);
>>>> 	Else
>>>> 		return real_kmalloc(size);
>>>> }
>>>>
>>>> Kfree(ptr)
>>>> {
>>>> 	If (is_vmalloc_addr(ptr))
>>>> 		Vfree(ptr);
>>>> 	Else
>>>> 		Kfree(ptr);
>>>> }
>>>>
>>>>
>>>> But we need add some flags to ensure always use kmalloc for Some 
>>>> special use (dma etc..)
>>>>
>>>> How do you think of it ?
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>

Re: change kmalloc into vmalloc for large memory allocations

[Lars-Peter Clausen]

On 03/03/2014 12:36 PM, Wang, Yalin wrote:
> Hi
>
> I am curious what number range is correct here for card->num_links ?

It depends on your board. But typically I'd say <= 5, for most boards it's 
either 1 or 2.

>
>
> Thanks
>
> -----Original Message-----
> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> Sent: Monday, March 03, 2014 7:35 PM
> To: Wang, Yalin
> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 'linux-kernel@vger.kernel.org'; Liam Girdwood
> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations
>
> On 03/03/2014 12:13 PM, Wang, Yalin wrote:
>> Hi
>>
>>
>> I add logs in kernel , the result is :
>>
>> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
>>
>> card->num_links=42 , is it correct ?
>
> Very unlikely. Make sure that it is properly initialized in your machine driver.
>
>>
>> our hardware is qcom msm8x26 platform .
>
> This one doesn't seem to be in upstream.
>
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Takashi Iwai [mailto:tiwai@suse.de]
>> Sent: Monday, March 03, 2014 5:23 PM
>> To: Wang, Yalin
>> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz';
>> 'alsa-devel@alsa-project.org'; Liam Girdwood
>> Subject: Re: change kmalloc into vmalloc for large memory allocations
>>
>> At Mon, 03 Mar 2014 10:19:58 +0100,
>> Takashi Iwai wrote:
>>>
>>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
>>
>> Grrr, even Mark's address was obsoleted.  Here corrected.
>>
>>
>> Takashi
>>
>>>
>>> At Mon, 3 Mar 2014 09:55:15 +0800,
>>> Wang, Yalin wrote:
>>>>
>>>> Hi  Takashi,
>>>>
>>>> For this one:
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sou
>>>> nd/soc/soc-core.c?h=master#n3772
>>>>
>>>> in my kernel log,
>>>> the allocation size 62496,
>>>> about 62KB,
>>>
>>> This looks way too much than expected.  I thought each struct
>>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an arary
>>> of snd_soc_pcm_runtime, so this might be due to some invalid values
>>> there.
>>>
>>>> I think need to  change to vmalloc .
>>>> How do you think of it ?
>>>
>>> This looks rather like a bug somewhere around it.
>>>
>>> Could you check the values there, sizeof(struct snd_soc_pcm_runtime),
>>> card->num_links and card->num_aux_devs?
>>>
>>> And, with which hardware is the issue examined?
>>>
>>>
>>> thanks,
>>>
>>> Takashi
>>>
>>>>
>>>> Logs:
>>>> <4>[   11.469789] ------------[ cut here ]------------
>>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
>>>> <4>[   11.469811] size=62496 flags=80d0
>>>> <4>[   11.469815] Modules linked in:
>>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
>>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
>>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
>>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
>>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
>>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
>>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
>>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
>>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
>>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
>>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
>>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
>>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
>>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
>>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
>>>>
>>>> BRs/ Yalin
>>>> -----Original Message-----
>>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>>> Sent: Friday, February 28, 2014 10:20 PM
>>>> To: Wang, Yalin
>>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
>>>> Subject: Re: change kmalloc into vmalloc for large memory
>>>> allocations
>>>>
>>>> At Fri, 28 Feb 2014 16:15:23 +0800,
>>>> Wang, Yalin wrote:
>>>>>
>>>>> Hi
>>>>>
>>>>>
>>>>> I find there is some drivers use kmalloc to allocate large Memorys
>>>>> during module_init,  some can be changed to use vmalloc To save
>>>>> some low mem, I add log in kernel to track , And list them here:
>>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>>> rive
>>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
>>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
>>>>> ound
>>>>> /soc/soc-core.c?h=master#n3772
>>>>
>>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
>>>>
>>>>
>>>> thanks,
>>>>
>>>> Takashi
>>>>
>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_irc.c?h=master#n247
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_sane.c?h=master#n195
>>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>>> rive
>>>>> rs/input/evdev.c?h=master#n403
>>>>>
>>>>>
>>>>> they allocate large memory from 10k~64K , this will use lots of low
>>>>> mem, instead if we use vmalloc for these drivers , it will be good
>>>>> for some devices like smart phone, we often encounter some errors
>>>>> like kmalloc failed because there is not enough low mem ,
>>>>> especially when the device has physical memory less than 1GB .
>>>>>
>>>>>
>>>>> could this module change the memory allocation into vmalloc ?
>>>>>
>>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
>>>>>
>>>>> Kmalloc(size, flags)
>>>>> {
>>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
>>>>> 		return vmalloc(size);
>>>>> 	Else
>>>>> 		return real_kmalloc(size);
>>>>> }
>>>>>
>>>>> Kfree(ptr)
>>>>> {
>>>>> 	If (is_vmalloc_addr(ptr))
>>>>> 		Vfree(ptr);
>>>>> 	Else
>>>>> 		Kfree(ptr);
>>>>> }
>>>>>
>>>>>
>>>>> But we need add some flags to ensure always use kmalloc for Some
>>>>> special use (dma etc..)
>>>>>
>>>>> How do you think of it ?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>
>> _______________________________________________
>> Alsa-devel mailing list
>> Alsa-devel@alsa-project.org
>> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>>
>

RE: [alsa-devel] change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi 


I find the num_links are defined here:
https://www.codeaurora.org/cgit/quic/la/kernel/msm-3.10/tree/sound/soc/msm/msm8226.c?h=msm-3.10&id=d681c8e79c597a5e1ae79803dc0215d02df8fe34#n1812


the ARRAY_SIZE is 42 , you mean the driver can't define so many elements ?
how to implement it if we can't do like this ?

Thanks 

-----Original Message-----
From: Lars-Peter Clausen [mailto:lars@metafoo.de] 
Sent: Monday, March 03, 2014 7:46 PM
To: Wang, Yalin
Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 'linux-kernel@vger.kernel.org'; Liam Girdwood
Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

On 03/03/2014 12:36 PM, Wang, Yalin wrote:
> Hi
>
> I am curious what number range is correct here for card->num_links ?

It depends on your board. But typically I'd say <= 5, for most boards it's either 1 or 2.

>
>
> Thanks
>
> -----Original Message-----
> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> Sent: Monday, March 03, 2014 7:35 PM
> To: Wang, Yalin
> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
> 'linux-kernel@vger.kernel.org'; Liam Girdwood
> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory 
> allocations
>
> On 03/03/2014 12:13 PM, Wang, Yalin wrote:
>> Hi
>>
>>
>> I add logs in kernel , the result is :
>>
>> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
>>
>> card->num_links=42 , is it correct ?
>
> Very unlikely. Make sure that it is properly initialized in your machine driver.
>
>>
>> our hardware is qcom msm8x26 platform .
>
> This one doesn't seem to be in upstream.
>
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Takashi Iwai [mailto:tiwai@suse.de]
>> Sent: Monday, March 03, 2014 5:23 PM
>> To: Wang, Yalin
>> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz'; 
>> 'alsa-devel@alsa-project.org'; Liam Girdwood
>> Subject: Re: change kmalloc into vmalloc for large memory allocations
>>
>> At Mon, 03 Mar 2014 10:19:58 +0100,
>> Takashi Iwai wrote:
>>>
>>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
>>
>> Grrr, even Mark's address was obsoleted.  Here corrected.
>>
>>
>> Takashi
>>
>>>
>>> At Mon, 3 Mar 2014 09:55:15 +0800,
>>> Wang, Yalin wrote:
>>>>
>>>> Hi  Takashi,
>>>>
>>>> For this one:
>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/so
>>>> u
>>>> nd/soc/soc-core.c?h=master#n3772
>>>>
>>>> in my kernel log,
>>>> the allocation size 62496,
>>>> about 62KB,
>>>
>>> This looks way too much than expected.  I thought each struct 
>>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an 
>>> arary of snd_soc_pcm_runtime, so this might be due to some invalid 
>>> values there.
>>>
>>>> I think need to  change to vmalloc .
>>>> How do you think of it ?
>>>
>>> This looks rather like a bug somewhere around it.
>>>
>>> Could you check the values there, sizeof(struct 
>>> snd_soc_pcm_runtime),
>>> card->num_links and card->num_aux_devs?
>>>
>>> And, with which hardware is the issue examined?
>>>
>>>
>>> thanks,
>>>
>>> Takashi
>>>
>>>>
>>>> Logs:
>>>> <4>[   11.469789] ------------[ cut here ]------------
>>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
>>>> <4>[   11.469811] size=62496 flags=80d0
>>>> <4>[   11.469815] Modules linked in:
>>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
>>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
>>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
>>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
>>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
>>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
>>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
>>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
>>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
>>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
>>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
>>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
>>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
>>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
>>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
>>>>
>>>> BRs/ Yalin
>>>> -----Original Message-----
>>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>>> Sent: Friday, February 28, 2014 10:20 PM
>>>> To: Wang, Yalin
>>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
>>>> Subject: Re: change kmalloc into vmalloc for large memory 
>>>> allocations
>>>>
>>>> At Fri, 28 Feb 2014 16:15:23 +0800, Wang, Yalin wrote:
>>>>>
>>>>> Hi
>>>>>
>>>>>
>>>>> I find there is some drivers use kmalloc to allocate large Memorys 
>>>>> during module_init,  some can be changed to use vmalloc To save 
>>>>> some low mem, I add log in kernel to track , And list them here:
>>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>>> rive
>>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
>>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
>>>>> ound
>>>>> /soc/soc-core.c?h=master#n3772
>>>>
>>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
>>>>
>>>>
>>>> thanks,
>>>>
>>>> Takashi
>>>>
>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_irc.c?h=master#n247
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>> et/n
>>>>> etfilter/nf_conntrack_sane.c?h=master#n195
>>>>>
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>>> rive
>>>>> rs/input/evdev.c?h=master#n403
>>>>>
>>>>>
>>>>> they allocate large memory from 10k~64K , this will use lots of 
>>>>> low mem, instead if we use vmalloc for these drivers , it will be 
>>>>> good for some devices like smart phone, we often encounter some 
>>>>> errors like kmalloc failed because there is not enough low mem , 
>>>>> especially when the device has physical memory less than 1GB .
>>>>>
>>>>>
>>>>> could this module change the memory allocation into vmalloc ?
>>>>>
>>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
>>>>>
>>>>> Kmalloc(size, flags)
>>>>> {
>>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
>>>>> 		return vmalloc(size);
>>>>> 	Else
>>>>> 		return real_kmalloc(size);
>>>>> }
>>>>>
>>>>> Kfree(ptr)
>>>>> {
>>>>> 	If (is_vmalloc_addr(ptr))
>>>>> 		Vfree(ptr);
>>>>> 	Else
>>>>> 		Kfree(ptr);
>>>>> }
>>>>>
>>>>>
>>>>> But we need add some flags to ensure always use kmalloc for Some 
>>>>> special use (dma etc..)
>>>>>
>>>>> How do you think of it ?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>
>> _______________________________________________
>> Alsa-devel mailing list
>> Alsa-devel@alsa-project.org
>> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>>
>

Re: change kmalloc into vmalloc for large memory allocations

['gregkh@linuxfoundation.org']

On Mon, Mar 03, 2014 at 04:00:59PM +0800, Wang, Yalin wrote:
> Hi  greg,

Please read Documentation/SubmittingPatches for how to properly submit
kernel patches in a form that they could be accepted.

thanks,

greg k-h

Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

[Lars-Peter Clausen]

On 03/03/2014 02:29 PM, Wang, Yalin wrote:
> Hi
>
>
> I find the num_links are defined here:
> https://www.codeaurora.org/cgit/quic/la/kernel/msm-3.10/tree/sound/soc/msm/msm8226.c?h=msm-3.10&id=d681c8e79c597a5e1ae79803dc0215d02df8fe34#n1812
>
>
> the ARRAY_SIZE is 42 , you mean the driver can't define so many elements ?
> how to implement it if we can't do like this ?
>

Looks like your platform is a bit special in this regard and requires quite 
a few more links than usual.

> Thanks
>
> -----Original Message-----
> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> Sent: Monday, March 03, 2014 7:46 PM
> To: Wang, Yalin
> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 'linux-kernel@vger.kernel.org'; Liam Girdwood
> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations
>
> On 03/03/2014 12:36 PM, Wang, Yalin wrote:
>> Hi
>>
>> I am curious what number range is correct here for card->num_links ?
>
> It depends on your board. But typically I'd say <= 5, for most boards it's either 1 or 2.
>
>>
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
>> Sent: Monday, March 03, 2014 7:35 PM
>> To: Wang, Yalin
>> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown;
>> 'linux-kernel@vger.kernel.org'; Liam Girdwood
>> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory
>> allocations
>>
>> On 03/03/2014 12:13 PM, Wang, Yalin wrote:
>>> Hi
>>>
>>>
>>> I add logs in kernel , the result is :
>>>
>>> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
>>>
>>> card->num_links=42 , is it correct ?
>>
>> Very unlikely. Make sure that it is properly initialized in your machine driver.
>>
>>>
>>> our hardware is qcom msm8x26 platform .
>>
>> This one doesn't seem to be in upstream.
>>
>>>
>>> Thanks
>>>
>>> -----Original Message-----
>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>> Sent: Monday, March 03, 2014 5:23 PM
>>> To: Wang, Yalin
>>> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz';
>>> 'alsa-devel@alsa-project.org'; Liam Girdwood
>>> Subject: Re: change kmalloc into vmalloc for large memory allocations
>>>
>>> At Mon, 03 Mar 2014 10:19:58 +0100,
>>> Takashi Iwai wrote:
>>>>
>>>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
>>>
>>> Grrr, even Mark's address was obsoleted.  Here corrected.
>>>
>>>
>>> Takashi
>>>
>>>>
>>>> At Mon, 3 Mar 2014 09:55:15 +0800,
>>>> Wang, Yalin wrote:
>>>>>
>>>>> Hi  Takashi,
>>>>>
>>>>> For this one:
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/so
>>>>> u
>>>>> nd/soc/soc-core.c?h=master#n3772
>>>>>
>>>>> in my kernel log,
>>>>> the allocation size 62496,
>>>>> about 62KB,
>>>>
>>>> This looks way too much than expected.  I thought each struct
>>>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an
>>>> arary of snd_soc_pcm_runtime, so this might be due to some invalid
>>>> values there.
>>>>
>>>>> I think need to  change to vmalloc .
>>>>> How do you think of it ?
>>>>
>>>> This looks rather like a bug somewhere around it.
>>>>
>>>> Could you check the values there, sizeof(struct
>>>> snd_soc_pcm_runtime),
>>>> card->num_links and card->num_aux_devs?
>>>>
>>>> And, with which hardware is the issue examined?
>>>>
>>>>
>>>> thanks,
>>>>
>>>> Takashi
>>>>
>>>>>
>>>>> Logs:
>>>>> <4>[   11.469789] ------------[ cut here ]------------
>>>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
>>>>> <4>[   11.469811] size=62496 flags=80d0
>>>>> <4>[   11.469815] Modules linked in:
>>>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
>>>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
>>>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
>>>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
>>>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
>>>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
>>>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
>>>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
>>>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
>>>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
>>>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
>>>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
>>>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
>>>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
>>>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
>>>>>
>>>>> BRs/ Yalin
>>>>> -----Original Message-----
>>>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>>>> Sent: Friday, February 28, 2014 10:20 PM
>>>>> To: Wang, Yalin
>>>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
>>>>> Subject: Re: change kmalloc into vmalloc for large memory
>>>>> allocations
>>>>>
>>>>> At Fri, 28 Feb 2014 16:15:23 +0800, Wang, Yalin wrote:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>>
>>>>>> I find there is some drivers use kmalloc to allocate large Memorys
>>>>>> during module_init,  some can be changed to use vmalloc To save
>>>>>> some low mem, I add log in kernel to track , And list them here:
>>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>>>> rive
>>>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
>>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
>>>>>> ound
>>>>>> /soc/soc-core.c?h=master#n3772
>>>>>
>>>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
>>>>>
>>>>>
>>>>> thanks,
>>>>>
>>>>> Takashi
>>>>>
>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_irc.c?h=master#n247
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_sane.c?h=master#n195
>>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/d
>>>>>> rive
>>>>>> rs/input/evdev.c?h=master#n403
>>>>>>
>>>>>>
>>>>>> they allocate large memory from 10k~64K , this will use lots of
>>>>>> low mem, instead if we use vmalloc for these drivers , it will be
>>>>>> good for some devices like smart phone, we often encounter some
>>>>>> errors like kmalloc failed because there is not enough low mem ,
>>>>>> especially when the device has physical memory less than 1GB .
>>>>>>
>>>>>>
>>>>>> could this module change the memory allocation into vmalloc ?
>>>>>>
>>>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
>>>>>>
>>>>>> Kmalloc(size, flags)
>>>>>> {
>>>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
>>>>>> 		return vmalloc(size);
>>>>>> 	Else
>>>>>> 		return real_kmalloc(size);
>>>>>> }
>>>>>>
>>>>>> Kfree(ptr)
>>>>>> {
>>>>>> 	If (is_vmalloc_addr(ptr))
>>>>>> 		Vfree(ptr);
>>>>>> 	Else
>>>>>> 		Kfree(ptr);
>>>>>> }
>>>>>>
>>>>>>
>>>>>> But we need add some flags to ensure always use kmalloc for Some
>>>>>> special use (dma etc..)
>>>>>>
>>>>>> How do you think of it ?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>> _______________________________________________
>>> Alsa-devel mailing list
>>> Alsa-devel@alsa-project.org
>>> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>>>
>>
>

RE: [alsa-devel] change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi  

I see,
Is it possible to change this part:
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772


so that it use vmalloc if it need allocate large memory for
struct snd_soc_pcm_runtime ?

Thanks 

-----Original Message-----
From: Lars-Peter Clausen [mailto:lars@metafoo.de] 
Sent: Monday, March 03, 2014 10:23 PM
To: Wang, Yalin
Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 'linux-kernel@vger.kernel.org'; Liam Girdwood
Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

On 03/03/2014 02:29 PM, Wang, Yalin wrote:
> Hi
>
>
> I find the num_links are defined here:
> https://www.codeaurora.org/cgit/quic/la/kernel/msm-3.10/tree/sound/soc
> /msm/msm8226.c?h=msm-3.10&id=d681c8e79c597a5e1ae79803dc0215d02df8fe34#
> n1812
>
>
> the ARRAY_SIZE is 42 , you mean the driver can't define so many elements ?
> how to implement it if we can't do like this ?
>

Looks like your platform is a bit special in this regard and requires quite a few more links than usual.

> Thanks
>
> -----Original Message-----
> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> Sent: Monday, March 03, 2014 7:46 PM
> To: Wang, Yalin
> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
> 'linux-kernel@vger.kernel.org'; Liam Girdwood
> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory 
> allocations
>
> On 03/03/2014 12:36 PM, Wang, Yalin wrote:
>> Hi
>>
>> I am curious what number range is correct here for card->num_links ?
>
> It depends on your board. But typically I'd say <= 5, for most boards it's either 1 or 2.
>
>>
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
>> Sent: Monday, March 03, 2014 7:35 PM
>> To: Wang, Yalin
>> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
>> 'linux-kernel@vger.kernel.org'; Liam Girdwood
>> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large 
>> memory allocations
>>
>> On 03/03/2014 12:13 PM, Wang, Yalin wrote:
>>> Hi
>>>
>>>
>>> I add logs in kernel , the result is :
>>>
>>> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
>>>
>>> card->num_links=42 , is it correct ?
>>
>> Very unlikely. Make sure that it is properly initialized in your machine driver.
>>
>>>
>>> our hardware is qcom msm8x26 platform .
>>
>> This one doesn't seem to be in upstream.
>>
>>>
>>> Thanks
>>>
>>> -----Original Message-----
>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>> Sent: Monday, March 03, 2014 5:23 PM
>>> To: Wang, Yalin
>>> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz'; 
>>> 'alsa-devel@alsa-project.org'; Liam Girdwood
>>> Subject: Re: change kmalloc into vmalloc for large memory 
>>> allocations
>>>
>>> At Mon, 03 Mar 2014 10:19:58 +0100,
>>> Takashi Iwai wrote:
>>>>
>>>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
>>>
>>> Grrr, even Mark's address was obsoleted.  Here corrected.
>>>
>>>
>>> Takashi
>>>
>>>>
>>>> At Mon, 3 Mar 2014 09:55:15 +0800,
>>>> Wang, Yalin wrote:
>>>>>
>>>>> Hi  Takashi,
>>>>>
>>>>> For this one:
>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
>>>>> o
>>>>> u
>>>>> nd/soc/soc-core.c?h=master#n3772
>>>>>
>>>>> in my kernel log,
>>>>> the allocation size 62496,
>>>>> about 62KB,
>>>>
>>>> This looks way too much than expected.  I thought each struct 
>>>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an 
>>>> arary of snd_soc_pcm_runtime, so this might be due to some invalid 
>>>> values there.
>>>>
>>>>> I think need to  change to vmalloc .
>>>>> How do you think of it ?
>>>>
>>>> This looks rather like a bug somewhere around it.
>>>>
>>>> Could you check the values there, sizeof(struct 
>>>> snd_soc_pcm_runtime),
>>>> card->num_links and card->num_aux_devs?
>>>>
>>>> And, with which hardware is the issue examined?
>>>>
>>>>
>>>> thanks,
>>>>
>>>> Takashi
>>>>
>>>>>
>>>>> Logs:
>>>>> <4>[   11.469789] ------------[ cut here ]------------
>>>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
>>>>> <4>[   11.469811] size=62496 flags=80d0
>>>>> <4>[   11.469815] Modules linked in:
>>>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
>>>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
>>>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
>>>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
>>>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
>>>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
>>>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
>>>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
>>>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
>>>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
>>>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
>>>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
>>>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
>>>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
>>>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
>>>>>
>>>>> BRs/ Yalin
>>>>> -----Original Message-----
>>>>> From: Takashi Iwai [mailto:tiwai@suse.de]
>>>>> Sent: Friday, February 28, 2014 10:20 PM
>>>>> To: Wang, Yalin
>>>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
>>>>> Subject: Re: change kmalloc into vmalloc for large memory 
>>>>> allocations
>>>>>
>>>>> At Fri, 28 Feb 2014 16:15:23 +0800, Wang, Yalin wrote:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>>
>>>>>> I find there is some drivers use kmalloc to allocate large 
>>>>>> Memorys during module_init,  some can be changed to use vmalloc 
>>>>>> To save some low mem, I add log in kernel to track , And list them here:
>>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
>>>>>> d
>>>>>> rive
>>>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
>>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
>>>>>> s
>>>>>> ound
>>>>>> /soc/soc-core.c?h=master#n3772
>>>>>
>>>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
>>>>>
>>>>>
>>>>> thanks,
>>>>>
>>>>> Takashi
>>>>>
>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
>>>>>> n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
>>>>>> n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
>>>>>> n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_irc.c?h=master#n247
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
>>>>>> n
>>>>>> et/n
>>>>>> etfilter/nf_conntrack_sane.c?h=master#n195
>>>>>>
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
>>>>>> d
>>>>>> rive
>>>>>> rs/input/evdev.c?h=master#n403
>>>>>>
>>>>>>
>>>>>> they allocate large memory from 10k~64K , this will use lots of 
>>>>>> low mem, instead if we use vmalloc for these drivers , it will be 
>>>>>> good for some devices like smart phone, we often encounter some 
>>>>>> errors like kmalloc failed because there is not enough low mem , 
>>>>>> especially when the device has physical memory less than 1GB .
>>>>>>
>>>>>>
>>>>>> could this module change the memory allocation into vmalloc ?
>>>>>>
>>>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
>>>>>>
>>>>>> Kmalloc(size, flags)
>>>>>> {
>>>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
>>>>>> 		return vmalloc(size);
>>>>>> 	Else
>>>>>> 		return real_kmalloc(size);
>>>>>> }
>>>>>>
>>>>>> Kfree(ptr)
>>>>>> {
>>>>>> 	If (is_vmalloc_addr(ptr))
>>>>>> 		Vfree(ptr);
>>>>>> 	Else
>>>>>> 		Kfree(ptr);
>>>>>> }
>>>>>>
>>>>>>
>>>>>> But we need add some flags to ensure always use kmalloc for Some 
>>>>>> special use (dma etc..)
>>>>>>
>>>>>> How do you think of it ?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>> _______________________________________________
>>> Alsa-devel mailing list
>>> Alsa-devel@alsa-project.org
>>> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>>>
>>
>

Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

[Takashi Iwai]

At Mon, 3 Mar 2014 22:29:19 +0800,
Wang, Yalin wrote:
> 
> Hi  
> 
> I see,
> Is it possible to change this part:
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772
> 
> 
> so that it use vmalloc if it need allocate large memory for
> struct snd_soc_pcm_runtime ?

As I mentioned, this won't help much for your problem, unless this
allocation really failed.  This is performed only once at boot.
Allocating this size via kmalloc once shouldn't be a big problem for a
system with 1GB RAM.  The other places allocate large buffers at each
open or such frequent operation, thus they might influence more,
though.

IMO, a cleaner solution would to rewrite the code to allocate each RTD
dynamically at binding.  But this would need lots of fundamental code
changes.


Takashi

> 
> Thanks 
> 
> -----Original Message-----
> From: Lars-Peter Clausen [mailto:lars@metafoo.de] 
> Sent: Monday, March 03, 2014 10:23 PM
> To: Wang, Yalin
> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 'linux-kernel@vger.kernel.org'; Liam Girdwood
> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations
> 
> On 03/03/2014 02:29 PM, Wang, Yalin wrote:
> > Hi
> >
> >
> > I find the num_links are defined here:
> > https://www.codeaurora.org/cgit/quic/la/kernel/msm-3.10/tree/sound/soc
> > /msm/msm8226.c?h=msm-3.10&id=d681c8e79c597a5e1ae79803dc0215d02df8fe34#
> > n1812
> >
> >
> > the ARRAY_SIZE is 42 , you mean the driver can't define so many elements ?
> > how to implement it if we can't do like this ?
> >
> 
> Looks like your platform is a bit special in this regard and requires quite a few more links than usual.
> 
> > Thanks
> >
> > -----Original Message-----
> > From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> > Sent: Monday, March 03, 2014 7:46 PM
> > To: Wang, Yalin
> > Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
> > 'linux-kernel@vger.kernel.org'; Liam Girdwood
> > Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory 
> > allocations
> >
> > On 03/03/2014 12:36 PM, Wang, Yalin wrote:
> >> Hi
> >>
> >> I am curious what number range is correct here for card->num_links ?
> >
> > It depends on your board. But typically I'd say <= 5, for most boards it's either 1 or 2.
> >
> >>
> >>
> >> Thanks
> >>
> >> -----Original Message-----
> >> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> >> Sent: Monday, March 03, 2014 7:35 PM
> >> To: Wang, Yalin
> >> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
> >> 'linux-kernel@vger.kernel.org'; Liam Girdwood
> >> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large 
> >> memory allocations
> >>
> >> On 03/03/2014 12:13 PM, Wang, Yalin wrote:
> >>> Hi
> >>>
> >>>
> >>> I add logs in kernel , the result is :
> >>>
> >>> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
> >>>
> >>> card->num_links=42 , is it correct ?
> >>
> >> Very unlikely. Make sure that it is properly initialized in your machine driver.
> >>
> >>>
> >>> our hardware is qcom msm8x26 platform .
> >>
> >> This one doesn't seem to be in upstream.
> >>
> >>>
> >>> Thanks
> >>>
> >>> -----Original Message-----
> >>> From: Takashi Iwai [mailto:tiwai@suse.de]
> >>> Sent: Monday, March 03, 2014 5:23 PM
> >>> To: Wang, Yalin
> >>> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz'; 
> >>> 'alsa-devel@alsa-project.org'; Liam Girdwood
> >>> Subject: Re: change kmalloc into vmalloc for large memory 
> >>> allocations
> >>>
> >>> At Mon, 03 Mar 2014 10:19:58 +0100,
> >>> Takashi Iwai wrote:
> >>>>
> >>>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
> >>>
> >>> Grrr, even Mark's address was obsoleted.  Here corrected.
> >>>
> >>>
> >>> Takashi
> >>>
> >>>>
> >>>> At Mon, 3 Mar 2014 09:55:15 +0800,
> >>>> Wang, Yalin wrote:
> >>>>>
> >>>>> Hi  Takashi,
> >>>>>
> >>>>> For this one:
> >>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/s
> >>>>> o
> >>>>> u
> >>>>> nd/soc/soc-core.c?h=master#n3772
> >>>>>
> >>>>> in my kernel log,
> >>>>> the allocation size 62496,
> >>>>> about 62KB,
> >>>>
> >>>> This looks way too much than expected.  I thought each struct 
> >>>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an 
> >>>> arary of snd_soc_pcm_runtime, so this might be due to some invalid 
> >>>> values there.
> >>>>
> >>>>> I think need to  change to vmalloc .
> >>>>> How do you think of it ?
> >>>>
> >>>> This looks rather like a bug somewhere around it.
> >>>>
> >>>> Could you check the values there, sizeof(struct 
> >>>> snd_soc_pcm_runtime),
> >>>> card->num_links and card->num_aux_devs?
> >>>>
> >>>> And, with which hardware is the issue examined?
> >>>>
> >>>>
> >>>> thanks,
> >>>>
> >>>> Takashi
> >>>>
> >>>>>
> >>>>> Logs:
> >>>>> <4>[   11.469789] ------------[ cut here ]------------
> >>>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
> >>>>> <4>[   11.469811] size=62496 flags=80d0
> >>>>> <4>[   11.469815] Modules linked in:
> >>>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
> >>>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
> >>>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
> >>>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
> >>>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
> >>>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
> >>>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
> >>>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
> >>>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
> >>>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
> >>>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
> >>>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
> >>>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
> >>>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
> >>>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
> >>>>>
> >>>>> BRs/ Yalin
> >>>>> -----Original Message-----
> >>>>> From: Takashi Iwai [mailto:tiwai@suse.de]
> >>>>> Sent: Friday, February 28, 2014 10:20 PM
> >>>>> To: Wang, Yalin
> >>>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
> >>>>> Subject: Re: change kmalloc into vmalloc for large memory 
> >>>>> allocations
> >>>>>
> >>>>> At Fri, 28 Feb 2014 16:15:23 +0800, Wang, Yalin wrote:
> >>>>>>
> >>>>>> Hi
> >>>>>>
> >>>>>>
> >>>>>> I find there is some drivers use kmalloc to allocate large 
> >>>>>> Memorys during module_init,  some can be changed to use vmalloc 
> >>>>>> To save some low mem, I add log in kernel to track , And list them here:
> >>>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
> >>>>>> d
> >>>>>> rive
> >>>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
> >>>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
> >>>>>> s
> >>>>>> ound
> >>>>>> /soc/soc-core.c?h=master#n3772
> >>>>>
> >>>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
> >>>>>
> >>>>>
> >>>>> thanks,
> >>>>>
> >>>>> Takashi
> >>>>>
> >>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_irc.c?h=master#n247
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_sane.c?h=master#n195
> >>>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/
> >>>>>> d
> >>>>>> rive
> >>>>>> rs/input/evdev.c?h=master#n403
> >>>>>>
> >>>>>>
> >>>>>> they allocate large memory from 10k~64K , this will use lots of 
> >>>>>> low mem, instead if we use vmalloc for these drivers , it will be 
> >>>>>> good for some devices like smart phone, we often encounter some 
> >>>>>> errors like kmalloc failed because there is not enough low mem , 
> >>>>>> especially when the device has physical memory less than 1GB .
> >>>>>>
> >>>>>>
> >>>>>> could this module change the memory allocation into vmalloc ?
> >>>>>>
> >>>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
> >>>>>>
> >>>>>> Kmalloc(size, flags)
> >>>>>> {
> >>>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
> >>>>>> 		return vmalloc(size);
> >>>>>> 	Else
> >>>>>> 		return real_kmalloc(size);
> >>>>>> }
> >>>>>>
> >>>>>> Kfree(ptr)
> >>>>>> {
> >>>>>> 	If (is_vmalloc_addr(ptr))
> >>>>>> 		Vfree(ptr);
> >>>>>> 	Else
> >>>>>> 		Kfree(ptr);
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>> But we need add some flags to ensure always use kmalloc for Some 
> >>>>>> special use (dma etc..)
> >>>>>>
> >>>>>> How do you think of it ?
> >>>>>>
> >>>>>> Thanks
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>> _______________________________________________
> >>> Alsa-devel mailing list
> >>> Alsa-devel@alsa-project.org
> >>> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
> >>>
> >>
> >
> 

Re: change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi Takashi,

I see,

I just want to save low memory for kmalloc use,
So I want to filter out large memory allocations
To use vmalloc instead ,

It will be preferable, if there is a better solution :)

Thank you for your clarification.

-----Original Message-----
From: Takashi Iwai [mailto:tiwai@suse.de] 
Sent: Monday, March 03, 2014 10:44 PM
To: Wang, Yalin
Cc: 'Lars-Peter Clausen'; 'alsa-devel@alsa-project.org'; Mark Brown; 'linux-kernel@vger.kernel.org'; Liam Girdwood
Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

At Mon, 3 Mar 2014 22:29:19 +0800,
Wang, Yalin wrote:
> 
> Hi
> 
> I see,
> Is it possible to change this part:
> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound
> /soc/soc-core.c?h=master#n3772
> 
> 
> so that it use vmalloc if it need allocate large memory for struct 
> snd_soc_pcm_runtime ?

As I mentioned, this won't help much for your problem, unless this allocation really failed.  This is performed only once at boot.
Allocating this size via kmalloc once shouldn't be a big problem for a system with 1GB RAM.  The other places allocate large buffers at each open or such frequent operation, thus they might influence more, though.

IMO, a cleaner solution would to rewrite the code to allocate each RTD dynamically at binding.  But this would need lots of fundamental code changes.


Takashi

> 
> Thanks
> 
> -----Original Message-----
> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> Sent: Monday, March 03, 2014 10:23 PM
> To: Wang, Yalin
> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
> 'linux-kernel@vger.kernel.org'; Liam Girdwood
> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large memory 
> allocations
> 
> On 03/03/2014 02:29 PM, Wang, Yalin wrote:
> > Hi
> >
> >
> > I find the num_links are defined here:
> > https://www.codeaurora.org/cgit/quic/la/kernel/msm-3.10/tree/sound/s
> > oc 
> > /msm/msm8226.c?h=msm-3.10&id=d681c8e79c597a5e1ae79803dc0215d02df8fe3
> > 4#
> > n1812
> >
> >
> > the ARRAY_SIZE is 42 , you mean the driver can't define so many elements ?
> > how to implement it if we can't do like this ?
> >
> 
> Looks like your platform is a bit special in this regard and requires quite a few more links than usual.
> 
> > Thanks
> >
> > -----Original Message-----
> > From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> > Sent: Monday, March 03, 2014 7:46 PM
> > To: Wang, Yalin
> > Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
> > 'linux-kernel@vger.kernel.org'; Liam Girdwood
> > Subject: Re: [alsa-devel] change kmalloc into vmalloc for large 
> > memory allocations
> >
> > On 03/03/2014 12:36 PM, Wang, Yalin wrote:
> >> Hi
> >>
> >> I am curious what number range is correct here for card->num_links ?
> >
> > It depends on your board. But typically I'd say <= 5, for most boards it's either 1 or 2.
> >
> >>
> >>
> >> Thanks
> >>
> >> -----Original Message-----
> >> From: Lars-Peter Clausen [mailto:lars@metafoo.de]
> >> Sent: Monday, March 03, 2014 7:35 PM
> >> To: Wang, Yalin
> >> Cc: 'Takashi Iwai'; 'alsa-devel@alsa-project.org'; Mark Brown; 
> >> 'linux-kernel@vger.kernel.org'; Liam Girdwood
> >> Subject: Re: [alsa-devel] change kmalloc into vmalloc for large 
> >> memory allocations
> >>
> >> On 03/03/2014 12:13 PM, Wang, Yalin wrote:
> >>> Hi
> >>>
> >>>
> >>> I add logs in kernel , the result is :
> >>>
> >>> 532 <4>[    5.690094] @@ sizeof(struct snd_soc_pcm_runtime)=1488 card->num_links=42 card->num_aux_devs=0
> >>>
> >>> card->num_links=42 , is it correct ?
> >>
> >> Very unlikely. Make sure that it is properly initialized in your machine driver.
> >>
> >>>
> >>> our hardware is qcom msm8x26 platform .
> >>
> >> This one doesn't seem to be in upstream.
> >>
> >>>
> >>> Thanks
> >>>
> >>> -----Original Message-----
> >>> From: Takashi Iwai [mailto:tiwai@suse.de]
> >>> Sent: Monday, March 03, 2014 5:23 PM
> >>> To: Wang, Yalin
> >>> Cc: 'linux-kernel@vger.kernel.org'; Mark Brown; 'perex@perex.cz'; 
> >>> 'alsa-devel@alsa-project.org'; Liam Girdwood
> >>> Subject: Re: change kmalloc into vmalloc for large memory 
> >>> allocations
> >>>
> >>> At Mon, 03 Mar 2014 10:19:58 +0100, Takashi Iwai wrote:
> >>>>
> >>>> [Dropped unrelated subsystem MLs, and corrected Liam's address]
> >>>
> >>> Grrr, even Mark's address was obsoleted.  Here corrected.
> >>>
> >>>
> >>> Takashi
> >>>
> >>>>
> >>>> At Mon, 3 Mar 2014 09:55:15 +0800, Wang, Yalin wrote:
> >>>>>
> >>>>> Hi  Takashi,
> >>>>>
> >>>>> For this one:
> >>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree
> >>>>> /s
> >>>>> o
> >>>>> u
> >>>>> nd/soc/soc-core.c?h=master#n3772
> >>>>>
> >>>>> in my kernel log,
> >>>>> the allocation size 62496,
> >>>>> about 62KB,
> >>>>
> >>>> This looks way too much than expected.  I thought each struct 
> >>>> snd_soc_pcm_runtime should be 1 or 2kB.  The code allocating an 
> >>>> arary of snd_soc_pcm_runtime, so this might be due to some 
> >>>> invalid values there.
> >>>>
> >>>>> I think need to  change to vmalloc .
> >>>>> How do you think of it ?
> >>>>
> >>>> This looks rather like a bug somewhere around it.
> >>>>
> >>>> Could you check the values there, sizeof(struct 
> >>>> snd_soc_pcm_runtime),
> >>>> card->num_links and card->num_aux_devs?
> >>>>
> >>>> And, with which hardware is the issue examined?
> >>>>
> >>>>
> >>>> thanks,
> >>>>
> >>>> Takashi
> >>>>
> >>>>>
> >>>>> Logs:
> >>>>> <4>[   11.469789] ------------[ cut here ]------------
> >>>>> <4>[   11.469803] WARNING: at /media/00DE7FE2DE7FCE82/jb-mr2-yukon/kernel/include/linux/slub_def.h:265 __kmalloc+0x38/0x29c()
> >>>>> <4>[   11.469811] size=62496 flags=80d0
> >>>>> <4>[   11.469815] Modules linked in:
> >>>>> <4>[   11.469835] [<c010c534>] (unwind_backtrace+0x0/0x11c) from [<c018ab04>] (warn_slowpath_common+0x4c/0x64)
> >>>>> <4>[   11.469850] [<c018ab04>] (warn_slowpath_common+0x4c/0x64) from [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c)
> >>>>> <4>[   11.469864] [<c018ab9c>] (warn_slowpath_fmt+0x2c/0x3c) from [<c0249ab4>] (__kmalloc+0x38/0x29c)
> >>>>> <4>[   11.469879] [<c0249ab4>] (__kmalloc+0x38/0x29c) from [<c06a0124>] (snd_soc_register_card+0x184/0x104c)
> >>>>> <4>[   11.469896] [<c06a0124>] (snd_soc_register_card+0x184/0x104c) from [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0)
> >>>>> <4>[   11.469913] [<c088183c>] (msm8226_asoc_machine_probe+0x298/0x7f0) from [<c044b1fc>] (platform_drv_probe+0x14/0x18)
> >>>>> <4>[   11.469927] [<c044b1fc>] (platform_drv_probe+0x14/0x18) from [<c0449f2c>] (driver_probe_device+0x134/0x334)
> >>>>> <4>[   11.469940] [<c0449f2c>] (driver_probe_device+0x134/0x334) from [<c044a194>] (__driver_attach+0x68/0x8c)
> >>>>> <4>[   11.469953] [<c044a194>] (__driver_attach+0x68/0x8c) from [<c044849c>] (bus_for_each_dev+0x48/0x80)
> >>>>> <4>[   11.469965] [<c044849c>] (bus_for_each_dev+0x48/0x80) from [<c0449528>] (bus_add_driver+0x100/0x26c)
> >>>>> <4>[   11.469978] [<c0449528>] (bus_add_driver+0x100/0x26c) from [<c044a66c>] (driver_register+0x9c/0x120)
> >>>>> <4>[   11.469992] [<c044a66c>] (driver_register+0x9c/0x120) from [<c010052c>] (do_one_initcall+0x90/0x160)
> >>>>> <4>[   11.470007] [<c010052c>] (do_one_initcall+0x90/0x160) from [<c0d00b74>] (kernel_init+0xec/0x1a8)
> >>>>> <4>[   11.470022] [<c0d00b74>] (kernel_init+0xec/0x1a8) from [<c0106aec>] (kernel_thread_exit+0x0/0x8)
> >>>>> <4>[   11.470030] ---[ end trace 1b75b31a2719ed4d ]---
> >>>>>
> >>>>> BRs/ Yalin
> >>>>> -----Original Message-----
> >>>>> From: Takashi Iwai [mailto:tiwai@suse.de]
> >>>>> Sent: Friday, February 28, 2014 10:20 PM
> >>>>> To: Wang, Yalin
> >>>>> Cc: 'linux-kernel@vger.kernel.org'; 'linux-arm-msm@vger.kernel.org'; 'linux-arm-kernel@lists.infradead.org'; 'linux-input@vger.kernel.org'; 'balbi@ti.com'; 'gregkh@linuxfoundation.org'; 'lrg@ti.com'; 'broonie@opensource.wolfsonmicro.com'; 'perex@perex.cz'; 'pablo@netfilter.org'; 'kaber@trash.net'; 'davem@davemloft.net'; 'rostedt@goodmis.org'; 'fweisbec@gmail.com'; 'mingo@redhat.com'; 'dmitry.torokhov@gmail.com'; 'rydberg@euromail.se'; 'linux-usb@vger.kernel.org'; 'alsa-devel@alsa-project.org'; 'netfilter-devel@vger.kernel.org'; 'netfilter@vger.kernel.org'; 'coreteam@netfilter.org'; 'netdev@vger.kernel.org'
> >>>>> Subject: Re: change kmalloc into vmalloc for large memory 
> >>>>> allocations
> >>>>>
> >>>>> At Fri, 28 Feb 2014 16:15:23 +0800, Wang, Yalin wrote:
> >>>>>>
> >>>>>> Hi
> >>>>>>
> >>>>>>
> >>>>>> I find there is some drivers use kmalloc to allocate large 
> >>>>>> Memorys during module_init,  some can be changed to use vmalloc 
> >>>>>> To save some low mem, I add log in kernel to track , And list them here:
> >>>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tre
> >>>>>> e/
> >>>>>> d
> >>>>>> rive
> >>>>>> rs/usb/gadget/f_mass_storage.c?h=master#n2724
> >>>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tre
> >>>>>> e/
> >>>>>> s
> >>>>>> ound
> >>>>>> /soc/soc-core.c?h=master#n3772
> >>>>>
> >>>>> At least the ASoC runtime case doesn't use the allocated memory as buffer, and they are allocated only once per device, thus it shouldn't be the problem you stated.  If it really consumes so much memory, we need to rethink, instead of allocating an array but allocate each object, for example.
> >>>>>
> >>>>>
> >>>>> thanks,
> >>>>>
> >>>>> Takashi
> >>>>>
> >>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tre
> >>>>>> e/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_ftp.c?h=master#n603
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tre
> >>>>>> e/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_h323_main.c?h=master#n1849
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tre
> >>>>>> e/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_irc.c?h=master#n247
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tre
> >>>>>> e/
> >>>>>> n
> >>>>>> et/n
> >>>>>> etfilter/nf_conntrack_sane.c?h=master#n195
> >>>>>>
> >>>>>> https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tre
> >>>>>> e/
> >>>>>> d
> >>>>>> rive
> >>>>>> rs/input/evdev.c?h=master#n403
> >>>>>>
> >>>>>>
> >>>>>> they allocate large memory from 10k~64K , this will use lots of 
> >>>>>> low mem, instead if we use vmalloc for these drivers , it will 
> >>>>>> be good for some devices like smart phone, we often encounter 
> >>>>>> some errors like kmalloc failed because there is not enough low 
> >>>>>> mem , especially when the device has physical memory less than 1GB .
> >>>>>>
> >>>>>>
> >>>>>> could this module change the memory allocation into vmalloc ?
> >>>>>>
> >>>>>> I was thinking that if we can introduce a helper function in kmalloc.h like this :
> >>>>>>
> >>>>>> Kmalloc(size, flags)
> >>>>>> {
> >>>>>> 	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
> >>>>>> 		return vmalloc(size);
> >>>>>> 	Else
> >>>>>> 		return real_kmalloc(size);
> >>>>>> }
> >>>>>>
> >>>>>> Kfree(ptr)
> >>>>>> {
> >>>>>> 	If (is_vmalloc_addr(ptr))
> >>>>>> 		Vfree(ptr);
> >>>>>> 	Else
> >>>>>> 		Kfree(ptr);
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>> But we need add some flags to ensure always use kmalloc for 
> >>>>>> Some special use (dma etc..)
> >>>>>>
> >>>>>> How do you think of it ?
> >>>>>>
> >>>>>> Thanks
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>> _______________________________________________
> >>> Alsa-devel mailing list
> >>> Alsa-devel@alsa-project.org
> >>> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
> >>>
> >>
> >
> 

Re: [alsa-devel] change kmalloc into vmalloc for large memory allocations

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]

On Mon, Mar 03, 2014 at 11:33:24PM +0800, Wang, Yalin wrote:
> Hi Takashi,
> 
> I see,
> 
> I just want to save low memory for kmalloc use,
> So I want to filter out large memory allocations
> To use vmalloc instead ,
> 
> It will be preferable, if there is a better solution :)
> 
> Thank you for your clarification.

Don't top post.  

Without having seen the earlier mails in the thread I'm not 100% sure on
the specifics here but wouldn't a more generic solution be to deal with
this in the memory management code?  

As far as I can tell the concern is that the runtime structures are
taking up too much memory when allocated using kzallloc?  We're also
doing lots of small allocations for things like DAPM which will
presumably also be burning this memory and will I expect for most
systems end up being more than that consumed by the runtime structures.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH] netfilter:Change nf_conntrack modules to use vmalloc

[Wang, Yalin]

This patch change some nf_conntrack modules to
use vmalloc to allocate large memory instead of kmalloc,
to save low memorys.

Signed-off-by: Yalin Wang <yalin.wang@sonymobile.com>
---
 net/netfilter/nf_conntrack_ftp.c       | 6 +++---
 net/netfilter/nf_conntrack_h323_main.c | 8 ++++----
 net/netfilter/nf_conntrack_irc.c       | 6 +++---
 net/netfilter/nf_conntrack_sane.c      | 6 +++---
 4 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/net/netfilter/nf_conntrack_ftp.c b/net/netfilter/nf_conntrack_ftp.c
index b8a0924..aad7eca 100644
--- a/net/netfilter/nf_conntrack_ftp.c
+++ b/net/netfilter/nf_conntrack_ftp.c
@@ -14,7 +14,7 @@
 #include <linux/moduleparam.h>
 #include <linux/netfilter.h>
 #include <linux/ip.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 #include <linux/ipv6.h>
 #include <linux/ctype.h>
 #include <linux/inet.h>
@@ -593,14 +593,14 @@ static void nf_conntrack_ftp_fini(void)
 		}
 	}
 
-	kfree(ftp_buffer);
+	vfree(ftp_buffer);
 }
 
 static int __init nf_conntrack_ftp_init(void)
 {
 	int i, j = -1, ret = 0;
 
-	ftp_buffer = kmalloc(65536, GFP_KERNEL);
+	ftp_buffer = vmalloc(65536);
 	if (!ftp_buffer)
 		return -ENOMEM;
 
diff --git a/net/netfilter/nf_conntrack_h323_main.c b/net/netfilter/nf_conntrack_h323_main.c
index 70866d1..5baee1a 100644
--- a/net/netfilter/nf_conntrack_h323_main.c
+++ b/net/netfilter/nf_conntrack_h323_main.c
@@ -18,7 +18,7 @@
 #include <linux/inet.h>
 #include <linux/in.h>
 #include <linux/ip.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 #include <linux/udp.h>
 #include <linux/tcp.h>
 #include <linux/skbuff.h>
@@ -1837,7 +1837,7 @@ static void __exit nf_conntrack_h323_fini(void)
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_q931[1]);
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_q931[0]);
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_h245);
-	kfree(h323_buffer);
+	vfree(h323_buffer);
 	pr_debug("nf_ct_h323: fini\n");
 }
 
@@ -1846,7 +1846,7 @@ static int __init nf_conntrack_h323_init(void)
 {
 	int ret;
 
-	h323_buffer = kmalloc(65536, GFP_KERNEL);
+	h323_buffer = vmalloc(65536);
 	if (!h323_buffer)
 		return -ENOMEM;
 	ret = nf_conntrack_helper_register(&nf_conntrack_helper_h245);
@@ -1876,7 +1876,7 @@ err3:
 err2:
 	nf_conntrack_helper_unregister(&nf_conntrack_helper_h245);
 err1:
-	kfree(h323_buffer);
+	vfree(h323_buffer);
 	return ret;
 }
 
diff --git a/net/netfilter/nf_conntrack_irc.c b/net/netfilter/nf_conntrack_irc.c
index 0fd2976..b57df10 100644
--- a/net/netfilter/nf_conntrack_irc.c
+++ b/net/netfilter/nf_conntrack_irc.c
@@ -16,7 +16,7 @@
 #include <linux/ip.h>
 #include <linux/tcp.h>
 #include <linux/netfilter.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 
 #include <net/netfilter/nf_conntrack.h>
 #include <net/netfilter/nf_conntrack_expect.h>
@@ -244,7 +244,7 @@ static int __init nf_conntrack_irc_init(void)
 	irc_exp_policy.max_expected = max_dcc_channels;
 	irc_exp_policy.timeout = dcc_timeout;
 
-	irc_buffer = kmalloc(65536, GFP_KERNEL);
+	irc_buffer = vmalloc(65536);
 	if (!irc_buffer)
 		return -ENOMEM;
 
@@ -285,7 +285,7 @@ static void nf_conntrack_irc_fini(void)
 
 	for (i = 0; i < ports_c; i++)
 		nf_conntrack_helper_unregister(&irc[i]);
-	kfree(irc_buffer);
+	vfree(irc_buffer);
 }
 
 module_init(nf_conntrack_irc_init);
diff --git a/net/netfilter/nf_conntrack_sane.c b/net/netfilter/nf_conntrack_sane.c
index 4a2134f..a4c8bf3 100644
--- a/net/netfilter/nf_conntrack_sane.c
+++ b/net/netfilter/nf_conntrack_sane.c
@@ -20,7 +20,7 @@
 #include <linux/module.h>
 #include <linux/moduleparam.h>
 #include <linux/netfilter.h>
-#include <linux/slab.h>
+#include <linux/vmalloc.h>
 #include <linux/in.h>
 #include <linux/tcp.h>
 #include <net/netfilter/nf_conntrack.h>
@@ -185,14 +185,14 @@ static void nf_conntrack_sane_fini(void)
 		}
 	}
 
-	kfree(sane_buffer);
+	vfree(sane_buffer);
 }
 
 static int __init nf_conntrack_sane_init(void)
 {
 	int i, j = -1, ret = 0;
 
-	sane_buffer = kmalloc(65536, GFP_KERNEL);
+	sane_buffer = vmalloc(65536);
 	if (!sane_buffer)
 		return -ENOMEM;
 
-- 
1.8.2.2

change kmalloc into vmalloc for large memory allocations

[Wang, Yalin]

Hi 


I find there is some drivers use kmalloc to allocate large 
Memorys during module_init,  some can be changed to use vmalloc
To save some low mem, I add log in kernel to track ,
And list them here:

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/usb/gadget/f_mass_storage.c?h=master#n2724

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/sound/soc/soc-core.c?h=master#n3772

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_ftp.c?h=master#n603
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_h323_main.c?h=master#n1849
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_irc.c?h=master#n247
https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/net/netfilter/nf_conntrack_sane.c?h=master#n195

https://git.kernel.org/cgit/linux/kernel/git/will/linux.git/tree/drivers/input/evdev.c?h=master#n403


they allocate large memory from 10k~64K , 
this will use lots of low mem, instead if we use 
vmalloc for these drivers , it will be good for some devices
like smart phone, we often encounter some errors like kmalloc failed 
because there is not enough low mem , especially when the device has physical 
memory less than 1GB .


could this module change the memory allocation into vmalloc ?

I was thinking that if we can introduce a helper function in kmalloc.h like this :

Kmalloc(size, flags)
{
	If (size > PAGE_SIZE && flags&CAN_USE_VMALLOC_FLAG)
		return vmalloc(size);
	Else
		return real_kmalloc(size);
}

Kfree(ptr)
{
	If (is_vmalloc_addr(ptr))
		Vfree(ptr);
	Else
		Kfree(ptr);
}


But we need add some flags to ensure always use kmalloc for
Some special use (dma etc..)

How do you think of it ?

Thanks