Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?

Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?

[孙世龙]

[-- Attachment #1.1: Type: text/plain, Size: 2373 bytes --]

Hi,

Why does "page allocation failure" occur whereas there are still "58*4096kB
(C)"(*I think it indicates there are 58 order 10 memory could be used*)
could be used?

Here is the related log:

[ 2161.623563] xxxx: page allocation failure: order:10,
mode:0x2084020(GFP_ATOMIC|__GFP_COMP)
[ 2161.632085] CPU: 0 PID: 179 Comm: AiApp Not tainted 4.9.56 #53
[ 2161.637947]
Call Trace:
[<802f63f2>] dump_stack+0x1e/0x3c
[<800f6cf4>] warn_alloc+0x100/0x148
[<800f709c>] __alloc_pages_nodemask+0x2bc/0xb5c
[<801120fe>] kmalloc_order+0x26/0x48
[<80112158>] kmalloc_order_trace+0x38/0x98
[<8012c5d8>] __kmalloc+0xf4/0x12c
[<8048ac78>] alloc_ep_req+0x5c/0x98
[<8048f232>] source_sink_recv+0x2a/0xe0
[<8048f35e>] usb_sourcesink_bulk_read+0x76/0x1c8
[<8048f770>] usb_sourcesink_read+0xfc/0x2c8
[<80134d58>] __vfs_read+0x30/0x108
[<80135c14>] vfs_read+0x94/0x128
[<80136d12>] SyS_read+0x52/0xd4
[<8004a246>] csky_systemcall+0x96/0xe0
[ 2161.689204] Mem-Info:
[ 2161.691518] active_anon:3268 inactive_anon:2 isolated_anon:0
[ 2161.691518]  active_file:1271 inactive_file:89286 isolated_file:0
[ 2161.691518]  unevictable:0 dirty:343 writeback:0 unstable:0
[ 2161.691518]  slab_reclaimable:2019 slab_unreclaimable:644
[ 2161.691518]  mapped:4282 shmem:4 pagetables:59 bounce:0
[ 2161.691518]  free:62086 free_pcp:199 free_cma:60234

[ 2161.724334] Node 0 active_anon:13072kB inactive_anon:8kB
active_file:5084kB inactive_file:357144kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB mapped:17128kB dirty:1372kB
writeback:0kB shmem:16kB writeback_tmp:0kB unstable:0kB
pages_scanned:0 all_unreclaimable? no

[ 2161.748626] Normal free:248344kB min:2444kB low:3052kB high:3660kB
active_anon:13072kB inactive_anon:8kB active_file:5084kB
inactive_file:357144kB unevictable:0kB writepending:1372kB
present:1048572kB managed:734568kB mlocked:0kB slab_reclaimable:8076kB
slab_unreclaimable:2576kB kernel_stack:608kB pagetables:236kB
bounce:0kB free_pcp:796kB local_pcp:796kB free_cma:240936kB
[ 2161.781670] lowmem_reserve[]: 0 0 0

[ 2161.785225] Normal: 4*4kB (UEC) 3*8kB (EC) 3*16kB (UEC) 2*32kB (UE)
2*64kB (UE) 2*128kB (UE) 2*256kB (EC) 1*512kB (E) 3*1024kB (UEC)
3*2048kB (UEC) 58*4096kB (C) = 248344kB
90573 total pagecache pages

[ 2161.803526] 262143 pages RAM
[ 2161.806410] 0 pages HighMem/MovableOnly
[ 2161.810264] 78501 pages reserved
[ 2161.813509] 90112 pages cma reserved

[-- Attachment #1.2: Type: text/html, Size: 4537 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?

[Valdis Klētnieks]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1: Type: text/plain; charset=us-ascii, Size: 1346 bytes --]

On Thu, 18 Jun 2020 14:21:05 +0800, 孙世龙 said:

> Why does "page allocation failure" occur whereas there are still "58*4096kB
> (C)"(*I think it indicates there are 58 order 10 memory could be used*)
> could be used?
>
> Here is the related log:
>
> [ 2161.623563] xxxx: page allocation failure: order:10,
> mode:0x2084020(GFP_ATOMIC|__GFP_COMP)

Most likely, the allocation wanted some other type of allocation.
The (C) on the order-10 says it's an CMA area.

        static const char types[MIGRATE_TYPES] = {
                [MIGRATE_UNMOVABLE]     = 'U',
                [MIGRATE_MOVABLE]       = 'M',
                [MIGRATE_RECLAIMABLE]   = 'E',
                [MIGRATE_HIGHATOMIC]    = 'H',
#ifdef CONFIG_CMA
                [MIGRATE_CMA]           = 'C',
#endif
#ifdef CONFIG_MEMORY_ISOLATION
                [MIGRATE_ISOLATE]       = 'I',
#endif

If the call was for an unmovable, movable, reclaimable, or highatomic
allocation, you lose.

If you look at the source for alloc_ap_req(), you find it wants GFP_ATOMIC, not
CMA.  And your box is fresh out of contiguous order-10 spaces that aren't CMA,
and you're down to your last 3 order-9 flagged as (UEC).

I admit I find it a tad suspicious that the USB gadget driver asks for a 4M
chunk of memory.  Does USB actually support single transfers that large? (I'm
not a USB expert)


[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?

[孙世龙 sunshilong]

[-- Attachment #1.1: Type: text/plain, Size: 2757 bytes --]

>> Why does "page allocation failure" occur whereas there are still
"58*4096kB
>> (C)"(*I think it indicates there are 58 order 10 memory could be used*)
>> could be used?
>>
>> Here is the related log:
>>
>> [ 2161.623563] xxxx: page allocation failure: order:10,
>> mode:0x2084020(GFP_ATOMIC|__GFP_COMP)
>
>If you look at the source for alloc_ap_req(), you find it wants
GFP_ATOMIC, not
>CMA.  And your box is fresh out of contiguous order-10 spaces that aren't
CMA,
>and you're down to your last 3 order-9 flagged as (UEC).

Thank you for the clarification.
I understand it on a deeper level with your help.

Why doesn't the kernel use two memory blocks whose size is 2048KB(i.e.*oder
9 *)
instead of one block *order 10 *(you see, there are still three free blocks
and
 2048KB*2=4096KB equivalent to the memory size of order 10)?

>If you look at the source for alloc_ap_req(), you find it wants
GFP_ATOMIC, not
>CMA.
I followed your advice and read the related source code carefully.
It's corresponding to the log(i.e.   mode:0x2084020(GFP_ATOMIC|__GFP_COMP)
).

Thank you for your attention to this matter.
Look forward to hearing from you.
Best regards.

Valdis Klētnieks <valdis.kletnieks@vt.edu> 于2020年6月19日周五 下午12:48写道：

> On Thu, 18 Jun 2020 14:21:05 +0800, sunshiong said:
>
> > Why does "page allocation failure" occur whereas there are still
> "58*4096kB
> > (C)"(*I think it indicates there are 58 order 10 memory could be used*)
> > could be used?
> >
> > Here is the related log:
> >
> > [ 2161.623563] xxxx: page allocation failure: order:10,
> > mode:0x2084020(GFP_ATOMIC|__GFP_COMP)
>
> Most likely, the allocation wanted some other type of allocation.
> The (C) on the order-10 says it's an CMA area.
>
>         static const char types[MIGRATE_TYPES] = {
>                 [MIGRATE_UNMOVABLE]     = 'U',
>                 [MIGRATE_MOVABLE]       = 'M',
>                 [MIGRATE_RECLAIMABLE]   = 'E',
>                 [MIGRATE_HIGHATOMIC]    = 'H',
> #ifdef CONFIG_CMA
>                 [MIGRATE_CMA]           = 'C',
> #endif
> #ifdef CONFIG_MEMORY_ISOLATION
>                 [MIGRATE_ISOLATE]       = 'I',
> #endif
>
> If the call was for an unmovable, movable, reclaimable, or highatomic
> allocation, you lose.
>
> If you look at the source for alloc_ap_req(), you find it wants
> GFP_ATOMIC, not
> CMA.  And your box is fresh out of contiguous order-10 spaces that aren't
> CMA,
> and you're down to your last 3 order-9 flagged as (UEC).
>
> I admit I find it a tad suspicious that the USB gadget driver asks for a 4M
> chunk of memory.  Does USB actually support single transfers that large?
> (I'm
> not a USB expert)
>
>

[-- Attachment #1.2: Type: text/html, Size: 3732 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?

[Valdis Klētnieks]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1: Type: text/plain; charset=us-ascii, Size: 1015 bytes --]

On Fri, 19 Jun 2020 14:56:20 +0800, 孙世龙 sunshilong said:

> Why doesn't the kernel use two memory blocks whose size is 2048KB(i.e.*oder 9 *)
> instead of one block *order 10 *(you see, there are still three free blocks and
>  2048KB*2=4096KB equivalent to the memory size of order 10)?

Most parts of the kernel, when asking for very high-order allocations, *will*
have a fallback strategy to use smaller chunks. So, for instance,  if a device
need a 1M buffer and supports scatter-gather operations, if 1M of contiguous
memory isn't available, the kernel can ask for 4 256K chunks and have the I/O
directed into the 4 areas.  However, if the memory *has* to be contiguous (for
example, no scatter/gather available, or it's for an array data structure),
then it can't do that.

And in fact, that fallback could very well have happened in this case - I
didn't bother chasing back to see if the gadget driver does recovery by
allocating multiple smaller chunks.

(That's a good "exercise for the student"... :)

[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?

[孙世龙 sunshilong]

[-- Attachment #1.1: Type: text/plain, Size: 2347 bytes --]

>> Why doesn't the kernel use two memory blocks whose size is
2048KB(i.e.*oder 9 *)
>> instead of one block *order 10 *(you see, there are still three free
blocks and
>>  2048KB*2=4096KB equivalent to the memory size of order 10)?
>
>Most parts of the kernel, when asking for very high-order allocations,
*will*
>have a fallback strategy to use smaller chunks. So, for instance,  if a
device
>need a 1M buffer and supports scatter-gather operations, if 1M of
contiguous
>memory isn't available, the kernel can ask for 4 256K chunks and have the
I/O
>directed into the 4 areas.  *However, if the memory *has* to be contiguous
(for*
>*example, no scatter/gather available, or it's for an array data
structure),*
>then it can't do that.

Thank you for the clarification.
I understand it on a deeper level with your help.

How can I know whether scatter/gather is available or not?
In another word, when it's available and when it's not?
I do not intend to ask the behavior of gadget driver.
I just wonder how I can confirm it in general.

Thank you for your attention to this matter.
Look forward to hearing from you.
Best regards.

Valdis Klētnieks <valdis.kletnieks@vt.edu> 于2020年6月19日周五 下午3:14写道：

> On Fri, 19 Jun 2020 14:56:20 +0800, 孙世龙 sunshilong said:
>
> > Why doesn't the kernel use two memory blocks whose size is
> 2048KB(i.e.*oder 9 *)
> > instead of one block *order 10 *(you see, there are still three free
> blocks and
> >  2048KB*2=4096KB equivalent to the memory size of order 10)?
>
> Most parts of the kernel, when asking for very high-order allocations,
> *will*
> have a fallback strategy to use smaller chunks. So, for instance,  if a
> device
> need a 1M buffer and supports scatter-gather operations, if 1M of
> contiguous
> memory isn't available, the kernel can ask for 4 256K chunks and have the
> I/O
> directed into the 4 areas.  However, if the memory *has* to be contiguous
> (for
> example, no scatter/gather available, or it's for an array data structure),
> then it can't do that.
>
> And in fact, that fallback could very well have happened in this case - I
> didn't bother chasing back to see if the gadget driver does recovery by
> allocating multiple smaller chunks.
>
> (That's a good "exercise for the student"... :)
>

[-- Attachment #1.2: Type: text/html, Size: 2850 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Why does “page allocation failure” occur whereas there are still “58*4096kB (C)” could be used?

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 2001 bytes --]

> How can I know whether scatter/gather is available or not?
> In another word, when it's available and when it's not?
> I do not intend to ask the behavior of gadget driver.
> I just wonder how I can confirm it in general.

The following applies for device features in general, not just scatter/gather.

Sometimes you get lucky, and the bus standard in use (PCI, USB, SPI/I2C, etc)
mandates that the feature is supported in a standard way.

So if you're writing a device driver for a specific card that uses a Wombat
9348j chipset, you'll probably need to get in touch with the company that makes
the Wombat 9348j chipset and get the data sheet for it, and that will hopefully
have the chipset programming details needed.

If you're unlucky, the company won't give you the datasheet, or it won't
contain the info, so you get to reverse engineer the chipset. One popular
technique is to look at *other* chips in the Wombat 93 series, determine via
some means whether those chipsets support scatter-gather, and how it's enabled.

Then you put the same code in your driver, and see what happens.

Possibility 1: Scatter-gather works just fine and the 16K I/O ends up in the 4
pages you allocated. Success!

Possibility 2: It doesn't work,and the 16K of I/O hits your first 4K page, and
the next 3 sequential pages even though they're something else, and your kernel
crashes and burns because you just overlaid 12K worth of page tables or struct
tasks or something important.

Possibility 3: It doesn't work, but it takes you a while to find out because
during your tests, the 4 pages happened to be allocated next to each other.
It has happened that nobody notices until the driver is part of the mainline
kernel, which is usually embarrassing...

Possibility 4: It doesn't work, and the bit that says "use scatter/gather" in
the IO command structure on the other chipsets actually enables the
undocumented auto-defenestration capability of the chipset. This is usually
worse than just embarrassing...


[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies