Kernel Newbies archive on lore.kernel.org
 help / color / Atom feed
* radix_tree_next_chunk: redundant search for next slot in hole
@ 2019-05-03 21:03 Probir Roy
  2019-05-03 22:38 ` Valdis Klētnieks
  0 siblings, 1 reply; 4+ messages in thread
From: Probir Roy @ 2019-05-03 21:03 UTC (permalink / raw)
  To: kernelnewbies, willy

Hi,

I am running Phoronix-fio benchmark on Linux kernel 4.18.0-rc5. I
observe that radix_tree_next_chunk search shows some form of a
redundant walk while called from radix_tree_for_each_slot iterator.
While searching for next slot in a hole, it walks through the same
slots over n over.

void __rcu **radix_tree_next_chunk(const struct radix_tree_root *root,
                 struct radix_tree_iter *iter, unsigned flags)
{
...
      if ((flags & RADIX_TREE_ITER_TAGGED) ?
                !tag_get(node, tag, offset) : !child) {
         ...
          while (++offset < RADIX_TREE_MAP_SIZE) {
                    void *slot = rcu_dereference_raw(   /* redundant
slot walk */
                            node->slots[offset]);
                    if (slot)
                        break;
                }
        }
...
}

Can you please explain such behavior? Any feedback/recommendation
about this case is very welcome.



-- 
Probir Roy

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: radix_tree_next_chunk: redundant search for next slot in hole
  2019-05-03 21:03 radix_tree_next_chunk: redundant search for next slot in hole Probir Roy
@ 2019-05-03 22:38 ` Valdis Klētnieks
  2019-05-04  0:00   ` Probir Roy
  0 siblings, 1 reply; 4+ messages in thread
From: Valdis Klētnieks @ 2019-05-03 22:38 UTC (permalink / raw)
  To: Probir Roy; +Cc: willy, kernelnewbies

[-- Attachment #1.1: Type: text/plain, Size: 722 bytes --]

On Fri, 03 May 2019 16:03:46 -0500, Probir Roy said:

> While searching for next slot in a hole, it walks through the same
> slots over n over.

How did you determine this?

>           while (++offset < RADIX_TREE_MAP_SIZE) {
>                     void *slot = rcu_dereference_raw(   /* redundant slot walk */
>                             node->slots[offset]);
>                     if (slot)
>                         break;
>                 }

Looks to me like the ++offset will walk through each potential slot once,
and break out if it finds one.

I haven't looked at the code closely, perhaps what you're seeing is repeated
scan/merge/rescan behavior?  Often, compacting a data structure requires
multiple passes.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: radix_tree_next_chunk: redundant search for next slot in hole
  2019-05-03 22:38 ` Valdis Klētnieks
@ 2019-05-04  0:00   ` Probir Roy
  2019-05-04  0:56     ` Valdis Klētnieks
  0 siblings, 1 reply; 4+ messages in thread
From: Probir Roy @ 2019-05-04  0:00 UTC (permalink / raw)
  To: Valdis Klētnieks; +Cc: kernelnewbies

> > While searching for next slot in a hole, it walks through the same
> > slots over n over.
>
> How did you determine this?

I am working on a tool that identifies repeated load of an address.
Often these repeated loads are redundant and can be avoided with data
structure modification. The tool points me to this line.

> Looks to me like the ++offset will walk through each potential slot once,
> and break out if it finds one.


This function is being called by the radix_tree_for_each_slot
iterator, defined as follows:

#define radix_tree_for_each_slot(slot, root, iter, start)       \
for (slot = radix_tree_iter_init(iter, start) ;         \
     slot || (slot = radix_tree_next_chunk(root, iter, 0)) ;    \
//   <<<<-------^^^
     slot = radix_tree_next_slot(slot, iter, 0))

Here is the calling context I get:
|_ depth: 1 :0, method: ext4_block_write_begin+0x335/0x4f0(),
  |_ depth: 2 :0, method: alloc_buffer_head+0x21/0x60(),
   |_ depth: 3 :0, method: ext4_da_get_block_prep+0x1a6/0x490(),
    |_ depth: 4 :0, method: clean_bdev_aliases+0x9a/0x210(),
     |_ depth: 5 :0, method: pagevec_lookup_range+0x24/0x30(),
      |_ depth: 6 :0, method: find_get_pages_range+0x151/0x2d0(),
       |_ depth: 7 :0, method: radix_tree_next_chunk+0x10f/0x360()

Does it explain the case?

> I haven't looked at the code closely, perhaps what you're seeing is repeated
> scan/merge/rescan behavior?  Often, compacting a data structure requires
> multiple passes.

If that's the case, can this repeated search for the next valid slot
be avoided by caching/storing the next valid slot?

--
Probir Roy

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: radix_tree_next_chunk: redundant search for next slot in hole
  2019-05-04  0:00   ` Probir Roy
@ 2019-05-04  0:56     ` Valdis Klētnieks
  0 siblings, 0 replies; 4+ messages in thread
From: Valdis Klētnieks @ 2019-05-04  0:56 UTC (permalink / raw)
  To: Probir Roy; +Cc: kernelnewbies

[-- Attachment #1.1: Type: text/plain, Size: 2165 bytes --]

On Fri, 03 May 2019 19:00:26 -0500, Probir Roy said:
> > > While searching for next slot in a hole, it walks through the same
> > > slots over n over.
> >
> > How did you determine this?
>
> I am working on a tool that identifies repeated load of an address.
> Often these repeated loads are redundant and can be avoided with data
> structure modification. The tool points me to this line.

Is this doing static analysis, or actually doing run-time tracing?

> > Looks to me like the ++offset will walk through each potential slot once,
> > and break out if it finds one.
>
>
> This function is being called by the radix_tree_for_each_slot
> iterator, defined as follows:
>
> #define radix_tree_for_each_slot(slot, root, iter, start)       \
> for (slot = radix_tree_iter_init(iter, start) ;         \
>      slot || (slot = radix_tree_next_chunk(root, iter, 0)) ;    \
> //   <<<<-------^^^
>      slot = radix_tree_next_slot(slot, iter, 0))
>
> Here is the calling context I get:
> |_ depth: 1 :0, method: ext4_block_write_begin+0x335/0x4f0(),
>   |_ depth: 2 :0, method: alloc_buffer_head+0x21/0x60(),
>    |_ depth: 3 :0, method: ext4_da_get_block_prep+0x1a6/0x490(),
>     |_ depth: 4 :0, method: clean_bdev_aliases+0x9a/0x210(),
>      |_ depth: 5 :0, method: pagevec_lookup_range+0x24/0x30(),
>       |_ depth: 6 :0, method: find_get_pages_range+0x151/0x2d0(),
>        |_ depth: 7 :0, method: radix_tree_next_chunk+0x10f/0x360()
>
> Does it explain the case?

Actually, that calling context doesn't tell us much of anything till depth 7.

Yes, next_chunk() and next_slot() can get called repeatedly, especially if it's
a large radix tree. The important question is: Is it being called with the
*same value* of 'slot' repeatedly? Looking at the code, it's pretty obvious
that 'slot' will be updated at least once through every pass through the
for_each_slot(), unless the radix tree is corrupted.

If you're trying to do static analysis, your code may be confused by either the
'slot || next_chunk()' iterator, or the fact that 'slot' is assigned both in the for loops
iterator and in the body of the loop, and thus failing to detect that slot is updated.



[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, back to index

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-03 21:03 radix_tree_next_chunk: redundant search for next slot in hole Probir Roy
2019-05-03 22:38 ` Valdis Klētnieks
2019-05-04  0:00   ` Probir Roy
2019-05-04  0:56     ` Valdis Klētnieks

Kernel Newbies archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kernelnewbies/0 kernelnewbies/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kernelnewbies kernelnewbies/ https://lore.kernel.org/kernelnewbies \
		kernelnewbies@kernelnewbies.org kernelnewbies@archiver.kernel.org
	public-inbox-index kernelnewbies


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernelnewbies.kernelnewbies


AGPL code for this site: git clone https://public-inbox.org/ public-inbox