linux-next: manual merge of the kmemcheck tree with the tracing tree

linux-next: manual merge of the kmemcheck tree with the tracing tree

[Stephen Rothwell]

Hi all,

Today's linux-next merge of the kmemcheck tree got a conflict in
include/linux/ring_buffer.h between commit
334d4169a6592d3fcd863bbe822a8f6985ffa9af ("ring_buffer: compressed event
header") from the tracing tree and commits
9b7ff384ee76ced9638ab236db588a6f13916336 ("trace: annotate bitfields in
struct ring_buffer_event") and 3467e18b1cf34c7d316af5717e7053ce845d014e
("kmemcheck: make bitfield annotations be valid C") from the kmemcheck
tree.

I fixed it up (see below) and can carry the fix as necessary.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc include/linux/ring_buffer.h
index f134582,771ee90..0000000
--- a/include/linux/ring_buffer.h
+++ b/include/linux/ring_buffer.h
@@@ -11,7 -12,10 +12,10 @@@ struct ring_buffer_iter
   * Don't refer to this struct directly, use functions below.
   */
  struct ring_buffer_event {
+ 	kmemcheck_bitfield_begin(bitfield);
 -	u32		type:2, len:3, time_delta:27;
 +	u32		type_len:5, time_delta:27;
+ 	kmemcheck_bitfield_end(bitfield);
+ 
  	u32		array[];
  };
  

Re: linux-next: manual merge of the kmemcheck tree with the tracing tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

Hi Steve,

On Mon, 01 Jun 2009 10:33:19 -0400 Steven Rostedt <srostedt@redhat.com> wrote:
>
> >  +	/*
> >  +	 * The actual tail page has moved forward.
> >  +	 */
> >  +	if (tail < BUF_PAGE_SIZE) {
> >  +		/* Mark the rest of the page with padding */
> >  +		event = __rb_page_index(tail_page, tail);
> > ++		kmemcheck_annotate_bitfield(event, bitfield);
> 
> Is this...

Just this (since the code was outdented and moved slightly).

> > @@@ -1311,12 -1289,9 +1313,13 @@@
> >   		return NULL;
> >   
> >   	event = __rb_page_index(tail_page, tail);
> > + 	kmemcheck_annotate_bitfield(event, bitfield);
> 
> and this the only changes you needed to fix?

This one git merged on its own.

The "++" is the clue.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: manual merge of the kmemcheck tree with the tracing tree

[Ingo Molnar]

* Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi all,
> 
> Today's linux-next merge of the kmemcheck tree got a conflict in
> kernel/trace/ring_buffer.c between commit
> aa20ae8444fc6c318272c643f856d8d8ad3e198d ("ring-buffer: move big if
> statement down") from the tracing tree and commits
> 9b7ff384ee76ced9638ab236db588a6f13916336 ("trace: annotate bitfields in
> struct ring_buffer_event") and 3467e18b1cf34c7d316af5717e7053ce845d014e
> ("kmemcheck: make bitfield annotations be valid C") from the kmemcheck
> tree.
> 
> I fixed it up (see below) and can carry the fix as necessary.

Would be nice if you indicated whether you cross-checked it against 
tip:master, which had most of these conflicts resolved already (for 
weeks).

( this has relevance for the x86 and tracing tree conflicts - 
  kmemleak is not in -tip)

	Ingo

Re: linux-next: manual merge of the kmemcheck tree with the tracing tree

[Steven Rostedt]

On Mon, 2009-06-01 at 17:55 +1000, Stephen Rothwell wrote:
> Hi all,
> 
> Today's linux-next merge of the kmemcheck tree got a conflict in
> kernel/trace/ring_buffer.c between commit
> aa20ae8444fc6c318272c643f856d8d8ad3e198d ("ring-buffer: move big if
> statement down") from the tracing tree and commits
> 9b7ff384ee76ced9638ab236db588a6f13916336 ("trace: annotate bitfields in
> struct ring_buffer_event") and 3467e18b1cf34c7d316af5717e7053ce845d014e
> ("kmemcheck: make bitfield annotations be valid C") from the kmemcheck
> tree.
> 
> I fixed it up (see below) and can carry the fix as necessary.

I'm not great at reading git conflict diffs.

> -- 
> Cheers,
> Stephen Rothwell                    sfr@canb.auug.org.au
> 
> diff --cc kernel/trace/ring_buffer.c
> index 3611706,c22506f..0000000
> --- a/kernel/trace/ring_buffer.c
> +++ b/kernel/trace/ring_buffer.c
> @@@ -1154,156 -1156,132 +1155,157 @@@ static unsigned rb_calculate_event_leng
>   	return length;
>   }

[...]

>  +	/*
>  +	 * The actual tail page has moved forward.
>  +	 */
>  +	if (tail < BUF_PAGE_SIZE) {
>  +		/* Mark the rest of the page with padding */
>  +		event = __rb_page_index(tail_page, tail);
> ++		kmemcheck_annotate_bitfield(event, bitfield);

Is this...

>  +		rb_event_set_padding(event);
>  +	}
>   
>  -		/*
>  -		 * The actual tail page has moved forward.
>  -		 */
>  -		if (tail < BUF_PAGE_SIZE) {
>  -			/* Mark the rest of the page with padding */
>  -			event = __rb_page_index(tail_page, tail);
>  -			kmemcheck_annotate_bitfield(event, bitfield);
>  -			rb_event_set_padding(event);
>  -		}

[...]

>   
> @@@ -1311,12 -1289,9 +1313,13 @@@
>   		return NULL;
>   
>   	event = __rb_page_index(tail_page, tail);
> + 	kmemcheck_annotate_bitfield(event, bitfield);

and this the only changes you needed to fix?

-- Steve

>   	rb_update_event(event, type, length);
>   
>  +	/* The passed in type is zero for DATA */
>  +	if (likely(!type))
>  +		local_inc(&tail_page->entries);
>  +
>   	/*
>   	 * If this is a commit and the tail is zero, then update
>   	 * this page's time stamp.

linux-next: manual merge of the kmemcheck tree with the tracing tree

[Stephen Rothwell]

Hi all,

Today's linux-next merge of the kmemcheck tree got a conflict in
kernel/trace/ring_buffer.c between commit
aa20ae8444fc6c318272c643f856d8d8ad3e198d ("ring-buffer: move big if
statement down") from the tracing tree and commits
9b7ff384ee76ced9638ab236db588a6f13916336 ("trace: annotate bitfields in
struct ring_buffer_event") and 3467e18b1cf34c7d316af5717e7053ce845d014e
("kmemcheck: make bitfield annotations be valid C") from the kmemcheck
tree.

I fixed it up (see below) and can carry the fix as necessary.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

diff --cc kernel/trace/ring_buffer.c
index 3611706,c22506f..0000000
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@@ -1154,156 -1156,132 +1155,157 @@@ static unsigned rb_calculate_event_leng
  	return length;
  }
  
 +
  static struct ring_buffer_event *
 -__rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer,
 -		  unsigned type, unsigned long length, u64 *ts)
 +rb_move_tail(struct ring_buffer_per_cpu *cpu_buffer,
 +	     unsigned long length, unsigned long tail,
 +	     struct buffer_page *commit_page,
 +	     struct buffer_page *tail_page, u64 *ts)
  {
 -	struct buffer_page *tail_page, *head_page, *reader_page, *commit_page;
 -	unsigned long tail, write;
 +	struct buffer_page *next_page, *head_page, *reader_page;
  	struct ring_buffer *buffer = cpu_buffer->buffer;
  	struct ring_buffer_event *event;
 -	unsigned long flags;
  	bool lock_taken = false;
 +	unsigned long flags;
  
 -	commit_page = cpu_buffer->commit_page;
 -	/* we just need to protect against interrupts */
 -	barrier();
 -	tail_page = cpu_buffer->tail_page;
 -	write = local_add_return(length, &tail_page->write);
 -	tail = write - length;
 +	next_page = tail_page;
  
 -	/* See if we shot pass the end of this buffer page */
 -	if (write > BUF_PAGE_SIZE) {
 -		struct buffer_page *next_page = tail_page;
 +	local_irq_save(flags);
 +	/*
 +	 * Since the write to the buffer is still not
 +	 * fully lockless, we must be careful with NMIs.
 +	 * The locks in the writers are taken when a write
 +	 * crosses to a new page. The locks protect against
 +	 * races with the readers (this will soon be fixed
 +	 * with a lockless solution).
 +	 *
 +	 * Because we can not protect against NMIs, and we
 +	 * want to keep traces reentrant, we need to manage
 +	 * what happens when we are in an NMI.
 +	 *
 +	 * NMIs can happen after we take the lock.
 +	 * If we are in an NMI, only take the lock
 +	 * if it is not already taken. Otherwise
 +	 * simply fail.
 +	 */
 +	if (unlikely(in_nmi())) {
 +		if (!__raw_spin_trylock(&cpu_buffer->lock)) {
 +			cpu_buffer->nmi_dropped++;
 +			goto out_reset;
 +		}
 +	} else
 +		__raw_spin_lock(&cpu_buffer->lock);
  
 -		local_irq_save(flags);
 -		/*
 -		 * Since the write to the buffer is still not
 -		 * fully lockless, we must be careful with NMIs.
 -		 * The locks in the writers are taken when a write
 -		 * crosses to a new page. The locks protect against
 -		 * races with the readers (this will soon be fixed
 -		 * with a lockless solution).
 -		 *
 -		 * Because we can not protect against NMIs, and we
 -		 * want to keep traces reentrant, we need to manage
 -		 * what happens when we are in an NMI.
 -		 *
 -		 * NMIs can happen after we take the lock.
 -		 * If we are in an NMI, only take the lock
 -		 * if it is not already taken. Otherwise
 -		 * simply fail.
 -		 */
 -		if (unlikely(in_nmi())) {
 -			if (!__raw_spin_trylock(&cpu_buffer->lock))
 -				goto out_reset;
 -		} else
 -			__raw_spin_lock(&cpu_buffer->lock);
 +	lock_taken = true;
  
 -		lock_taken = true;
 +	rb_inc_page(cpu_buffer, &next_page);
  
 -		rb_inc_page(cpu_buffer, &next_page);
 +	head_page = cpu_buffer->head_page;
 +	reader_page = cpu_buffer->reader_page;
  
 -		head_page = cpu_buffer->head_page;
 -		reader_page = cpu_buffer->reader_page;
 +	/* we grabbed the lock before incrementing */
 +	if (RB_WARN_ON(cpu_buffer, next_page == reader_page))
 +		goto out_reset;
  
 -		/* we grabbed the lock before incrementing */
 -		if (RB_WARN_ON(cpu_buffer, next_page == reader_page))
 -			goto out_reset;
 +	/*
 +	 * If for some reason, we had an interrupt storm that made
 +	 * it all the way around the buffer, bail, and warn
 +	 * about it.
 +	 */
 +	if (unlikely(next_page == commit_page)) {
 +		cpu_buffer->commit_overrun++;
 +		goto out_reset;
 +	}
  
 -		/*
 -		 * If for some reason, we had an interrupt storm that made
 -		 * it all the way around the buffer, bail, and warn
 -		 * about it.
 -		 */
 -		if (unlikely(next_page == commit_page)) {
 -			WARN_ON_ONCE(1);
 +	if (next_page == head_page) {
 +		if (!(buffer->flags & RB_FL_OVERWRITE))
  			goto out_reset;
 -		}
 -
 -		if (next_page == head_page) {
 -			if (!(buffer->flags & RB_FL_OVERWRITE))
 -				goto out_reset;
  
 -			/* tail_page has not moved yet? */
 -			if (tail_page == cpu_buffer->tail_page) {
 -				/* count overflows */
 -				rb_update_overflow(cpu_buffer);
 +		/* tail_page has not moved yet? */
 +		if (tail_page == cpu_buffer->tail_page) {
 +			/* count overflows */
 +			cpu_buffer->overrun +=
 +				local_read(&head_page->entries);
  
 -				rb_inc_page(cpu_buffer, &head_page);
 -				cpu_buffer->head_page = head_page;
 -				cpu_buffer->head_page->read = 0;
 -			}
 +			rb_inc_page(cpu_buffer, &head_page);
 +			cpu_buffer->head_page = head_page;
 +			cpu_buffer->head_page->read = 0;
  		}
 +	}
  
 -		/*
 -		 * If the tail page is still the same as what we think
 -		 * it is, then it is up to us to update the tail
 -		 * pointer.
 -		 */
 -		if (tail_page == cpu_buffer->tail_page) {
 -			local_set(&next_page->write, 0);
 -			local_set(&next_page->page->commit, 0);
 -			cpu_buffer->tail_page = next_page;
 +	/*
 +	 * If the tail page is still the same as what we think
 +	 * it is, then it is up to us to update the tail
 +	 * pointer.
 +	 */
 +	if (tail_page == cpu_buffer->tail_page) {
 +		local_set(&next_page->write, 0);
 +		local_set(&next_page->entries, 0);
 +		local_set(&next_page->page->commit, 0);
 +		cpu_buffer->tail_page = next_page;
 +
 +		/* reread the time stamp */
 +		*ts = ring_buffer_time_stamp(buffer, cpu_buffer->cpu);
 +		cpu_buffer->tail_page->page->time_stamp = *ts;
 +	}
  
 -			/* reread the time stamp */
 -			*ts = ring_buffer_time_stamp(buffer, cpu_buffer->cpu);
 -			cpu_buffer->tail_page->page->time_stamp = *ts;
 -		}
 +	/*
 +	 * The actual tail page has moved forward.
 +	 */
 +	if (tail < BUF_PAGE_SIZE) {
 +		/* Mark the rest of the page with padding */
 +		event = __rb_page_index(tail_page, tail);
++		kmemcheck_annotate_bitfield(event, bitfield);
 +		rb_event_set_padding(event);
 +	}
  
 -		/*
 -		 * The actual tail page has moved forward.
 -		 */
 -		if (tail < BUF_PAGE_SIZE) {
 -			/* Mark the rest of the page with padding */
 -			event = __rb_page_index(tail_page, tail);
 -			kmemcheck_annotate_bitfield(event, bitfield);
 -			rb_event_set_padding(event);
 -		}
 +	/* Set the write back to the previous setting */
 +	local_sub(length, &tail_page->write);
  
 -		if (tail <= BUF_PAGE_SIZE)
 -			/* Set the write back to the previous setting */
 -			local_set(&tail_page->write, tail);
 +	/*
 +	 * If this was a commit entry that failed,
 +	 * increment that too
 +	 */
 +	if (tail_page == cpu_buffer->commit_page &&
 +	    tail == rb_commit_index(cpu_buffer)) {
 +		rb_set_commit_to_write(cpu_buffer);
 +	}
  
 -		/*
 -		 * If this was a commit entry that failed,
 -		 * increment that too
 -		 */
 -		if (tail_page == cpu_buffer->commit_page &&
 -		    tail == rb_commit_index(cpu_buffer)) {
 -			rb_set_commit_to_write(cpu_buffer);
 -		}
 +	__raw_spin_unlock(&cpu_buffer->lock);
 +	local_irq_restore(flags);
 +
 +	/* fail and let the caller try again */
 +	return ERR_PTR(-EAGAIN);
 +
 + out_reset:
 +	/* reset write */
 +	local_sub(length, &tail_page->write);
  
 +	if (likely(lock_taken))
  		__raw_spin_unlock(&cpu_buffer->lock);
 -		local_irq_restore(flags);
 +	local_irq_restore(flags);
 +	return NULL;
 +}
  
 -		/* fail and let the caller try again */
 -		return ERR_PTR(-EAGAIN);
 -	}
 +static struct ring_buffer_event *
 +__rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer,
 +		  unsigned type, unsigned long length, u64 *ts)
 +{
 +	struct buffer_page *tail_page, *commit_page;
 +	struct ring_buffer_event *event;
 +	unsigned long tail, write;
 +
 +	commit_page = cpu_buffer->commit_page;
 +	/* we just need to protect against interrupts */
 +	barrier();
 +	tail_page = cpu_buffer->tail_page;
 +	write = local_add_return(length, &tail_page->write);
 +	tail = write - length;
 +
 +	/* See if we shot pass the end of this buffer page */
 +	if (write > BUF_PAGE_SIZE)
 +		return rb_move_tail(cpu_buffer, length, tail,
 +				    commit_page, tail_page, ts);
  
  	/* We reserved something on the buffer */
  
@@@ -1311,12 -1289,9 +1313,13 @@@
  		return NULL;
  
  	event = __rb_page_index(tail_page, tail);
+ 	kmemcheck_annotate_bitfield(event, bitfield);
  	rb_update_event(event, type, length);
  
 +	/* The passed in type is zero for DATA */
 +	if (likely(!type))
 +		local_inc(&tail_page->entries);
 +
  	/*
  	 * If this is a commit and the tail is zero, then update
  	 * this page's time stamp.

Re: linux-next: manual merge of the kmemcheck tree with the tracing tree

[Ingo Molnar]

* Vegard Nossum <vegard.nossum@gmail.com> wrote:

> 2009/3/6 Stephen Rothwell <sfr@canb.auug.org.au>:
> > Hi all,
> >
> > Today's linux-next merge of the kmemcheck tree got a conflict in
> > kernel/trace/ring_buffer.c between commit
> > a81bd80a0b0a405dc0483e2c428332d69da2c79f ("ring-buffer: use generic
> > version of in_nmi") from the tracing tree and commit
> > 9b7ff384ee76ced9638ab236db588a6f13916336 ("trace: annotate bitfields in
> > struct ring_buffer_event") from the kmemcheck tree.
> >
> > Just simple overlapping additions.  I fixed it up (see below) and can
> > carry the fix as necessary.
> > --
> > Cheers,
> > Stephen Rothwell                    sfr@canb.auug.org.au
> > http://www.canb.auug.org.au/~sfr/
> >
> > diff --cc kernel/trace/ring_buffer.c
> > index f747364,b1f2f60..0000000
> > --- a/kernel/trace/ring_buffer.c
> > +++ b/kernel/trace/ring_buffer.c
> > @@@ -9,7 -7,7 +9,8 @@@
> >  #include <linux/spinlock.h>
> >  #include <linux/debugfs.h>
> >  #include <linux/uaccess.h>
> >  +#include <linux/hardirq.h>
> > + #include <linux/kmemcheck.h>
> >  #include <linux/module.h>
> >  #include <linux/percpu.h>
> >  #include <linux/mutex.h>
> >
> 
> Isn't it amazing how we both managed to put our new include in 
> exactly the same spot? :-D

Yeah - i tend to shuffle patches and order include file sections 
of headers to reduce it - but most of the time it doesnt matter 
as fixing such a conflict is about 2 seconds.

> Anyway, thanks for fixing these up, it looks good to me!

I suspect it's the same resolution like what i did in tip:master 
a few days ago or so? I dont send out notifications about 
trivial conflicts, only about non-trivial conflicts that i'm 
unsure about. That's rare - when i see a non-trivial conflict i 
work on eliminating it.

	Ingo

Re: linux-next: manual merge of the kmemcheck tree with the tracing tree

[Vegard Nossum]

2009/3/6 Stephen Rothwell <sfr@canb.auug.org.au>:
> Hi all,
>
> Today's linux-next merge of the kmemcheck tree got a conflict in
> kernel/trace/ring_buffer.c between commit
> a81bd80a0b0a405dc0483e2c428332d69da2c79f ("ring-buffer: use generic
> version of in_nmi") from the tracing tree and commit
> 9b7ff384ee76ced9638ab236db588a6f13916336 ("trace: annotate bitfields in
> struct ring_buffer_event") from the kmemcheck tree.
>
> Just simple overlapping additions.  I fixed it up (see below) and can
> carry the fix as necessary.
> --
> Cheers,
> Stephen Rothwell                    sfr@canb.auug.org.au
> http://www.canb.auug.org.au/~sfr/
>
> diff --cc kernel/trace/ring_buffer.c
> index f747364,b1f2f60..0000000
> --- a/kernel/trace/ring_buffer.c
> +++ b/kernel/trace/ring_buffer.c
> @@@ -9,7 -7,7 +9,8 @@@
>  #include <linux/spinlock.h>
>  #include <linux/debugfs.h>
>  #include <linux/uaccess.h>
>  +#include <linux/hardirq.h>
> + #include <linux/kmemcheck.h>
>  #include <linux/module.h>
>  #include <linux/percpu.h>
>  #include <linux/mutex.h>
>

Isn't it amazing how we both managed to put our new include in exactly
the same spot? :-D

Anyway, thanks for fixing these up, it looks good to me!


Vegard

-- 
"The animistic metaphor of the bug that maliciously sneaked in while
the programmer was not looking is intellectually dishonest as it
disguises that the error is the programmer's own creation."
	-- E. W. Dijkstra, EWD1036

linux-next: manual merge of the kmemcheck tree with the tracing tree

[Stephen Rothwell]

Hi all,

Today's linux-next merge of the kmemcheck tree got a conflict in
kernel/trace/ring_buffer.c between commit
a81bd80a0b0a405dc0483e2c428332d69da2c79f ("ring-buffer: use generic
version of in_nmi") from the tracing tree and commit
9b7ff384ee76ced9638ab236db588a6f13916336 ("trace: annotate bitfields in
struct ring_buffer_event") from the kmemcheck tree.

Just simple overlapping additions.  I fixed it up (see below) and can
carry the fix as necessary.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

diff --cc kernel/trace/ring_buffer.c
index f747364,b1f2f60..0000000
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@@ -9,7 -7,7 +9,8 @@@
  #include <linux/spinlock.h>
  #include <linux/debugfs.h>
  #include <linux/uaccess.h>
 +#include <linux/hardirq.h>
+ #include <linux/kmemcheck.h>
  #include <linux/module.h>
  #include <linux/percpu.h>
  #include <linux/mutex.h>