All of lore.kernel.org
 help / color / mirror / Atom feed
From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: Patrick Ohly <patrick.ohly@intel.com>,
	Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [PATCH 1/2] commands: send stderr to a new pipe
Date: Thu, 22 Jun 2017 17:59:35 +0100	[thread overview]
Message-ID: <1498150775.24449.120.camel@linuxfoundation.org> (raw)
In-Reply-To: <1498147164.22706.30.camel@intel.com>

On Thu, 2017-06-22 at 17:59 +0200, Patrick Ohly wrote:
> On Thu, 2017-06-22 at 10:37 -0500, Leonardo Sandoval wrote:
> > 
> > On Thu, 2017-06-22 at 17:14 +0200, Patrick Ohly wrote:
> > > 
> > > On Thu, 2017-06-22 at 09:58 -0500, Leonardo Sandoval wrote:
> > > > 
> > > > On Thu, 2017-06-22 at 16:17 +0200, Patrick Ohly wrote:
> > > > > 
> > > > > On Mon, 2017-06-19 at 07:39 -0700,
> > > > > leonardo.sandoval.gonzalez@linux.intel.com wrote:
> > > > > > 
> > > > > > From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.i
> > > > > > ntel.com>
> > > > > > 
> > > > > > Do not mix the stderr into stdout, allowing test cases to
> > > > > > query
> > > > > > the specific output.
> > > > > This changes the behavior of functions that are also used
> > > > > outside of
> > > > > OE-core in a way that won't be easy to notice. I also don't
> > > > > think that
> > > > > it is the right default. For example, for bitbake it is
> > > > > easier to
> > > > > understand where an error occurred when stderr goes to the
> > > > > same stream
> > > > > as stdout.
> > > > how would that make it easier?
> > > Because then output will be properly interleaved, as it would be
> > > on a
> > > console.
> > > 
> > > Actually, the entire error reporting in runCmd() only prints
> > > result.output, so with stderr going to result.error by default,
> > > you
> > > won't get the actual errors reported anymore at all, will you? 
> > > 
> > process stderr will go into result.error and process stdout into
> > result.output. So when the process is executed ignoring the return
> > status, then test must check result.error. I find the latter
> > cleaner
> > that checking errors into stdout.
> It depends on how the result is used. That you prefer split output
> for some tests does not mean that everyone wants the same in their
> tests. I don't want it in my own usage of runCmd() or bitbake()
> because I don't care about where a message was printed. I just want
> it in proper order.
> 
> If you change the default, then you will also have to enhance
> runCmd()'s error handling to include results.error. That's currently
> missing in your patch.
> 
> > 
> > > 
> > > > 
> > > > > 
> > > > > Can't you keep the current semantic and just override it
> > > > > explicitly in
> > > > > those tests that need separate stdout/stderr?
> > > > > 
> > > > My proposed patch was mainly based on a RP's comment [1],
> > > > suggesting to
> > > > split std[out|err].
> > > He did not suggest to change the default behavior. I agree that
> > > using
> > > split stdout/stderr in those specific tests which specifically
> > > want to
> > > check for error messages makes sense, but only in those tests.
> > No tests require splitting the output (all tests pass with and
> > without
> > this series). The series is actually an enhancement and without it,
> > we
> > saw (specially when the python 2 to 3 was going on) past warnings
> > going
> > into stdout, so Chris and RP suggested the approach.
> Richard, can you please comment on whether changing the default is
> really what you meant?

The problem here is that stdout and stderr are being mixed and that
mixing is not accounting for newlines. bitbake itself does print
different messages to both. The mixing depends on kernel page size and
when the buffers are flushed.

The key thing I'm not sure about from reading the bug is whether this
output is from the same bitbake process or different ones?

I can think of three things we can do here:

a) Add some flush calls somewhere assuming this is different calls.
That somehow seems unlikely

b) Keep stdout and stderr separate as per Leo's change

c) Have a command version which merges stdout and stderr respecting
newlines. I suspect this happens on a console but likely not on inter
process pipes...

It does sound like we need to answer some questions on what exactly is
happening before we can know which solution(s) might work.

Cheers,

Richard




  parent reply	other threads:[~2017-06-22 16:59 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-19 14:39 [PATCH 1/2] commands: send stderr to a new pipe leonardo.sandoval.gonzalez
2017-06-19 14:39 ` [PATCH 2/2] selftest/cases: use stderr data when querying for errors leonardo.sandoval.gonzalez
2017-06-21 10:56 ` [PATCH 1/2] commands: send stderr to a new pipe Jussi Kukkonen
2017-06-21 11:08   ` Burton, Ross
2017-06-21 15:01   ` Leonardo Sandoval
2017-06-22 14:17 ` Patrick Ohly
2017-06-22 14:58   ` Leonardo Sandoval
2017-06-22 15:14     ` Patrick Ohly
2017-06-22 15:37       ` Leonardo Sandoval
2017-06-22 15:59         ` Patrick Ohly
2017-06-22 16:18           ` Leonardo Sandoval
2017-06-22 17:39             ` Patrick Ohly
2017-06-22 20:47               ` Leonardo Sandoval
2017-06-22 21:07                 ` Patrick Ohly
2017-06-22 21:27                   ` Leonardo Sandoval
2017-06-22 16:59           ` Richard Purdie [this message]
2017-06-22 19:35             ` Patrick Ohly
2017-06-22 20:51               ` Leonardo Sandoval

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1498150775.24449.120.camel@linuxfoundation.org \
    --to=richard.purdie@linuxfoundation.org \
    --cc=leonardo.sandoval.gonzalez@linux.intel.com \
    --cc=openembedded-core@lists.openembedded.org \
    --cc=patrick.ohly@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.