Re: [PATCH v5 1/5] iotests: remove 'linux' from default supported platforms

[Kevin Wolf <kwolf@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 4048 bytes --]

Am 02.10.2019 um 19:47 hat Max Reitz geschrieben:
> On 02.10.19 18:44, Kevin Wolf wrote:
> > Am 02.10.2019 um 13:57 hat Max Reitz geschrieben:
> >> It usually worked fine for me because it’s rather rare that non-block
> >> patches broke the iotests.
> > 
> > I disagree. It happened all the time that someone else broke the iotests
> > in master and we needed to fix it up.
> 
> OK.
> 
> (In my experience, it’s still mostly block patches, only that they tend
> to go through non-block trees.)

Fair enough, it's usually code that touches block code. I assumed "block
patches" to mean patches that go through one of the block trees and for
which iotests are run before sending a pull request.

In the end, I don't care what code these patches touched. I do an
innocent git pull, and when I finally see that it's master that breaks
iotests and not my patches on top of it, I'm annoyed.

> >> Maybe my main problem is that I feel like now I have to deal with all of
> >> the fallout, even though adding the iotests to make check wasn’t my idea
> >> and neither me nor Kevin ever consented.  (I don’t know whether Kevin
> >> consented implicitly, but I don’t see his name in bdd95e47844f2d8b.)
> >>
> >> You can’t just give me more responsibility without my consent and then
> >> call me egoistic for not liking it.
> > 
> > I think I may have implicitly consented by helping Alex with the changes
> > to 'check' that made its output fit better in 'make check'.
> > 
> > Basically, my stance is that I share your dislike for the effects on me
> > personally (also, I can't run 'make check' any more before a pull
> > request without testing half of the iotests twice because I still need a
> > full iotests run), but I also think that having iotests in 'make check'
> > is really the right thing for the project despite the inconvenience it
> > means for me.
> 
> Then I expect you to NACK Thomas’s patch, and I take it to mean that I
> can rely on you to handle basically all make check iotests failures that
> are not caused by my own pull requests.
> 
> Works for me.

Ah, we can also play this game the other way round.

So if you merge that revert, when iotests break in master, I take it I
can rely on you to fix it up before it impacts my working branch?

> >> I know you’ll say that we just need to ensure we can add more tests,
> >> then.  But for one thing, the most important tests are the ones that
> >> take the longest, like 041.  And the other of course is that adding any
> >> more tests to make check just brings me more pain, so I won’t do it.
> > 
> > That's something that can be fixed by tweaking the auto group.
> > 
> > Can we run tests in 'auto' that require the system emulator? If so,
> > let's add 030 040 041 to the default set. They take long, but they are
> > among the most valuable tests we have. If this makes the tests take too
> > much time, let's remove some less important ones instead.
> > 
> >> [1] There is the recent case of Kevin’s pull request having been
> >> rejected because his test failed on anything but x64.  I’m torn here,
> >> because I would have seen that failure on my -m32 build.  So it isn’t
> >> like it would have evaded our view for long.
> > 
> > I messed up, so this pull request was rightly stopped. I consider this
> > one a feature, not a bug.
> 
> Sure, that was a good thing.  I just wonder whether that instance is
> enough to justify the pain.

Yes, making iotests stable on CI probably involves some pain, especially
initially. However, if we don't fix them upstream, they'll end up on our
desk again downstream because people run tests neither on your nor on my
laptop.

I think we might actually save time by fixing them once and for all,
even if they are problems in the test cases and not in QEMU, and making
new test cases portable from the start, instead of getting individual
reports one at a time and having to look at the same test cases again
and again.

Kevin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]