From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29737C433EF for ; Wed, 11 May 2022 18:40:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346416AbiEKSkN (ORCPT ); Wed, 11 May 2022 14:40:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240901AbiEKSkL (ORCPT ); Wed, 11 May 2022 14:40:11 -0400 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C0C922DA12; Wed, 11 May 2022 11:40:09 -0700 (PDT) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-2ef5380669cso31429877b3.9; Wed, 11 May 2022 11:40:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=gacAQnuTjXI3dz5TBJm0cWWRDuAqQVNEJfTWj1bBIpg=; b=QcTRIebnvYiCLvVVye/uet+vfZJgWbzfp1uLdVVnlMnPKm01s0LuUJUZ91Xq0MB9Vr UvJ31JfGq+SEWZ/FnBrJstqzQahaDs4gfi2jYP96zXZMoyWJRCbQjC+8BTGNblnigo4i TvhQFLN+5/vohU/bN8z9bLHVLaZLJS3z+dZYTZdnb3KD65mJqrn37NewG3VMLPxk0JdJ hSRNeZyyAddkDt1GafBCleGW7l8iaZWLLpop8J/VUfnRHMwixxlz9JXsSWtrgqV4zwJx 4xhFLuYsbOhpVnyJBYcM/wKPrOu/YABlYqo35H2KKherBlAV90VG0HVQlT3v75juxVZX EfNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gacAQnuTjXI3dz5TBJm0cWWRDuAqQVNEJfTWj1bBIpg=; b=wLjFVD9jh2PGwCuon2WfVO0CdF2NPaDOKsdQ86fp+W/hNdF6AbKS0ITmIlUoQ9ZBfb VNOAhf2PiSzx33HMajBtKqG3Djv9Foy6nwoC/+X8XMs+2uiFsaU4D1D3e0yugLmZERdD 4SBPuhDGVFYYwBY7szBu5tWyl6C+3LEc9txy9jvFGSvSfKqAmfR4uEctxHde8lGJI7Ts jgILNd3o1Wup6wbiP/O6612ZNYhWmkfp+yx0U/lMSn5FJ/rBvuCcdmJ9x/EkjJKB2qPq JA8UEdh9H9xC+/dK1xeqGKkwEiNb4c9IOBq06HuGgtfOnTtX8OukvqmPwaay9RIrd/Hj XhlA== X-Gm-Message-State: AOAM532r1K/vwomugWu8AwwbpaGdF7ZAPB7rDhWQ3n6Pbt4eRDoMhTUT lbVYow2ZFOrpJJ8eQQhLFkiScW7uRz/KcPLaapc= X-Google-Smtp-Source: ABdhPJyMUehcufRfIX14setohpSGBpeP18SXvT0KMG1vCIhPypdsZ3bdLIhs3tcx/6SKNJM6VYHywe/u8k/I9as5dlI= X-Received: by 2002:a0d:e645:0:b0:2f4:dbd6:261e with SMTP id p66-20020a0de645000000b002f4dbd6261emr26676104ywe.7.1652294408357; Wed, 11 May 2022 11:40:08 -0700 (PDT) MIME-Version: 1.0 References: <20220510070140.45407-1-tomeu.vizoso@collabora.com> <20220510141329.54414-1-tomeu.vizoso@collabora.com> In-Reply-To: From: Rob Clark Date: Wed, 11 May 2022 11:39:56 -0700 Message-ID: Subject: Re: Adding CI results to the kernel tree was Re: [RFC v2] drm/msm: Add initial ci/ subdirectory To: Linus Torvalds Cc: Dave Airlie , Tomeu Vizoso , Greg Kroah-Hartman , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Daniel Vetter , Jonathan Corbet , Sean Paul , Abhinav Kumar , "open list:DOCUMENTATION" , linux-arm-msm , LKML , dri-devel , freedreno Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Wed, May 11, 2022 at 10:33 AM Linus Torvalds wrote: > > On Tue, May 10, 2022 at 10:07 PM Dave Airlie wrote: > > > > > And use it to store expectations about what the drm/msm driver is > > > supposed to pass in the IGT test suite. > > > > I wanted to loop in Linus/Greg to see if there are any issues raised > > by adding CI results file to the tree in their minds, or if any other > > subsystem has done this already, and it's all fine. > > > > I think this is a good thing after our Mesa experience, but Mesa has a > > lot tighter integration here, so I want to get some more opinions > > outside the group. > > Honestly, my immediate reaction is that I think it might be ok, but > > (a) are these things going to absolutely balloon over time? > > (b) should these not be separated out? > > Those two issues kind of interact. > > If it's a small and targeted test-suite, by all means keep it in the > kernel, but why not make it part of "tools/testing/selftests" > > But if people expect this to balloon and we end up having megabytes of > test output, then I really think it should be a separate git tree. > > A diffstat like this: > > > 7 files changed, 791 insertions(+) > > is not a problem at all. But I get the feeling that this is just the > tip of the iceberg, and people will want to not just have the result > files, but start adding actual *input* files that may be largely > automated stuff and may be tens of megabytes in size. > > Because the result files on their own aren't really self-contained, > and then people will want to keep them in sync with the test-files > themselves, and start adding those, and now it *really* is likely very > unwieldy. It is missing in this revision of the RFC, but the intention is to have the gitlab-ci.yml point to a specific commit SHA in the gfx-ci/drm-ci[1] tree, to solve the problem of keeping the results in sync with the expectations. Ie. a kernel commit would control moving to a new version of i-g-t (and eventually deqp and/or piglit), and at the same time make any necessary updates in the expectations files. BR, -R [1] https://gitlab.freedesktop.org/gfx-ci/drm-ci > Or if that doesn't happen, and the actual input test files stay in a > separate CI repo, and then you end up having random coherency issues > with that CI repo, and it all gets to be either horribly messy, or the > result files in the kernel end up really stale. > > So honestly, I personally don't see a good end result here. This > particular small patch? *This* one looks fine to me, except I really > think tools/testing/selftests/gpu would be a much more logical place > for it. > > But I don't see a way forward that is sane. > > Can somebody argue otherwise? > > Linus