From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B488CA90AF for ; Wed, 13 May 2020 11:13:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D0DB420740 for ; Wed, 13 May 2020 11:13:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729686AbgEMLNa (ORCPT ); Wed, 13 May 2020 07:13:30 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:47620 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726645AbgEMLNa (ORCPT ); Wed, 13 May 2020 07:13:30 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: gtucker) with ESMTPSA id BFAF72711FF To: kernelci@groups.io, kernel-build-reports@lists.linaro.org, automated-testing@lists.yoctoproject.org, linux-next@vger.kernel.org, stable@vger.kernel.org, Ard Biesheuvel , agross@kernel.org, qcomlt-patches@lists.linaro.org, "linux-gpio@vger.kernel.org" , Tony Lindgren , ulf.hansson@linaro.org, "linux-arm-kernel@lists.infradead.org" , linux-pm@vger.kernel.org, vireshk@kernel.org, Krzysztof Kozlowski , kernel@collabora.com, kernelci@baylibre.com, "linux-kernel@vger.kernel.org" From: Guillaume Tucker Subject: kernelci.org transitioning to functional testing Message-ID: <66aae710-1ee9-fb67-1a1b-997eeb70dc04@collabora.com> Date: Wed, 13 May 2020 12:13:24 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-next-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-next@vger.kernel.org As kernelci.org is expanding its functional testing capabilities, the concept of boot testing is now being deprecated. Next Monday 18th May, the web dashboard on https://kernelci.org will be updated to primarily show functional test results rather than boot results. The Boots tab will still be available until 5th June to ease the transition. The new equivalent to boot testing is the *baseline* test suite which also runs sanity checks using dmesg and bootrr[1]. Boot email reports will eventually be replaced with baseline reports. For those of you already familiar with the test email reports, they will be simplified to only show regressions with links to the dashboard for all the details. Some functional tests are already being run by kernelci.org, results have only been shared by email so far but they will become visible on the web dashboard next week. In particular: v4l2-compliance, i-g-t for DRM/KMS and Panfrost, suspend/resume... And of course, a lot of functional test suites are in the process of being added: kselftest, KUnit, LTP, xfstests, extended i-g-t coverage and many more. The detailed schedule is available on a GitHub issue[2]. Please let us know if you have any questions, comments or concerns either in this thread, on kernelci@groups.io or IRC #kernelci on Freenode. Stay tuned! Thanks, Guillaume [1] bootrr: https://github.com/kernelci/bootrr [2] schedule: https://github.com/kernelci/kernelci-backend/issues/238