From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B065C00140 for ; Fri, 12 Aug 2022 21:12:31 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 7E1568493D; Fri, 12 Aug 2022 23:12:29 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=amarulasolutions.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=amarulasolutions.com header.i=@amarulasolutions.com header.b="IjEVxeIT"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id D2C2B84A54; Fri, 12 Aug 2022 23:12:27 +0200 (CEST) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 45FB0848E0 for ; Fri, 12 Aug 2022 23:12:25 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=amarulasolutions.com Authentication-Results: phobos.denx.de; spf=none smtp.helo=mail-wr1-x429.google.com Received: by mail-wr1-x429.google.com with SMTP id q30so2360571wra.11 for ; Fri, 12 Aug 2022 14:12:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amarulasolutions.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=qSiyadV5GfuQFqAX3CME+xNzRKxTD07Hh5o0RybgM2o=; b=IjEVxeITqJC2ysy4MRgJ9ITFEgvX83YSDNUvvSM5wh6MRWA0G67nthKI4JO7Zx+N7y iT5KqUdCYlaJJ4InwQw2XKpgRapdbsYXAdx3zoL/H/k8HqP0GedJO0riu8Ce3lFpRKDA Qeza9LqLfUTOXeRxLht84Mpt7qvgxSg2SKQkk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=qSiyadV5GfuQFqAX3CME+xNzRKxTD07Hh5o0RybgM2o=; b=qQX/R/Ayc5S2MGTG5OoXY0/oJiMtppjhGKL08vkwrlZH50eoX821SI6zHJL8UQI13d 0gtbv2VuzRSOFUuNBjB6LlR3vNrn0xQWhOCCBO8mOd/EJq44cUdmsasGdWPnSLEKdO0c YHFcC2iE0snkIQUbDxXuzDSw9RAUH5oMaDJ1bWADke4ffxuHRAgmNaN00LqjaedEp7ar YVm+nPwiRKOBNcbmDau1N7NN+CUcHiIzdcI1kT1hSEPI3Rz+AeKXC92iYvlH/xgiy39z OaiCAXtjix0zwB3YMRbOn4IaxOfbC5zn9AMH+NkzATvSNTxZ/cU69Nnsu6v/SnVEqgx5 8Ejw== X-Gm-Message-State: ACgBeo3vG/69m8UvX2RLCLUb/vwAnXjK1WvbDlSwKIRm4fwdk4dZdaV3 EQFVWc5XyoCRmNQba5ALhvqhOw== X-Google-Smtp-Source: AA6agR6rROclAJjQ2mRVwkeuizFgHpZQ4LpEO4nSLJlhFwjgJE0Ehv4GzYtaYBBzWGgffT0HqJo36g== X-Received: by 2002:a5d:64c9:0:b0:220:79cc:b7c9 with SMTP id f9-20020a5d64c9000000b0022079ccb7c9mr2819723wri.103.1660338744888; Fri, 12 Aug 2022 14:12:24 -0700 (PDT) Received: from pwmachine.localnet (85-170-37-153.rev.numericable.fr. [85.170.37.153]) by smtp.gmail.com with ESMTPSA id e5-20020adfe7c5000000b0021e5cc26dd0sm371789wrn.62.2022.08.12.14.12.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 14:12:24 -0700 (PDT) From: Francis Laniel To: Tom Rini Cc: u-boot@lists.denx.de, Marek Behun , Michael Nazzareno Trimarchi , Simon Glass , Wolfgang Denk , Harald Seiler Subject: Re: [RFC PATCH v4 26/28] for test purpose only: Comment out dollar tests which prints error messages. Date: Fri, 12 Aug 2022 23:12:23 +0200 Message-ID: <13077013.uLZWGnKmhe@pwmachine> In-Reply-To: <20220617140235.GW2484912@bill-the-cat> References: <20220616223158.9225-1-francis.laniel@amarulasolutions.com> <20220616223158.9225-27-francis.laniel@amarulasolutions.com> <20220617140235.GW2484912@bill-the-cat> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean Le vendredi 17 juin 2022, 16:02:35 CEST Tom Rini a =E9crit : > On Fri, Jun 17, 2022 at 12:31:56AM +0200, Francis Laniel wrote: > > Error messages like "unknown command" will make the CI fails. > > So, for the moment, we comment these tests. > >=20 > > Signed-off-by: Francis Laniel >=20 > Why don't we have the test expect "unknown command" as the output here, > when on the newer shell? Or if both shells print that now, in both > cases? We want tests for when things are expected to fail. If you run `ut hush` from U-Boot prompt, this test does not fail (I added t= he=20 failure on purpose to test that it should fail). But when run from the CI, it is run within test.py which checks from "unkno= wn=20 command" to not be present in the output [1]. I am not really familiar with the python testing framework but I think I ca= n=20 find another solution using disable_check() when running this test with 202= 1=20 shell. =2D-- [1] https://elixir.bootlin.com/u-boot/v2022.07/source/test/py/ u_boot_console_base.py#L35