From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1mIyts-0004Aj-8q for mharc-grub-devel@gnu.org; Wed, 25 Aug 2021 15:49:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48356) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mIytq-0004AT-4d for grub-devel@gnu.org; Wed, 25 Aug 2021 15:49:26 -0400 Received: from mail-qt1-x834.google.com ([2607:f8b0:4864:20::834]:46902) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mIyto-0004cZ-3G for grub-devel@gnu.org; Wed, 25 Aug 2021 15:49:25 -0400 Received: by mail-qt1-x834.google.com with SMTP id x5so455925qtq.13 for ; Wed, 25 Aug 2021 12:49:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=efficientek-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=mSscYx7dUhrzNwmjaLYDT1GyT3QRiP8QQvmoKbUg7W0=; b=V6GkNCIIIO4MV2zzhDiHDramS+jElQNG/LRTEthg9oWmR7CHp7BaS+hQz1PPeg0LOP 6GALxO+Bx3SElQnTFjeu2PvPmwrpNHOtrOIjrB5xBUdVegSwUqWlJASHmDyqWxDajor7 ydNSIJ2x6vkiIaYwerea9u2yfmyuSKr6CdJQ3+f1ENd+pVpRtSmnsodORs721w/+FL1a PwXN7zBo8KoTloOgLPpriWgCZg35RLQwKxZzmf8fZWFA21STdRGZPIDUgSyh7JwIpU4t DX10uqZNoqa/yPBGt0kvilXeYfJO538PCjxUYz7ajZYecrngkCO8efR8ZV0l5pJ0hSam 85lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=mSscYx7dUhrzNwmjaLYDT1GyT3QRiP8QQvmoKbUg7W0=; b=MTvFN60jBuXw6aQsi7kIP//Mt7hN5R7HE+R8mUQSyiLMO1zE3GiSTtdCjCC6O9R/Ql ft8knwcsm9gm8NBFqbTQtTHokNMIUmNFK94nyTnCVdddhGJ7PMvVaYlqucCyj+fG6xrx VCJpBHeiw4DzbQVazOZrwM5rcCiGuNLnDBA48UHOe8Q3/ErhXiqgUgF/fSJd2Pk3px1S UV+1WKmhR8Po3zpkGCPf7++F6/AqrRihKidUPZAM5HtzoshTc8a+L2wGXJOVhpWRzzhI UKo8MqHn4YOLSwUVWAz50fh9m4SXlC4rGUUDc1R0Qdq7Va0Cq/1p1BC+ZjKBX6rbgbX0 14mQ== X-Gm-Message-State: AOAM5330aB/Sqy/D3g8RxCFwvvIBR21PbMFnwMVZfVuMsLkk/XCiJ7ct Gyzh0T3HkqFgnTNigvX0Q3bTjw== X-Google-Smtp-Source: ABdhPJyRh008OaokntpotM0Wh4hidvAh1ZwijiN82MpcrfpSk+Ev0/zO2qyZjYsfbb4BCImO59Zm0A== X-Received: by 2002:ac8:47d6:: with SMTP id d22mr4167612qtr.87.1629920962845; Wed, 25 Aug 2021 12:49:22 -0700 (PDT) Received: from crass-HP-ZBook-15-G2 ([199.58.83.11]) by smtp.gmail.com with ESMTPSA id v14sm708268qkb.88.2021.08.25.12.49.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Aug 2021 12:49:22 -0700 (PDT) Date: Wed, 25 Aug 2021 14:49:09 -0500 From: Glenn Washburn To: "Thomas Schmitt" Cc: The development of GNU GRUB Subject: Re: [PATCH v2 1/8] tests: Make sure LANG is set properly for iso9660_test Message-ID: <20210825144909.25959b8a@crass-HP-ZBook-15-G2> In-Reply-To: <18749360916363390519@scdbackup.webframe.org> References: <17ddccb2ea35790b2effaf6bbe7d9acc4628f48c.1629874373.git.development@efficientek.com> <18749360916363390519@scdbackup.webframe.org> Reply-To: development@efficientek.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::834; envelope-from=development@efficientek.com; helo=mail-qt1-x834.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Aug 2021 19:49:26 -0000 Hi Thomas, On Wed, 25 Aug 2021 11:34:47 +0200 "Thomas Schmitt" wrote: > Hi, >=20 > Glenn Washburn wrote: > > LANG must be set to something that supports international > > characters, otherwise xorriso will refuse to include the file with > > name having international characters, causing the test to fail. >=20 > Can you tell me the exact error message from xorriso ? > I have some difficulties with reproducing the situation. There is no error message from xorriso, it will do as you noticed below, which is to default to replacing non-ASCII characters with underscores. The problem is that that behavior causes the tests to fail (rightly so because non-ASCII names are not being tested). The following causes the test to fail, but removing the LANG=3D causes success (because my system has LANG set to a UTF8 encoding). make SUBDIRS=3D LANG=3D TESTS=3Diso9660_test check > Your description is plausible, because libisofs takes its character > set default from nl_langinfo(3) with item CODESET and uses iconv(3) to > convert the file names to the output charset. >=20 > I would like to bring up for discussion an alternative remedy by these > xorrisofs options: >=20 > -input-charset UTF-8 -output-charset UTF-8 >=20 > ----------------------------------------------------------------------- >=20 > I am quite sure that it will work, but have problems with creating the > situation where it shall be the remedy. > On a system with configured LANG=3Den_US.UTF-8 i fail to reproduce > the issue with LANG=3DC or LANG=3D"" unless i add the explicit demand to > convert to UTF-8. >=20 > My disappointingly unproblematic xorriso test run is >=20 > ( LANG=3D xorrisofs -o test.iso '=C3=84=C3=96=C3=9C=C3=9F' ) >=20 > with various attempts to set the locale charset away from UTF-8: > LANG, LANGUAGE, LC_ALL set to empty text or to "C". > LANG=3D causes nl_langinfo(CODESET) to return "ANSI_X3.4-1968" but > iconv(3) throws no error when it shall convert > "\303\204\303\226\303\234\303\237". >=20 > I get warning messages if i tell xorriso to aim for output charset > UTF-8 while LANG is empty and the file name contains non-ASCII > characters: >=20 > (LANG=3D xorrisofs -o test.iso -output-charset UTF-8 '=C3=84=C3=96=C3= =9C=C3=9F') >=20 > yields three times >=20 > libisofs: WARNING : Charset conversion error. Cannot convert =C3=84=C3= =96=C3=9C=C3=9F > from ANSI_X3.4-1968 to UTF-8 libisofs: NOTE : > Caused by: Charset > conversion error >=20 > Inspection of test.iso shows that the file name '=C3=84=C3=96=C3=9C=C3=9F= ' was defaulted > to name '________'. >=20 > Adding the other charset option >=20 > (LANG=3D xorrisofs -o test.iso \ > -input-charset UTF-8 -output-charset UTF-8 '=C3=84=C3=96=C3= =9C=C3=9F') >=20 > yields a run without warning messages resulting in Rock Ridge file > name '=C3=84=C3=96=C3=9C=C3=9F'. >=20 > So i assume that my proposal is a valid alternative to setting > LANG=3DUTF-8. Whether it is preferrable over setting LANG to UTF-8 > would have to be discussed. > At least it would be nice for me to know why LANG=3D causes trouble in > the GRUB tests but not on my command line. > (I see nothing in the xorriso runs of tests/util/grub-fs-tester.in > which would be equivalent to the run which fails to convert '=C3=84=C3=96= =C3=9C=C3=9F' > here.) The takeaway here is that '=C3=84=C3=96=C3=9C=C3=9F' is being converted and= its is precisely that which is causing the test to fail. So unless I'm mistaken, you are confirming the issue this patch resolves. It looks like you have a viable alternative to my proposed patch, though I've not tested it myself. I really don't care about the implementation as long as the test succeeds when LANG is unset or empty. Despite having an alternative approach, do you have concerns or objections to my patch? Also would you like to create a patch that I would be happy to test? Glenn