* [PATCH] tests: Let xorriso fixely assume UTF-8 as local character set
@ 2021-08-27 21:05 Thomas Schmitt
2021-09-01 13:11 ` Daniel Kiper
0 siblings, 1 reply; 4+ messages in thread
From: Thomas Schmitt @ 2021-08-27 21:05 UTC (permalink / raw)
To: grub-devel, daniel.kiper, development; +Cc: Thomas Schmitt
iso9660_test fails if the effective locale is not UTF-8. This happens
because xorriso needs to convert file names and FSLABEL to UCS-2 when
preparing a Joliet tree. grub-fs-tester obviously intends to use UTF-8
as character set, but xorriso assumes by default the result of
nl_langinfo(3) with item CODESET.
So override the result of nl_langinfo(CODESET) by options of xorriso -as
mkisofs.
Signed-off-by: Thomas Schmitt <scdbackup@gmx.net>
---
tests/util/grub-fs-tester.in | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/tests/util/grub-fs-tester.in b/tests/util/grub-fs-tester.in
index bfc425e1f..4f581b638 100644
--- a/tests/util/grub-fs-tester.in
+++ b/tests/util/grub-fs-tester.in
@@ -8,6 +8,10 @@ GRUBFSTEST="@builddir@/grub-fstest"
tempdir=`mktemp -d "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"` || exit 1
+# xorriso -as mkisofs options to ignore locale when processing file names and
+# FSLABEL. This is especially needed for the conversion to Joliet UCS-2.
+XORRISOFS_CHARSET="-input-charset UTF-8 -output-charset UTF-8"
+
# This wrapper is to ease insertion of valgrind or time statistics
run_it () {
LC_ALL=C "$GRUBFSTEST" "$@"
@@ -1020,31 +1024,31 @@ for LOGSECSIZE in $(range "$MINLOGSECSIZE" "$MAXLOGSECSIZE" 1); do
(cd "$MASTER"; find . | cpio -o -H "$(echo ${fs} | sed 's@^cpio_@@')" > "${FSIMAGEP}0.img" ) ;;
x"ziso9660")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso -compliance rec_mtime -set_filter_r --zisofs -- -zisofs default -as mkisofs -iso-level 3 -graft-points -R -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" -- -set_filter_r --zisofs -- -zisofs default -add /="$MASTER" ;;
+ xorriso -compliance rec_mtime -set_filter_r --zisofs -- -zisofs default -as mkisofs $XORRISOFS_CHARSET -iso-level 3 -graft-points -R -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" -- -set_filter_r --zisofs -- -zisofs default -add /="$MASTER" ;;
x"iso9660")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge off -compliance rec_mtime -as mkisofs -iso-level 3 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge off -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 3 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"joliet")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge off -compliance rec_mtime -as mkisofs -iso-level 3 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge off -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 3 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"rockridge")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge on -compliance rec_mtime -as mkisofs -iso-level 3 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge on -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 3 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"rockridge_joliet")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge on -compliance rec_mtime -as mkisofs -iso-level 3 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge on -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 3 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"iso9660_1999")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge off -compliance rec_mtime -as mkisofs -iso-level 4 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge off -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 4 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"joliet_1999")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge off -compliance rec_mtime -as mkisofs -iso-level 4 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge off -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 4 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"rockridge_1999")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge on -compliance rec_mtime -as mkisofs -iso-level 4 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge on -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 4 -graft-points -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"rockridge_joliet_1999")
FSUUID=$(date -u +%Y-%m-%d-%H-%M-%S-00);
- xorriso --rockridge on -compliance rec_mtime -as mkisofs -iso-level 4 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
+ xorriso --rockridge on -compliance rec_mtime -as mkisofs $XORRISOFS_CHARSET -iso-level 4 -graft-points -J -joliet-long -V "$FSLABEL" --modification-date=$(echo ${FSUUID} | sed 's/-//g;') -o "${FSIMAGEP}0.img" /="$MASTER" ;;
x"romfs")
genromfs -V "$FSLABEL" -f "${FSIMAGEP}0.img" -d "$MASTER" ;;
xsquash4_*)
--
2.20.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] tests: Let xorriso fixely assume UTF-8 as local character set
2021-08-27 21:05 [PATCH] tests: Let xorriso fixely assume UTF-8 as local character set Thomas Schmitt
@ 2021-09-01 13:11 ` Daniel Kiper
2021-09-07 21:15 ` Glenn Washburn
0 siblings, 1 reply; 4+ messages in thread
From: Daniel Kiper @ 2021-09-01 13:11 UTC (permalink / raw)
To: Thomas Schmitt; +Cc: grub-devel, development
On Fri, Aug 27, 2021 at 11:05:06PM +0200, Thomas Schmitt wrote:
> iso9660_test fails if the effective locale is not UTF-8. This happens
> because xorriso needs to convert file names and FSLABEL to UCS-2 when
> preparing a Joliet tree. grub-fs-tester obviously intends to use UTF-8
> as character set, but xorriso assumes by default the result of
> nl_langinfo(3) with item CODESET.
> So override the result of nl_langinfo(CODESET) by options of xorriso -as
> mkisofs.
>
> Signed-off-by: Thomas Schmitt <scdbackup@gmx.net>
Thomas, Glenn, thank you for doing such deep investigation!
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Glenn, did you test this patch?
Daniel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] tests: Let xorriso fixely assume UTF-8 as local character set
2021-09-01 13:11 ` Daniel Kiper
@ 2021-09-07 21:15 ` Glenn Washburn
2021-09-08 19:06 ` Daniel Kiper
0 siblings, 1 reply; 4+ messages in thread
From: Glenn Washburn @ 2021-09-07 21:15 UTC (permalink / raw)
To: Daniel Kiper; +Cc: Thomas Schmitt, grub-devel
On Wed, 1 Sep 2021 15:11:38 +0200
Daniel Kiper <dkiper@net-space.pl> wrote:
> On Fri, Aug 27, 2021 at 11:05:06PM +0200, Thomas Schmitt wrote:
> > iso9660_test fails if the effective locale is not UTF-8. This
> > happens because xorriso needs to convert file names and FSLABEL to
> > UCS-2 when preparing a Joliet tree. grub-fs-tester obviously
> > intends to use UTF-8 as character set, but xorriso assumes by
> > default the result of nl_langinfo(3) with item CODESET.
> > So override the result of nl_langinfo(CODESET) by options of
> > xorriso -as mkisofs.
> >
> > Signed-off-by: Thomas Schmitt <scdbackup@gmx.net>
>
> Thomas, Glenn, thank you for doing such deep investigation!
>
> Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
>
> Glenn, did you test this patch?
Coming to this late, had some issues which kept me away. I notice that
the patch is already in master, but for what its worth, I have now
tested with the patch and LANG unset and the test now succeeds. Thanks
Thomas.
Glenn
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] tests: Let xorriso fixely assume UTF-8 as local character set
2021-09-07 21:15 ` Glenn Washburn
@ 2021-09-08 19:06 ` Daniel Kiper
0 siblings, 0 replies; 4+ messages in thread
From: Daniel Kiper @ 2021-09-08 19:06 UTC (permalink / raw)
To: Glenn Washburn; +Cc: Thomas Schmitt, grub-devel
On Tue, Sep 07, 2021 at 09:15:58PM +0000, Glenn Washburn wrote:
> On Wed, 1 Sep 2021 15:11:38 +0200
> Daniel Kiper <dkiper@net-space.pl> wrote:
>
> > On Fri, Aug 27, 2021 at 11:05:06PM +0200, Thomas Schmitt wrote:
> > > iso9660_test fails if the effective locale is not UTF-8. This
> > > happens because xorriso needs to convert file names and FSLABEL to
> > > UCS-2 when preparing a Joliet tree. grub-fs-tester obviously
> > > intends to use UTF-8 as character set, but xorriso assumes by
> > > default the result of nl_langinfo(3) with item CODESET.
> > > So override the result of nl_langinfo(CODESET) by options of
> > > xorriso -as mkisofs.
> > >
> > > Signed-off-by: Thomas Schmitt <scdbackup@gmx.net>
> >
> > Thomas, Glenn, thank you for doing such deep investigation!
> >
> > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
> >
> > Glenn, did you test this patch?
>
> Coming to this late, had some issues which kept me away. I notice that
No worries...
> the patch is already in master, but for what its worth, I have now
> tested with the patch and LANG unset and the test now succeeds. Thanks
Thank you for doing test.
Daniel
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-09-08 19:07 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-27 21:05 [PATCH] tests: Let xorriso fixely assume UTF-8 as local character set Thomas Schmitt
2021-09-01 13:11 ` Daniel Kiper
2021-09-07 21:15 ` Glenn Washburn
2021-09-08 19:06 ` Daniel Kiper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.