[PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[Emil Velikov]

Greetings everyone,

For a while now, the kernel has been accepting compressed firmware - be that xz
or zstd. There is a patch for the former on the ML, yet the latter is still
lacking.

While working through I've noticed a few issues - of various severity - so
decided to fix them all.

Mind you, I'm sending everything as single patch series to gauge feedback. I'm
quite happy to split and rework as per the community's feedback.

In a gist:
 - remove quoted filenames in WHENCE - escape white space as needed
 - remove _nearly_ all duplicate entries in WHENCE && add check
   - NOTE: one entry remains, which I'm not sure how to resolve cleanly
 - check if directory is listed as File - had a mistake like that locally
 - check we have no symlinks and drop symlink workarounds
 - quote deskdir (genuine bug) + few more shellcheck inspired fixes
 - support xz and zstd compressed firmware - no intermediate files, pipe
   directly into destdir


Kind regards
Emil


Emil Velikov (15):
  WHENCE: remove trailing white space
  WHENCE: remove unnecessary filename quotation
  check_whence, WHENCE, copy-firmware: escape filenames with spaces
  WHENCE: remove duplicate File entries
  check_whence: error on duplicate file entries
  check_whence: error on directory listed as File
  copy-firmware: remove non-applicable file presence test
  check_whence: error if File: is actually a link
  check_whence: error if symlinks are in-tree
  copy-firmware: remove unreachable symlink workarounds
  copy-firmware: quote deskdir to prevent word splitting
  copy-firmware: tweak sed invocation
  copy-firmware: quote the output of dirname
  copy-firmware: silence the last shellcheck warnings
  Makefile, copy-firmware: support xz/zstd compressed firmware

 Makefile         |  8 ++++
 WHENCE           | 97 +++++++++++++++++++-----------------------------
 check_whence.py  | 44 +++++++++++++++++++---
 copy-firmware.sh | 71 +++++++++++++++++------------------
 4 files changed, 119 insertions(+), 101 deletions(-)

-- 
2.39.2

[PATCH 01/15] WHENCE: remove trailing white space

[Emil Velikov]

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 WHENCE | 36 ++++++++++++++++++------------------
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/WHENCE b/WHENCE
index b6199d6..23c8847 100644
--- a/WHENCE
+++ b/WHENCE
@@ -395,7 +395,7 @@ Version: 8.07.00 MIDQ
 
 Licence: Redistributable. See LICENCE.qla2xxx for details
 
-Available from http://ldriver.qlogic.com/firmware/ 
+Available from http://ldriver.qlogic.com/firmware/
 
 --------------------------------------------------------------------------
 
@@ -2563,29 +2563,29 @@ Licence: Redistributable. See LICENCE.ueagle-atm4-firmware for details
 Driver: ueagle-atm - Driver for USB ADSL Modems based on Eagle I,II,III
 
 File: ueagle-atm/930-fpga.bin
-File: ueagle-atm/CMVeiWO.bin    
-File: ueagle-atm/CMVepFR10.bin  
-File: ueagle-atm/DSP9p.bin	
+File: ueagle-atm/CMVeiWO.bin
+File: ueagle-atm/CMVepFR10.bin
+File: ueagle-atm/DSP9p.bin
 File: ueagle-atm/eagleIII.fw
-File: ueagle-atm/adi930.fw     
-File: ueagle-atm/CMVep.bin      
-File: ueagle-atm/CMVepFR.bin    
+File: ueagle-atm/adi930.fw
+File: ueagle-atm/CMVep.bin
+File: ueagle-atm/CMVepFR.bin
 File: ueagle-atm/DSPei.bin
-File: ueagle-atm/CMV9i.bin     
-File: ueagle-atm/CMVepES03.bin  
-File: ueagle-atm/CMVepIT.bin    
+File: ueagle-atm/CMV9i.bin
+File: ueagle-atm/CMVepES03.bin
+File: ueagle-atm/CMVepIT.bin
 File: ueagle-atm/DSPep.bin
-File: ueagle-atm/CMV9p.bin     
-File: ueagle-atm/CMVepES.bin    
-File: ueagle-atm/CMVepWO.bin    
+File: ueagle-atm/CMV9p.bin
+File: ueagle-atm/CMVepES.bin
+File: ueagle-atm/CMVepWO.bin
 File: ueagle-atm/eagleI.fw
-File: ueagle-atm/CMVei.bin     
-File: ueagle-atm/CMVepFR04.bin  
-File: ueagle-atm/DSP9i.bin	    
+File: ueagle-atm/CMVei.bin
+File: ueagle-atm/CMVepFR04.bin
+File: ueagle-atm/DSP9i.bin
 File: ueagle-atm/eagleII.fw
 Version: 1.1
 
-Licence: Redistributable. Based on 
+Licence: Redistributable. Based on
          https://mail.gna.org/public/eagleusb-dev/2004-11/msg00172.html
 
 --------------------------------------------------------------------------
@@ -3949,7 +3949,7 @@ License:
 
   This SDD ("Static Dynamic Data") file is licensed strictly for use with
   the Sagrad WiFi modules (such as the SG901-1091/1098) that utilize the
-  cw1200 driver. There is no warranty expressed or implied about its 
+  cw1200 driver. There is no warranty expressed or implied about its
   fitness for any purpose.
 
   Permission is hereby granted for the distribution of this SDD file as
-- 
2.39.2

[PATCH 02/15] WHENCE: remove unnecessary filename quotation

[Emil Velikov]

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 WHENCE | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/WHENCE b/WHENCE
index 23c8847..fe5e423 100644
--- a/WHENCE
+++ b/WHENCE
@@ -2691,20 +2691,20 @@ Link: brcm/brcmfmac54591-pcie.clm_blob -> ../cypress/cyfmac54591-pcie.clm_blob
 Licence: Redistributable. See LICENCE.cypress for details.
 
 File: "brcm/brcmfmac43241b4-sdio.Intel Corp.-VALLEYVIEW C0 PLATFORM.txt"
-File: "brcm/brcmfmac4330-sdio.Prowise-PT301.txt"
+File: brcm/brcmfmac4330-sdio.Prowise-PT301.txt
 File: "brcm/brcmfmac43340-sdio.ASUSTeK COMPUTER INC.-TF103CE.txt"
-File: "brcm/brcmfmac43340-sdio.meegopad-t08.txt"
-File: "brcm/brcmfmac43340-sdio.pov-tab-p1006w-data.txt"
-File: "brcm/brcmfmac43340-sdio.predia-basic.txt"
-File: "brcm/brcmfmac43362-sdio.WC121.txt"
-File: "brcm/brcmfmac43362-sdio.cubietech,cubietruck.txt"
+File: brcm/brcmfmac43340-sdio.meegopad-t08.txt
+File: brcm/brcmfmac43340-sdio.pov-tab-p1006w-data.txt
+File: brcm/brcmfmac43340-sdio.predia-basic.txt
+File: brcm/brcmfmac43362-sdio.WC121.txt
+File: brcm/brcmfmac43362-sdio.cubietech,cubietruck.txt
 Link: brcm/brcmfmac43362-sdio.kobo,aura.txt -> brcmfmac43362-sdio.WC121.txt
 Link: brcm/brcmfmac43362-sdio.kobo,tolino-shine2hd.txt -> brcmfmac43362-sdio.WC121.txt
 Link: brcm/brcmfmac43362-sdio.lemaker,bananapro.txt -> brcmfmac43362-sdio.cubietech,cubietruck.txt
-File: "brcm/brcmfmac43430a0-sdio.ilife-S806.txt"
-File: "brcm/brcmfmac43430a0-sdio.jumper-ezpad-mini3.txt"
+File: brcm/brcmfmac43430a0-sdio.ilife-S806.txt
+File: brcm/brcmfmac43430a0-sdio.jumper-ezpad-mini3.txt
 File: "brcm/brcmfmac43430a0-sdio.ONDA-V80 PLUS.txt"
-File: "brcm/brcmfmac43430-sdio.AP6212.txt"
+File: brcm/brcmfmac43430-sdio.AP6212.txt
 Link: brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-plus.txt -> brcmfmac43430-sdio.AP6212.txt
 Link: brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-zero.txt -> brcmfmac43430-sdio.AP6212.txt
 Link: brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-ultra.txt -> brcmfmac43430-sdio.AP6212.txt
@@ -2713,19 +2713,19 @@ Link: brcm/brcmfmac43430-sdio.friendlyarm,nanopi-r1.txt -> brcmfmac43430-sdio.AP
 Link: brcm/brcmfmac43430-sdio.starfive,visionfive-v1.txt -> brcmfmac43430-sdio.AP6212.txt
 Link: brcm/brcmfmac43430-sdio.beagle,beaglev-starlight-jh7100-a1.txt -> brcmfmac43430-sdio.AP6212.txt
 Link: brcm/brcmfmac43430-sdio.beagle,beaglev-starlight-jh7100-r0.txt -> brcmfmac43430-sdio.AP6212.txt
-File: "brcm/brcmfmac43430-sdio.Hampoo-D2D3_Vi8A1.txt"
-File: "brcm/brcmfmac43430-sdio.MUR1DX.txt"
-File: "brcm/brcmfmac43430-sdio.raspberrypi,3-model-b.txt"
+File: brcm/brcmfmac43430-sdio.Hampoo-D2D3_Vi8A1.txt
+File: brcm/brcmfmac43430-sdio.MUR1DX.txt
+File: brcm/brcmfmac43430-sdio.raspberrypi,3-model-b.txt
 Link: brcm/brcmfmac43430-sdio.raspberrypi,model-zero-w.txt -> brcmfmac43430-sdio.raspberrypi,3-model-b.txt
 Link: brcm/brcmfmac43430-sdio.raspberrypi,model-zero-2-w.txt -> brcmfmac43430-sdio.raspberrypi,3-model-b.txt
-File: "brcm/brcmfmac43455-sdio.acepc-t8.txt"
-File: "brcm/brcmfmac43455-sdio.raspberrypi,3-model-b-plus.txt"
+File: brcm/brcmfmac43455-sdio.acepc-t8.txt
+File: brcm/brcmfmac43455-sdio.raspberrypi,3-model-b-plus.txt
 Link: brcm/brcmfmac43455-sdio.raspberrypi,3-model-a-plus.txt -> brcmfmac43455-sdio.raspberrypi,3-model-b-plus.txt
-File: "brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt"
+File: brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt
 Link: brcm/brcmfmac43455-sdio.Raspberry\ Pi\ Foundation-Raspberry\ Pi\ 4\ Model\ B.txt -> brcmfmac43455-sdio.raspberrypi,4-model-b.txt
 Link: brcm/brcmfmac43455-sdio.Raspberry\ Pi\ Foundation-Raspberry\ Pi\ Compute\ Module\ 4.txt -> brcmfmac43455-sdio.raspberrypi,4-model-b.txt
 File: "brcm/brcmfmac43455-sdio.MINIX-NEO Z83-4.txt"
-File: "brcm/brcmfmac4356-pcie.gpd-win-pocket.txt"
+File: brcm/brcmfmac4356-pcie.gpd-win-pocket.txt
 File: "brcm/brcmfmac4356-pcie.Xiaomi Inc-Mipad2.txt"
 File: brcm/brcmfmac4356-sdio.AP6356S.txt
 Link: brcm/brcmfmac4356-sdio.firefly,firefly-rk3399.txt -> brcmfmac4356-sdio.AP6356S.txt
-- 
2.39.2

[PATCH 03/15] check_whence, WHENCE, copy-firmware: escape filenames with spaces

[Emil Velikov]

We have a handful of instances which have space in the filename - just
escape them (like we do for the symlinks) and drop the extra re pattern.

With this we can also drop the extra sed pattern, stripping out the
quotation marks.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 WHENCE           | 10 +++++-----
 check_whence.py  |  8 ++------
 copy-firmware.sh |  2 +-
 3 files changed, 8 insertions(+), 12 deletions(-)

diff --git a/WHENCE b/WHENCE
index fe5e423..13013f8 100644
--- a/WHENCE
+++ b/WHENCE
@@ -2690,9 +2690,9 @@ Link: brcm/brcmfmac54591-pcie.clm_blob -> ../cypress/cyfmac54591-pcie.clm_blob
 
 Licence: Redistributable. See LICENCE.cypress for details.
 
-File: "brcm/brcmfmac43241b4-sdio.Intel Corp.-VALLEYVIEW C0 PLATFORM.txt"
+File: brcm/brcmfmac43241b4-sdio.Intel\ Corp.-VALLEYVIEW\ C0\ PLATFORM.txt
 File: brcm/brcmfmac4330-sdio.Prowise-PT301.txt
-File: "brcm/brcmfmac43340-sdio.ASUSTeK COMPUTER INC.-TF103CE.txt"
+File: brcm/brcmfmac43340-sdio.ASUSTeK\ COMPUTER\ INC.-TF103CE.txt
 File: brcm/brcmfmac43340-sdio.meegopad-t08.txt
 File: brcm/brcmfmac43340-sdio.pov-tab-p1006w-data.txt
 File: brcm/brcmfmac43340-sdio.predia-basic.txt
@@ -2703,7 +2703,7 @@ Link: brcm/brcmfmac43362-sdio.kobo,tolino-shine2hd.txt -> brcmfmac43362-sdio.WC1
 Link: brcm/brcmfmac43362-sdio.lemaker,bananapro.txt -> brcmfmac43362-sdio.cubietech,cubietruck.txt
 File: brcm/brcmfmac43430a0-sdio.ilife-S806.txt
 File: brcm/brcmfmac43430a0-sdio.jumper-ezpad-mini3.txt
-File: "brcm/brcmfmac43430a0-sdio.ONDA-V80 PLUS.txt"
+File: brcm/brcmfmac43430a0-sdio.ONDA-V80\ PLUS.txt
 File: brcm/brcmfmac43430-sdio.AP6212.txt
 Link: brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-plus.txt -> brcmfmac43430-sdio.AP6212.txt
 Link: brcm/brcmfmac43430-sdio.sinovoip,bpi-m2-zero.txt -> brcmfmac43430-sdio.AP6212.txt
@@ -2724,9 +2724,9 @@ Link: brcm/brcmfmac43455-sdio.raspberrypi,3-model-a-plus.txt -> brcmfmac43455-sd
 File: brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt
 Link: brcm/brcmfmac43455-sdio.Raspberry\ Pi\ Foundation-Raspberry\ Pi\ 4\ Model\ B.txt -> brcmfmac43455-sdio.raspberrypi,4-model-b.txt
 Link: brcm/brcmfmac43455-sdio.Raspberry\ Pi\ Foundation-Raspberry\ Pi\ Compute\ Module\ 4.txt -> brcmfmac43455-sdio.raspberrypi,4-model-b.txt
-File: "brcm/brcmfmac43455-sdio.MINIX-NEO Z83-4.txt"
+File: brcm/brcmfmac43455-sdio.MINIX-NEO\ Z83-4.txt
 File: brcm/brcmfmac4356-pcie.gpd-win-pocket.txt
-File: "brcm/brcmfmac4356-pcie.Xiaomi Inc-Mipad2.txt"
+File: brcm/brcmfmac4356-pcie.Xiaomi\ Inc-Mipad2.txt
 File: brcm/brcmfmac4356-sdio.AP6356S.txt
 Link: brcm/brcmfmac4356-sdio.firefly,firefly-rk3399.txt -> brcmfmac4356-sdio.AP6356S.txt
 Link: brcm/brcmfmac4356-sdio.khadas,vim2.txt -> brcmfmac4356-sdio.AP6356S.txt
diff --git a/check_whence.py b/check_whence.py
index 8805e99..f347f0e 100755
--- a/check_whence.py
+++ b/check_whence.py
@@ -6,13 +6,9 @@ from io import open
 def list_whence():
     with open('WHENCE', encoding='utf-8') as whence:
         for line in whence:
-            match = re.match(r'(?:File|Source):\s*"(.*)"', line)
+            match = re.match(r'(?:File|Source):\s*(.*)', line)
             if match:
-                yield match.group(1)
-                continue
-            match = re.match(r'(?:File|Source):\s*(\S*)', line)
-            if match:
-                yield match.group(1)
+                yield match.group(1).replace("\ ", " ")
                 continue
             match = re.match(r'Licen[cs]e: (?:.*\bSee (.*) for details\.?|(\S*))\n',
                              line)
diff --git a/copy-firmware.sh b/copy-firmware.sh
index bbacb92..6bceef3 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -31,7 +31,7 @@ while test $# -gt 0; do
     esac
 done
 
-grep '^File:' WHENCE | sed -e's/^File: *//g' -e's/"//g' | while read f; do
+grep '^File:' WHENCE | sed -e's/^File: *//g' | while read f; do
     test -f "$f" || continue
     $verbose "copying file $f"
     install -d $destdir/$(dirname "$f")
-- 
2.39.2

[PATCH 04/15] WHENCE: remove duplicate File entries

[Emil Velikov]

XXX: still one outstanding - mediatek/mt7663pr2h.bin

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 WHENCE | 19 -------------------
 1 file changed, 19 deletions(-)

diff --git a/WHENCE b/WHENCE
index 13013f8..d1cc3f8 100644
--- a/WHENCE
+++ b/WHENCE
@@ -1073,10 +1073,6 @@ Version: 75.27f1c37b.0
 File: iwlwifi-so-a0-jf-b0-77.ucode
 Version: 74.f92b5fed.0
 
-File: iwlwifi-so-a0-gf-a0.pnvm
-
-File: iwlwifi-so-a0-gf4-a0.pnvm
-
 File: iwlwifi-ty-a0-gf-a0.pnvm
 
 Licence: Redistributable. See LICENCE.iwlwifi_firmware for details
@@ -2980,19 +2976,6 @@ Licence: Redistributable. See LICENCE.rtlwifi_firmware.txt for details.
 
 --------------------------------------------------------------------------
 
-Driver: r8723au - Realtek 802.11n WLAN driver for RTL8723AU
-
-Info: Taken from Realtek driver rtl8723A_WiFi_linux_v4.1.3_6044.20121224
-      Firmware is embedded in the driver as data statements. This info
-      has been extracted into a binary file.
-File: rtlwifi/rtl8723aufw_A.bin
-File: rtlwifi/rtl8723aufw_B.bin
-File: rtlwifi/rtl8723aufw_B_NoBT.bin
-
-Licence: Redistributable. See LICENCE.rtlwifi_firmware.txt for details.
-
---------------------------------------------------------------------------
-
 Driver: rtl8188ee - Realtek 802.11n WLAN driver for RTL8188EE
 
 Info: Taken from Realtek version rtl_92ce_92se_92de_8723ae_88ee_linux_mac80211_0010.0109.2013
@@ -5943,8 +5926,6 @@ Link: cirrus/cs35l41-dsp1-spk-prot-103c8995.wmfw -> cs35l41/v6.61.1/halo_cspl_RA
 Link: cirrus/cs35l41-dsp1-spk-cali-103c8995.wmfw -> cs35l41/v6.61.1/halo_cspl_RAM_revB2_29.63.1.wmfw
 Link: cirrus/cs35l41-dsp1-spk-prot-103c89c6.wmfw -> cs35l41/v6.61.1/halo_cspl_RAM_revB2_29.63.1.wmfw
 Link: cirrus/cs35l41-dsp1-spk-cali-103c89c6.wmfw -> cs35l41/v6.61.1/halo_cspl_RAM_revB2_29.63.1.wmfw
-File: cirrus/cs35l41-dsp1-spk-cali-103c8971.bin
-File: cirrus/cs35l41-dsp1-spk-prot-103c8971.bin
 File: cirrus/cs35l41-dsp1-spk-prot-103c8971.bin
 File: cirrus/cs35l41-dsp1-spk-cali-103c8971.bin
 File: cirrus/cs35l41-dsp1-spk-prot-103c8972.bin
-- 
2.39.2

[PATCH 05/15] check_whence: error on duplicate file entries

[Emil Velikov]

There's little point in copying (or compressing with later patches) the
same files multiple times. So let's error out when duplicate entries are
present.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 check_whence.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/check_whence.py b/check_whence.py
index f347f0e..7ff21f6 100755
--- a/check_whence.py
+++ b/check_whence.py
@@ -24,6 +24,14 @@ def list_whence():
                         yield match.group(2)
                         continue
 
+def list_whence_files():
+    with open('WHENCE', encoding='utf-8') as whence:
+        for line in whence:
+            match = re.match(r'File:\s*(.*)', line)
+            if match:
+                yield match.group(1).replace("\ ", " ")
+                continue
+
 def list_git():
     with os.popen('git ls-files') as git_files:
         for line in git_files:
@@ -32,12 +40,17 @@ def list_git():
 def main():
     ret = 0
     whence_list = list(list_whence())
+    whence_files = list(list_whence_files())
     known_files = set(name for name in whence_list if not name.endswith('/')) | \
                   set(['check_whence.py', 'configure', 'Makefile',
                        'README', 'copy-firmware.sh', 'WHENCE'])
     known_prefixes = set(name for name in whence_list if name.endswith('/'))
     git_files = set(list_git())
 
+    for name in set(fw for fw in whence_files if whence_files.count(fw) > 1):
+        sys.stderr.write('E: %s listed in WHENCE twice\n' % name)
+        ret = 1
+
     for name in sorted(list(known_files - git_files)):
         sys.stderr.write('E: %s listed in WHENCE does not exist\n' % name)
         ret = 1
-- 
2.39.2

[PATCH 06/15] check_whence: error on directory listed as File

[Emil Velikov]

No occurrences in-tree, but it makes sense to sanity check that.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 check_whence.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/check_whence.py b/check_whence.py
index 7ff21f6..130966e 100755
--- a/check_whence.py
+++ b/check_whence.py
@@ -47,6 +47,11 @@ def main():
     known_prefixes = set(name for name in whence_list if name.endswith('/'))
     git_files = set(list_git())
 
+    for name in set(name for name in whence_files if name.endswith('/')):
+        sys.stderr.write('E: %s listed in WHENCE as File, but is directory\n' %
+                         name)
+        ret = 1
+
     for name in set(fw for fw in whence_files if whence_files.count(fw) > 1):
         sys.stderr.write('E: %s listed in WHENCE twice\n' % name)
         ret = 1
-- 
2.39.2

[PATCH 07/15] copy-firmware: remove non-applicable file presence test

[Emil Velikov]

The check_whence.py script already ensures that the files listed in
WHENCE are present on-disk. Thus the extra check isn't needed.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 copy-firmware.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/copy-firmware.sh b/copy-firmware.sh
index 6bceef3..656b0ec 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -32,7 +32,6 @@ while test $# -gt 0; do
 done
 
 grep '^File:' WHENCE | sed -e's/^File: *//g' | while read f; do
-    test -f "$f" || continue
     $verbose "copying file $f"
     install -d $destdir/$(dirname "$f")
     cp -d "$f" $destdir/"$f"
-- 
2.39.2

[PATCH 08/15] check_whence: error if File: is actually a link

[Emil Velikov]

This allows us to remove the -d aka --no-dereference --preserve=links
from the copy command.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 check_whence.py  | 5 +++++
 copy-firmware.sh | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/check_whence.py b/check_whence.py
index 130966e..92b104e 100755
--- a/check_whence.py
+++ b/check_whence.py
@@ -56,6 +56,11 @@ def main():
         sys.stderr.write('E: %s listed in WHENCE twice\n' % name)
         ret = 1
 
+    for name in set(link for link in whence_files if os.path.islink(link)):
+        sys.stderr.write('E: %s listed in WHENCE as File, but is a symlink\n' %
+                         name)
+        ret = 1
+
     for name in sorted(list(known_files - git_files)):
         sys.stderr.write('E: %s listed in WHENCE does not exist\n' % name)
         ret = 1
diff --git a/copy-firmware.sh b/copy-firmware.sh
index 656b0ec..e3f3176 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -34,7 +34,7 @@ done
 grep '^File:' WHENCE | sed -e's/^File: *//g' | while read f; do
     $verbose "copying file $f"
     install -d $destdir/$(dirname "$f")
-    cp -d "$f" $destdir/"$f"
+    cp "$f" $destdir/"$f"
 done
 
 grep -E '^Link:' WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do
-- 
2.39.2

[PATCH 09/15] check_whence: error if symlinks are in-tree

[Emil Velikov]

Currently we have no symlinks in-tree. Add a simple check, ensuring they
don't get added in the future.

This allows us to remove the clunky symlink checking code in
copy-firmware.sh

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 check_whence.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/check_whence.py b/check_whence.py
index 92b104e..75fd3ad 100755
--- a/check_whence.py
+++ b/check_whence.py
@@ -32,6 +32,14 @@ def list_whence_files():
                 yield match.group(1).replace("\ ", " ")
                 continue
 
+def list_whence_links():
+    with open('WHENCE', encoding='utf-8') as whence:
+        for line in whence:
+            match = re.match(r'Link:\s*(.*) ->', line)
+            if match:
+                yield match.group(1).replace("\ ", " ")
+                continue
+
 def list_git():
     with os.popen('git ls-files') as git_files:
         for line in git_files:
@@ -41,6 +49,7 @@ def main():
     ret = 0
     whence_list = list(list_whence())
     whence_files = list(list_whence_files())
+    whence_links = list(list_whence_links())
     known_files = set(name for name in whence_list if not name.endswith('/')) | \
                   set(['check_whence.py', 'configure', 'Makefile',
                        'README', 'copy-firmware.sh', 'WHENCE'])
@@ -61,6 +70,10 @@ def main():
                          name)
         ret = 1
 
+    for name in set(link for link in whence_links if os.path.islink(link)):
+        sys.stderr.write('E: %s listed in WHENCE as Link, is in tree\n' % name)
+        ret = 1
+
     for name in sorted(list(known_files - git_files)):
         sys.stderr.write('E: %s listed in WHENCE does not exist\n' % name)
         ret = 1
-- 
2.39.2

[PATCH 10/15] copy-firmware: remove unreachable symlink workarounds

[Emil Velikov]

There are no symlinks in-tree and the earlier commit to check_whence.py
ensures new ones don't get added. Thus we can remove the consistency
checks and pruning code.

With this done we can also drop:
 - the --force argument for ln
 - the prune argument and related handling

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 copy-firmware.sh | 36 +++---------------------------------
 1 file changed, 3 insertions(+), 33 deletions(-)

diff --git a/copy-firmware.sh b/copy-firmware.sh
index e3f3176..0132244 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -5,7 +5,6 @@
 #
 
 verbose=:
-prune=no
 
 while test $# -gt 0; do
     case $1 in
@@ -14,11 +13,6 @@ while test $# -gt 0; do
             shift
             ;;
 
-        -P | --prune)
-            prune=yes
-            shift
-            ;;
-
         *)
             if test "x$destdir" != "x"; then
                 echo "ERROR: unknown command-line options: $@"
@@ -38,33 +32,9 @@ grep '^File:' WHENCE | sed -e's/^File: *//g' | while read f; do
 done
 
 grep -E '^Link:' WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do
-    if test -L "$f"; then
-        test -f "$destdir/$f" && continue
-        $verbose "copying link $f"
-        install -d $destdir/$(dirname "$f")
-        cp -d "$f" $destdir/"$f"
-
-        if test "x$d" != "x"; then
-            target=`readlink "$f"`
-
-            if test "x$target" != "x$d"; then
-                $verbose "WARNING: inconsistent symlink target: $target != $d"
-            else
-                if test "x$prune" != "xyes"; then
-                    $verbose "WARNING: unneeded symlink detected: $f"
-                else
-                    $verbose "WARNING: pruning unneeded symlink $f"
-                    rm -f "$f"
-                fi
-            fi
-        else
-            $verbose "WARNING: missing target for symlink $f"
-        fi
-    else
-        $verbose "creating link $f -> $d"
-        install -d $destdir/$(dirname "$f")
-        ln -sf "$d" "$destdir/$f"
-    fi
+    $verbose "creating link $f -> $d"
+    install -d $destdir/$(dirname "$f")
+    ln -s "$d" "$destdir/$f"
 done
 
 exit 0
-- 
2.39.2

[PATCH 11/15] copy-firmware: quote deskdir to prevent word splitting

[Emil Velikov]

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 copy-firmware.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/copy-firmware.sh b/copy-firmware.sh
index 0132244..c6a2fe9 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -27,13 +27,13 @@ done
 
 grep '^File:' WHENCE | sed -e's/^File: *//g' | while read f; do
     $verbose "copying file $f"
-    install -d $destdir/$(dirname "$f")
-    cp "$f" $destdir/"$f"
+    install -d "$destdir"/$(dirname "$f")
+    cp "$f" "$destdir/$f"
 done
 
 grep -E '^Link:' WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do
     $verbose "creating link $f -> $d"
-    install -d $destdir/$(dirname "$f")
+    install -d "$destdir"/$(dirname "$f")
     ln -s "$d" "$destdir/$f"
 done
 
-- 
2.39.2

[PATCH 12/15] copy-firmware: tweak sed invocation

[Emil Velikov]

Add space between the arguments and pattern and combine patterns where
possible.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 copy-firmware.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/copy-firmware.sh b/copy-firmware.sh
index c6a2fe9..722227f 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -25,13 +25,13 @@ while test $# -gt 0; do
     esac
 done
 
-grep '^File:' WHENCE | sed -e's/^File: *//g' | while read f; do
+grep '^File:' WHENCE | sed -e 's/^File: *//g' | while read f; do
     $verbose "copying file $f"
     install -d "$destdir"/$(dirname "$f")
     cp "$f" "$destdir/$f"
 done
 
-grep -E '^Link:' WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do
+grep -E '^Link:' WHENCE | sed -e 's/^Link: *//g;s/-> //g' | while read f d; do
     $verbose "creating link $f -> $d"
     install -d "$destdir"/$(dirname "$f")
     ln -s "$d" "$destdir/$f"
-- 
2.39.2

[PATCH 13/15] copy-firmware: quote the output of dirname

[Emil Velikov]

As highlighted by shellcheck - as-is one could get word splitting. That
isn't an issue, yet, since none of the in-tree directories have space in
their name.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 copy-firmware.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/copy-firmware.sh b/copy-firmware.sh
index 722227f..5a5e1a8 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -27,13 +27,13 @@ done
 
 grep '^File:' WHENCE | sed -e 's/^File: *//g' | while read f; do
     $verbose "copying file $f"
-    install -d "$destdir"/$(dirname "$f")
+    install -d "$destdir/$(dirname "$f")"
     cp "$f" "$destdir/$f"
 done
 
 grep -E '^Link:' WHENCE | sed -e 's/^Link: *//g;s/-> //g' | while read f d; do
     $verbose "creating link $f -> $d"
-    install -d "$destdir"/$(dirname "$f")
+    install -d "$destdir/$(dirname "$f")"
     ln -s "$d" "$destdir/$f"
 done
 
-- 
2.39.2

[PATCH 14/15] copy-firmware: silence the last shellcheck warnings

[Emil Velikov]

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 copy-firmware.sh | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/copy-firmware.sh b/copy-firmware.sh
index 5a5e1a8..a52b847 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -9,13 +9,14 @@ verbose=:
 while test $# -gt 0; do
     case $1 in
         -v | --verbose)
+            # shellcheck disable=SC2209
             verbose=echo
             shift
             ;;
 
         *)
             if test "x$destdir" != "x"; then
-                echo "ERROR: unknown command-line options: $@"
+                echo "ERROR: unknown command-line options: $*"
                 exit 1
             fi
 
@@ -25,12 +26,14 @@ while test $# -gt 0; do
     esac
 done
 
+# shellcheck disable=SC2162
 grep '^File:' WHENCE | sed -e 's/^File: *//g' | while read f; do
     $verbose "copying file $f"
     install -d "$destdir/$(dirname "$f")"
     cp "$f" "$destdir/$f"
 done
 
+# shellcheck disable=SC2162
 grep -E '^Link:' WHENCE | sed -e 's/^Link: *//g;s/-> //g' | while read f d; do
     $verbose "creating link $f -> $d"
     install -d "$destdir/$(dirname "$f")"
-- 
2.39.2

[PATCH 15/15] Makefile, copy-firmware: support xz/zstd compressed firmware

[Emil Velikov]

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
---
 Makefile         |  8 ++++++++
 copy-firmware.sh | 33 +++++++++++++++++++++++++++++----
 2 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 83a0ec6..50a4c29 100644
--- a/Makefile
+++ b/Makefile
@@ -11,3 +11,11 @@ check:
 install:
 	install -d $(DESTDIR)$(FIRMWAREDIR)
 	./copy-firmware.sh $(DESTDIR)$(FIRMWAREDIR)
+
+install-xz:
+	install -d $(DESTDIR)$(FIRMWAREDIR)
+	./copy-firmware.sh --xz $(DESTDIR)$(FIRMWAREDIR)
+
+install-zst:
+	install -d $(DESTDIR)$(FIRMWAREDIR)
+	./copy-firmware.sh --zstd $(DESTDIR)$(FIRMWAREDIR)
diff --git a/copy-firmware.sh b/copy-firmware.sh
index a52b847..6c78e27 100755
--- a/copy-firmware.sh
+++ b/copy-firmware.sh
@@ -5,6 +5,9 @@
 #
 
 verbose=:
+# shellcheck disable=SC2209
+compress=cat
+compext=
 
 while test $# -gt 0; do
     case $1 in
@@ -14,6 +17,28 @@ while test $# -gt 0; do
             shift
             ;;
 
+        --xz)
+            if test "x$compext" == "x.zst"; then
+                echo "ERROR: cannot mix XZ and ZSTD compression"
+                exit 1
+            fi
+            # shellcheck disable=SC2209
+            compress="xz --compress --quiet --stdout --check=crc32"
+            compext=".xz"
+            shift
+            ;;
+
+        --zstd)
+            if test "x$compext" == "x.xz"; then
+                echo "ERROR: cannot mix XZ and ZSTD compression"
+                exit 1
+            fi
+            # shellcheck disable=SC2209
+            compress="zstd --compress --quiet --stdout"
+            compext=".zst"
+            shift
+            ;;
+
         *)
             if test "x$destdir" != "x"; then
                 echo "ERROR: unknown command-line options: $*"
@@ -28,16 +53,16 @@ done
 
 # shellcheck disable=SC2162
 grep '^File:' WHENCE | sed -e 's/^File: *//g' | while read f; do
-    $verbose "copying file $f"
     install -d "$destdir/$(dirname "$f")"
-    cp "$f" "$destdir/$f"
+    $verbose "copying/compressing file $f$compext"
+    $compress "$f" > "$destdir/$f$compext"
 done
 
 # shellcheck disable=SC2162
 grep -E '^Link:' WHENCE | sed -e 's/^Link: *//g;s/-> //g' | while read f d; do
-    $verbose "creating link $f -> $d"
     install -d "$destdir/$(dirname "$f")"
-    ln -s "$d" "$destdir/$f"
+    $verbose "creating link $f$compext -> $d$compext"
+    ln -s "$d$compext" "$destdir/$f$compext"
 done
 
 exit 0
-- 
2.39.2

Re: [PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[Emil Velikov]

On Sat, 18 Feb 2023 at 18:46, Emil Velikov <emil.l.velikov@gmail.com> wrote:
>
> Greetings everyone,
>
> For a while now, the kernel has been accepting compressed firmware - be that xz
> or zstd. There is a patch for the former on the ML, yet the latter is still
> lacking.
>
> While working through I've noticed a few issues - of various severity - so
> decided to fix them all.
>
> Mind you, I'm sending everything as single patch series to gauge feedback. I'm
> quite happy to split and rework as per the community's feedback.
>
> In a gist:
>  - remove quoted filenames in WHENCE - escape white space as needed
>  - remove _nearly_ all duplicate entries in WHENCE && add check
>    - NOTE: one entry remains, which I'm not sure how to resolve cleanly
>  - check if directory is listed as File - had a mistake like that locally
>  - check we have no symlinks and drop symlink workarounds
>  - quote deskdir (genuine bug) + few more shellcheck inspired fixes
>  - support xz and zstd compressed firmware - no intermediate files, pipe
>    directly into destdir
>
>
> Kind regards
> Emil
>
>
> Emil Velikov (15):
>   WHENCE: remove trailing white space
>   WHENCE: remove unnecessary filename quotation
>   check_whence, WHENCE, copy-firmware: escape filenames with spaces
>   WHENCE: remove duplicate File entries
>   check_whence: error on duplicate file entries
>   check_whence: error on directory listed as File
>   copy-firmware: remove non-applicable file presence test
>   check_whence: error if File: is actually a link
>   check_whence: error if symlinks are in-tree
>   copy-firmware: remove unreachable symlink workarounds
>   copy-firmware: quote deskdir to prevent word splitting
>   copy-firmware: tweak sed invocation
>   copy-firmware: quote the output of dirname
>   copy-firmware: silence the last shellcheck warnings
>   Makefile, copy-firmware: support xz/zstd compressed firmware
>
>  Makefile         |  8 ++++
>  WHENCE           | 97 +++++++++++++++++++-----------------------------
>  check_whence.py  | 44 +++++++++++++++++++---
>  copy-firmware.sh | 71 +++++++++++++++++------------------
>  4 files changed, 119 insertions(+), 101 deletions(-)
>

Adding Josh Boyer - the maintainer, according to git log.

Josh, do you have any objections to the patches - part or in full? Can
you pick these off the ML or shall I send you a git request-pull?

Thanks
Emil

Re: [PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2718 bytes --]

On Tue, 2023-02-28 at 14:18 +0000, Emil Velikov wrote:
> On Sat, 18 Feb 2023 at 18:46, Emil Velikov <emil.l.velikov@gmail.com>
> wrote:
> > 
> > Greetings everyone,
> > 
> > For a while now, the kernel has been accepting compressed firmware - be that xz
> > or zstd. There is a patch for the former on the ML, yet the latter is still
> > lacking.
> > 
> > While working through I've noticed a few issues - of various severity - so
> > decided to fix them all.
> > 
> > Mind you, I'm sending everything as single patch series to gauge feedback. I'm
> > quite happy to split and rework as per the community's feedback.
> > 
> > In a gist:
> >  - remove quoted filenames in WHENCE - escape white space as needed
> >  - remove _nearly_ all duplicate entries in WHENCE && add check
> >    - NOTE: one entry remains, which I'm not sure how to resolve cleanly
> >  - check if directory is listed as File - had a mistake like that locally
> >  - check we have no symlinks and drop symlink workarounds
> >  - quote deskdir (genuine bug) + few more shellcheck inspired fixes
> >  - support xz and zstd compressed firmware - no intermediate files, pipe
> >    directly into destdir
> > 
> > 
> > Kind regards
> > Emil
> > 
> > 
> > Emil Velikov (15):
> >   WHENCE: remove trailing white space
> >   WHENCE: remove unnecessary filename quotation
> >   check_whence, WHENCE, copy-firmware: escape filenames with spaces
> >   WHENCE: remove duplicate File entries
> >   check_whence: error on duplicate file entries
> >   check_whence: error on directory listed as File
> >   copy-firmware: remove non-applicable file presence test
> >   check_whence: error if File: is actually a link
> >   check_whence: error if symlinks are in-tree
> >   copy-firmware: remove unreachable symlink workarounds
> >   copy-firmware: quote deskdir to prevent word splitting
> >   copy-firmware: tweak sed invocation
> >   copy-firmware: quote the output of dirname
> >   copy-firmware: silence the last shellcheck warnings
> >   Makefile, copy-firmware: support xz/zstd compressed firmware
> > 
> >  Makefile         |  8 ++++
> >  WHENCE           | 97 +++++++++++++++++++-----------------------------
> >  check_whence.py  | 44 +++++++++++++++++++---
> >  copy-firmware.sh | 71 +++++++++++++++++------------------
> >  4 files changed, 119 insertions(+), 101 deletions(-)
> > 
> 
> Adding Josh Boyer - the maintainer, according to git log.
> 
> Josh, do you have any objections to the patches - part or in full? Can
> you pick these off the ML or shall I send you a git request-pull?

Does this work even for early microcode loading?


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

Re: [PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[Emil Velikov]

On Tue, 28 Feb 2023 at 14:37, David Woodhouse <dwmw2@infradead.org> wrote:
>
> On Tue, 2023-02-28 at 14:18 +0000, Emil Velikov wrote:
> > On Sat, 18 Feb 2023 at 18:46, Emil Velikov <emil.l.velikov@gmail.com>
> > wrote:
> > >
> > > Greetings everyone,
> > >
> > > For a while now, the kernel has been accepting compressed firmware - be that xz
> > > or zstd. There is a patch for the former on the ML, yet the latter is still
> > > lacking.
> > >
> > > While working through I've noticed a few issues - of various severity - so
> > > decided to fix them all.
> > >
> > > Mind you, I'm sending everything as single patch series to gauge feedback. I'm
> > > quite happy to split and rework as per the community's feedback.
> > >
> > > In a gist:
> > >  - remove quoted filenames in WHENCE - escape white space as needed
> > >  - remove _nearly_ all duplicate entries in WHENCE && add check
> > >    - NOTE: one entry remains, which I'm not sure how to resolve cleanly
> > >  - check if directory is listed as File - had a mistake like that locally
> > >  - check we have no symlinks and drop symlink workarounds
> > >  - quote deskdir (genuine bug) + few more shellcheck inspired fixes
> > >  - support xz and zstd compressed firmware - no intermediate files, pipe
> > >    directly into destdir
> > >
> > >
> > > Kind regards
> > > Emil
> > >
> > >
> > > Emil Velikov (15):
> > >   WHENCE: remove trailing white space
> > >   WHENCE: remove unnecessary filename quotation
> > >   check_whence, WHENCE, copy-firmware: escape filenames with spaces
> > >   WHENCE: remove duplicate File entries
> > >   check_whence: error on duplicate file entries
> > >   check_whence: error on directory listed as File
> > >   copy-firmware: remove non-applicable file presence test
> > >   check_whence: error if File: is actually a link
> > >   check_whence: error if symlinks are in-tree
> > >   copy-firmware: remove unreachable symlink workarounds
> > >   copy-firmware: quote deskdir to prevent word splitting
> > >   copy-firmware: tweak sed invocation
> > >   copy-firmware: quote the output of dirname
> > >   copy-firmware: silence the last shellcheck warnings
> > >   Makefile, copy-firmware: support xz/zstd compressed firmware
> > >
> > >  Makefile         |  8 ++++
> > >  WHENCE           | 97 +++++++++++++++++++-----------------------------
> > >  check_whence.py  | 44 +++++++++++++++++++---
> > >  copy-firmware.sh | 71 +++++++++++++++++------------------
> > >  4 files changed, 119 insertions(+), 101 deletions(-)
> > >
> >
> > Adding Josh Boyer - the maintainer, according to git log.
> >
> > Josh, do you have any objections to the patches - part or in full? Can
> > you pick these off the ML or shall I send you a git request-pull?
>
> Does this work even for early microcode loading?
>

Greetings David

Don't think I follow - the series is completely unrelated to microcode
loading. There isn't any mention of microcode in there, so I'm not
sure what inspired the question. Perhaps it was meant for another
thread?

Thanks
Emil

Re: [PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 366 bytes --]

On Tue, 2023-02-28 at 16:46 +0000, Emil Velikov wrote:
> 
> Greetings David
> 
> Don't think I follow - the series is completely unrelated to microcode
> loading. There isn't any mention of microcode in there, so I'm not
> sure what inspired the question. Perhaps it was meant for another
> thread?

cf. https://bugzilla.redhat.com/show_bug.cgi?id=2122271

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

Re: [PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[Emil Velikov]

On Tue, 28 Feb 2023 at 16:48, David Woodhouse <dwmw2@infradead.org> wrote:
>
> On Tue, 2023-02-28 at 16:46 +0000, Emil Velikov wrote:
> >
> > Greetings David
> >
> > Don't think I follow - the series is completely unrelated to microcode
> > loading. There isn't any mention of microcode in there, so I'm not
> > sure what inspired the question. Perhaps it was meant for another
> > thread?
>
> cf. https://bugzilla.redhat.com/show_bug.cgi?id=2122271

Nor really a RH/Fedora person, so the link is appreciated.

On my Arch machines, only the AMD PSP firmware is included in the
(mkinitcpio) generated init ramdisk. And as outlined in the bug that
works just fine. Will have a look into this.
Mind you - nearly all the patches can be applied independently of one
another. So dropping the final patch is one option in the short term.

Thanks again,
Emil

Re: [PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1042 bytes --]

On Tue, 2023-02-28 at 17:02 +0000, Emil Velikov wrote:
> On Tue, 28 Feb 2023 at 16:48, David Woodhouse <dwmw2@infradead.org> wrote:
> > 
> > On Tue, 2023-02-28 at 16:46 +0000, Emil Velikov wrote:
> > > 
> > > Greetings David
> > > 
> > > Don't think I follow - the series is completely unrelated to microcode
> > > loading. There isn't any mention of microcode in there, so I'm not
> > > sure what inspired the question. Perhaps it was meant for another
> > > thread?
> > 
> > cf. https://bugzilla.redhat.com/show_bug.cgi?id=2122271
> 
> Nor really a RH/Fedora person, so the link is appreciated.
> 
> On my Arch machines, only the AMD PSP firmware is included in the
> (mkinitcpio) generated init ramdisk. And as outlined in the bug that
> works just fine. Will have a look into this.
> Mind you - nearly all the patches can be applied independently of one
> another. So dropping the final patch is one option in the short term.

Maybe the answer is just to have a way to exempt *certain* files from
the compression?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

Re: [PATCH 00/15] Misc fixes, sanity checks and xz/zstd compression

[Emil Velikov]

On Tue, 28 Feb 2023 at 17:03, David Woodhouse <dwmw2@infradead.org> wrote:
>
> On Tue, 2023-02-28 at 17:02 +0000, Emil Velikov wrote:
> > On Tue, 28 Feb 2023 at 16:48, David Woodhouse <dwmw2@infradead.org> wrote:
> > >
> > > On Tue, 2023-02-28 at 16:46 +0000, Emil Velikov wrote:
> > > >
> > > > Greetings David
> > > >
> > > > Don't think I follow - the series is completely unrelated to microcode
> > > > loading. There isn't any mention of microcode in there, so I'm not
> > > > sure what inspired the question. Perhaps it was meant for another
> > > > thread?
> > >
> > > cf. https://bugzilla.redhat.com/show_bug.cgi?id=2122271
> >
> > Nor really a RH/Fedora person, so the link is appreciated.
> >
> > On my Arch machines, only the AMD PSP firmware is included in the
> > (mkinitcpio) generated init ramdisk. And as outlined in the bug that
> > works just fine. Will have a look into this.
> > Mind you - nearly all the patches can be applied independently of one
> > another. So dropping the final patch is one option in the short term.
>
> Maybe the answer is just to have a way to exempt *certain* files from
> the compression?

Arch has been concatenating the amd ucode into a standalone initrd for
5+ years, thus making sure dracut or similar tools don't interfere.
Having some way to opt-out, would overall be beneficial. This way one
doesn't depend on distro specific solutions/hacks.

Skimming through the list - there's another set of WHENCE related
fixes [1], with a slightly better helper. Let me re-spin my series, so
things are less likely to clash.

-Emil

[1] https://lore.kernel.org/linux-firmware/20230216005602.18838-1-ats@offog.org/T/#t