[PATCH 0/3] Support RawFile in check_whence and update WHENCE

[PATCH 0/3] Support RawFile in check_whence and update WHENCE

[Juerg Haefliger]

This series updates check_whence to understand the RawFile keyword and
updates WHENCE to use that for currently known firmware files.

Juerg Haefliger (3):
  check_whence: Recognize RawFile keyword
  WHENCE: amd-ucode: Use new RawFile keyword
  WHENCE: Don't compress qcom json files

 WHENCE          | 47 +++++++++++++++++++++--------------------------
 check_whence.py |  6 +++---
 2 files changed, 24 insertions(+), 29 deletions(-)

-- 
2.39.2

[PATCH 1/3] check_whence: Recognize RawFile keyword

[Juerg Haefliger]

WHENCE and copy_firmware.sh now support a new keyword RawFile to list
files that must not be compressed. Update check_whence.py to recognize
that.

Signed-off-by: Juerg Haefliger <juerg.haefliger@canonical.com>
---
 check_whence.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/check_whence.py b/check_whence.py
index d45d90f8d720..936a3c480e33 100755
--- a/check_whence.py
+++ b/check_whence.py
@@ -6,11 +6,11 @@ from io import open
 def list_whence():
     with open('WHENCE', encoding='utf-8') as whence:
         for line in whence:
-            match = re.match(r'(?:File|Source):\s*"(.*)"', line)
+            match = re.match(r'(?:RawFile|File|Source):\s*"(.*)"', line)
             if match:
                 yield match.group(1)
                 continue
-            match = re.match(r'(?:File|Source):\s*(\S*)', line)
+            match = re.match(r'(?:RawFile|File|Source):\s*(\S*)', line)
             if match:
                 yield match.group(1)
                 continue
@@ -31,7 +31,7 @@ def list_whence():
 def list_whence_files():
     with open('WHENCE', encoding='utf-8') as whence:
         for line in whence:
-            match = re.match(r'File:\s*(.*)', line)
+            match = re.match(r'(?:RawFile|File):\s*(.*)', line)
             if match:
                 yield match.group(1).replace("\ ", " ").replace("\"", "")
                 continue
-- 
2.39.2

[PATCH 2/3] WHENCE: amd-ucode: Use new RawFile keyword

[Juerg Haefliger]

Replace old File/Raw entries with a single RawFile entry now that
copy-firmware and check_whence support that.

Signed-off-by: Juerg Haefliger <juerg.haefliger@canonical.com>
---
 WHENCE | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/WHENCE b/WHENCE
index 390a5689408e..852e88f55341 100644
--- a/WHENCE
+++ b/WHENCE
@@ -3912,20 +3912,15 @@ License: Redistributable. See LICENSE.amd-sev for details
 
 Driver: microcode_amd - AMD CPU Microcode Update Driver for Linux
 
-File: amd-ucode/microcode_amd.bin
-Raw: amd-ucode/microcode_amd.bin
+RawFile: amd-ucode/microcode_amd.bin
 Version: 2013-07-10
-File: amd-ucode/microcode_amd_fam15h.bin
-Raw: amd-ucode/microcode_amd_fam15h.bin
+RawFile: amd-ucode/microcode_amd_fam15h.bin
 Version: 2018-05-24
-File: amd-ucode/microcode_amd_fam16h.bin
-Raw: amd-ucode/microcode_amd_fam16h.bin
+RawFile: amd-ucode/microcode_amd_fam16h.bin
 Version: 2014-10-28
-File: amd-ucode/microcode_amd_fam17h.bin
-Raw: amd-ucode/microcode_amd_fam17h.bin
+RawFile: amd-ucode/microcode_amd_fam17h.bin
 Version: 2023-07-19
-File: amd-ucode/microcode_amd_fam19h.bin
-Raw: amd-ucode/microcode_amd_fam19h.bin
+RawFile: amd-ucode/microcode_amd_fam19h.bin
 Version: 2023-08-08
 File: amd-ucode/README
 
-- 
2.39.2

[PATCH 3/3] WHENCE: Don't compress qcom json files

[Juerg Haefliger]

Don't compress qcom json files. They are loaded by the pd-mapper userspace
app which doesn't know about compression.

Signed-off-by: Juerg Haefliger <juerg.haefliger@canonical.com>
---
 WHENCE | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/WHENCE b/WHENCE
index 852e88f55341..e057a5e49931 100644
--- a/WHENCE
+++ b/WHENCE
@@ -5630,28 +5630,28 @@ File: qcom/apq8016/modem.mbn
 File: qcom/apq8016/wcnss.mbn
 File: qcom/apq8016/WCNSS_qcom_wlan_nv_sbc.bin
 File: qcom/apq8096/adsp.mbn
-File: qcom/apq8096/adspr.jsn
-File: qcom/apq8096/adspua.jsn
+RawFile: qcom/apq8096/adspr.jsn
+RawFile: qcom/apq8096/adspua.jsn
 File: qcom/apq8096/mba.mbn
 File: qcom/apq8096/modem.mbn
-File: qcom/apq8096/modemr.jsn
+RawFile: qcom/apq8096/modemr.jsn
 File: qcom/sdm845/adsp.mbn
-File: qcom/sdm845/adspr.jsn
-File: qcom/sdm845/adspua.jsn
+RawFile: qcom/sdm845/adspr.jsn
+RawFile: qcom/sdm845/adspua.jsn
 File: qcom/sdm845/cdsp.mbn
-File: qcom/sdm845/cdspr.jsn
+RawFile: qcom/sdm845/cdspr.jsn
 File: qcom/sdm845/Thundercomm/db845c/slpi.mbn
-File: qcom/sdm845/Thundercomm/db845c/slpir.jsn
-File: qcom/sdm845/Thundercomm/db845c/slpius.jsn
+RawFile: qcom/sdm845/Thundercomm/db845c/slpir.jsn
+RawFile: qcom/sdm845/Thundercomm/db845c/slpius.jsn
 File: qcom/sm8250/adsp.mbn
-File: qcom/sm8250/adspr.jsn
-File: qcom/sm8250/adspua.jsn
+RawFile: qcom/sm8250/adspr.jsn
+RawFile: qcom/sm8250/adspua.jsn
 File: qcom/sm8250/cdsp.mbn
-File: qcom/sm8250/cdspr.jsn
-File: qcom/sc8280xp/LENOVO/21BX/adspr.jsn
-File: qcom/sc8280xp/LENOVO/21BX/adspua.jsn
-File: qcom/sc8280xp/LENOVO/21BX/battmgr.jsn
-File: qcom/sc8280xp/LENOVO/21BX/cdspr.jsn
+RawFile: qcom/sm8250/cdspr.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/adspr.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/adspua.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/battmgr.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/cdspr.jsn
 File: qcom/sc8280xp/LENOVO/21BX/qcadsp8280.mbn
 File: qcom/sc8280xp/LENOVO/21BX/qccdsp8280.mbn
 File: qcom/sc8280xp/LENOVO/21BX/qcslpi8280.mbn
@@ -5675,7 +5675,7 @@ Driver: qcom_q6v5_mss - Qualcomm modem subsystem firmware
 
 File: qcom/sdm845/mba.mbn
 File: qcom/sdm845/modem_nm.mbn
-File: qcom/sdm845/modemuw.jsn
+RawFile: qcom/sdm845/modemuw.jsn
 Link: qcom/sdm845/modem.mbn -> modem_nm.mbn
 
 Licence: Redistributable. See LICENSE.qcom and qcom/NOTICE.txt for details
-- 
2.39.2

Re: [PATCH 0/3] Support RawFile in check_whence and update WHENCE

[Mario Limonciello]

On 8/31/2023 03:13, Juerg Haefliger wrote:
> This series updates check_whence to understand the RawFile keyword and
> updates WHENCE to use that for currently known firmware files.
> 
> Juerg Haefliger (3):
>    check_whence: Recognize RawFile keyword
>    WHENCE: amd-ucode: Use new RawFile keyword
>    WHENCE: Don't compress qcom json files
> 
>   WHENCE          | 47 +++++++++++++++++++++--------------------------
>   check_whence.py |  6 +++---
>   2 files changed, 24 insertions(+), 29 deletions(-)
> 

I tried to apply this series to check it, but I think it got caught up 
in the new CI changes that we are introducing.

$ git am 
20230831_juerg_haefliger_support_rawfile_in_check_whence_and_update_whence.mbx
Applying: check_whence: Recognize RawFile keyword
error: patch failed: check_whence.py:6
error: check_whence.py: patch does not apply
Patch failed at 0001 check_whence: Recognize RawFile keyword
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

Can you re-base it?  If it's easier for you, you can also open a MR here 
now:

https://gitlab.com/kernel-firmware/linux-firmware

[PATCH v2 0/3] Support RawFile in check_whence and update WHENCE

[Juerg Haefliger]

This series updates check_whence to understand the RawFile keyword and
updates WHENCE to use that for currently known firmware files.

v2:
  - Rebase to latest upstream.

Juerg Haefliger (3):
  check_whence: Recognize RawFile keyword
  WHENCE: amd-ucode: Use new RawFile keyword
  WHENCE: Don't compress qcom json files

 WHENCE          | 47 +++++++++++++++++++++--------------------------
 check_whence.py |  6 +++---
 2 files changed, 24 insertions(+), 29 deletions(-)

-- 
2.39.2

[PATCH v2 1/3] check_whence: Recognize RawFile keyword

[Juerg Haefliger]

WHENCE and copy_firmware.sh now support a new keyword RawFile to list
files that must not be compressed. Update check_whence.py to recognize
that.

Signed-off-by: Juerg Haefliger <juerg.haefliger@canonical.com>
---
 check_whence.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/check_whence.py b/check_whence.py
index db6ac0f5d1c5..7e0a04f964da 100755
--- a/check_whence.py
+++ b/check_whence.py
@@ -7,11 +7,11 @@ from io import open
 def list_whence():
     with open("WHENCE", encoding="utf-8") as whence:
         for line in whence:
-            match = re.match(r'(?:File|Source):\s*"(.*)"', line)
+            match = re.match(r'(?:RawFile|File|Source):\s*"(.*)"', line)
             if match:
                 yield match.group(1)
                 continue
-            match = re.match(r"(?:File|Source):\s*(\S*)", line)
+            match = re.match(r"(?:RawFile|File|Source):\s*(\S*)", line)
             if match:
                 yield match.group(1)
                 continue
@@ -35,7 +35,7 @@ def list_whence():
 def list_whence_files():
     with open("WHENCE", encoding="utf-8") as whence:
         for line in whence:
-            match = re.match(r"File:\s*(.*)", line)
+            match = re.match(r"(?:RawFile|File):\s*(.*)", line)
             if match:
                 yield match.group(1).replace("\ ", " ").replace('"', "")
                 continue
-- 
2.39.2

[PATCH v2 2/3] WHENCE: amd-ucode: Use new RawFile keyword

[Juerg Haefliger]

Replace old File/Raw entries with a single RawFile entry now that
copy-firmware and check_whence supports that.

Signed-off-by: Juerg Haefliger <juerg.haefliger@canonical.com>
---
 WHENCE | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/WHENCE b/WHENCE
index 390a5689408e..852e88f55341 100644
--- a/WHENCE
+++ b/WHENCE
@@ -3912,20 +3912,15 @@ License: Redistributable. See LICENSE.amd-sev for details
 
 Driver: microcode_amd - AMD CPU Microcode Update Driver for Linux
 
-File: amd-ucode/microcode_amd.bin
-Raw: amd-ucode/microcode_amd.bin
+RawFile: amd-ucode/microcode_amd.bin
 Version: 2013-07-10
-File: amd-ucode/microcode_amd_fam15h.bin
-Raw: amd-ucode/microcode_amd_fam15h.bin
+RawFile: amd-ucode/microcode_amd_fam15h.bin
 Version: 2018-05-24
-File: amd-ucode/microcode_amd_fam16h.bin
-Raw: amd-ucode/microcode_amd_fam16h.bin
+RawFile: amd-ucode/microcode_amd_fam16h.bin
 Version: 2014-10-28
-File: amd-ucode/microcode_amd_fam17h.bin
-Raw: amd-ucode/microcode_amd_fam17h.bin
+RawFile: amd-ucode/microcode_amd_fam17h.bin
 Version: 2023-07-19
-File: amd-ucode/microcode_amd_fam19h.bin
-Raw: amd-ucode/microcode_amd_fam19h.bin
+RawFile: amd-ucode/microcode_amd_fam19h.bin
 Version: 2023-08-08
 File: amd-ucode/README
 
-- 
2.39.2

[PATCH v2 3/3] WHENCE: Don't compress qcom json files

[Juerg Haefliger]

Don't compress qcom json files. They are loaded by the pd-mapper userspace
app which doesn't know about compression.

Signed-off-by: Juerg Haefliger <juerg.haefliger@canonical.com>
---
 WHENCE | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/WHENCE b/WHENCE
index 852e88f55341..e057a5e49931 100644
--- a/WHENCE
+++ b/WHENCE
@@ -5630,28 +5630,28 @@ File: qcom/apq8016/modem.mbn
 File: qcom/apq8016/wcnss.mbn
 File: qcom/apq8016/WCNSS_qcom_wlan_nv_sbc.bin
 File: qcom/apq8096/adsp.mbn
-File: qcom/apq8096/adspr.jsn
-File: qcom/apq8096/adspua.jsn
+RawFile: qcom/apq8096/adspr.jsn
+RawFile: qcom/apq8096/adspua.jsn
 File: qcom/apq8096/mba.mbn
 File: qcom/apq8096/modem.mbn
-File: qcom/apq8096/modemr.jsn
+RawFile: qcom/apq8096/modemr.jsn
 File: qcom/sdm845/adsp.mbn
-File: qcom/sdm845/adspr.jsn
-File: qcom/sdm845/adspua.jsn
+RawFile: qcom/sdm845/adspr.jsn
+RawFile: qcom/sdm845/adspua.jsn
 File: qcom/sdm845/cdsp.mbn
-File: qcom/sdm845/cdspr.jsn
+RawFile: qcom/sdm845/cdspr.jsn
 File: qcom/sdm845/Thundercomm/db845c/slpi.mbn
-File: qcom/sdm845/Thundercomm/db845c/slpir.jsn
-File: qcom/sdm845/Thundercomm/db845c/slpius.jsn
+RawFile: qcom/sdm845/Thundercomm/db845c/slpir.jsn
+RawFile: qcom/sdm845/Thundercomm/db845c/slpius.jsn
 File: qcom/sm8250/adsp.mbn
-File: qcom/sm8250/adspr.jsn
-File: qcom/sm8250/adspua.jsn
+RawFile: qcom/sm8250/adspr.jsn
+RawFile: qcom/sm8250/adspua.jsn
 File: qcom/sm8250/cdsp.mbn
-File: qcom/sm8250/cdspr.jsn
-File: qcom/sc8280xp/LENOVO/21BX/adspr.jsn
-File: qcom/sc8280xp/LENOVO/21BX/adspua.jsn
-File: qcom/sc8280xp/LENOVO/21BX/battmgr.jsn
-File: qcom/sc8280xp/LENOVO/21BX/cdspr.jsn
+RawFile: qcom/sm8250/cdspr.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/adspr.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/adspua.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/battmgr.jsn
+RawFile: qcom/sc8280xp/LENOVO/21BX/cdspr.jsn
 File: qcom/sc8280xp/LENOVO/21BX/qcadsp8280.mbn
 File: qcom/sc8280xp/LENOVO/21BX/qccdsp8280.mbn
 File: qcom/sc8280xp/LENOVO/21BX/qcslpi8280.mbn
@@ -5675,7 +5675,7 @@ Driver: qcom_q6v5_mss - Qualcomm modem subsystem firmware
 
 File: qcom/sdm845/mba.mbn
 File: qcom/sdm845/modem_nm.mbn
-File: qcom/sdm845/modemuw.jsn
+RawFile: qcom/sdm845/modemuw.jsn
 Link: qcom/sdm845/modem.mbn -> modem_nm.mbn
 
 Licence: Redistributable. See LICENSE.qcom and qcom/NOTICE.txt for details
-- 
2.39.2

Re: [PATCH v2 0/3] Support RawFile in check_whence and update WHENCE

[Josh Boyer]

On Fri, Sep 1, 2023 at 2:52 AM Juerg Haefliger
<juerg.haefliger@canonical.com> wrote:
>
> This series updates check_whence to understand the RawFile keyword and
> updates WHENCE to use that for currently known firmware files.
>
> v2:
>   - Rebase to latest upstream.

All applied and pushed out.

josh

>
> Juerg Haefliger (3):
>   check_whence: Recognize RawFile keyword
>   WHENCE: amd-ucode: Use new RawFile keyword
>   WHENCE: Don't compress qcom json files
>
>  WHENCE          | 47 +++++++++++++++++++++--------------------------
>  check_whence.py |  6 +++---
>  2 files changed, 24 insertions(+), 29 deletions(-)
>
> --
> 2.39.2
>