[PATCH 0/2] tests/acceptance: Test Aspeed ARM machines

[PATCH 0/2] tests/acceptance: Test Aspeed ARM machines

[Joel Stanley]

This adds tests for the Aspeed ARM SoCs. The AST2400 and AST2500 tests
use OpenBMC images from that project, fetched from github releases. The
AST2600 test uses a Debian arm32 kernel.

Note that the ast2600 test will fail if [1] is not applied. I have
tested locally and all seems good.

I note that checkpatch warns about a couple of lines over 80 columns in
this file.  Can we agree to relax this restriction for this file, as it
helps the readability of the constants such as sha hashes and URLs?

[1] https://lore.kernel.org/qemu-devel/20210303010505.635621-1-joel@jms.id.au/T/#u

Joel Stanley (2):
  tests/acceptance: Test ast2400 and ast2500 machines
  tests/acceptance: Test ast2600 machine

 tests/acceptance/boot_linux_console.py | 72 ++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)

-- 
2.30.1

[PATCH 1/2] tests/acceptance: Test ast2400 and ast2500 machines

[Joel Stanley]

Test MTD images from the OpenBMC project on AST2400 and AST2500 SoCs
from ASPEED, by booting Palmetto and Romulus BMC machines.

The images are fetched from OpenBMC's release directory on github.

Co-developed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Joel Stanley <joel@jms.id.au>
---
 tests/acceptance/boot_linux_console.py | 46 ++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index eb012867997f..2f46a08fdc10 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -1048,6 +1048,52 @@ def test_arm_vexpressa9(self):
         self.vm.add_args('-dtb', self.workdir + '/day16/vexpress-v2p-ca9.dtb')
         self.do_test_advcal_2018('16', tar_hash, 'winter.zImage')
 
+    def test_arm_ast2400_palmetto_openbmc_v2_9_0(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:palmetto-bmc
+        """
+
+        image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
+                     'obmc-phosphor-image-palmetto.static.mtd')
+        image_hash = ('3e13bbbc28e424865dc42f35ad672b10f2e82cdb11846bb28fa625b48beafd0d')
+        image_path = self.fetch_asset(image_url, asset_hash=image_hash,
+                                      algorithm='sha256')
+
+        self.do_test_arm_aspeed(image_path)
+
+    def test_arm_ast2500_romulus_openbmc_v2_9_0(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:romulus-bmc
+        """
+
+        image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
+                     'obmc-phosphor-image-romulus.static.mtd')
+        image_hash = ('820341076803f1955bc31e647a512c79f9add4f5233d0697678bab4604c7bb25')
+        image_path = self.fetch_asset(image_url, asset_hash=image_hash,
+                                      algorithm='sha256')
+
+        self.do_test_arm_aspeed(image_path)
+
+    def do_test_arm_aspeed(self, image):
+        self.vm.set_console()
+        self.vm.add_args('-drive', 'file=' + image + ',if=mtd,format=raw',
+                         '-net', 'nic')
+        self.vm.launch()
+
+        self.wait_for_console_pattern("U-Boot 2016.07")
+        self.wait_for_console_pattern("## Loading kernel from FIT Image at 20080000")
+        self.wait_for_console_pattern("Starting kernel ...")
+        self.wait_for_console_pattern("Booting Linux on physical CPU 0x0")
+        self.wait_for_console_pattern(
+                "aspeed-smc 1e620000.spi: read control register: 203b0641")
+        self.wait_for_console_pattern("ftgmac100 1e660000.ethernet eth0: irq ")
+        self.wait_for_console_pattern("systemd[1]: Set hostname to")
+        # This often takes longer than the 90s timeout
+        # self.wait_for_console_pattern("login:")
+        self.vm.shutdown()
+
     def test_m68k_mcf5208evb(self):
         """
         :avocado: tags=arch:m68k
-- 
2.30.1

[PATCH 2/2] tests/acceptance: Test ast2600 machine

[Joel Stanley]

This tests a Debian multi-soc arm32 Linux kernel on the AST2600 based
Tacoma BMC machine.

There is no root file system so the test terminates when boot reaches
the stage where it attempts and fails to mount something.

Signed-off-by: Joel Stanley <joel@jms.id.au>
---
 tests/acceptance/boot_linux_console.py | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 2f46a08fdc10..6b7d7f5df984 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -1094,6 +1094,32 @@ def do_test_arm_aspeed(self, image):
         # self.wait_for_console_pattern("login:")
         self.vm.shutdown()
 
+    def test_arm_ast2600_debian(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:tacoma-bmc
+        """
+        deb_url = ('http://snapshot.debian.org/archive/debian/'
+                   '20210302T203551Z/'
+                   'pool/main/l/linux/'
+                   'linux-image-5.10.0-3-armmp_5.10.13-1_armhf.deb')
+        deb_hash = 'db40d32fe39255d05482bea48d72467b67d6225bb2a2a4d6f618cb8976f1e09e'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash,
+                                    algorithm='sha256')
+        kernel_path = self.extract_from_deb(deb_path, '/boot/vmlinuz-5.10.0-3-armmp')
+        dtb_path = self.extract_from_deb(deb_path,
+                '/usr/lib/linux-image-5.10.0-3-armmp/aspeed-bmc-opp-tacoma.dtb')
+
+        self.vm.set_console()
+        self.vm.add_args('-kernel', kernel_path,
+                         '-dtb', dtb_path)
+        self.vm.launch()
+        self.wait_for_console_pattern("Booting Linux on physical CPU 0xf00")
+        self.wait_for_console_pattern("SMP: Total of 2 processors activated")
+        self.wait_for_console_pattern("ftgmac100 1e670000.ftgmac eth0: irq")
+        self.wait_for_console_pattern("VFS: Cannot open root device")
+        self.vm.shutdown()
+
     def test_m68k_mcf5208evb(self):
         """
         :avocado: tags=arch:m68k
-- 
2.30.1

Re: [PATCH 2/2] tests/acceptance: Test ast2600 machine

[Cédric Le Goater]

On 3/3/21 2:22 AM, Joel Stanley wrote:
> This tests a Debian multi-soc arm32 Linux kernel on the AST2600 based
> Tacoma BMC machine.
> 
> There is no root file system so the test terminates when boot reaches
> the stage where it attempts and fails to mount something.

because of that, the test always fails  :

console: [    3.615193] No filesystem could mount root, tried:
console: [    3.615269]
console: [    3.615688] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
FAIL: Failure message found in console: Kernel panic - not syncing (22.45 s)
RESULTS    : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0


Is there a way around this problem ? 

C. 

> 
> Signed-off-by: Joel Stanley <joel@jms.id.au>
> ---
>  tests/acceptance/boot_linux_console.py | 26 ++++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 2f46a08fdc10..6b7d7f5df984 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -1094,6 +1094,32 @@ def do_test_arm_aspeed(self, image):
>          # self.wait_for_console_pattern("login:")
>          self.vm.shutdown()
>  
> +    def test_arm_ast2600_debian(self):
> +        """
> +        :avocado: tags=arch:arm
> +        :avocado: tags=machine:tacoma-bmc
> +        """
> +        deb_url = ('http://snapshot.debian.org/archive/debian/'
> +                   '20210302T203551Z/'
> +                   'pool/main/l/linux/'
> +                   'linux-image-5.10.0-3-armmp_5.10.13-1_armhf.deb')
> +        deb_hash = 'db40d32fe39255d05482bea48d72467b67d6225bb2a2a4d6f618cb8976f1e09e'
> +        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash,
> +                                    algorithm='sha256')
> +        kernel_path = self.extract_from_deb(deb_path, '/boot/vmlinuz-5.10.0-3-armmp')
> +        dtb_path = self.extract_from_deb(deb_path,
> +                '/usr/lib/linux-image-5.10.0-3-armmp/aspeed-bmc-opp-tacoma.dtb')
> +
> +        self.vm.set_console()
> +        self.vm.add_args('-kernel', kernel_path,
> +                         '-dtb', dtb_path)
> +        self.vm.launch()
> +        self.wait_for_console_pattern("Booting Linux on physical CPU 0xf00")
> +        self.wait_for_console_pattern("SMP: Total of 2 processors activated")
> +        self.wait_for_console_pattern("ftgmac100 1e670000.ftgmac eth0: irq")
> +        self.wait_for_console_pattern("VFS: Cannot open root device")
> +        self.vm.shutdown()
> +
>      def test_m68k_mcf5208evb(self):
>          """
>          :avocado: tags=arch:m68k
> 

Re: [PATCH 1/2] tests/acceptance: Test ast2400 and ast2500 machines

[Cédric Le Goater]

On 3/3/21 2:22 AM, Joel Stanley wrote:
> Test MTD images from the OpenBMC project on AST2400 and AST2500 SoCs
> from ASPEED, by booting Palmetto and Romulus BMC machines.
> 
> The images are fetched from OpenBMC's release directory on github.
> 
> Co-developed-by: Cédric Le Goater <clg@kaod.org>
> Signed-off-by: Joel Stanley <joel@jms.id.au>


Reviewed-by: Cédric Le Goater <clg@kaod.org>
Tested-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C. 


> ---
>  tests/acceptance/boot_linux_console.py | 46 ++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index eb012867997f..2f46a08fdc10 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -1048,6 +1048,52 @@ def test_arm_vexpressa9(self):
>          self.vm.add_args('-dtb', self.workdir + '/day16/vexpress-v2p-ca9.dtb')
>          self.do_test_advcal_2018('16', tar_hash, 'winter.zImage')
>  
> +    def test_arm_ast2400_palmetto_openbmc_v2_9_0(self):
> +        """
> +        :avocado: tags=arch:arm
> +        :avocado: tags=machine:palmetto-bmc
> +        """
> +
> +        image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
> +                     'obmc-phosphor-image-palmetto.static.mtd')
> +        image_hash = ('3e13bbbc28e424865dc42f35ad672b10f2e82cdb11846bb28fa625b48beafd0d')
> +        image_path = self.fetch_asset(image_url, asset_hash=image_hash,
> +                                      algorithm='sha256')
> +
> +        self.do_test_arm_aspeed(image_path)
> +
> +    def test_arm_ast2500_romulus_openbmc_v2_9_0(self):
> +        """
> +        :avocado: tags=arch:arm
> +        :avocado: tags=machine:romulus-bmc
> +        """
> +
> +        image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
> +                     'obmc-phosphor-image-romulus.static.mtd')
> +        image_hash = ('820341076803f1955bc31e647a512c79f9add4f5233d0697678bab4604c7bb25')
> +        image_path = self.fetch_asset(image_url, asset_hash=image_hash,
> +                                      algorithm='sha256')
> +
> +        self.do_test_arm_aspeed(image_path)
> +
> +    def do_test_arm_aspeed(self, image):
> +        self.vm.set_console()
> +        self.vm.add_args('-drive', 'file=' + image + ',if=mtd,format=raw',
> +                         '-net', 'nic')
> +        self.vm.launch()
> +
> +        self.wait_for_console_pattern("U-Boot 2016.07")
> +        self.wait_for_console_pattern("## Loading kernel from FIT Image at 20080000")
> +        self.wait_for_console_pattern("Starting kernel ...")
> +        self.wait_for_console_pattern("Booting Linux on physical CPU 0x0")
> +        self.wait_for_console_pattern(
> +                "aspeed-smc 1e620000.spi: read control register: 203b0641")
> +        self.wait_for_console_pattern("ftgmac100 1e660000.ethernet eth0: irq ")
> +        self.wait_for_console_pattern("systemd[1]: Set hostname to")
> +        # This often takes longer than the 90s timeout
> +        # self.wait_for_console_pattern("login:")
> +        self.vm.shutdown()
> +
>      def test_m68k_mcf5208evb(self):
>          """
>          :avocado: tags=arch:m68k
> 

Re: [PATCH 2/2] tests/acceptance: Test ast2600 machine

[Joel Stanley]

On Wed, 3 Mar 2021 at 07:43, Cédric Le Goater <clg@kaod.org> wrote:
>
> On 3/3/21 2:22 AM, Joel Stanley wrote:
> > This tests a Debian multi-soc arm32 Linux kernel on the AST2600 based
> > Tacoma BMC machine.
> >
> > There is no root file system so the test terminates when boot reaches
> > the stage where it attempts and fails to mount something.
>
> because of that, the test always fails  :
>
> console: [    3.615193] No filesystem could mount root, tried:
> console: [    3.615269]
> console: [    3.615688] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
> FAIL: Failure message found in console: Kernel panic - not syncing (22.45 s)
> RESULTS    : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

You're right. I must have been testing a different setup when I put
these checks in:

> > +        self.wait_for_console_pattern("ftgmac100 1e670000.ftgmac eth0: irq")
> > +        self.wait_for_console_pattern("VFS: Cannot open root device")

The second string is different to what we see with the kernel we're
booting, and more importantly the first string doesn't ever appear.
This is because the kernel driver is built as a module, and with no
filesystem we don't have any modules to load. IIRC the test is still
waiting on the ftgmac string when it sees the panic, so even if the
VFS string was correct it wouldn't save us.

This will sort it out:

 -        self.wait_for_console_pattern("ftgmac100 1e670000.ftgmac eth0: irq")
 -        self.wait_for_console_pattern("VFS: Cannot open root device")
 +        self.wait_for_console_pattern("No filesystem could mount root")

I'll send a v2. Thanks for testing.

Cheers,

Joel

Re: [PATCH 1/2] tests/acceptance: Test ast2400 and ast2500 machines

[Philippe Mathieu-Daudé]

On 3/3/21 2:22 AM, Joel Stanley wrote:
> Test MTD images from the OpenBMC project on AST2400 and AST2500 SoCs
> from ASPEED, by booting Palmetto and Romulus BMC machines.
> 
> The images are fetched from OpenBMC's release directory on github.

You need to justify here why this is safe to run that
on anyone workstation, or use AVOCADO_ALLOW_UNTRUSTED_CODE.

> 
> Co-developed-by: Cédric Le Goater <clg@kaod.org>
> Signed-off-by: Joel Stanley <joel@jms.id.au>
> ---
>  tests/acceptance/boot_linux_console.py | 46 ++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index eb012867997f..2f46a08fdc10 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -1048,6 +1048,52 @@ def test_arm_vexpressa9(self):
>          self.vm.add_args('-dtb', self.workdir + '/day16/vexpress-v2p-ca9.dtb')
>          self.do_test_advcal_2018('16', tar_hash, 'winter.zImage')
>  
> +    def test_arm_ast2400_palmetto_openbmc_v2_9_0(self):
> +        """
> +        :avocado: tags=arch:arm
> +        :avocado: tags=machine:palmetto-bmc
> +        """
> +
> +        image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
> +                     'obmc-phosphor-image-palmetto.static.mtd')
> +        image_hash = ('3e13bbbc28e424865dc42f35ad672b10f2e82cdb11846bb28fa625b48beafd0d')
> +        image_path = self.fetch_asset(image_url, asset_hash=image_hash,
> +                                      algorithm='sha256')
> +
> +        self.do_test_arm_aspeed(image_path)

Re: [PATCH 1/2] tests/acceptance: Test ast2400 and ast2500 machines

[Joel Stanley]

On Wed, 3 Mar 2021 at 10:19, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> On 3/3/21 2:22 AM, Joel Stanley wrote:
> > Test MTD images from the OpenBMC project on AST2400 and AST2500 SoCs
> > from ASPEED, by booting Palmetto and Romulus BMC machines.
> >
> > The images are fetched from OpenBMC's release directory on github.
>
> You need to justify here why this is safe to run that
> on anyone workstation, or use AVOCADO_ALLOW_UNTRUSTED_CODE.

I don't completely understand. What circumstances would it be unsafe
to run a qemu guest, aside from a bug in qemu itself?

Cheers,

Joel

Re: [PATCH 1/2] tests/acceptance: Test ast2400 and ast2500 machines

[Philippe Mathieu-Daudé]

On 3/3/21 12:19 PM, Joel Stanley wrote:
> On Wed, 3 Mar 2021 at 10:19, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>>
>> On 3/3/21 2:22 AM, Joel Stanley wrote:
>>> Test MTD images from the OpenBMC project on AST2400 and AST2500 SoCs
>>> from ASPEED, by booting Palmetto and Romulus BMC machines.
>>>
>>> The images are fetched from OpenBMC's release directory on github.
>>
>> You need to justify here why this is safe to run that
>> on anyone workstation, or use AVOCADO_ALLOW_UNTRUSTED_CODE.
> 
> I don't completely understand. What circumstances would it be unsafe
> to run a qemu guest, aside from a bug in qemu itself?

See Alex's rationale here:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg774096.html

Point taken, this should be better documented.