Re: [OE-core] [RFC][PATCH] oeqa/runtime/cases: new image_upgrade test

[Alexander Kanavin <alex.kanavin@gmail.com>]

On Thu, 25 Apr 2024 at 17:46, Michael Opdenacker via
lists.openembedded.org
<michael.opdenacker=bootlin.com@lists.openembedded.org> wrote:
> +++ b/meta/lib/oeqa/runtime/cases/opkg_sysupgrade.py

This should perhaps go to meta-selftest/lib/oeqa/runtime/cases/ as it
needs special setup via selftest, and not useful in standalone '-c
testimage' runs.

> +    def test_image_upgrade(self):
> +        """
> +        Summary: Test that generated ipk packages can
> +        be used to upgrade an older image version.
> +        This is done by generating an image but then replacing it
> +        by an older image shared by the Yocto Project autobuilder.
> +        We then run QEMU on the old image and replace the original
> +        original package feeds by our own.
> +        """
> +
> +        image = 'core-image-full-cmdline'
> +        machine = get_bb_var("MACHINE")
> +
> +        features = 'DISTRO = "poky-altcfg"\n'
> +        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
> +        features += 'PACKAGE_CLASSES = "package_ipk"\n'
> +        features += 'IMAGE_CLASSES += "testimage"\n'
> +        features += 'TEST_SUITES="opkg_sysupgrade"\n'
> +        self.write_config(features)
> +
> +        # Need to build a full image to build the .json file needed by QEMU.
> +        # Therefore, it is not sufficient to run only "package_write_ipk" for the image.
> +
> +        self.logger.info("Generating '%s' and package index for latest commit in this branch..." % image)
> +        bitbake(image)
> +        bitbake('package-index')
> +
> +        # Download previously generated image
> +
> +        image_file = '%s-%s.rootfs.ext4' % (image, machine)
> +        image_path = '%s/tmp/deploy/images/%s/%s' % (self.builddir, machine, image_file)
> +        machine_variant = '-alt'
> +
> +        os.remove(image_path)
> +        image_url = get_latest_image_url(machine, machine_variant, image_file)
> +        self.logger.info("Downloading image: %s..." % image_url)
> +        cmd = 'wget -O %s %s' % (image_path, image_url)
> +        result = runCmd(cmd)
> +        self.assertEqual(0, result.status, cmd + ' returned a non 0 status: %s' % result.output)
> +
> +        # Now run the upgrade tests on the old image
> +
> +        self.logger.info("Running upgrade tests on the downloaded image, using the package feeds generated here...")
> +        bitbake(image + ' -c testimage')

RP perhaps already pointed out this, but this mashes together
parameters and implementation. Please split the implementation into
'non-opinionated' functions that take everything that can change via
arguments, and have a short and sweet top level test definition that
does something like this:

def test_poky_altcfg_cmdline_update(self):
   config = 'DISTRO=..., etc'
   imagelocation = find_yocto_image(...)
   run_update_test(config, imagelocation, ...)

As an example, CDN sstate tests are arranged similarly:
https://git.yoctoproject.org/poky/tree/meta/lib/oeqa/selftest/cases/sstatetests.py#n920

Then we can further improve this by defining those parameters
somewhere external, that's a separate decision.

Alex