[PATCH 0/2] Adding cmake selftest

[PATCH 0/2] Adding cmake selftest

[Armin Kuster]

This is my first selftest so any feedback is welcome.

This selftest should help catch regressions when boost is updated and cmake is not. It appears cmake may be version sensitive to some packages so we may want to expand these selftests for other packages that live in core.  I got the testcase from Dennis Menschel via Yocto bug 12762 so many thanks.
 

Armin Kuster (2):
  selftest: add cmake test
  selftest: add cmake oeqa portion

 .../recipes-test/cmake/cmake-boost-version-test.bb | 20 ++++++++++
 .../recipes-test/cmake/files/CMakeLists.txt        | 24 ++++++++++++
 .../recipes-test/cmake/files/helloworld.cpp        |  7 ++++
 meta/lib/oeqa/selftest/cases/cmake.py              | 43 ++++++++++++++++++++++
 4 files changed, 94 insertions(+)
 create mode 100644 meta-selftest/recipes-test/cmake/cmake-boost-version-test.bb
 create mode 100644 meta-selftest/recipes-test/cmake/files/CMakeLists.txt
 create mode 100644 meta-selftest/recipes-test/cmake/files/helloworld.cpp
 create mode 100644 meta/lib/oeqa/selftest/cases/cmake.py

-- 
2.7.4

[PATCH 1/2] selftest: add cmake test

[Armin Kuster]

This came out of Yocto bug 12762. We need to ensure cmake will support
various packages in core as they update.

Test case provided by Dennis Menschel <menschel-d@posteo.de>, thanks

Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 .../recipes-test/cmake/cmake-boost-version-test.bb | 20 ++++++++++++++++++
 .../recipes-test/cmake/files/CMakeLists.txt        | 24 ++++++++++++++++++++++
 .../recipes-test/cmake/files/helloworld.cpp        |  7 +++++++
 3 files changed, 51 insertions(+)
 create mode 100644 meta-selftest/recipes-test/cmake/cmake-boost-version-test.bb
 create mode 100644 meta-selftest/recipes-test/cmake/files/CMakeLists.txt
 create mode 100644 meta-selftest/recipes-test/cmake/files/helloworld.cpp

diff --git a/meta-selftest/recipes-test/cmake/cmake-boost-version-test.bb b/meta-selftest/recipes-test/cmake/cmake-boost-version-test.bb
new file mode 100644
index 0000000..7bfd6ef
--- /dev/null
+++ b/meta-selftest/recipes-test/cmake/cmake-boost-version-test.bb
@@ -0,0 +1,20 @@
+#
+# This file was derived from the 'Hello World!' example recipe in the
+# Yocto Project Development Manual.
+#
+
+SUMMARY = "Simple helloworld application"
+SECTION = "examples"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
+
+DEPENDS = "boost"
+
+SRC_URI = " \
+    file://helloworld.cpp \
+    file://CMakeLists.txt \
+"
+
+S = "${WORKDIR}"
+
+inherit cmake
diff --git a/meta-selftest/recipes-test/cmake/files/CMakeLists.txt b/meta-selftest/recipes-test/cmake/files/CMakeLists.txt
new file mode 100644
index 0000000..2dc2c3f
--- /dev/null
+++ b/meta-selftest/recipes-test/cmake/files/CMakeLists.txt
@@ -0,0 +1,24 @@
+cmake_minimum_required(VERSION 3.8)
+
+project(example)
+
+find_package(Boost 1.60 REQUIRED
+    COMPONENTS
+        unit_test_framework
+)
+
+add_executable(example
+    helloworld.cpp
+)
+
+target_link_libraries(example
+    PRIVATE
+        Boost::unit_test_framework
+)
+
+install(
+    TARGETS
+        example
+    DESTINATION
+        bin
+)
diff --git a/meta-selftest/recipes-test/cmake/files/helloworld.cpp b/meta-selftest/recipes-test/cmake/files/helloworld.cpp
new file mode 100644
index 0000000..81ef9b9
--- /dev/null
+++ b/meta-selftest/recipes-test/cmake/files/helloworld.cpp
@@ -0,0 +1,7 @@
+#include <iostream>
+
+int main(int argc, char *argv[])
+{
+    std::cout << "Hello World!" << std::endl;
+    return 0;
+}
-- 
2.7.4

[PATCH 2/2] selftest: add cmake oeqa portion

[Armin Kuster]

add boost
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/lib/oeqa/selftest/cases/cmake.py | 43 +++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/cmake.py

diff --git a/meta/lib/oeqa/selftest/cases/cmake.py b/meta/lib/oeqa/selftest/cases/cmake.py
new file mode 100644
index 0000000..aaf7f95
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/cmake.py
@@ -0,0 +1,43 @@
+#!/usr/bin/env python
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+
+"""Test cases for cmake."""
+
+import os
+import sys
+import unittest
+from glob import glob
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, runCmd
+from oeqa.core.decorator.oeid import OETestID
+
+class CmakeSelfTests(OESelftestTestCase):
+    """cmake test class."""
+
+    @classmethod
+    def setUpClass(cls):
+        super(CmakeSelfTests, cls).setUpClass()
+        bitbake("core-image-minimal")
+
+    @OETestID(3000)
+    def test_boost_vesrion_check(self):
+        """Build Hello world with boost support"""
+        result = runCmd("bitbake cmake-boost-version-test")
+        self.assertEqual(0, result.status)
+
-- 
2.7.4

Re: [PATCH 2/2] selftest: add cmake oeqa portion

[Alexander Kanavin]

2018-06-09 23:00 GMT+03:00 Armin Kuster <akuster808@gmail.com>:
> +        bitbake("core-image-minimal")

This shouldn't be needed as the test doesn't do anything with the image.

Generally, when writing selftests please do pay attention to making
them as fast as possible, as the full selftest run already takes 3-4
hours even on the powerful machine (the AB for instance), and thus is
on the critical path for merging things in master.

Alex

Re: [PATCH 1/2] selftest: add cmake test

[Alexander Kanavin]

2018-06-09 23:00 GMT+03:00 Armin Kuster <akuster808@gmail.com>:
> +DEPENDS = "boost"



> +find_package(Boost 1.60 REQUIRED
> +    COMPONENTS
> +        unit_test_framework
> +)


Is it possible to use something else than boost here, if we want to
test cmake's ability to find compoments? So that the build time is as
quick as possible - boost isn't particularly fast.

Alex

Re: [PATCH 1/2] selftest: add cmake test

[akuster808]

On 06/10/2018 12:25 AM, Alexander Kanavin wrote:
> 2018-06-09 23:00 GMT+03:00 Armin Kuster <akuster808@gmail.com>:
>> +DEPENDS = "boost"
>
>
>> +find_package(Boost 1.60 REQUIRED
>> +    COMPONENTS
>> +        unit_test_framework
>> +)
>
> Is it possible to use something else than boost here, if we want to
> test cmake's ability to find compoments? 
I am not a cmake expert by any means. I just took a testcase given that
found a problem that I fixed earlier this week. I have not other idea
one would ensure folks don't break cmake when the update packages.

> So that the build time is as
> quick as possible - boost isn't particularly fast.

- armin
>
> Alex

Re: [PATCH 1/2] selftest: add cmake test

[Burton, Ross]

Well the problem in 12762 is that cmake *hardcodes a list of valid
boost versions*, and I'm struggling to understand why this would seem
like a good idea.

Some thoughts:
1) We definitely need a selftest for cmake.  We have a few build-stuff
test cases already (lzip, cpio, galculator in sdk and runtime, some
other in devtool's selftest).  I believe they need consolidation
(repeated for runtime and SDK tests) and review (they're all
autotools-based).  We should definitely be exercising that the build
systems work on target.
2) 12762 is a very specific problem with cmake.  Do we want to add a
whole selftest just for this?  Would it be easier to just patch cmake
to handle all versions?

Ross

On 10 June 2018 at 17:04, akuster808 <akuster808@gmail.com> wrote:
>
>
> On 06/10/2018 12:25 AM, Alexander Kanavin wrote:
>> 2018-06-09 23:00 GMT+03:00 Armin Kuster <akuster808@gmail.com>:
>>> +DEPENDS = "boost"
>>
>>
>>> +find_package(Boost 1.60 REQUIRED
>>> +    COMPONENTS
>>> +        unit_test_framework
>>> +)
>>
>> Is it possible to use something else than boost here, if we want to
>> test cmake's ability to find compoments?
> I am not a cmake expert by any means. I just took a testcase given that
> found a problem that I fixed earlier this week. I have not other idea
> one would ensure folks don't break cmake when the update packages.
>
>> So that the build time is as
>> quick as possible - boost isn't particularly fast.
>
> - armin
>>
>> Alex
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 2/2] selftest: add cmake oeqa portion

[Burton, Ross]

On 9 June 2018 at 21:00, Armin Kuster <akuster808@gmail.com> wrote:
> add boost

Explain what this test is meant to do so in two years time we're not
struggling to understand why.

>  create mode 100644 meta/lib/oeqa/selftest/cases/cmake.py

Squash with the previous commit.

> +from glob import glob

Not needed.

> +from oeqa.core.decorator.oeid import OETestID

Not needed becase...

> +class CmakeSelfTests(OESelftestTestCase):
> +    @OETestID(3000)

https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=3000
"Test case 3000 does not exist"

Don't invent test IDs.

Ross