[oe][Patch 0/2] udev: fix ptest to not influence qemu

[oe][Patch 0/2] udev: fix ptest to not influence qemu

[Max Krummenacher]

From: Max Krummenacher <max.oss.09@gmail.com<>

Hi

On a OE setup configured to build ptest packages udev's ptest test files are
staged to the sysroot.
A run of qemu chockes on this test files and never returns.

A ptest run on the target before any changes failed.
Patch 1/2 first fixes ptest.
Patch 2/2 keeps the test files in its tar-ball. run-ptest unpacks it before
a test and cleans up afterwards.

This fixes the issues with qemu while the ptests can still be run witout
failure.

Tested with distro-less oe-core master on qemux86-64 and 
colibri-t30 (armv7a) machines

Regards
Max

Max Krummenacher (2):
  udev: fix ptest rule syntax check
  udev: don't keep ptest testdata laying around

 meta/recipes-core/udev/udev.inc                    |  3 +-
 .../recipes-core/udev/udev/add-install-ptest.patch | 19 +++++--
 .../udev/udev/fix_rule-syntax-regex-ptest.patch    | 59 ++++++++++++++++++++++
 meta/recipes-core/udev/udev/run-ptest              |  2 +
 4 files changed, 79 insertions(+), 4 deletions(-)
 create mode 100644 meta/recipes-core/udev/udev/fix_rule-syntax-regex-ptest.patch

-- 
1.8.4.5

[oe][Patch 1/2] udev: fix ptest rule syntax check

[Max Krummenacher]

From: Max Krummenacher <max.oss.09@gmail.com>

The ptest which checks for correct udev rules fails.
Missing files and paths for the build host caused this.

Signed-off-by: Max Krummenacher <max.oss.09@gmail.com>
---
 meta/recipes-core/udev/udev.inc                    |  3 +-
 .../recipes-core/udev/udev/add-install-ptest.patch | 17 ++++++-
 .../udev/udev/fix_rule-syntax-regex-ptest.patch    | 59 ++++++++++++++++++++++
 3 files changed, 76 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-core/udev/udev/fix_rule-syntax-regex-ptest.patch

diff --git a/meta/recipes-core/udev/udev.inc b/meta/recipes-core/udev/udev.inc
index 280da10..24463b1 100644
--- a/meta/recipes-core/udev/udev.inc
+++ b/meta/recipes-core/udev/udev.inc
@@ -27,11 +27,12 @@ SRC_URI = "${KERNELORG_MIRROR}/linux/utils/kernel/hotplug/udev-${PV}.tar.gz \
            file://udev-cache \
            file://udev-cache.default \
            file://add-install-ptest.patch \
+           file://fix_rule-syntax-regex-ptest.patch \
            file://run-ptest \
            file://init"
 
 inherit autotools pkgconfig update-rc.d ptest
-RDEPENDS_${PN}-ptest += "make perl"
+RDEPENDS_${PN}-ptest += "make perl python"
 
 libexecdir = "${base_libdir}"
 EXTRA_OECONF = "--disable-introspection \
diff --git a/meta/recipes-core/udev/udev/add-install-ptest.patch b/meta/recipes-core/udev/udev/add-install-ptest.patch
index 0f8e9b6..755946a 100644
--- a/meta/recipes-core/udev/udev/add-install-ptest.patch
+++ b/meta/recipes-core/udev/udev/add-install-ptest.patch
@@ -8,19 +8,32 @@ Signed-off-by: Björn Stenberg <bjst@enea.com>
 Signed-off-by: Alexandra Safta <alst@enea.com>
 Upstream-Status: Pending
 
+Add missing files for rule-syntax-check
+- Add rule-syntax-check.py
+- Add the deployed udev rules to the testdata
+
+Signed-off-by: Max Krummenacher <max.oss.09@gmail.com>
+Upstream-Status: Pending
 --- a/Makefile.am	2012-03-18 16:28:14.000000000 +0100
 +++ b/Makefile.am	2013-02-18 10:03:36.531101244 +0100
-@@ -708,3 +708,11 @@
+@@ -708,3 +708,18 @@
  	for i in src/docs/html/*.{html,css,png}; do echo $$i; kup put $$i $$i.sign /pub/linux/utils/kernel/hotplug/libudev/; done
  	for i in src/gudev/docs/html/*.{html,css,png}; do rm -f $$i.sign; gpg --armor --detach-sign --output=$$i.sign $$i; done
  	for i in src/gudev/docs/html/*.{html,css,png}; do echo $$i; kup put $$i $$i.sign /pub/linux/utils/kernel/hotplug/gudev/; done
 +
++RULES = rules/* src/accelerometer/61-accelerometer.rules \
++	 src/cdrom_id/60-cdrom_id.rules \
++	 src/keymap/95-keyboard-force-release.rules src/keymap/95-keymap.rules \
++	 src/mtd_probe/75-probe_mtd.rules src/v4l_id/60-persistent-v4l.rules
++
 +install-ptest:
 +	install test-udev $(DESTDIR)
 +	cp Makefile $(DESTDIR)
 +	sed -i -e 's|^Makefile:|_Makefile:|' $(DESTDIR)/Makefile
-+	$(MKDIR_P) $(DESTDIR)/test
++	install -d $(DESTDIR)/test $(DESTDIR)/rules
 +	(cd $(top_srcdir) && install $(TESTS) $(DESTDIR)/test)
++	(cd $(top_srcdir) && install test/rule-syntax-check.py $(DESTDIR)/test)
++	(cd $(top_srcdir) && install $(RULES) $(DESTDIR)/rules)
 +	tar -C $(DESTDIR)/test/ -xJf $(top_srcdir)/test/sys.tar.xz
 --- a/test/udev-test.pl	2012-03-18 16:43:36.000000000 +0100
 +++ b/test/udev-test.pl	2013-02-18 10:31:29.706357321 +0100
diff --git a/meta/recipes-core/udev/udev/fix_rule-syntax-regex-ptest.patch b/meta/recipes-core/udev/udev/fix_rule-syntax-regex-ptest.patch
new file mode 100644
index 0000000..548a241
--- /dev/null
+++ b/meta/recipes-core/udev/udev/fix_rule-syntax-regex-ptest.patch
@@ -0,0 +1,59 @@
+The rule-syntax-check script fails with errors like this:
+
+Invalid line /lib/udev/rules.d/95-keymap.rules:49: ENV{DMI_VENDOR}=="ASUS*", KERNELS=="input*", ATTRS{name}=="Asus Extra Buttons", ATTR{[dmi/id]product_name}=="W3J", RUN+="keymap $name module-asus-w3j"
+('  clause:', 'ATTR{[dmi/id]product_name}=="W3J"')
+()
+
+
+Move line comment from end of rules file to its own line, the regex used to
+test correct syntax choke on it.
+
+The rule-syntax-check.py uses regex which errournessly complains on ATTR rules
+of the form ATTR{[dmi/id]board_name}=="30B7"
+Use the regex from systemd's script which allow [] characters and additional
+compare operators
+
+The Makefile passes rules-test.sh script a build host path to the testdata.
+Ignore the argument and use a relative path instead.
+
+
+Signed-off-by: Max Krummenacher <max.oss.09@gmail.com>
+Upstream status pending
+Upstream status Inappropriate (cross environment path)
+
+diff -Naur udev-182.orig/src/keymap/95-keymap.rules udev-182/src/keymap/95-keymap.rules
+--- udev-182.orig/src/keymap/95-keymap.rules	2012-02-07 00:01:55.154640792 +0100
++++ udev-182/src/keymap/95-keymap.rules	2015-01-24 20:58:40.156930520 +0100
+@@ -94,7 +94,8 @@
+ ENV{DMI_VENDOR}=="Hewlett-Packard", ATTR{[dmi/id]product_name}=="HP G62 Notebook PC", RUN+="keymap $name 0xB2 www"
+ ENV{DMI_VENDOR}=="Hewlett-Packard", ATTR{[dmi/id]product_name}=="HP ProBook*", RUN+="keymap $name 0xF8 rfkill"
+ # HP Pavillion dv6315ea has empty DMI_VENDOR
+-ATTR{[dmi/id]board_vendor}=="Quanta", ATTR{[dmi/id]board_name}=="30B7", ATTR{[dmi/id]board_version}=="65.2B", RUN+="keymap $name 0x88 media" # "quick play
++# "quick play"
++ATTR{[dmi/id]board_vendor}=="Quanta", ATTR{[dmi/id]board_name}=="30B7", ATTR{[dmi/id]board_version}=="65.2B", RUN+="keymap $name 0x88 media"
+ 
+ # Gateway clone of Acer Aspire One AOA110/AOA150
+ ENV{DMI_VENDOR}=="Gateway*", ATTR{[dmi/id]product_name}=="*AOA1*", RUN+="keymap $name acer"
+diff -Naur udev-182.orig/test/rule-syntax-check.py udev-182/test/rule-syntax-check.py
+--- udev-182.orig/test/rule-syntax-check.py	2012-02-15 20:10:12.872333342 +0100
++++ udev-182/test/rule-syntax-check.py	2015-01-24 21:08:00.496049600 +0100
+@@ -28,7 +28,7 @@
+ no_args_tests = re.compile('(ACTION|DEVPATH|KERNELS?|NAME|SYMLINK|SUBSYSTEMS?|DRIVERS?|TAG|RESULT|TEST)\s*(?:=|!)=\s*"([^"]*)"$')
+ args_tests = re.compile('(ATTRS?|ENV|TEST){([a-zA-Z0-9/_.*%-]+)}\s*(?:=|!)=\s*"([^"]*)"$')
+ no_args_assign = re.compile('(NAME|SYMLINK|OWNER|GROUP|MODE|TAG|PROGRAM|RUN|LABEL|GOTO|WAIT_FOR|OPTIONS|IMPORT)\s*(?:\+=|:=|=)\s*"([^"]*)"$')
+-args_assign = re.compile('(ATTR|ENV|IMPORT){([a-zA-Z0-9/_.*%-]+)}\s*=\s*"([^"]*)"$')
++args_assign = re.compile('(ATTR|ENV|IMPORT|RUN){([][a-zA-Z0-9/_.*%-]+)}\s*(=|==|\+=)\s*"([^"]*)"$')
+ 
+ result = 0
+ buffer = ''
+--- udev-182.orig/test/rules-test.sh	2012-01-29 01:15:46.000000000 +0100
++++ udev-182/test/rules-test.sh	2015-01-24 17:53:51.201858658 +0100
+@@ -4,7 +4,7 @@
+ # (C) 2010 Canonical Ltd.
+ # Author: Martin Pitt <martin.pitt@ubuntu.com>
+ 
+-[ -n "$srcdir" ] || srcdir=`dirname $0`/..
++srcdir=`dirname $0`/..
+ 
+ # skip if we don't have python
+ type python >/dev/null 2>&1 || {
-- 
1.8.4.5

[oe][Patch 2/2] udev: don't keep ptest testdata laying around

[Max Krummenacher]

From: Max Krummenacher <max.oss.09@gmail.com>

Only unpack udev's testdata right before executing the tests and cleanup
afterwards.

udev's testsuite can be used by ptest. However currently the testdata against
which its functionality is tested is installed in the sysroot at udev install
time.
If the sysroot is used with qemu the testdata makes qemu entering an infinite
loop.
http://lists.openembedded.org/pipermail/openembedded-core/2014-September/097098.html

This has already been fixed for the systemd udev flavour.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5664

Signed-off-by: Max Krummenacher <max.oss.09@gmail.com>
---
 meta/recipes-core/udev/udev/add-install-ptest.patch | 2 +-
 meta/recipes-core/udev/udev/run-ptest               | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/add-install-ptest.patch b/meta/recipes-core/udev/udev/add-install-ptest.patch
index 755946a..bfc2e94 100644
--- a/meta/recipes-core/udev/udev/add-install-ptest.patch
+++ b/meta/recipes-core/udev/udev/add-install-ptest.patch
@@ -34,7 +34,7 @@ Upstream-Status: Pending
 +	(cd $(top_srcdir) && install $(TESTS) $(DESTDIR)/test)
 +	(cd $(top_srcdir) && install test/rule-syntax-check.py $(DESTDIR)/test)
 +	(cd $(top_srcdir) && install $(RULES) $(DESTDIR)/rules)
-+	tar -C $(DESTDIR)/test/ -xJf $(top_srcdir)/test/sys.tar.xz
++	cp $(top_srcdir)/test/sys.tar.xz $(DESTDIR)/test/
 --- a/test/udev-test.pl	2012-03-18 16:43:36.000000000 +0100
 +++ b/test/udev-test.pl	2013-02-18 10:31:29.706357321 +0100
 @@ -1459,11 +1459,13 @@
diff --git a/meta/recipes-core/udev/udev/run-ptest b/meta/recipes-core/udev/udev/run-ptest
index c6961ce..0e39806 100644
--- a/meta/recipes-core/udev/udev/run-ptest
+++ b/meta/recipes-core/udev/udev/run-ptest
@@ -1,3 +1,5 @@
 #!/bin/sh
 
+tar -C test/ -xJf test/sys.tar.xz
 make -k check-TESTS
+make test-sys-distclean
-- 
1.8.4.5

Re: [oe][Patch 0/2] udev: fix ptest to not influence qemu

[Otavio Salvador]

Hello Max,

On Sun, Jan 25, 2015 at 1:33 PM, Max Krummenacher <max.oss.09@gmail.com> wrote:
...
> On a OE setup configured to build ptest packages udev's ptest test files are
> staged to the sysroot.
> A run of qemu chockes on this test files and never returns.
...

This fixes a hard to debug issue in meta-java as it ended triggering a
QEMU which never ends and is eventually killed by Linux kernel by
consuming all available memory in the system.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [oe][Patch 0/2] udev: fix ptest to not influence qemu

[Nathan Lynch]

On 01/25/2015 09:33 AM, Max Krummenacher wrote:
> From: Max Krummenacher <max.oss.09@gmail.com<>
> 
> Hi
> 
> On a OE setup configured to build ptest packages udev's ptest test files are
> staged to the sysroot.
> A run of qemu chockes on this test files and never returns.
> 
> A ptest run on the target before any changes failed.
> Patch 1/2 first fixes ptest.
> Patch 2/2 keeps the test files in its tar-ball. run-ptest unpacks it before
> a test and cleans up afterwards.
> 
> This fixes the issues with qemu while the ptests can still be run witout
> failure.

This looks relevant to

https://bugzilla.yoctoproject.org/show_bug.cgi?id=6734
do_rootfs: The postinstall intercept hook 'update_pixbuf_cache' failed

Re: [oe][Patch 0/2] udev: fix ptest to not influence qemu

[Max Krummenacher]

2015-01-26 14:42 GMT+01:00 Nathan Lynch <Nathan_Lynch@mentor.com>:
> On 01/25/2015 09:33 AM, Max Krummenacher wrote:
>> From: Max Krummenacher <max.oss.09@gmail.com<>
>>
>> Hi
>>
>> On a OE setup configured to build ptest packages udev's ptest test files are
>> staged to the sysroot.
>> A run of qemu chockes on this test files and never returns.
>>
>> A ptest run on the target before any changes failed.
>> Patch 1/2 first fixes ptest.
>> Patch 2/2 keeps the test files in its tar-ball. run-ptest unpacks it before
>> a test and cleans up afterwards.
>>
>> This fixes the issues with qemu while the ptests can still be run witout
>> failure.
>
> This looks relevant to
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=6734
> do_rootfs: The postinstall intercept hook 'update_pixbuf_cache' failed
>
>
I did not specifically look for the update_pixbuf_cache postinstalls.
Nontheless from the bug description I think the patch likely fixes
id=6734.
A cheap test on an affected build is to:
 rm -rf ..../sysroots/<machine>/usr/lib/udev/ptest
If that fixes the issue the patch will help.

Max