[Fuego] Fuego error reason: Error while ROOTFS_LOGREAD command execution on target

[Fuego] Fuego error reason: Error while ROOTFS_LOGREAD command execution on target

[dhinakar k]

[-- Attachment #1: Type: text/plain, Size: 2420 bytes --]

Hi,

I am trying to run a simple functional test (Functional.hello_world) after
setting up fuego. I  have created a target named 'dhina-test-arm' taking a
copy of 'qemu-test-arm'. I ran into error mentioned in the subject (Fuego
error reason: Error while ROOTFS_LOGREAD command execution on target). A
truncated log has been pasted below for reference. Please let me know where
is the issue?

Regards,
Dhinakar

+++ case "$TRANSPORT" in
+++ sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o
UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -p 22 dhinakar@172.17.0.1
'logger Starting test Functional.hello_world'
Warning: Permanently added '172.17.0.1' (ECDSA) to the list of known hosts.
+++ dump_syslogs Functional.hello_world before
+++ is_empty Functional.hello_world
+++ '[' -z Functional.hello_world ']'
+++ is_empty before
+++ '[' -z before ']'
+++ ov_rootfs_logread Functional.hello_world before
+++ cmd 'mkdir -p /tmp/fuego.Functional.hello_world && cd
/tmp/fuego.Functional.hello_world && cat /var/log/messages >
dhina-test-arm.2017-03-08_11-30-03.1.before'
+++ report_devlog 'cmd: mkdir -p /tmp/fuego.Functional.hello_world && cd
/tmp/fuego.Functional.hello_world && cat /var/log/messages >
dhina-test-arm.2017-03-08_11-30-03.1.before'
+++ mkdir -p /home/jenkins/logs/Functional.hello_world/devlogs
+++
devlog_file=/home/jenkins/logs/Functional.hello_world/devlogs/dhina-test-arm.2017-03-08_11-30-03.1.txt
+++ echo 'cmd: mkdir -p /tmp/fuego.Functional.hello_world && cd
/tmp/fuego.Functional.hello_world && cat /var/log/messages >
dhina-test-arm.2017-03-08_11-30-03.1.before'
+++ ov_transport_cmd 'mkdir -p /tmp/fuego.Functional.hello_world && cd
/tmp/fuego.Functional.hello_world && cat /var/log/messages >
dhina-test-arm.2017-03-08_11-30-03.1.before'
+++ case "$TRANSPORT" in
+++ sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o
UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -p 22 dhinakar@172.17.0.1
'mkdir -p /tmp/fuego.Functional.hello_world && cd
/tmp/fuego.Functional.hello_world && cat /var/log/messages >
dhina-test-arm.2017-03-08_11-30-03.1.before'
Warning: Permanently added '172.17.0.1' (ECDSA) to the list of known hosts.
cat: /var/log/messages: No such file or directory
+++ abort_job 'Error while ROOTFS_LOGREAD command execution on target'
+++ set +x

*** ABORTED ***

Fuego error reason: Error while ROOTFS_LOGREAD command execution on target

[-- Attachment #2: Type: text/html, Size: 2927 bytes --]

Re: [Fuego] Fuego error reason: Error while ROOTFS_LOGREAD command execution on target

[Bird, Timothy]

See comments inline below.

> -----Original Message-----
> From  dhinakar k on Wednesday, March 08, 2017 3:44 AM
>
> I am trying to run a simple functional test (Functional.hello_world) after
> setting up fuego. I  have created a target named 'dhina-test-arm' taking a
> copy of 'qemu-test-arm'. I ran into error mentioned in the subject (Fuego
> error reason: Error while ROOTFS_LOGREAD command execution on target).
> A truncated log has been pasted below for reference. Please let me know
> where is the issue?
> 
> Regards,
> Dhinakar
> 
> +++ case "$TRANSPORT" in
> +++ sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o
> UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -p 22
> dhinakar@172.17.0.1 <mailto:dhinakar@172.17.0.1>  'logger Starting test
> Functional.hello_world'
> Warning: Permanently added '172.17.0.1' (ECDSA) to the list of known hosts.
> +++ dump_syslogs Functional.hello_world before
> +++ is_empty Functional.hello_world
> +++ '[' -z Functional.hello_world ']'
> +++ is_empty before
> +++ '[' -z before ']'
> +++ ov_rootfs_logread Functional.hello_world before
> +++ cmd 'mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ report_devlog 'cmd: mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ mkdir -p /home/jenkins/logs/Functional.hello_world/devlogs
> +++ devlog_file=/home/jenkins/logs/Functional.hello_world/devlogs/dhina-
> test-arm.2017-03-08_11-30-03.1.txt
> +++ echo 'cmd: mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ ov_transport_cmd 'mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ case "$TRANSPORT" in
> +++ sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o
> UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -p 22
> dhinakar@172.17.0.1 <mailto:dhinakar@172.17.0.1>  'mkdir -p
> /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> Warning: Permanently added '172.17.0.1' (ECDSA) to the list of known hosts.
> cat: /var/log/messages: No such file or directory

This is the actual error.   The job definition for qemu-arm.board specifies a DISTRIB
of 'distribs/nologread.dist'.  It looks like this is the wrong one to use for your board.
This distribution file is for use with a board that does not have the 'logread' command, but
that still uses standard (old-style/syslogd-style) logging that puts system log messages
into /var/log/messages.

I'm going to guess that you have a board that uses systemd for logging.

The quick workaround for your situation is to change DISTRIB to 'distribs/nosyslogd.dist'.
This will generate a 'fake' system log, that Fuego can read and write during the test.
It never accesses /var/log/messages.

A better solution, and one I might implement if you can confirm your situation,
is to make a new distribution file which actually handles systemd logging correctly.
It should not be very hard to do.

But first, please change your definition of DISTRIB for your target, and let me know
if this fixes the problem for you.  You can do this as follows:  In the Jenkins interface
select your target under 'target status' on the main dashboard (left-side panel).
The in the left-side menu, choose "Configure target".  In the section: "Node Properties"
there should be a list of environment variables, one for BOARD_OVERLAY and one
for DISTRIB.  Change the value for DISTRIB to distribs/nosyslogd.dist.

Then try the test and again and let me know what happens.

> +++ abort_job 'Error while ROOTFS_LOGREAD command execution on target'
> +++ set +x
> 
> *** ABORTED ***
> 
> Fuego error reason: Error while ROOTFS_LOGREAD command execution on
> target

Do you know if your target uses systemd for system logging?  If so, would you
be willing to help me test a new distribs/systemd.dist file, to handle this case?

Thanks,
 -- Tim

Re: [Fuego] Fuego error reason: Error while ROOTFS_LOGREAD command execution on target

[dhinakar k]

[-- Attachment #1: Type: text/plain, Size: 4971 bytes --]

Hi,

Please see my comments inline (in blue colour).

Regards,
Dhinakar


On Mar 8, 2017 9:00 PM, "Bird, Timothy" <Tim.Bird@sony.com> wrote:

See comments inline below.

> -----Original Message-----
> From  dhinakar k on Wednesday, March 08, 2017 3:44 AM
>
> I am trying to run a simple functional test (Functional.hello_world) after
> setting up fuego. I  have created a target named 'dhina-test-arm' taking a
> copy of 'qemu-test-arm'. I ran into error mentioned in the subject (Fuego
> error reason: Error while ROOTFS_LOGREAD command execution on target).
> A truncated log has been pasted below for reference. Please let me know
> where is the issue?
>
> Regards,
> Dhinakar
>
> +++ case "$TRANSPORT" in
> +++ sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no
-o
> UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -p 22
> dhinakar@172.17.0.1 <mailto:dhinakar@172.17.0.1>  'logger Starting test
> Functional.hello_world'
> Warning: Permanently added '172.17.0.1' (ECDSA) to the list of known
hosts.
> +++ dump_syslogs Functional.hello_world before
> +++ is_empty Functional.hello_world
> +++ '[' -z Functional.hello_world ']'
> +++ is_empty before
> +++ '[' -z before ']'
> +++ ov_rootfs_logread Functional.hello_world before
> +++ cmd 'mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ report_devlog 'cmd: mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ mkdir -p /home/jenkins/logs/Functional.hello_world/devlogs
> +++ devlog_file=/home/jenkins/logs/Functional.hello_world/devlogs/dhina-
> test-arm.2017-03-08_11-30-03.1.txt
> +++ echo 'cmd: mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ ov_transport_cmd 'mkdir -p /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> +++ case "$TRANSPORT" in
> +++ sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no
-o
> UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -p 22
> dhinakar@172.17.0.1 <mailto:dhinakar@172.17.0.1>  'mkdir -p
> /tmp/fuego.Functional.hello_world && cd
> /tmp/fuego.Functional.hello_world && cat /var/log/messages > dhina-test-
> arm.2017-03-08_11-30-03.1.before'
> Warning: Permanently added '172.17.0.1' (ECDSA) to the list of known
hosts.
> cat: /var/log/messages: No such file or directory


Thanks, I will try the below suggestion...

This is the actual error.   The job definition for qemu-arm.board specifies
a DISTRIB
of 'distribs/nologread.dist'.  It looks like this is the wrong one to use
for your board.
This distribution file is for use with a board that does not have the
'logread' command, but
that still uses standard (old-style/syslogd-style) logging that puts system
log messages
into /var/log/messages.

I'm going to guess that you have a board that uses systemd for logging.

Actually I am evaluating Fuego, so haven't yet connected a target. At
first, I just wanted to run some simple tests on emulator such as qemu...

The quick workaround for your situation is to change DISTRIB to
'distribs/nosyslogd.dist'.
This will generate a 'fake' system log, that Fuego can read and write
during the test.
It never accesses /var/log/messages.

My host is ubuntu 16.04 and I wanted to run some tests in qemu emulator
running inside the docker container.

A better solution, and one I might implement if you can confirm your
situation,
is to make a new distribution file which actually handles systemd logging
correctly.
It should not be very hard to do.

But first, please change your definition of DISTRIB for your target, and
let me know
if this fixes the problem for you.  You can do this as follows:  In the
Jenkins interface
select your target under 'target status' on the main dashboard (left-side
panel).
The in the left-side menu, choose "Configure target".  In the section:
"Node Properties"
there should be a list of environment variables, one for BOARD_OVERLAY and
one
for DISTRIB.  Change the value for DISTRIB to distribs/nosyslogd.dist.

Then try the test and again and let me know what happens.

> +++ abort_job 'Error while ROOTFS_LOGREAD command execution on target'
> +++ set +x
>
> *** ABORTED ***
>
> Fuego error reason: Error while ROOTFS_LOGREAD command execution on
> target

Do you know if your target uses systemd for system logging?  If so, would
you
be willing to help me test a new distribs/systemd.dist file, to handle this
case?

As mentioned before I am trying to evaluate fuego on qemu emulator.
Having said that in future when I try out on targets that support systemd
for system logging,
I will surely help you test your distribs/systemd.dist file.

Thanks,
 -- Tim

[-- Attachment #2: Type: text/html, Size: 8080 bytes --]