All of lore.kernel.org
 help / color / mirror / Atom feed
* [Fuego] [PATCH 1/3] install-debian: add fuego-core link
@ 2019-03-13  9:14 Daniel Sangorrin
  2019-03-13  9:14 ` [Fuego] [PATCH 2/3] install-debian: set nojenkins configuration automatically Daniel Sangorrin
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Daniel Sangorrin @ 2019-03-13  9:14 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 install-debian.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/install-debian.sh b/install-debian.sh
index a8be709..349f62e 100755
--- a/install-debian.sh
+++ b/install-debian.sh
@@ -83,6 +83,7 @@ if [ ! -d "/fuego" ]; then
 	ln -s /fuego/fuego-rw /fuego-rw
 	cd fuego
 	git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
+	ln -s /fuego/fuego-core /fuego-core
 fi
 
 # ==============================================================================
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Fuego] [PATCH 2/3] install-debian: set nojenkins configuration automatically
  2019-03-13  9:14 [Fuego] [PATCH 1/3] install-debian: add fuego-core link Daniel Sangorrin
@ 2019-03-13  9:14 ` Daniel Sangorrin
  2019-03-13 22:43   ` Tim.Bird
  2019-03-13  9:14 ` [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker Daniel Sangorrin
  2019-03-13 22:41 ` [Fuego] [PATCH 1/3] install-debian: add fuego-core link Tim.Bird
  2 siblings, 1 reply; 8+ messages in thread
From: Daniel Sangorrin @ 2019-03-13  9:14 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 install-debian.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/install-debian.sh b/install-debian.sh
index 349f62e..1597535 100755
--- a/install-debian.sh
+++ b/install-debian.sh
@@ -186,6 +186,8 @@ if [ $nojenkins -eq 0 ]; then
 	cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
 	sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
 	chown -R jenkins:jenkins $JENKINS_HOME/
+else
+	sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
 fi
 
 # ==============================================================================
@@ -228,7 +230,6 @@ ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
 # ==============================================================================
 # Small guide
 # ==============================================================================
-echo "Edit /fuego-ro/conf/fuego.conf, and set jenkins_enabled=0"
 echo "Run 'service netperf start' to start a netperf server"
 echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
 echo "Run 'ftc list-boards' to see the available boards"
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker
  2019-03-13  9:14 [Fuego] [PATCH 1/3] install-debian: add fuego-core link Daniel Sangorrin
  2019-03-13  9:14 ` [Fuego] [PATCH 2/3] install-debian: set nojenkins configuration automatically Daniel Sangorrin
@ 2019-03-13  9:14 ` Daniel Sangorrin
  2019-03-13 23:16   ` Tim.Bird
  2019-03-14  1:40   ` Tim.Bird
  2019-03-13 22:41 ` [Fuego] [PATCH 1/3] install-debian: add fuego-core link Tim.Bird
  2 siblings, 2 replies; 8+ messages in thread
From: Daniel Sangorrin @ 2019-03-13  9:14 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 install-debian.sh | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/install-debian.sh b/install-debian.sh
index 1597535..62bf799 100755
--- a/install-debian.sh
+++ b/install-debian.sh
@@ -232,8 +232,8 @@ ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
 # ==============================================================================
 echo "Run 'service netperf start' to start a netperf server"
 echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
-echo "Run 'ftc list-boards' to see the available boards"
-echo "Run 'ftc list-tests' to see the available tests"
-echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
-echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
-echo "Run 'ftc gen-report' to get results"
+echo "Run 'ftc -x list-boards' to see the available boards"
+echo "Run 'ftc -x list-tests' to see the available tests"
+echo "Run 'ftc -x run-test -b local -t Functional.hello_world' to run a hello world"
+echo "Run 'ftc -x run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
+echo "Run 'ftc -x gen-report' to get results"
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [Fuego] [PATCH 1/3] install-debian: add fuego-core link
  2019-03-13  9:14 [Fuego] [PATCH 1/3] install-debian: add fuego-core link Daniel Sangorrin
  2019-03-13  9:14 ` [Fuego] [PATCH 2/3] install-debian: set nojenkins configuration automatically Daniel Sangorrin
  2019-03-13  9:14 ` [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker Daniel Sangorrin
@ 2019-03-13 22:41 ` Tim.Bird
  2 siblings, 0 replies; 8+ messages in thread
From: Tim.Bird @ 2019-03-13 22:41 UTC (permalink / raw)
  To: daniel.sangorrin, fuego

Applied.  Thanks.
 -- Tim

> -----Original Message-----
> From: Daniel Sangorrin
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  install-debian.sh | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/install-debian.sh b/install-debian.sh
> index a8be709..349f62e 100755
> --- a/install-debian.sh
> +++ b/install-debian.sh
> @@ -83,6 +83,7 @@ if [ ! -d "/fuego" ]; then
>  	ln -s /fuego/fuego-rw /fuego-rw
>  	cd fuego
>  	git clone --branch next --depth=1
> https://bitbucket.org/nirrognas/fuego-core.git
> +	ln -s /fuego/fuego-core /fuego-core
>  fi
> 
>  #
> ==========================================================
> ====================
> --
> 2.7.4
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Fuego] [PATCH 2/3] install-debian: set nojenkins configuration automatically
  2019-03-13  9:14 ` [Fuego] [PATCH 2/3] install-debian: set nojenkins configuration automatically Daniel Sangorrin
@ 2019-03-13 22:43   ` Tim.Bird
  0 siblings, 0 replies; 8+ messages in thread
From: Tim.Bird @ 2019-03-13 22:43 UTC (permalink / raw)
  To: daniel.sangorrin, fuego

Applied.  Thanks,
 -- Tim


> -----Original Message-----
> From: Daniel Sangorrin

Would be nice to have something in the body of the commit message.

> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  install-debian.sh | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/install-debian.sh b/install-debian.sh
> index 349f62e..1597535 100755
> --- a/install-debian.sh
> +++ b/install-debian.sh
> @@ -186,6 +186,8 @@ if [ $nojenkins -eq 0 ]; then
>  	cp /fuego/frontend-
> install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
>  	sed -i -e "s#8080#$JENKINS_PORT#g"
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
>  	chown -R jenkins:jenkins $JENKINS_HOME/
> +else
> +	sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-
> ro/conf/fuego.conf
>  fi
> 
>  #
> ==========================================================
> ====================
> @@ -228,7 +230,6 @@ ln -s /fuego-ro/scripts/fuego-lava-target-teardown
> /usr/local/bin
>  #
> ==========================================================
> ====================
>  # Small guide
>  #
> ==========================================================
> ====================
> -echo "Edit /fuego-ro/conf/fuego.conf, and set jenkins_enabled=0"
>  echo "Run 'service netperf start' to start a netperf server"
>  echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
>  echo "Run 'ftc list-boards' to see the available boards"
> --
> 2.7.4
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker
  2019-03-13  9:14 ` [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker Daniel Sangorrin
@ 2019-03-13 23:16   ` Tim.Bird
  2019-03-14  1:40   ` Tim.Bird
  1 sibling, 0 replies; 8+ messages in thread
From: Tim.Bird @ 2019-03-13 23:16 UTC (permalink / raw)
  To: daniel.sangorrin, fuego

See comments below.

> -----Original Message-----
> From: Daniel Sangorrin
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  install-debian.sh | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/install-debian.sh b/install-debian.sh
> index 1597535..62bf799 100755
> --- a/install-debian.sh
> +++ b/install-debian.sh
> @@ -232,8 +232,8 @@ ln -s /fuego-ro/scripts/fuego-lava-target-teardown
> /usr/local/bin
>  #
> ==========================================================
> ====================
>  echo "Run 'service netperf start' to start a netperf server"
>  echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> -echo "Run 'ftc list-boards' to see the available boards"
> -echo "Run 'ftc list-tests' to see the available tests"
> -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello
> world"
> -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> Dhrystone"
> -echo "Run 'ftc gen-report' to get results"
> +echo "Run 'ftc -x list-boards' to see the available boards"
> +echo "Run 'ftc -x list-tests' to see the available tests"
> +echo "Run 'ftc -x run-test -b local -t Functional.hello_world' to run a hello
> world"
> +echo "Run 'ftc -x run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> Dhrystone"
> +echo "Run 'ftc -x gen-report' to get results"
Ugh.  This inconsistency is ugly.  And I don't think these instructions
will actually work, will they?  I thought 'ftc -x' always required a '-c <confpath>'
to indicate the path to the configuration file.  See the FIXTHIS on line 4950
of 'ftc'.

For some of these, it would be better to just run them externally always.
'ftc list-boards' , 'ftc list-tests' fall in that category.

Here's how I'd like 'ftc run-test' to work:
'ftc run-test' should be run internally if there is a docker container present,
and externally if not (allowing the user to override and force it to run
externally even if there is a container present, using the '-x' argument.)

Can we reliably detect that a Fuego docker container is running?
We seem to have some code for this in 'container_command()'.

I think the logic I'll use here is as follows:
 - if we're outside the container, and there exists /fuego-ro/conf/fuego.conf, then
    if there is no container running, then execute the command external (don't require the '-x' flag)

I'll put some code in ftc to do this, which should allow us to not require the '-x' flag
for native use of ftc.  Let me know what you think.
 -- Tim

P.S.  This patch is not applied.

> --
> 2.7.4
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker
  2019-03-13  9:14 ` [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker Daniel Sangorrin
  2019-03-13 23:16   ` Tim.Bird
@ 2019-03-14  1:40   ` Tim.Bird
  2019-03-14  8:36     ` daniel.sangorrin
  1 sibling, 1 reply; 8+ messages in thread
From: Tim.Bird @ 2019-03-14  1:40 UTC (permalink / raw)
  To: daniel.sangorrin, fuego

> -----Original Message-----
> From: Bird, Timothy
> 
> See comments below.
> 
> > -----Original Message-----
> > From: Daniel Sangorrin
> >
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > ---
> >  install-debian.sh | 10 +++++-----
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/install-debian.sh b/install-debian.sh
> > index 1597535..62bf799 100755
> > --- a/install-debian.sh
> > +++ b/install-debian.sh
> > @@ -232,8 +232,8 @@ ln -s /fuego-ro/scripts/fuego-lava-target-teardown
> > /usr/local/bin
> >  #
> >
> ==========================================================
> > ====================
> >  echo "Run 'service netperf start' to start a netperf server"
> >  echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > -echo "Run 'ftc list-boards' to see the available boards"
> > -echo "Run 'ftc list-tests' to see the available tests"
> > -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello
> > world"
> > -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> > Dhrystone"
> > -echo "Run 'ftc gen-report' to get results"
> > +echo "Run 'ftc -x list-boards' to see the available boards"
> > +echo "Run 'ftc -x list-tests' to see the available tests"
> > +echo "Run 'ftc -x run-test -b local -t Functional.hello_world' to run a hello
> > world"
> > +echo "Run 'ftc -x run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> > Dhrystone"
> > +echo "Run 'ftc -x gen-report' to get results"
> Ugh.  This inconsistency is ugly.  And I don't think these instructions
> will actually work, will they?  I thought 'ftc -x' always required a '-c
> <confpath>'
> to indicate the path to the configuration file.  See the FIXTHIS on line 4950
> of 'ftc'.
> 
> For some of these, it would be better to just run them externally always.
> 'ftc list-boards' , 'ftc list-tests' fall in that category.
> 
> Here's how I'd like 'ftc run-test' to work:
> 'ftc run-test' should be run internally if there is a docker container present,
> and externally if not (allowing the user to override and force it to run
> externally even if there is a container present, using the '-x' argument.)
> 
> Can we reliably detect that a Fuego docker container is running?
> We seem to have some code for this in 'container_command()'.
> 
> I think the logic I'll use here is as follows:
>  - if we're outside the container, and there exists /fuego-ro/conf/fuego.conf,
> then
>     if there is no container running, then execute the command external (don't
> require the '-x' flag)
> 
> I'll put some code in ftc to do this, which should allow us to not require the '-
> x' flag
> for native use of ftc.  Let me know what you think.

Here's what I came up with.  It's in 'next', and pushed to bitbucket.
Can you give it a test?
 -- Tim


Subject: [PATCH] ftc: do better handling for native invocations

If ftc is started outside the container, check if it's likely we're
running ftc natively.  If so, and a container is not running, then
default to running ftc outside the container rather than inside.

This is a bit complicated, but I think it captures the most common
use cases (and hopefully is not too brain-bending in weird situations).

This requires that the native installation have the fuego.conf file
in the directory /fuego-ro/conf, which is what the current debian
install script configures.

Move 'import jenkins' and routines that require it to later in
the command execution sequence, so that routines that don't depend
on it won't fail if python-jenkins is not installed on the host.

Signed-off-by: Tim Bird <tim.bird@sony.com>
---
 scripts/ftc | 150 ++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 86 insertions(+), 64 deletions(-)

diff --git a/scripts/ftc b/scripts/ftc
index e0a0a79..9d35c14 100755
--- a/scripts/ftc
+++ b/scripts/ftc
@@ -4302,6 +4302,25 @@ def inside_docker():
     else:
         return True
 
+# cache the name, if found, to avoid doing 'docker ps' twice
+cached_container_name = None
+
+# return fuego container name, or None if not found
+def get_running_fuego_container_name():
+    global cached_container_name
+
+    if cached_container_name:
+        return cached_container_name
+
+    # return the first container with "fuego" in the image or container name
+    dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n')
+    for line in dps_lines:
+        if "fuego" in line:
+            cached_container_name = line.strip().split(" ")[-1]
+            break
+
+    return cached_container_name
+
 # Here's a convenience function to allow running ftc outside the container
 def container_command(cmd_list):
     # get name of docker container
@@ -4310,12 +4329,9 @@ def container_command(cmd_list):
     else:
         # use the first container with "fuego" in the image or container name
         # if none found, use "fuego-container"
-        fuego_container = "fuego-container"
-        dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n')
-        for line in dps_lines:
-            if "fuego" in line:
-                fuego_container = line.strip().split(" ")[-1]
-                break
+        fuego_container = get_running_fuego_container_name()
+        if not fuego_container:
+            fuego_container = "fuego-container"
 
     # pass through the FUEGO_BATCH_ID environment variable
     env_arg_list = []
@@ -4894,7 +4910,7 @@ def main():
     global ll_verbose
     global ll_debug
 
-    # most commands must be run inside the docker container
+    # some commands must be run inside the docker container
     run_inside = True
     if "-x" in sys.argv:
         # allow user to force running the command outside the container
@@ -4905,7 +4921,7 @@ def main():
     if "-h" in sys.argv or 'help' in sys.argv:
         run_inside = False
 
-    # unless user requests it, run commands inside the container
+    # check whether to run the command inside the container
     outside_docker = not inside_docker()
     if outside_docker:
         # check for operations that can't be run from outside the container
@@ -4913,6 +4929,11 @@ def main():
             error_out("Can't do rm-jobs outside the container!\n" +
                     "Please re-run the command inside the container.")
 
+        # check if it's likely we're in a native install
+        container_name = get_running_fuego_container_name()
+        if os.path.exists("/fuego-ro/conf/fuego.conf") and not container_name:
+            run_inside = False
+
         if run_inside:
             # run this command inside the container
             cmd_list = ["ftc"] + sys.argv[1:]
@@ -5048,62 +5069,6 @@ def main():
         # shows fuego boards
         do_list_boards(conf)
 
-    if conf.jenkins_enabled:
-        import jenkins
-        server = jenkins.Jenkins(conf.JENKINS_URL)
-
-    if command.startswith("add-job"):
-        # adds Jenkins jobs
-        user_check(conf)
-        try:
-            do_add_jobs(conf, options)
-        except Exception as e:
-            # this assumes the problem is something with the options
-            sys.exit(str(e) + '\n' + command_help['add-jobs'][1])
-
-    if command.startswith("rm-job"):
-        # removes Jenkins jobs
-        user_check(conf)
-        try:
-            do_rm_jobs(conf, options)
-        except Exception as e:
-            sys.exit(str(e) + '\n' + command_help['rm-jobs'][1])
-
-    if command.startswith("add-node"):
-        # adds Jenkins nodes
-        user_check(conf)
-        try:
-            do_add_nodes(conf, options)
-        except Exception as e:
-            sys.exit(str(e) + '\n' + command_help['add-nodes'][1])
-
-    if command.startswith("rm-node"):
-        # removes Jenkins nodes
-        user_check(conf)
-        try:
-            do_rm_nodes(conf, options)
-        except Exception as e:
-            sys.exit(str(e) + '\n' + command_help['rm-nodes'][1])
-
-    if command == "list-nodes":
-        # shows jenkins nodes
-        user_check(conf)
-        do_list_nodes(conf)
-
-    if command == "list-jobs":
-        # shows jenkins jobs
-        user_check(conf)
-        do_list_jobs(conf)
-
-    if command.startswith("build-job"):
-        # build jenkins jobs
-        user_check(conf)
-        do_build_jobs(conf, options)
-
-    if command == "add-view":
-        user_check(conf)
-        do_add_view(conf, options)
-
     if command == "list-plans":
         do_list_plans(conf)
 
@@ -5168,6 +5133,63 @@ def main():
         rcode = do_release_resource(conf, options)
         sys.exit(rcode)
 
+
+    if conf.jenkins_enabled:
+        import jenkins
+        server = jenkins.Jenkins(conf.JENKINS_URL)
+
+    if command.startswith("add-job"):
+        # adds Jenkins jobs
+        user_check(conf)
+        try:
+            do_add_jobs(conf, options)
+        except Exception as e:
+            # this assumes the problem is something with the options
+            sys.exit(str(e) + '\n' + command_help['add-jobs'][1])
+
+    if command.startswith("rm-job"):
+        # removes Jenkins jobs
+        user_check(conf)
+        try:
+            do_rm_jobs(conf, options)
+        except Exception as e:
+            sys.exit(str(e) + '\n' + command_help['rm-jobs'][1])
+
+    if command.startswith("add-node"):
+        # adds Jenkins nodes
+        user_check(conf)
+        try:
+            do_add_nodes(conf, options)
+        except Exception as e:
+            sys.exit(str(e) + '\n' + command_help['add-nodes'][1])
+
+    if command.startswith("rm-node"):
+        # removes Jenkins nodes
+        user_check(conf)
+        try:
+            do_rm_nodes(conf, options)
+        except Exception as e:
+            sys.exit(str(e) + '\n' + command_help['rm-nodes'][1])
+
+    if command == "list-nodes":
+        # shows jenkins nodes
+        user_check(conf)
+        do_list_nodes(conf)
+
+    if command == "list-jobs":
+        # shows jenkins jobs
+        user_check(conf)
+        do_list_jobs(conf)
+
+    if command.startswith("build-job"):
+        # build jenkins jobs
+        user_check(conf)
+        do_build_jobs(conf, options)
+
+    if command == "add-view":
+        user_check(conf)
+        do_add_view(conf, options)
+
     # all non-board commands have been handled
     #if command in board_mod_commands:
     #   check_reservation(bmap, command)
-- 
2.1.4


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker
  2019-03-14  1:40   ` Tim.Bird
@ 2019-03-14  8:36     ` daniel.sangorrin
  0 siblings, 0 replies; 8+ messages in thread
From: daniel.sangorrin @ 2019-03-14  8:36 UTC (permalink / raw)
  To: Tim.Bird, fuego

> -----Original Message-----
> From: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Sent: Thursday, March 14, 2019 10:41 AM
> To: sangorrin daniel(サンゴリン ダニエル ○SWC□OST) <daniel.sangorrin@toshiba.co.jp>;
> fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker
> 
> > -----Original Message-----
> > From: Bird, Timothy
> >
> > See comments below.
> >
> > > -----Original Message-----
> > > From: Daniel Sangorrin
> > >
> > > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > > ---
> > >  install-debian.sh | 10 +++++-----
> > >  1 file changed, 5 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/install-debian.sh b/install-debian.sh
> > > index 1597535..62bf799 100755
> > > --- a/install-debian.sh
> > > +++ b/install-debian.sh
> > > @@ -232,8 +232,8 @@ ln -s /fuego-ro/scripts/fuego-lava-target-teardown
> > > /usr/local/bin
> > >  #
> > >
> > ==========================================================
> > > ====================
> > >  echo "Run 'service netperf start' to start a netperf server"
> > >  echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > > -echo "Run 'ftc list-boards' to see the available boards"
> > > -echo "Run 'ftc list-tests' to see the available tests"
> > > -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello
> > > world"
> > > -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> > > Dhrystone"
> > > -echo "Run 'ftc gen-report' to get results"
> > > +echo "Run 'ftc -x list-boards' to see the available boards"
> > > +echo "Run 'ftc -x list-tests' to see the available tests"
> > > +echo "Run 'ftc -x run-test -b local -t Functional.hello_world' to run a hello
> > > world"
> > > +echo "Run 'ftc -x run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> > > Dhrystone"
> > > +echo "Run 'ftc -x gen-report' to get results"
> > Ugh.  This inconsistency is ugly.  And I don't think these instructions
> > will actually work, will they?  I thought 'ftc -x' always required a '-c
> > <confpath>'
> > to indicate the path to the configuration file.  See the FIXTHIS on line 4950
> > of 'ftc'.

Actually it did work, for some reason.

> > For some of these, it would be better to just run them externally always.
> > 'ftc list-boards' , 'ftc list-tests' fall in that category.
> >
> > Here's how I'd like 'ftc run-test' to work:
> > 'ftc run-test' should be run internally if there is a docker container present,
> > and externally if not (allowing the user to override and force it to run
> > externally even if there is a container present, using the '-x' argument.)
> >
> > Can we reliably detect that a Fuego docker container is running?
> > We seem to have some code for this in 'container_command()'.

Be careful not to call "docker ps" because the native host may not even have it installed. (Maybe you already changed that in your patch)

> > I think the logic I'll use here is as follows:
> >  - if we're outside the container, and there exists /fuego-ro/conf/fuego.conf,
> > then
> >     if there is no container running, then execute the command external (don't
> > require the '-x' flag)
> >
> > I'll put some code in ftc to do this, which should allow us to not require the '-
> > x' flag
> > for native use of ftc.  Let me know what you think.
> 
> Here's what I came up with.  It's in 'next', and pushed to bitbucket.
> Can you give it a test?

Thanks a lot. I will check tomorrow

Regards,
Daniel

>  -- Tim
> 
> 
> Subject: [PATCH] ftc: do better handling for native invocations
> 
> If ftc is started outside the container, check if it's likely we're
> running ftc natively.  If so, and a container is not running, then
> default to running ftc outside the container rather than inside.
> 
> This is a bit complicated, but I think it captures the most common
> use cases (and hopefully is not too brain-bending in weird situations).
> 
> This requires that the native installation have the fuego.conf file
> in the directory /fuego-ro/conf, which is what the current debian
> install script configures.
> 
> Move 'import jenkins' and routines that require it to later in
> the command execution sequence, so that routines that don't depend
> on it won't fail if python-jenkins is not installed on the host.
> 
> Signed-off-by: Tim Bird <tim.bird@sony.com>
> ---
>  scripts/ftc | 150 ++++++++++++++++++++++++++++++++++--------------------------
>  1 file changed, 86 insertions(+), 64 deletions(-)
> 
> diff --git a/scripts/ftc b/scripts/ftc
> index e0a0a79..9d35c14 100755
> --- a/scripts/ftc
> +++ b/scripts/ftc
> @@ -4302,6 +4302,25 @@ def inside_docker():
>      else:
>          return True
> 
> +# cache the name, if found, to avoid doing 'docker ps' twice
> +cached_container_name = None
> +
> +# return fuego container name, or None if not found
> +def get_running_fuego_container_name():
> +    global cached_container_name
> +
> +    if cached_container_name:
> +        return cached_container_name
> +
> +    # return the first container with "fuego" in the image or container name
> +    dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n')
> +    for line in dps_lines:
> +        if "fuego" in line:
> +            cached_container_name = line.strip().split(" ")[-1]
> +            break
> +
> +    return cached_container_name
> +
>  # Here's a convenience function to allow running ftc outside the container
>  def container_command(cmd_list):
>      # get name of docker container
> @@ -4310,12 +4329,9 @@ def container_command(cmd_list):
>      else:
>          # use the first container with "fuego" in the image or container name
>          # if none found, use "fuego-container"
> -        fuego_container = "fuego-container"
> -        dps_lines = subprocess.check_output("sudo docker ps", shell=True).split('\n')
> -        for line in dps_lines:
> -            if "fuego" in line:
> -                fuego_container = line.strip().split(" ")[-1]
> -                break
> +        fuego_container = get_running_fuego_container_name()
> +        if not fuego_container:
> +            fuego_container = "fuego-container"
> 
>      # pass through the FUEGO_BATCH_ID environment variable
>      env_arg_list = []
> @@ -4894,7 +4910,7 @@ def main():
>      global ll_verbose
>      global ll_debug
> 
> -    # most commands must be run inside the docker container
> +    # some commands must be run inside the docker container
>      run_inside = True
>      if "-x" in sys.argv:
>          # allow user to force running the command outside the container
> @@ -4905,7 +4921,7 @@ def main():
>      if "-h" in sys.argv or 'help' in sys.argv:
>          run_inside = False
> 
> -    # unless user requests it, run commands inside the container
> +    # check whether to run the command inside the container
>      outside_docker = not inside_docker()
>      if outside_docker:
>          # check for operations that can't be run from outside the container
> @@ -4913,6 +4929,11 @@ def main():
>              error_out("Can't do rm-jobs outside the container!\n" +
>                      "Please re-run the command inside the container.")
> 
> +        # check if it's likely we're in a native install
> +        container_name = get_running_fuego_container_name()
> +        if os.path.exists("/fuego-ro/conf/fuego.conf") and not container_name:
> +            run_inside = False
> +
>          if run_inside:
>              # run this command inside the container
>              cmd_list = ["ftc"] + sys.argv[1:]
> @@ -5048,62 +5069,6 @@ def main():
>          # shows fuego boards
>          do_list_boards(conf)
> 
> -    if conf.jenkins_enabled:
> -        import jenkins
> -        server = jenkins.Jenkins(conf.JENKINS_URL)
> -
> -    if command.startswith("add-job"):
> -        # adds Jenkins jobs
> -        user_check(conf)
> -        try:
> -            do_add_jobs(conf, options)
> -        except Exception as e:
> -            # this assumes the problem is something with the options
> -            sys.exit(str(e) + '\n' + command_help['add-jobs'][1])
> -
> -    if command.startswith("rm-job"):
> -        # removes Jenkins jobs
> -        user_check(conf)
> -        try:
> -            do_rm_jobs(conf, options)
> -        except Exception as e:
> -            sys.exit(str(e) + '\n' + command_help['rm-jobs'][1])
> -
> -    if command.startswith("add-node"):
> -        # adds Jenkins nodes
> -        user_check(conf)
> -        try:
> -            do_add_nodes(conf, options)
> -        except Exception as e:
> -            sys.exit(str(e) + '\n' + command_help['add-nodes'][1])
> -
> -    if command.startswith("rm-node"):
> -        # removes Jenkins nodes
> -        user_check(conf)
> -        try:
> -            do_rm_nodes(conf, options)
> -        except Exception as e:
> -            sys.exit(str(e) + '\n' + command_help['rm-nodes'][1])
> -
> -    if command == "list-nodes":
> -        # shows jenkins nodes
> -        user_check(conf)
> -        do_list_nodes(conf)
> -
> -    if command == "list-jobs":
> -        # shows jenkins jobs
> -        user_check(conf)
> -        do_list_jobs(conf)
> -
> -    if command.startswith("build-job"):
> -        # build jenkins jobs
> -        user_check(conf)
> -        do_build_jobs(conf, options)
> -
> -    if command == "add-view":
> -        user_check(conf)
> -        do_add_view(conf, options)
> -
>      if command == "list-plans":
>          do_list_plans(conf)
> 
> @@ -5168,6 +5133,63 @@ def main():
>          rcode = do_release_resource(conf, options)
>          sys.exit(rcode)
> 
> +
> +    if conf.jenkins_enabled:
> +        import jenkins
> +        server = jenkins.Jenkins(conf.JENKINS_URL)
> +
> +    if command.startswith("add-job"):
> +        # adds Jenkins jobs
> +        user_check(conf)
> +        try:
> +            do_add_jobs(conf, options)
> +        except Exception as e:
> +            # this assumes the problem is something with the options
> +            sys.exit(str(e) + '\n' + command_help['add-jobs'][1])
> +
> +    if command.startswith("rm-job"):
> +        # removes Jenkins jobs
> +        user_check(conf)
> +        try:
> +            do_rm_jobs(conf, options)
> +        except Exception as e:
> +            sys.exit(str(e) + '\n' + command_help['rm-jobs'][1])
> +
> +    if command.startswith("add-node"):
> +        # adds Jenkins nodes
> +        user_check(conf)
> +        try:
> +            do_add_nodes(conf, options)
> +        except Exception as e:
> +            sys.exit(str(e) + '\n' + command_help['add-nodes'][1])
> +
> +    if command.startswith("rm-node"):
> +        # removes Jenkins nodes
> +        user_check(conf)
> +        try:
> +            do_rm_nodes(conf, options)
> +        except Exception as e:
> +            sys.exit(str(e) + '\n' + command_help['rm-nodes'][1])
> +
> +    if command == "list-nodes":
> +        # shows jenkins nodes
> +        user_check(conf)
> +        do_list_nodes(conf)
> +
> +    if command == "list-jobs":
> +        # shows jenkins jobs
> +        user_check(conf)
> +        do_list_jobs(conf)
> +
> +    if command.startswith("build-job"):
> +        # build jenkins jobs
> +        user_check(conf)
> +        do_build_jobs(conf, options)
> +
> +    if command == "add-view":
> +        user_check(conf)
> +        do_add_view(conf, options)
> +
>      # all non-board commands have been handled
>      #if command in board_mod_commands:
>      #   check_reservation(bmap, command)
> --
> 2.1.4


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2019-03-14  8:36 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-13  9:14 [Fuego] [PATCH 1/3] install-debian: add fuego-core link Daniel Sangorrin
2019-03-13  9:14 ` [Fuego] [PATCH 2/3] install-debian: set nojenkins configuration automatically Daniel Sangorrin
2019-03-13 22:43   ` Tim.Bird
2019-03-13  9:14 ` [Fuego] [PATCH 3/3] install-debian: add -x because we dont use docker Daniel Sangorrin
2019-03-13 23:16   ` Tim.Bird
2019-03-14  1:40   ` Tim.Bird
2019-03-14  8:36     ` daniel.sangorrin
2019-03-13 22:41 ` [Fuego] [PATCH 1/3] install-debian: add fuego-core link Tim.Bird

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.