All of lore.kernel.org
 help / color / mirror / Atom feed
* [Fuego] Support for Squad
@ 2019-01-10  9:00 Daniel Sangorrin
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] ftc: implement put-run for squad Daniel Sangorrin
                   ` (3 more replies)
  0 siblings, 4 replies; 12+ messages in thread
From: Daniel Sangorrin @ 2019-01-10  9:00 UTC (permalink / raw)
  To: fuego

Hi Tim,

I have rebased my work to support Squad on top of Fuego v1.4.
Before it's too late I wanted to send it to you (the patches are on my repo as well).

[PATCH 2/2] ftc: implement put-run for squad
[PATCH 2/2] squad: add paramters for squad
Note: sorry about the patch numbering and mixing fuego and fuego-core patches.

The instructions to test these patches are on the latest fuego-core commit (on my repo)
but I reproduce them below. I can add them to the wiki page later on. I modified
some variables in fuego.conf as well that should be updated.

The code is experimental, I have only tested it for Dhrystone and IOzone :D.
But it is a good start and I think that it is better to discuss some important concepts early on.
For example, I think I need to add paramwters to the put-run interface (team, project)
Also, I think that all these server-related functions should be implemented as classes/objects.
Currently, I only added support for put-run but technically Squad can also support other
functions such as put-request.
Other than that, in Squad the "job_id" needs to be unique to a "project". We need to think
about how to do that if anyone in the team can submit jobs. 
You can check my implementation and the comments I wrote there.

Thanks,
Daniel

Test instructions
=================

Ref: https://squad.readthedocs.io (doc)

Install server application
	host$ sudo apt-get install rabbitmq-server
		[Note] this is required for squad to communicate with celery worker tasks
	host$ git clone https://github.com/Linaro/squad
	host$ cd squad/
	host$ mkvirtualenv --python=python3  mysquad
	host$ cd squad
	host$ pip3 install -r requirements-dev.txt
	host$ ./manage.py test
	host$ ./manage.py migrate
	host$ ./manage.py createsuperuser
	host$ ./manage.py runserver

[Note] The next time you use it use this:
	host$ cd squad
	host$ workon squad
	host$ ./manage.py runserver

Configure a team and project in Squad
	host$ firefox http://localhost:8000/
		- click upright > settings
			- profile: write your name
			- API token: copy the token 
				-> assign it to "server_squad_token" in fuego-ro/fuego.conf
		- click upright > Administration
			- Authentication and authorization
				- Add group (this is a user group!)
					- name: myusergroup
					- permissons: choose all
			- Core
				- Add group (this is a team)
					- slug=name=fuego
						-> assign it to "server_squad_team" in fuego-ro/fuego.conf
					- description: whatever
					- usergroups: myusergroup
				- Add project
					- group: fuego
					- slug=name=jessie
						-> assign it to "server_squad_project" in fuego-ro/fuego.conf

Upload a run from Fuego
	fuego-docker# ftc list-runs -q
		Benchmark.IOzone-default-1-bbb
	fuego-docker# ftc put-run Benchmark.IOzone-default-1-bbb
		Packaging run 'Benchmark.IOzone-default-1-bbb'
		run/
		run/devlog.after.txt
		run/syslog.before.txt
		run/build.xml
		run/devlog.txt
		run/testlog.txt
		run/consolelog.txt
		run/run.json
		Run packaged successfully, and is at: /tmp/run-Benchmark.IOzone-default-1-on-fuegohost:bbb.frp
		WARNING: not adding attachment syslog.after
		WARNING: not adding attachment test_spec
		POSTING to http://localhost:8000/api/submit/fuego/jessie/a80be74d1629bdb2455b08db453beba0-4.9.133/bbb-4.9
		Run package run-Benchmark.IOzone-default-1-on-fuegohost:bbb.frp was accepted by the server.




^ permalink raw reply	[flat|nested] 12+ messages in thread
* [Fuego] [PATCH 2/2] ftc: implement put-run for squad
  2019-01-10  9:00 [Fuego] Support for Squad Daniel Sangorrin
@ 2019-01-10  9:00 ` Daniel Sangorrin
  2019-01-11  0:23   ` daniel.sangorrin
  2019-01-11  1:50   ` Tim.Bird
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] squad: add paramters " Daniel Sangorrin
                   ` (2 subsequent siblings)
  3 siblings, 2 replies; 12+ messages in thread
From: Daniel Sangorrin @ 2019-01-10  9:00 UTC (permalink / raw)
  To: fuego

This is an early implementation, more debugging is needed
and there are several FIXTHIS included, but it works.
You can try it on Ubuntu Xenial with the instructions below.

Install server application
	host$ sudo apt-get install rabbitmq-server
		[Note] this is required for squad to communicate with celery worker tasks
	host$ git clone https://github.com/Linaro/squad
	host$ cd squad/
	host$ mkvirtualenv --python=python3  mysquad
	host$ cd squad
	host$ pip3 install -r requirements-dev.txt
	host$ ./manage.py test
	host$ ./manage.py migrate
	host$ ./manage.py createsuperuser
	host$ ./manage.py runserver

[Note] The next time you use it use this:
	host$ cd squad
	host$ workon squad
	host$ ./manage.py runserver

Configure a team and project in Squad
	host$ firefox http://localhost:8000/
		- click upright > settings
			- profile: write your name
			- API token: copy the token
				-> assign it to "server_squad_token" in fuego-ro/fuego.conf
		- click upright > Administration
			- Authentication and authorization
				- Add group (this is a user group!)
					- name: myusergroup
					- permissons: choose all
			- Core
				- Add group (this is a team)
					- slug=name=fuego
						-> assign it to "server_squad_team" in fuego-ro/fuego.conf
					- description: whatever
					- usergroups: myusergroup
				- Add project
					- group: fuego
					- slug=name=jessie
						-> assign it to "server_squad_project" in fuego-ro/fuego.conf

Upload a run from Fuego
	fuego-docker# ftc list-runs -q
		Benchmark.IOzone-default-1-bbb
	fuego-docker# ftc put-run Benchmark.IOzone-default-1-bbb
		Packaging run 'Benchmark.IOzone-default-1-bbb'
		run/
		run/devlog.after.txt
		run/syslog.before.txt
		run/build.xml
		run/devlog.txt
		run/testlog.txt
		run/consolelog.txt
		run/run.json
		Run packaged successfully, and is at: /tmp/run-Benchmark.IOzone-default-1-on-fuegohost:bbb.frp
		WARNING: not adding attachment syslog.after
		WARNING: not adding attachment test_spec
		POSTING to http://localhost:8000/api/submit/fuego/jessie/a80be74d1629bdb2455b08db453beba0-4.9.133/bbb-4.9
		Run package run-Benchmark.IOzone-default-1-on-fuegohost:bbb.frp was accepted by the server.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/scripts/ftc | 131 ++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 119 insertions(+), 12 deletions(-)

diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index 73fd98a..ab2d2cb 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -519,9 +519,16 @@ class config_class:
         # set values from conf_map
         self.host = conf_map.get("host_name", "localhost")
         self.host_name = conf_map.get("host_name", "localhost")
-        self.fuego_server = conf_map.get("fuego_server", "fuegotest.org")
-        #self.SERVER_URL_BASE = "http://%s/server/Fuego_Server?action=Fuego." % self.fuego_server
-        self.SERVER_URL_BASE = "http://%s/fserver.py/?action=" % self.fuego_server
+        self.server_type = conf_map.get("server_type", "fuego")
+        self.server_domain = conf_map.get("server_domain", "fuegotest.org")
+        #self.SERVER_URL_BASE = "http://%s/server/Fuego_Server?action=Fuego." % self.server_domain
+        if self.server_type == "fuego":
+            self.SERVER_URL_BASE = "http://%s/fserver.py/?action=" % self.server_domain
+        elif self.server_type == "squad":
+            self.SERVER_URL_BASE = "http://%s/api" % self.server_domain
+            self.token = conf_map.get("server_squad_token", "WRITE_TOKEN_IN_FUEGO.CONF")
+            self.team = conf_map.get("server_squad_team", "fuego")
+            self.project_id = conf_map.get("server_squad_project", "myproject")
 
         # handle paths (including relative ones)
         conf_dir = os.path.dirname(os.path.abspath(config_path))
@@ -3282,6 +3289,103 @@ def do_put_test(conf, options):
 
     print "Test package %s was accepted by the server." % os.path.basename(test_filepath)
 
+def put_run_fuego(conf, run_filepath):
+    url = conf.SERVER_URL_BASE+"put_run"
+    run_files = {"file1": open(run_filepath, "rb")}
+    resp = requests.post(url, files=run_files)
+    return resp.text.split('\n', 1)
+
+def put_run_squad(conf, run_filepath):
+    # squad doesn't know about fuego run formats
+    dirpath = tempfile.mkdtemp()
+    try:
+        subprocess.check_call('tar -C %s -xf %s' % (dirpath, run_filepath), shell=True)
+        with open(dirpath + '/run/run.json') as f:
+            json_data = json.load(f)
+        # create submit url
+        # Ex: http://localhost:8000/api/submit/fuego/Benchmark.Dhrystone/713c5342/bbb
+        # project_id: this is a name for something you want to test periodically.
+        #   For example, the major version of some software that will be updated
+        #   with bug fixes periodically such as the stable kernel v4.4.y
+        #   or a rootfs built with a specific yocto branch
+        #   -> For now, the project_id is specified in fuego.conf using server_squad_project.
+        #      [FIXTHIS] add a command line argument to override it (we don't have this concept in fuego)
+        # build_id: this is something like an unique identifier for a snapshot
+        #    of your project when you built it. For example, the commit hash of
+        #    the kernel/yocto repo, the kernel minor version or the date. You
+        #    can test the same build_id on multiple environments (boards/kernels)
+        #    for comparison.
+        #   -> For now, just use the testsuite_version and kernel_version
+        #      [FIXTHIS] add a command line argument to override it (we don't have this concept in fuego)
+        # env_id: this is the environment for the test
+        #   -> For now, use the board name and kernel_major_version
+        #      [FIXTHIS] add a command line argument to override it
+        project_id = conf.project_id
+        build_id = json_data['metadata']['testsuite_version'] + '-' + json_data['metadata']['kernel_version']
+        kernel_major_version = '.'.join(json_data['metadata']['kernel_version'].split('.')[:2])
+        env_id = json_data['metadata']['board'] + '-' + kernel_major_version
+        url = "%s/submit/%s/%s/%s/%s" % (conf.SERVER_URL_BASE, conf.team, project_id, build_id, env_id)
+
+        # prepare a multipart files dict for posting
+        files = []
+        ## prepare metadata file
+        metadata = json_data['metadata']
+        # the job_id needs to be unique within the project, it identifies a run
+        timestamp = metadata['timestamp'].replace(':', 'colon').replace('+', 'plus')
+        metadata['job_id'] = '-'.join((metadata['job_name'], metadata['build_number'], metadata['kernel_version'], timestamp))
+        metadata['job_status'] = json_data['status']
+        metadata['datetime'] = metadata['timestamp']
+        files.append(('metadata', ('metadata', json.dumps(metadata))))
+        ## prepare tests and metrics files
+        test_sets = json_data['test_sets']
+        tests = dict()
+        metrics = dict()
+        suite = json_data['name'].split('.')[1]
+        #tests[suite] = json_data['status']
+        for test_set in test_sets:
+            test_set_key = os.path.join(suite, test_set['name'])
+            #tests[test_set_key] = test_set['status']
+            for test_case in test_set['test_cases']:
+                test_case_key = os.path.join(test_set_key, test_case['name'])
+                if 'measurements' in test_case:
+                    for test_measure in test_case['measurements']:
+                        test_measure_key = os.path.join(test_case_key, test_measure['name'])
+                        # use the suite name to group all subtests
+                        squad_key = suite + '/' + test_measure_key.replace('/','_')
+                        tests[squad_key] = test_measure['status']
+                        if test_measure['status'] != 'SKIP':
+                            metrics[squad_key] = [test_measure['measure']]
+                else:
+                    squad_key= suite + '/' + test_case_key.replace('/','_')
+                    tests[squad_key] = test_case['status']
+        files.append(('tests', ('tests', json.dumps(tests))))
+        files.append(('metrics', ('metrics', json.dumps(metrics))))
+        ## prepare attachments
+        for attachment in metadata['attachments']:
+            abs_path = dirpath + '/run/' + attachment['path']
+            if os.path.isfile(abs_path):
+                files.append(('attachment', (attachment['path'], open(abs_path, 'rb'))))
+            else:
+                print("WARNING: not adding attachment " + attachment['name'])
+
+        # set token
+        headers = {}
+        headers['Auth-Token'] = conf.token
+
+        print("POSTING to " + url)
+        response = requests.post(url, headers=headers, files=files)
+        if str(response.status_code)[:1] == "2":
+            result = 'OK'
+        else:
+            result = 'NG'
+        content = response.text.split('\n', 1)
+    except Exception as e:
+        result = "NG"
+        content = 'Exception in ftc'
+        print_error(str(e))
+    finally:
+        shutil.rmtree(dirpath)
+    return result, content
 
 def do_put_run(conf, options):
     # FIXTHIS - could consolidate do_put_test and do_put_run into do_put_item
@@ -3302,16 +3406,19 @@ def do_put_run(conf, options):
         created_package = True
         run_filepath = do_package_run(conf, [run_arg, "-o", "/tmp"])
 
-    url = conf.SERVER_URL_BASE+"put_run"
-    run_files = {"file1": open(run_filepath, "rb")}
-    resp = requests.post(url, files=run_files)
-    if created_package:
-        os.remove(run_filepath)
-    result, content = resp.text.split('\n', 1)
-    if result != "OK":
-        error_out("Can't put run to server\nServer returned message: %s" % content)
+    try:
+        if conf.server_type == "fuego":
+            result, content = put_run_fuego(conf, run_filepath)
+        elif conf.server_type == "squad":
+            result, content = put_run_squad(conf, run_filepath)
+    finally:
+        if created_package:
+            os.remove(run_filepath)
 
-    print "Run package %s was accepted by the server." % os.path.basename(run_filepath)
+    if result != "OK":
+        print("Can't put run to server\nServer returned message: %s" % content)
+    else:
+        print "Run package %s was accepted by the server." % os.path.basename(run_filepath)
 
 # returns a matching item (which must be unique)
 # from a list.  A matching item is one with leading
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread
* [Fuego] [PATCH 2/2] squad: add paramters for squad
  2019-01-10  9:00 [Fuego] Support for Squad Daniel Sangorrin
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] ftc: implement put-run for squad Daniel Sangorrin
@ 2019-01-10  9:00 ` Daniel Sangorrin
  2019-01-10  9:04   ` daniel.sangorrin
  2019-01-11  1:28   ` Tim.Bird
  2019-01-11  2:17 ` [Fuego] Support for Squad Tim.Bird
  2019-01-11 22:46 ` Tim.Bird
  3 siblings, 2 replies; 12+ messages in thread
From: Daniel Sangorrin @ 2019-01-10  9:00 UTC (permalink / raw)
  To: fuego

Fuego results can be exported to multiple servers, one of them
Squad.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 fuego-ro/conf/fuego.conf | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
index 23fc312..c6c17db 100644
--- a/fuego-ro/conf/fuego.conf
+++ b/fuego-ro/conf/fuego.conf
@@ -6,10 +6,22 @@
 # internet domain name
 host_name=fuegohost
 
-### Fuego server
-# fuego_server is usually the central public Fuego server
-# this is where the "test store" is, and where you publish results to
-fuego_server=fuegotest.org/cgi-bin
+### Server
+# This contains configuration related to a server where you
+# can publish results (runs) or where the "test store" is.
+# - server_type: API to use (default:fuego, others: squad)
+# - server_domain: the URL domain of your server
+# - server_${server_type}_param: parameters specific to a server type
+# - server_squad_token: auth token needed by squad
+# - server_squad_team: default team to use when submitting jobs to squad
+# - server_squad_project: default project to use when submitting runs to squad
+#server_type=fuego
+#server_domain=fuegotest.org/cgi-bin
+server_type=squad
+server_domain=localhost:8000
+server_squad_token=sdf9s8s9fasd7fsagsdfg9asg9asdfasfas7dfas
+server_squad_team=fuego
+server_squad_project=jessie
 
 ### paths to fuego directories
 # Paths can be absolute, or relative.
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread
* Re: [Fuego] [PATCH 2/2] squad: add paramters for squad
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] squad: add paramters " Daniel Sangorrin
@ 2019-01-10  9:04   ` daniel.sangorrin
  2019-01-11  1:28   ` Tim.Bird
  1 sibling, 0 replies; 12+ messages in thread
From: daniel.sangorrin @ 2019-01-10  9:04 UTC (permalink / raw)
  To: fuego

Sorry I forgot to switch back to Fuego server by default.

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org <fuego-bounces@lists.linuxfoundation.org> On Behalf Of
> Daniel Sangorrin
> Sent: Thursday, January 10, 2019 6:01 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 2/2] squad: add paramters for squad
> 
> Fuego results can be exported to multiple servers, one of them
> Squad.
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  fuego-ro/conf/fuego.conf | 20 ++++++++++++++++----
>  1 file changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
> index 23fc312..c6c17db 100644
> --- a/fuego-ro/conf/fuego.conf
> +++ b/fuego-ro/conf/fuego.conf
> @@ -6,10 +6,22 @@
>  # internet domain name
>  host_name=fuegohost
> 
> -### Fuego server
> -# fuego_server is usually the central public Fuego server
> -# this is where the "test store" is, and where you publish results to
> -fuego_server=fuegotest.org/cgi-bin
> +### Server
> +# This contains configuration related to a server where you
> +# can publish results (runs) or where the "test store" is.
> +# - server_type: API to use (default:fuego, others: squad)
> +# - server_domain: the URL domain of your server
> +# - server_${server_type}_param: parameters specific to a server type
> +# - server_squad_token: auth token needed by squad
> +# - server_squad_team: default team to use when submitting jobs to squad
> +# - server_squad_project: default project to use when submitting runs to squad
> +#server_type=fuego
> +#server_domain=fuegotest.org/cgi-bin
> +server_type=squad
> +server_domain=localhost:8000
> +server_squad_token=sdf9s8s9fasd7fsagsdfg9asg9asdfasfas7dfas
> +server_squad_team=fuego
> +server_squad_project=jessie
> 
>  ### paths to fuego directories
>  # Paths can be absolute, or relative.
> --
> 2.7.4
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] [PATCH 2/2] ftc: implement put-run for squad
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] ftc: implement put-run for squad Daniel Sangorrin
@ 2019-01-11  0:23   ` daniel.sangorrin
  2019-01-11  1:50   ` Tim.Bird
  1 sibling, 0 replies; 12+ messages in thread
From: daniel.sangorrin @ 2019-01-11  0:23 UTC (permalink / raw)
  To: fuego



> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org <fuego-bounces@lists.linuxfoundation.org> On Behalf Of
> Daniel Sangorrin
> Sent: Thursday, January 10, 2019 6:01 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 2/2] ftc: implement put-run for squad
> 
> This is an early implementation, more debugging is needed
> and there are several FIXTHIS included, but it works.
> You can try it on Ubuntu Xenial with the instructions below.
> 
> Install server application
> 	host$ sudo apt-get install rabbitmq-server
> 		[Note] this is required for squad to communicate with celery worker tasks
> 	host$ git clone https://github.com/Linaro/squad
> 	host$ cd squad/
> 	host$ mkvirtualenv --python=python3  mysquad
> 	host$ cd squad
> 	host$ pip3 install -r requirements-dev.txt
> 	host$ ./manage.py test
> 	host$ ./manage.py migrate
> 	host$ ./manage.py createsuperuser
> 	host$ ./manage.py runserver
> 
> [Note] The next time you use it use this:
> 	host$ cd squad
> 	host$ workon squad
> 	host$ ./manage.py runserver
> 
> Configure a team and project in Squad
> 	host$ firefox http://localhost:8000/
> 		- click upright > settings
> 			- profile: write your name
> 			- API token: copy the token
> 				-> assign it to "server_squad_token" in fuego-ro/fuego.conf
> 		- click upright > Administration
> 			- Authentication and authorization
> 				- Add group (this is a user group!)
> 					- name: myusergroup
> 					- permissons: choose all
> 			- Core
> 				- Add group (this is a team)
> 					- slug=name=fuego
> 						-> assign it to "server_squad_team" in fuego-ro/fuego.conf
> 					- description: whatever
> 					- usergroups: myusergroup
> 				- Add project
> 					- group: fuego
> 					- slug=name=jessie
> 						-> assign it to "server_squad_project" in fuego-ro/fuego.conf
> 
> Upload a run from Fuego
> 	fuego-docker# ftc list-runs -q
> 		Benchmark.IOzone-default-1-bbb
> 	fuego-docker# ftc put-run Benchmark.IOzone-default-1-bbb
> 		Packaging run 'Benchmark.IOzone-default-1-bbb'
> 		run/
> 		run/devlog.after.txt
> 		run/syslog.before.txt
> 		run/build.xml
> 		run/devlog.txt
> 		run/testlog.txt
> 		run/consolelog.txt
> 		run/run.json
> 		Run packaged successfully, and is at:
> /tmp/run-Benchmark.IOzone-default-1-on-fuegohost:bbb.frp
> 		WARNING: not adding attachment syslog.after
> 		WARNING: not adding attachment test_spec
> 		POSTING to
> http://localhost:8000/api/submit/fuego/jessie/a80be74d1629bdb2455b08db453beba0-4.9.133/bbb-4.9
> 		Run package run-Benchmark.IOzone-default-1-on-fuegohost:bbb.frp was accepted by the
> server.
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/scripts/ftc | 131
> ++++++++++++++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 119 insertions(+), 12 deletions(-)
> 
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index 73fd98a..ab2d2cb 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -519,9 +519,16 @@ class config_class:
>          # set values from conf_map
>          self.host = conf_map.get("host_name", "localhost")
>          self.host_name = conf_map.get("host_name", "localhost")
> -        self.fuego_server = conf_map.get("fuego_server", "fuegotest.org")
> -        #self.SERVER_URL_BASE = "http://%s/server/Fuego_Server?action=Fuego." % self.fuego_server
> -        self.SERVER_URL_BASE = "http://%s/fserver.py/?action=" % self.fuego_server
> +        self.server_type = conf_map.get("server_type", "fuego")
> +        self.server_domain = conf_map.get("server_domain", "fuegotest.org")
> +        #self.SERVER_URL_BASE = "http://%s/server/Fuego_Server?action=Fuego." % self.server_domain
> +        if self.server_type == "fuego":
> +            self.SERVER_URL_BASE = "http://%s/fserver.py/?action=" % self.server_domain
> +        elif self.server_type == "squad":
> +            self.SERVER_URL_BASE = "http://%s/api" % self.server_domain
> +            self.token = conf_map.get("server_squad_token", "WRITE_TOKEN_IN_FUEGO.CONF")
> +            self.team = conf_map.get("server_squad_team", "fuego")
> +            self.project_id = conf_map.get("server_squad_project", "myproject")
> 
>          # handle paths (including relative ones)
>          conf_dir = os.path.dirname(os.path.abspath(config_path))
> @@ -3282,6 +3289,103 @@ def do_put_test(conf, options):
> 
>      print "Test package %s was accepted by the server." % os.path.basename(test_filepath)
> 
> +def put_run_fuego(conf, run_filepath):
> +    url = conf.SERVER_URL_BASE+"put_run"
> +    run_files = {"file1": open(run_filepath, "rb")}
> +    resp = requests.post(url, files=run_files)
> +    return resp.text.split('\n', 1)
> +
> +def put_run_squad(conf, run_filepath):
> +    # squad doesn't know about fuego run formats
> +    dirpath = tempfile.mkdtemp()
> +    try:
> +        subprocess.check_call('tar -C %s -xf %s' % (dirpath, run_filepath), shell=True)
> +        with open(dirpath + '/run/run.json') as f:
> +            json_data = json.load(f)
> +        # create submit url
> +        # Ex: http://localhost:8000/api/submit/fuego/Benchmark.Dhrystone/713c5342/bbb

Oops this URL is outdated. Benchmark.Dhrystone is a Suite, not project.
It should be something like in the example above:
http://localhost:8000/api/submit/fuego/jessie/a80be74d1629bdb2455b08db453beba0-4.9.133/bbb-4.9

> +        # project_id: this is a name for something you want to test periodically.
> +        #   For example, the major version of some software that will be updated
> +        #   with bug fixes periodically such as the stable kernel v4.4.y
> +        #   or a rootfs built with a specific yocto branch
> +        #   -> For now, the project_id is specified in fuego.conf using server_squad_project.
> +        #      [FIXTHIS] add a command line argument to override it (we don't have this concept in fuego)
> +        # build_id: this is something like an unique identifier for a snapshot
> +        #    of your project when you built it. For example, the commit hash of
> +        #    the kernel/yocto repo, the kernel minor version or the date. You
> +        #    can test the same build_id on multiple environments (boards/kernels)
> +        #    for comparison.
> +        #   -> For now, just use the testsuite_version and kernel_version
> +        #      [FIXTHIS] add a command line argument to override it (we don't have this concept in fuego)
> +        # env_id: this is the environment for the test
> +        #   -> For now, use the board name and kernel_major_version
> +        #      [FIXTHIS] add a command line argument to override it
> +        project_id = conf.project_id
> +        build_id = json_data['metadata']['testsuite_version'] + '-' + json_data['metadata']['kernel_version']
> +        kernel_major_version = '.'.join(json_data['metadata']['kernel_version'].split('.')[:2])
> +        env_id = json_data['metadata']['board'] + '-' + kernel_major_version
> +        url = "%s/submit/%s/%s/%s/%s" % (conf.SERVER_URL_BASE, conf.team, project_id, build_id,
> env_id)
> +
> +        # prepare a multipart files dict for posting
> +        files = []
> +        ## prepare metadata file
> +        metadata = json_data['metadata']
> +        # the job_id needs to be unique within the project, it identifies a run
> +        timestamp = metadata['timestamp'].replace(':', 'colon').replace('+', 'plus')
> +        metadata['job_id'] = '-'.join((metadata['job_name'], metadata['build_number'],
> metadata['kernel_version'], timestamp))
> +        metadata['job_status'] = json_data['status']
> +        metadata['datetime'] = metadata['timestamp']
> +        files.append(('metadata', ('metadata', json.dumps(metadata))))
> +        ## prepare tests and metrics files
> +        test_sets = json_data['test_sets']
> +        tests = dict()
> +        metrics = dict()
> +        suite = json_data['name'].split('.')[1]
> +        #tests[suite] = json_data['status']
> +        for test_set in test_sets:
> +            test_set_key = os.path.join(suite, test_set['name'])
> +            #tests[test_set_key] = test_set['status']
> +            for test_case in test_set['test_cases']:
> +                test_case_key = os.path.join(test_set_key, test_case['name'])
> +                if 'measurements' in test_case:
> +                    for test_measure in test_case['measurements']:
> +                        test_measure_key = os.path.join(test_case_key, test_measure['name'])
> +                        # use the suite name to group all subtests
> +                        squad_key = suite + '/' + test_measure_key.replace('/','_')
> +                        tests[squad_key] = test_measure['status']
> +                        if test_measure['status'] != 'SKIP':
> +                            metrics[squad_key] = [test_measure['measure']]
> +                else:
> +                    squad_key= suite + '/' + test_case_key.replace('/','_')
> +                    tests[squad_key] = test_case['status']
> +        files.append(('tests', ('tests', json.dumps(tests))))
> +        files.append(('metrics', ('metrics', json.dumps(metrics))))
> +        ## prepare attachments
> +        for attachment in metadata['attachments']:
> +            abs_path = dirpath + '/run/' + attachment['path']
> +            if os.path.isfile(abs_path):
> +                files.append(('attachment', (attachment['path'], open(abs_path, 'rb'))))
> +            else:
> +                print("WARNING: not adding attachment " + attachment['name'])
> +
> +        # set token
> +        headers = {}
> +        headers['Auth-Token'] = conf.token
> +
> +        print("POSTING to " + url)
> +        response = requests.post(url, headers=headers, files=files)
> +        if str(response.status_code)[:1] == "2":
> +            result = 'OK'
> +        else:
> +            result = 'NG'
> +        content = response.text.split('\n', 1)
> +    except Exception as e:
> +        result = "NG"
> +        content = 'Exception in ftc'
> +        print_error(str(e))
> +    finally:
> +        shutil.rmtree(dirpath)
> +    return result, content
> 
>  def do_put_run(conf, options):
>      # FIXTHIS - could consolidate do_put_test and do_put_run into do_put_item
> @@ -3302,16 +3406,19 @@ def do_put_run(conf, options):
>          created_package = True
>          run_filepath = do_package_run(conf, [run_arg, "-o", "/tmp"])
> 
> -    url = conf.SERVER_URL_BASE+"put_run"
> -    run_files = {"file1": open(run_filepath, "rb")}
> -    resp = requests.post(url, files=run_files)
> -    if created_package:
> -        os.remove(run_filepath)
> -    result, content = resp.text.split('\n', 1)
> -    if result != "OK":
> -        error_out("Can't put run to server\nServer returned message: %s" % content)
> +    try:
> +        if conf.server_type == "fuego":
> +            result, content = put_run_fuego(conf, run_filepath)
> +        elif conf.server_type == "squad":
> +            result, content = put_run_squad(conf, run_filepath)
> +    finally:
> +        if created_package:
> +            os.remove(run_filepath)
> 
> -    print "Run package %s was accepted by the server." % os.path.basename(run_filepath)
> +    if result != "OK":
> +        print("Can't put run to server\nServer returned message: %s" % content)
> +    else:
> +        print "Run package %s was accepted by the server." % os.path.basename(run_filepath)
> 
>  # returns a matching item (which must be unique)
>  # from a list.  A matching item is one with leading
> --
> 2.7.4
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] [PATCH 2/2] squad: add paramters for squad
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] squad: add paramters " Daniel Sangorrin
  2019-01-10  9:04   ` daniel.sangorrin
@ 2019-01-11  1:28   ` Tim.Bird
  1 sibling, 0 replies; 12+ messages in thread
From: Tim.Bird @ 2019-01-11  1:28 UTC (permalink / raw)
  To: daniel.sangorrin, fuego



> -----Original Message-----
> From: Daniel Sangorrin
> 
> Fuego results can be exported to multiple servers, one of them
> Squad.
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  fuego-ro/conf/fuego.conf | 20 ++++++++++++++++----
>  1 file changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
> index 23fc312..c6c17db 100644
> --- a/fuego-ro/conf/fuego.conf
> +++ b/fuego-ro/conf/fuego.conf
> @@ -6,10 +6,22 @@
>  # internet domain name
>  host_name=fuegohost
> 
> -### Fuego server
> -# fuego_server is usually the central public Fuego server
> -# this is where the "test store" is, and where you publish results to
> -fuego_server=fuegotest.org/cgi-bin
> +### Server
> +# This contains configuration related to a server where you
> +# can publish results (runs) or where the "test store" is.
> +# - server_type: API to use (default:fuego, others: squad)
> +# - server_domain: the URL domain of your server
> +# - server_${server_type}_param: parameters specific to a server type
> +# - server_squad_token: auth token needed by squad
> +# - server_squad_team: default team to use when submitting jobs to squad
> +# - server_squad_project: default project to use when submitting runs to
> squad
> +#server_type=fuego
> +#server_domain=fuegotest.org/cgi-bin
> +server_type=squad
> +server_domain=localhost:8000
> +server_squad_token=sdf9s8s9fasd7fsagsdfg9asg9asdfasfas7dfas
> +server_squad_team=fuego
> +server_squad_project=jessie
> 
>  ### paths to fuego directories
>  # Paths can be absolute, or relative.
> --
> 2.7.4

This looks OK.  I have a couple of thoughts (not really objections, just random ideas).

This is mostly about the configuration name space for this.

The Fuego server eventually intends to be:
 - a test store (and test information and review site)
 - a request dispatcher
 - a results store
 - a results presenter

But it might not be a good to coalesce the settings for all of these into one thing.
I think you mentioned that currently squad is used for a results store and a 
results presenter, and might in the future be used as a request dispatcher.

Along these lines, I'm not sure if the parameters for each server should intersect
or not.

Right now, the settings that fserver (that's what I'm calling the Fuego server now)
and squad have in common are type and  domain (with type just being used to
disambiguate the different servers)  Is domain the only thing they'll share?

I'm sure fserver will require additional parameters, like squad does.
For instance, I know that eventually I'll need an authentication token, and
maybe some other data for that server (like a client id).

You've separated the variables by putting the type into the variable name:
e.g. fuego_squad_token, fuego_squad_team.

Since this is a proof-of-concept, I guess we don't have enough examples to
see if we could consolidate these into a single name space or whether that's
even worthwhile.

But maybe we shouldn't even consolidate 'domain'?  How about just using
squad_server_address and fserver_url, and not having a type?  I guess
ftc uses the type variable to indicate which one to send to, in addition
to the protocol to use to communicate with it.

I'm thinking out loud - not actually requesting any changes.

In the meantime, I'll apply the patch.
 -- Tim


^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] [PATCH 2/2] ftc: implement put-run for squad
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] ftc: implement put-run for squad Daniel Sangorrin
  2019-01-11  0:23   ` daniel.sangorrin
@ 2019-01-11  1:50   ` Tim.Bird
  1 sibling, 0 replies; 12+ messages in thread
From: Tim.Bird @ 2019-01-11  1:50 UTC (permalink / raw)
  To: daniel.sangorrin, fuego

A few comments inline below.

> -----Original Message-----
> From: Daniel Sangorrin
>
> This is an early implementation, more debugging is needed
> and there are several FIXTHIS included, but it works.
> You can try it on Ubuntu Xenial with the instructions below.
> 
> Install server application
> 	host$ sudo apt-get install rabbitmq-server
> 		[Note] this is required for squad to communicate with celery
> worker tasks
> 	host$ git clone https://github.com/Linaro/squad
> 	host$ cd squad/
> 	host$ mkvirtualenv --python=python3  mysquad
> 	host$ cd squad
> 	host$ pip3 install -r requirements-dev.txt
> 	host$ ./manage.py test
> 	host$ ./manage.py migrate
> 	host$ ./manage.py createsuperuser
> 	host$ ./manage.py runserver
> 
> [Note] The next time you use it use this:
> 	host$ cd squad
> 	host$ workon squad
> 	host$ ./manage.py runserver
> 
> Configure a team and project in Squad
> 	host$ firefox http://localhost:8000/
> 		- click upright > settings
> 			- profile: write your name
> 			- API token: copy the token
> 				-> assign it to "server_squad_token" in
> fuego-ro/fuego.conf
> 		- click upright > Administration
> 			- Authentication and authorization
> 				- Add group (this is a user group!)
> 					- name: myusergroup
> 					- permissons: choose all
> 			- Core
> 				- Add group (this is a team)
> 					- slug=name=fuego
> 						-> assign it to
> "server_squad_team" in fuego-ro/fuego.conf
> 					- description: whatever
> 					- usergroups: myusergroup
> 				- Add project
> 					- group: fuego
> 					- slug=name=jessie
> 						-> assign it to
> "server_squad_project" in fuego-ro/fuego.conf
> 
> Upload a run from Fuego
> 	fuego-docker# ftc list-runs -q
> 		Benchmark.IOzone-default-1-bbb
> 	fuego-docker# ftc put-run Benchmark.IOzone-default-1-bbb
> 		Packaging run 'Benchmark.IOzone-default-1-bbb'
> 		run/
> 		run/devlog.after.txt
> 		run/syslog.before.txt
> 		run/build.xml
> 		run/devlog.txt
> 		run/testlog.txt
> 		run/consolelog.txt
> 		run/run.json
> 		Run packaged successfully, and is at: /tmp/run-
> Benchmark.IOzone-default-1-on-fuegohost:bbb.frp
> 		WARNING: not adding attachment syslog.after
> 		WARNING: not adding attachment test_spec
> 		POSTING to
> http://localhost:8000/api/submit/fuego/jessie/a80be74d1629bdb2455b08db
> 453beba0-4.9.133/bbb-4.9
> 		Run package run-Benchmark.IOzone-default-1-on-
> fuegohost:bbb.frp was accepted by the server.

OK - yeah - this definitely needs a wiki page.  Or, if we like squad enough
we preinstall it in the container and set all this up for at least the local 
instance.  (And we still need a wiki page for how people would change
the settings for a non-local or non-Fuego squad instance.)

> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  engine/scripts/ftc | 131
> ++++++++++++++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 119 insertions(+), 12 deletions(-)
> 
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index 73fd98a..ab2d2cb 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -519,9 +519,16 @@ class config_class:
>          # set values from conf_map
>          self.host = conf_map.get("host_name", "localhost")
>          self.host_name = conf_map.get("host_name", "localhost")
> -        self.fuego_server = conf_map.get("fuego_server", "fuegotest.org")
> -        #self.SERVER_URL_BASE =
> "http://%s/server/Fuego_Server?action=Fuego." % self.fuego_server
> -        self.SERVER_URL_BASE = "http://%s/fserver.py/?action=" %
> self.fuego_server
> +        self.server_type = conf_map.get("server_type", "fuego")
> +        self.server_domain = conf_map.get("server_domain", "fuegotest.org")
> +        #self.SERVER_URL_BASE =
> "http://%s/server/Fuego_Server?action=Fuego." % self.server_domain
> +        if self.server_type == "fuego":
> +            self.SERVER_URL_BASE = "http://%s/fserver.py/?action=" %
> self.server_domain
> +        elif self.server_type == "squad":
> +            self.SERVER_URL_BASE = "http://%s/api" % self.server_domain
> +            self.token = conf_map.get("server_squad_token",
> "WRITE_TOKEN_IN_FUEGO.CONF")

I like the stand-in value here. :-)

> +            self.team = conf_map.get("server_squad_team", "fuego")
> +            self.project_id = conf_map.get("server_squad_project",
> "myproject")
> 
>          # handle paths (including relative ones)
>          conf_dir = os.path.dirname(os.path.abspath(config_path))
> @@ -3282,6 +3289,103 @@ def do_put_test(conf, options):
> 
>      print "Test package %s was accepted by the server." %
> os.path.basename(test_filepath)
> 
> +def put_run_fuego(conf, run_filepath):
> +    url = conf.SERVER_URL_BASE+"put_run"
> +    run_files = {"file1": open(run_filepath, "rb")}
> +    resp = requests.post(url, files=run_files)
> +    return resp.text.split('\n', 1)
> +
> +def put_run_squad(conf, run_filepath):
> +    # squad doesn't know about fuego run formats
> +    dirpath = tempfile.mkdtemp()
> +    try:
> +        subprocess.check_call('tar -C %s -xf %s' % (dirpath, run_filepath),
> shell=True)
> +        with open(dirpath + '/run/run.json') as f:
> +            json_data = json.load(f)
> +        # create submit url
> +        # Ex:
> http://localhost:8000/api/submit/fuego/Benchmark.Dhrystone/713c5342/bb
> b
> +        # project_id: this is a name for something you want to test periodically.
> +        #   For example, the major version of some software that will be
> updated
> +        #   with bug fixes periodically such as the stable kernel v4.4.y
> +        #   or a rootfs built with a specific yocto branch
> +        #   -> For now, the project_id is specified in fuego.conf using
> server_squad_project.
> +        #      [FIXTHIS] add a command line argument to override it (we don't
> have this concept in fuego)

This begs the question of possibly supporting a plugin architecture
for ftc, for settings and communications protocols for backends.

> +        # build_id: this is something like an unique identifier for a snapshot
> +        #    of your project when you built it. For example, the commit hash of
> +        #    the kernel/yocto repo, the kernel minor version or the date. You
> +        #    can test the same build_id on multiple environments
> (boards/kernels)
interesting.

> +        #    for comparison.
> +        #   -> For now, just use the testsuite_version and kernel_version
> +        #      [FIXTHIS] add a command line argument to override it (we don't
> have this concept in fuego)
> +        # env_id: this is the environment for the test
> +        #   -> For now, use the board name and kernel_major_version
> +        #      [FIXTHIS] add a command line argument to override it
> +        project_id = conf.project_id
> +        build_id = json_data['metadata']['testsuite_version'] + '-' +
> json_data['metadata']['kernel_version']
> +        kernel_major_version =
> '.'.join(json_data['metadata']['kernel_version'].split('.')[:2])
> +        env_id = json_data['metadata']['board'] + '-' + kernel_major_version
> +        url = "%s/submit/%s/%s/%s/%s" % (conf.SERVER_URL_BASE,
> conf.team, project_id, build_id, env_id)
> +
> +        # prepare a multipart files dict for posting
> +        files = []
> +        ## prepare metadata file
> +        metadata = json_data['metadata']
> +        # the job_id needs to be unique within the project, it identifies a run
fserver requires something like this as well.  I don't recall if I'm using timestamp,
but I might be.

> +        timestamp = metadata['timestamp'].replace(':', 'colon').replace('+',
> 'plus')
OK - I presume that ':' and '+' are illegal chars for squad job ids?  this probably
produces pretty ugly job ids though.  Could you just strip them?

> +        metadata['job_id'] = '-'.join((metadata['job_name'],
> metadata['build_number'], metadata['kernel_version'], timestamp))
> +        metadata['job_status'] = json_data['status']
> +        metadata['datetime'] = metadata['timestamp']
> +        files.append(('metadata', ('metadata', json.dumps(metadata))))
> +        ## prepare tests and metrics files
> +        test_sets = json_data['test_sets']
> +        tests = dict()
> +        metrics = dict()
> +        suite = json_data['name'].split('.')[1]
> +        #tests[suite] = json_data['status']
> +        for test_set in test_sets:
> +            test_set_key = os.path.join(suite, test_set['name'])
> +            #tests[test_set_key] = test_set['status']
> +            for test_case in test_set['test_cases']:
> +                test_case_key = os.path.join(test_set_key, test_case['name'])
> +                if 'measurements' in test_case:
> +                    for test_measure in test_case['measurements']:
> +                        test_measure_key = os.path.join(test_case_key,
> test_measure['name'])
> +                        # use the suite name to group all subtests
> +                        squad_key = suite + '/' + test_measure_key.replace('/','_')
> +                        tests[squad_key] = test_measure['status']
> +                        if test_measure['status'] != 'SKIP':
> +                            metrics[squad_key] = [test_measure['measure']]
> +                else:
> +                    squad_key= suite + '/' + test_case_key.replace('/','_')
> +                    tests[squad_key] = test_case['status']
> +        files.append(('tests', ('tests', json.dumps(tests))))
> +        files.append(('metrics', ('metrics', json.dumps(metrics))))
> +        ## prepare attachments
> +        for attachment in metadata['attachments']:
> +            abs_path = dirpath + '/run/' + attachment['path']
> +            if os.path.isfile(abs_path):
> +                files.append(('attachment', (attachment['path'], open(abs_path,
> 'rb'))))
> +            else:
> +                print("WARNING: not adding attachment " + attachment['name'])
> +
> +        # set token
> +        headers = {}
> +        headers['Auth-Token'] = conf.token
> +
> +        print("POSTING to " + url)
> +        response = requests.post(url, headers=headers, files=files)
> +        if str(response.status_code)[:1] == "2":
> +            result = 'OK'
> +        else:
> +            result = 'NG'
> +        content = response.text.split('\n', 1)
> +    except Exception as e:
> +        result = "NG"
> +        content = 'Exception in ftc'
> +        print_error(str(e))
> +    finally:
> +        shutil.rmtree(dirpath)
> +    return result, content
> 
>  def do_put_run(conf, options):
>      # FIXTHIS - could consolidate do_put_test and do_put_run into
> do_put_item
> @@ -3302,16 +3406,19 @@ def do_put_run(conf, options):
>          created_package = True
>          run_filepath = do_package_run(conf, [run_arg, "-o", "/tmp"])
> 
> -    url = conf.SERVER_URL_BASE+"put_run"
> -    run_files = {"file1": open(run_filepath, "rb")}
> -    resp = requests.post(url, files=run_files)
> -    if created_package:
> -        os.remove(run_filepath)
> -    result, content = resp.text.split('\n', 1)
> -    if result != "OK":
> -        error_out("Can't put run to server\nServer returned message: %s" %
> content)
> +    try:
> +        if conf.server_type == "fuego":
> +            result, content = put_run_fuego(conf, run_filepath)
> +        elif conf.server_type == "squad":
> +            result, content = put_run_squad(conf, run_filepath)
> +    finally:
> +        if created_package:
> +            os.remove(run_filepath)
> 
> -    print "Run package %s was accepted by the server." %
> os.path.basename(run_filepath)
> +    if result != "OK":
> +        print("Can't put run to server\nServer returned message: %s" %
> content)
> +    else:
> +        print "Run package %s was accepted by the server." %
> os.path.basename(run_filepath)
> 
>  # returns a matching item (which must be unique)
>  # from a list.  A matching item is one with leading
> --
> 2.7.4


Looks good for a prototype.

I'm applying as is.
 -- Tim


^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] Support for Squad
  2019-01-10  9:00 [Fuego] Support for Squad Daniel Sangorrin
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] ftc: implement put-run for squad Daniel Sangorrin
  2019-01-10  9:00 ` [Fuego] [PATCH 2/2] squad: add paramters " Daniel Sangorrin
@ 2019-01-11  2:17 ` Tim.Bird
  2019-01-11 22:46 ` Tim.Bird
  3 siblings, 0 replies; 12+ messages in thread
From: Tim.Bird @ 2019-01-11  2:17 UTC (permalink / raw)
  To: daniel.sangorrin, fuego

> -----Original Message-----
> From: Daniel Sangorrin
> 
> Hi Tim,
> 
> I have rebased my work to support Squad on top of Fuego v1.4.
> Before it's too late I wanted to send it to you (the patches are on my repo as
> well).


Thanks.  It's applied in my next branch, and pushed to bitbucket.
Let me know if I messed anything up.

I'll try to give it a test tomorrow.
 -- Tim



^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] Support for Squad
  2019-01-10  9:00 [Fuego] Support for Squad Daniel Sangorrin
                   ` (2 preceding siblings ...)
  2019-01-11  2:17 ` [Fuego] Support for Squad Tim.Bird
@ 2019-01-11 22:46 ` Tim.Bird
  2019-01-15  4:37   ` daniel.sangorrin
  3 siblings, 1 reply; 12+ messages in thread
From: Tim.Bird @ 2019-01-11 22:46 UTC (permalink / raw)
  To: daniel.sangorrin, fuego



> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Thursday, January 10, 2019 1:01 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] Support for Squad
> 
> Hi Tim,
> 
> I have rebased my work to support Squad on top of Fuego v1.4.
> Before it's too late I wanted to send it to you (the patches are on my repo as
> well).
> 
> [PATCH 2/2] ftc: implement put-run for squad
> [PATCH 2/2] squad: add paramters for squad
> Note: sorry about the patch numbering and mixing fuego and fuego-core
> patches.
> 
> The instructions to test these patches are on the latest fuego-core commit
> (on my repo)
> but I reproduce them below. I can add them to the wiki page later on. I
> modified
> some variables in fuego.conf as well that should be updated.
> 
> The code is experimental, I have only tested it for Dhrystone and IOzone :D.
> But it is a good start and I think that it is better to discuss some important
> concepts early on.
> For example, I think I need to add paramwters to the put-run interface
> (team, project)
> Also, I think that all these server-related functions should be implemented as
> classes/objects.
> Currently, I only added support for put-run but technically Squad can also
> support other
> functions such as put-request.
> Other than that, in Squad the "job_id" needs to be unique to a "project". We
> need to think
> about how to do that if anyone in the team can submit jobs.
> You can check my implementation and the comments I wrote there.
> 
> Thanks,
> Daniel
> 
> Test instructions
> =================
> 
> Ref: https://squad.readthedocs.io (doc)
> 
> Install server application
> 	host$ sudo apt-get install rabbitmq-server
> 		[Note] this is required for squad to communicate with celery
> worker tasks
> 	host$ git clone https://github.com/Linaro/squad
> 	host$ cd squad/
> 	host$ mkvirtualenv --python=python3  mysquad
I get the following error at this step:

---
$ mkvirtualenv --python=python3 mysquad
The executable /home/tbird/work/fuego/squad/python3 (from --python=/home/tbird/work/fuego/squad/python3) does not exist
---

I'm not sure how to fix this.  
I haven't worked with python virtual environments.
There must be something simple I'm missing.
Please help.


 -- Tim



^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] Support for Squad
  2019-01-11 22:46 ` Tim.Bird
@ 2019-01-15  4:37   ` daniel.sangorrin
  2019-01-15 19:05     ` Tim.Bird
  0 siblings, 1 reply; 12+ messages in thread
From: daniel.sangorrin @ 2019-01-15  4:37 UTC (permalink / raw)
  To: Tim.Bird, fuego

> -----Original Message-----
> From: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Sent: Saturday, January 12, 2019 7:46 AM
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> >
> > Hi Tim,
> >
> > I have rebased my work to support Squad on top of Fuego v1.4.
> > Before it's too late I wanted to send it to you (the patches are on my repo as
> > well).
> >
> > [PATCH 2/2] ftc: implement put-run for squad
> > [PATCH 2/2] squad: add paramters for squad
> > Note: sorry about the patch numbering and mixing fuego and fuego-core
> > patches.
> >
> > The instructions to test these patches are on the latest fuego-core commit
> > (on my repo)
> > but I reproduce them below. I can add them to the wiki page later on. I
> > modified
> > some variables in fuego.conf as well that should be updated.
> >
> > The code is experimental, I have only tested it for Dhrystone and IOzone :D.
> > But it is a good start and I think that it is better to discuss some important
> > concepts early on.
> > For example, I think I need to add paramwters to the put-run interface
> > (team, project)
> > Also, I think that all these server-related functions should be implemented as
> > classes/objects.
> > Currently, I only added support for put-run but technically Squad can also
> > support other
> > functions such as put-request.
> > Other than that, in Squad the "job_id" needs to be unique to a "project". We
> > need to think
> > about how to do that if anyone in the team can submit jobs.
> > You can check my implementation and the comments I wrote there.
> >
> > Thanks,
> > Daniel
> >
> > Test instructions
> > =================
> >
> > Ref: https://squad.readthedocs.io (doc)
> >
> > Install server application
> > 	host$ sudo apt-get install rabbitmq-server
> > 		[Note] this is required for squad to communicate with celery
> > worker tasks
> > 	host$ git clone https://github.com/Linaro/squad
> > 	host$ cd squad/
> > 	host$ mkvirtualenv --python=python3  mysquad
> I get the following error at this step:
> 
> ---
> $ mkvirtualenv --python=python3 mysquad
> The executable /home/tbird/work/fuego/squad/python3 (from
> --python=/home/tbird/work/fuego/squad/python3) does not exist
> ---
> 
> I'm not sure how to fix this.
> I haven't worked with python virtual environments.
> There must be something simple I'm missing.
> Please help.

Assuming that you have python3 installed, it looks like virtualenv can't find your default python3 binary.
For example, in Ubuntu Xenial I have /usr/bin/python3 pointing to /usr/bin/python3.5.
Could you try changing --python=python3 to the version you are using (eg: --python=/usr/bin/python3.x)

If you prefer not to use a virtual environment you can install it on the host as well
https://squad.readthedocs.io/en/latest/quickstart.html

Another easy alternative is to install squad directly from pip3:
https://squad.readthedocs.io/en/latest/install.html
$ mkvirtualenv --python=python3.5 pipsquad
(squadpip) pip3 install squad
(squadpip) squad

Thanks,
Daniel





^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] Support for Squad
  2019-01-15  4:37   ` daniel.sangorrin
@ 2019-01-15 19:05     ` Tim.Bird
  2019-01-15 21:40       ` Tim.Bird
  0 siblings, 1 reply; 12+ messages in thread
From: Tim.Bird @ 2019-01-15 19:05 UTC (permalink / raw)
  To: daniel.sangorrin, fuego



> -----Original Message-----
> From: daniel.sangorrin@toshiba.co.jp
> [mailto:daniel.sangorrin@toshiba.co.jp]
> Sent: Monday, January 14, 2019 8:37 PM
> To: Bird, Timothy <Tim.Bird@sony.com>; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] Support for Squad
> 
> > -----Original Message-----
> > From: Tim.Bird@sony.com <Tim.Bird@sony.com>
> > Sent: Saturday, January 12, 2019 7:46 AM
> > > -----Original Message-----
> > > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > >
> > > Hi Tim,
> > >
> > > I have rebased my work to support Squad on top of Fuego v1.4.
> > > Before it's too late I wanted to send it to you (the patches are on my repo
> as
> > > well).
> > >
> > > [PATCH 2/2] ftc: implement put-run for squad
> > > [PATCH 2/2] squad: add paramters for squad
> > > Note: sorry about the patch numbering and mixing fuego and fuego-core
> > > patches.
> > >
> > > The instructions to test these patches are on the latest fuego-core
> commit
> > > (on my repo)
> > > but I reproduce them below. I can add them to the wiki page later on. I
> > > modified
> > > some variables in fuego.conf as well that should be updated.
> > >
> > > The code is experimental, I have only tested it for Dhrystone and IOzone
> :D.
> > > But it is a good start and I think that it is better to discuss some important
> > > concepts early on.
> > > For example, I think I need to add paramwters to the put-run interface
> > > (team, project)
> > > Also, I think that all these server-related functions should be
> implemented as
> > > classes/objects.
> > > Currently, I only added support for put-run but technically Squad can also
> > > support other
> > > functions such as put-request.
> > > Other than that, in Squad the "job_id" needs to be unique to a "project".
> We
> > > need to think
> > > about how to do that if anyone in the team can submit jobs.
> > > You can check my implementation and the comments I wrote there.
> > >
> > > Thanks,
> > > Daniel
> > >
> > > Test instructions
> > > =================
> > >
> > > Ref: https://squad.readthedocs.io (doc)
> > >
> > > Install server application
> > > 	host$ sudo apt-get install rabbitmq-server
> > > 		[Note] this is required for squad to communicate with celery
> > > worker tasks
> > > 	host$ git clone https://github.com/Linaro/squad
> > > 	host$ cd squad/
> > > 	host$ mkvirtualenv --python=python3  mysquad
> > I get the following error at this step:
> >
> > ---
> > $ mkvirtualenv --python=python3 mysquad
> > The executable /home/tbird/work/fuego/squad/python3 (from
> > --python=/home/tbird/work/fuego/squad/python3) does not exist
> > ---
> >
> > I'm not sure how to fix this.
> > I haven't worked with python virtual environments.
> > There must be something simple I'm missing.
> > Please help.
> 
> Assuming that you have python3 installed, it looks like virtualenv can't find
> your default python3 binary.
> For example, in Ubuntu Xenial I have /usr/bin/python3 pointing to
> /usr/bin/python3.5.
> Could you try changing --python=python3 to the version you are using (eg: --
> python=/usr/bin/python3.x)

This got me farther.  But then later I got this:

(mysquad) tbird@timdesk:~/work/fuego/squad$ ./manage.py test
I: angularjs installed from http://code.angularjs.org/1.6.6/angular-1.6.6.zip
I: bootstrap installed from https://github.com/twbs/bootstrap/releases/download/v3.3.7/bootstrap-3.3.7-dist.zip
I: chartjs/Chart.bundle.js installed from https://github.com/chartjs/Chart.js/releases/download/v2.7.0/Chart.bundle.js
I: font-awesome installed from http://http.debian.net/debian/pool/main/f/fonts-font-awesome/fonts-font-awesome_4.7.0~dfsg.orig.tar.gz
I: lodash.js installed from https://raw.githubusercontent.com/lodash/lodash/4.17.4/dist/lodash.js
I: jquery.js installed from https://code.jquery.com/jquery-3.2.1.js
I: floatThread installed from https://github.com/mkoryak/floatThead/archive/2.0.3.zip
I: select2.js/select2.min.js installed from https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.5/js/select2.min.js
I: select2.js/select2.css installed from https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.5/css/select2.css
I: chartjs/chartjs-plugin-annotation.min.js installed from https://cdnjs.cloudflare.com/ajax/libs/chartjs-plugin-annotation/0.5.7/chartjs-plugin-annotation.min.js
I: skipping architecture conformance tests (snakefood not available)
Creating test database for alias 'default'...
/home/tbird/virtualenvs/mysquad/lib/python3.5/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
System check identified no issues (0 silenced).
....................................................................................................E............................................................................................................................................................................................................................................................................................................................................................................
======================================================================
ERROR: test_lava_log_parsing (test.ci.backend.test_lava.LavaTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tbird/work/fuego/squad/test/ci/backend/test_lava.py", line 691, in test_lava_log_parsing
    log = lava.__parse_log__(log_data)
  File "/home/tbird/work/fuego/squad/squad/ci/backend/lava.py", line 209, in __parse_log__
    for event in yaml.parse(log_data, Loader=yaml.CLoader):
AttributeError: module 'yaml' has no attribute 'CLoader'

----------------------------------------------------------------------
Ran 465 tests in 27.430s

FAILED (errors=1)
Destroying test database for alias 'default'...
I: skipping javascript test (karma not available)

> 
> If you prefer not to use a virtual environment you can install it on the host as
> well
> https://squad.readthedocs.io/en/latest/quickstart.html
> 
> Another easy alternative is to install squad directly from pip3:
> https://squad.readthedocs.io/en/latest/install.html
> $ mkvirtualenv --python=python3.5 pipsquad
> (squadpip) pip3 install squad
> (squadpip) squad

Maybe I'll try one of these.
  -- Tim


^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [Fuego] Support for Squad
  2019-01-15 19:05     ` Tim.Bird
@ 2019-01-15 21:40       ` Tim.Bird
  0 siblings, 0 replies; 12+ messages in thread
From: Tim.Bird @ 2019-01-15 21:40 UTC (permalink / raw)
  To: Tim.Bird, daniel.sangorrin, fuego

> -----Original Message-----
> From: Tim Bird
> 
> >
> > If you prefer not to use a virtual environment you can install it on the host
> as
> > well
> > https://squad.readthedocs.io/en/latest/quickstart.html
> >
> > Another easy alternative is to install squad directly from pip3:
> > https://squad.readthedocs.io/en/latest/install.html
> > $ mkvirtualenv --python=python3.5 pipsquad
> > (squadpip) pip3 install squad
> > (squadpip) squad
> 
> Maybe I'll try one of these.

OK - I got it working, and put some instructions (basically yours with a few edits)
on the following wiki page:
  http://fuegotest.org/wiki/Squad_backend_notes

Now that it's up, I'm not sure what to do with it, but the data appears there.
 -- Tim



^ permalink raw reply	[flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-01-15 21:40 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-10  9:00 [Fuego] Support for Squad Daniel Sangorrin
2019-01-10  9:00 ` [Fuego] [PATCH 2/2] ftc: implement put-run for squad Daniel Sangorrin
2019-01-11  0:23   ` daniel.sangorrin
2019-01-11  1:50   ` Tim.Bird
2019-01-10  9:00 ` [Fuego] [PATCH 2/2] squad: add paramters " Daniel Sangorrin
2019-01-10  9:04   ` daniel.sangorrin
2019-01-11  1:28   ` Tim.Bird
2019-01-11  2:17 ` [Fuego] Support for Squad Tim.Bird
2019-01-11 22:46 ` Tim.Bird
2019-01-15  4:37   ` daniel.sangorrin
2019-01-15 19:05     ` Tim.Bird
2019-01-15 21:40       ` Tim.Bird

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.