[Fuego] ftc: merge create-delete node scripts

[Fuego] ftc: merge create-delete node scripts

[Daniel Sangorrin]

Hi,

These two patches merge my fuego-create/delete-node scripts into
ftc. I have used python-jenkins (installed with pip) for the 
implementation.

You can also find them on my tree.

Thanks,
Daniel

[Fuego] [PATCH] ftc: implement add-nodes and rm-nodes using python-jenkins

[Daniel Sangorrin]

Note that you can now add/remove multiple nodes at the same
time, or remove all nodes at once. This was not available
in the fuego-create-node and fuego-delete-node scripts.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/scripts/ftc | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)

diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index e064016..3a09dbe 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -55,6 +55,7 @@ import getopt
 import shutil
 import tempfile
 import yaml
+import jenkins
 
 # MAJOR, MINOR, REVISION
 VERSION = (1,0,9)
@@ -71,6 +72,7 @@ def pvar(name):
 quiet = 0
 verbose = 0
 use_statusouput = 1
+server = jenkins.Jenkins('http://localhost:8080/fuego')
 
 # keep configuration file in /fuego-ro/conf area
 config_dir = "/fuego-ro/conf"
@@ -82,6 +84,20 @@ TARGET_ENV_VAR="FTC_TARGET"
 
 # format for command_help mapping with: key=name, value=(summary, long description)
 command_help = {
+"rm-nodes":("Removes nodes from Jenkins.",
+    """Usage: ftc rm-nodes [<target1> <target2> ...]
+  Use list-nodes to see the existing nodes.
+
+  If no target is provided all existing nodes will be removed."""),
+
+"add-nodes":("Adds new nodes to Jenkins.",
+    """Usage: ftc add-nodes [-f] <target1> <target2> ...
+  By convention the name of the node and target board is the same. Also,
+  the corresponding board file (<target>.board) must exist. Use list-targets
+  to see available target boards.
+
+  Use -f for "force" mode. This tries to remove the node before adding it"""),
+
 "list-targets":("Show a list of available target boards.",
     """Usage: ftc list-targets [-q]
   Prints target board names and summary information, if any.
@@ -831,6 +847,62 @@ def get_includes(include_filename, conf):
 	inc_vars = parse_shell_file(inc_path, conf)
 	return inc_vars
 
+def do_add_nodes(conf, options):
+	global server
+
+	if '-f' in options:
+		force = True
+		options.remove('-f')
+	else:
+		force = False
+
+	target_list = get_fuego_targets(conf).keys()
+	for board in options:
+		if board not in target_list:
+			raise Exception('No %s.board found.' % board)
+
+	params = { 'command' : 'java -jar /fuego-core/engine/slave.jar'}
+	for board in options:
+		nodes = [node['name'] for node in server.get_nodes()]
+		if board in nodes:
+			if force:
+				server.delete_node(board)
+			else:
+				raise Exception('Node \'%s\' already exists' % board)
+
+		server.create_node(
+			board,
+			numExecutors = 1,
+			launcher = jenkins.LAUNCHER_COMMAND,
+			launcher_params = params
+		)
+		# bug?: it seems enable_node requires two calls
+		server.enable_node(board)
+		server.enable_node(board)
+	sys.exit(0)
+
+def do_rm_nodes(conf, options):
+	global server
+
+	nodes = [node['name'] for node in server.get_nodes()]
+	if not options:
+		options = nodes
+	else:
+		for board in options:
+			if board not in nodes:
+				raise Exception('Node \'%s\' not found.' % board)
+
+	for board in options:
+		nodes = [node['name'] for node in server.get_nodes()]
+		if board not in nodes:
+			# in case the same node is repeated in options
+			continue
+		if board == 'master':
+			# the master node is special in jenkins
+			continue
+		server.delete_node(board)
+	sys.exit(0)
+
 def do_list_nodes(conf):
 	global quiet, verbose
 
-- 
2.7.4

[Fuego] [PATCH] ftc: merge node creation and removal

[Daniel Sangorrin]

Node creation and removal were in separate scripts on
Toshiba's fork. Merge that functionality using the
python-jenkins library.

Although Jessie has a python-jenkins package, it was a bit
to old so I installed a newer version with pip.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 Dockerfile                               |  3 +-
 README                                   |  2 +-
 fuego-ro/scripts/nodes/fuego-create-node | 56 --------------------------------
 fuego-ro/scripts/nodes/fuego-delete-node | 36 --------------------
 4 files changed, 3 insertions(+), 94 deletions(-)
 delete mode 100755 fuego-ro/scripts/nodes/fuego-create-node
 delete mode 100755 fuego-ro/scripts/nodes/fuego-delete-node

diff --git a/Dockerfile b/Dockerfile
index a807cae..f0827ca 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -28,7 +28,8 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install \
 	libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf \
 	netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl \
 	inotify-tools g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev \
-	time
+	time python-pip
+RUN pip install python-jenkins==0.4.14
 RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash'
 RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
 
diff --git a/README b/README
index da89e7c..329da20 100644
--- a/README
+++ b/README
@@ -79,7 +79,7 @@ Initial test
 You can use the docker container as a fuego target in order to confirm that
 the installation was successful.
 
-# fuego-create-node --board docker
+# sudo -u jenkins ftc add-nodes docker
 # fuego-create-jobs --board docker --testplan testplan_docker --distrib nosyslogd.dist
 
 Build the job "docker.testplan_docker.batch", it will trigger the rest of
diff --git a/fuego-ro/scripts/nodes/fuego-create-node b/fuego-ro/scripts/nodes/fuego-create-node
deleted file mode 100755
index 6ca26c6..0000000
--- a/fuego-ro/scripts/nodes/fuego-create-node
+++ /dev/null
@@ -1,56 +0,0 @@
-#!/usr/bin/python
-#
-# fuego-create-node - a tool to create a jenkins node
-#
-# Copyright 2016 TOSHIBA Corporation
-#
-#   This program is free software; you can redistribute it and/or modify
-#   it under the terms of version 2 of the GNU General Public License as
-#   published by the Free Software Foundation.  The GNU General Public
-#   License is available online at: http://www.gnu.org/copyleft/gpl.html
-#
-#   This program is distributed in the hope that it will be useful,
-#   but WITHOUT ANY WARRANTY; without even the implied warranty of
-#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#   GNU General Public License for more details.
-#
-# Author: Daniel Sangorrin <daniel.sangorrin (at) toshiba.co.jp>
-#
-import subprocess
-import sys
-import argparse
-
-def create_node(board):
-    tmp = "/tmp/fuego_tmp_node"
-    fd = open(tmp, "w+")
-    fd.write("""<?xml version='1.0' encoding='UTF-8'?>
-<slave>
-    <name>{board}</name>
-    <description></description>
-    <remoteFS>/tmp/dev-slave1</remoteFS>
-    <numExecutors>1</numExecutors>
-    <mode>NORMAL</mode>
-    <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>
-    <launcher class="hudson.slaves.CommandLauncher">
-    <agentCommand>java -jar /fuego-core/engine/slave.jar</agentCommand>
-    </launcher>
-    <label></label>
-    <nodeProperties/>
-</slave>
-""".format(board=board))
-    fd.close()
-
-    print("Creating node " + board)
-    try:
-        subprocess.call('java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego create-node ' + board + ' < ' + tmp, shell=True)
-        subprocess.call('rm -f ' + tmp, shell=True)
-    except Exception as e:
-        print("Node already exists")
-        print(e)
-        sys.exit(1)
-
-if __name__=="__main__":
-    parser = argparse.ArgumentParser(description='Create a jenkins node')
-    parser.add_argument('-b', '--board', type=str, help='board name (e.g.: bbb for bbb.board)', required=True)
-    args = parser.parse_args()
-    create_node(args.board)
diff --git a/fuego-ro/scripts/nodes/fuego-delete-node b/fuego-ro/scripts/nodes/fuego-delete-node
deleted file mode 100755
index a4bd7a0..0000000
--- a/fuego-ro/scripts/nodes/fuego-delete-node
+++ /dev/null
@@ -1,36 +0,0 @@
-#!/usr/bin/python
-#
-# fuego-delete-node - a tool to delete a jenkins node
-#
-# Copyright 2016 TOSHIBA Corporation
-#
-#   This program is free software; you can redistribute it and/or modify
-#   it under the terms of version 2 of the GNU General Public License as
-#   published by the Free Software Foundation.  The GNU General Public
-#   License is available online at: http://www.gnu.org/copyleft/gpl.html
-#
-#   This program is distributed in the hope that it will be useful,
-#   but WITHOUT ANY WARRANTY; without even the implied warranty of
-#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#   GNU General Public License for more details.
-#
-# Author: Daniel Sangorrin <daniel.sangorrin (at) toshiba.co.jp>
-#
-import subprocess
-import sys
-import argparse
-
-def delete_node(board):
-    print("Deleting node " + board)
-    try:
-        subprocess.call('java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego delete-node ' + board, shell=True)
-    except Exception as e:
-        print("Node didn't exist")
-        print(e)
-        sys.exit(1)
-
-if __name__=="__main__":
-    parser = argparse.ArgumentParser(description='Delete a jenkins node')
-    parser.add_argument('-b', '--board', type=str, help='board name (e.g.: bbb for bbb.board)', required=True)
-    args = parser.parse_args()
-    delete_node(args.board)
-- 
2.7.4