[Fuego] [nirrognas][fuego-core][next 1/2] ftc: remove quotes from board and config variables

[Fuego] [nirrognas][fuego-core][next 1/2] ftc: remove quotes from board and config variables

[Daniel Sangorrin]

Although the configuration file is written without quotes,
board files use them to assign values to variables.
For example:
BOARD_CONTROL="ttc"

When we parse a board file, we want to get the value of
those variables as a string but without the quotes.
print(bvars['BOARD_CONTROL'))
 -> ttc instead of "ttc"

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 scripts/ftc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ftc b/scripts/ftc
index b81d511..f4db937 100755
--- a/scripts/ftc
+++ b/scripts/ftc
@@ -1240,7 +1240,7 @@ def parse_shell_file(path, conf):
             continue
 
         (name, value) = line.split('=', 1)
-        value = value.strip()
+        value = value.strip().replace('"', '')
         name = name.strip()
         var_map[name] = value
 
-- 
2.17.1

[Fuego] [nirrognas][fuego-core][next 2/2] power: add ftc power control commands

[Daniel Sangorrin]

This adds support for controlling the power of a board and
it is a step towards realizing a simple board provisioning
subsystem.

Test functions should use shell wrappers on the shared
shell library rather than calling ftc directly because of
modularity (tests should be easy to use in other frameworks)
and for sharing error handling code

[Note] I will add shell wrappers for ftc power-on/off if
a test needs them.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 overlays/base/base-board-test-vars.fuegoclass |  13 +-
 overlays/base/base-board.fuegoclass           |  13 +-
 scripts/ftc                                   | 122 ++++++++++++++++++
 scripts/ftc_completion.sh                     |   3 +
 tests/Functional.fuego_ftc_test/ftc-tests     |  13 +-
 5 files changed, 134 insertions(+), 30 deletions(-)

diff --git a/overlays/base/base-board-test-vars.fuegoclass b/overlays/base/base-board-test-vars.fuegoclass
index f834f47..31fb3b7 100644
--- a/overlays/base/base-board-test-vars.fuegoclass
+++ b/overlays/base/base-board-test-vars.fuegoclass
@@ -196,17 +196,10 @@ function ov_transport_cmd() {
 
 # function to reboot the board
 function ov_board_control_reboot() {
-  if [ -z "$TTC_TARGET" ] ; then
-    TTC_TARGET="$NODE_NAME"
+  ftc power-cycle -b $NODE_NAME
+  if [ $? -ne 0 ]; then
+    abort_job "Error: could not power-cycle the board"
   fi
-  case "$BOARD_CONTROL" in
-  "ttc")
-    $TTC $TTC_TARGET reboot
-    ;;
-  *)
-    abort_job "Error reason: unsupported BOARD_CONTROL ${BOARD_CONTROL}"
-    ;;
-  esac
 }
 
 # the following definitions are used to test variable precedence
diff --git a/overlays/base/base-board.fuegoclass b/overlays/base/base-board.fuegoclass
index ae35464..0d07bec 100644
--- a/overlays/base/base-board.fuegoclass
+++ b/overlays/base/base-board.fuegoclass
@@ -292,15 +292,8 @@ function ov_transport_cmd() {
 
 # function to reboot the board
 function ov_board_control_reboot() {
-  if [ -z "$TTC_TARGET" ] ; then
-    TTC_TARGET="$NODE_NAME"
+  ftc power-cycle -b $NODE_NAME
+  if [ $? -ne 0 ]; then
+    abort_job "Error: could not power-cycle the board"
   fi
-  case "$BOARD_CONTROL" in
-  "ttc")
-    $TTC $TTC_TARGET reboot
-    ;;
-  *)
-    abort_job "Error reason: unsupported BOARD_CONTROL ${BOARD_CONTROL}"
-    ;;
-  esac
 }
diff --git a/scripts/ftc b/scripts/ftc
index f4db937..ae43b4b 100755
--- a/scripts/ftc
+++ b/scripts/ftc
@@ -425,6 +425,33 @@ A message and the exit code indicate if the resource was released.
 
    ex: ftc release-resource -b beaglebone"""),
 
+"power-on": ("Power on the selected board.",
+    """Usage: ftc power-on -b <board1>[,<board2>...]
+
+This command can be used to power on a board. The board file must
+contain a definition for BOARD_CONTROL (e.g., pdudaemon or ttc) and
+the corresponding options (see README.pdu).
+
+Example: ftc power-on -b raspberrypi3"""),
+
+"power-off": ("Power off the selected board.",
+    """Usage: ftc power-off -b <board1>[,<board2>...]
+
+This command can be used to power off a board. The board file must
+contain a definition for BOARD_CONTROL (e.g., pdudaemon or ttc) and
+the corresponding options (see README.pdu).
+
+Example: ftc power-off -b raspberrypi3"""),
+
+"power-cycle": ("Power cycle the selected board.",
+    """Usage: ftc power-off -b <board1>[,<board2>...]
+
+This command can be used to power cycle (reboot) a board. The board file must
+contain a definition for BOARD_CONTROL (e.g., pdudaemon or ttc) and
+the corresponding options (see README.pdu).
+
+Example: ftc power-cycle -b raspberrypi3"""),
+
 "package-test": ("Package a test.",
     """Usage: ftc package-test <test_name> [options]
 
@@ -4951,6 +4978,94 @@ def do_release_resource(conf, options):
 
     return 0
 
+def do_pdudaemon(command, bvars):
+    # pdudaemon hostname
+    if 'PDUDAEMON_HOSTNAME' in bvars:
+        pdudaemon_hostname = bvars['PDUDAEMON_HOSTNAME']
+    else:
+        pdudaemon_hostname = 'localhost'
+
+    # pdudaemon port (optional, default 16421)
+    if 'PDUDAEMON_PORT' in bvars:
+        pdudaemon_port = bvars['PDUDAEMON_PORT']
+    else:
+        pdudaemon_port = '16421'
+
+    # pdu hostname
+    if 'PDU_HOSTNAME' in bvars:
+        pdu_hostname = bvars['PDU_HOSTNAME']
+    else:
+        error_out('PDU_HOSTNAME not specified in board file')
+
+    # pdu port
+    if 'PDU_PORT' in bvars:
+        pdu_port = bvars['PDU_PORT']
+    else:
+        error_out('PDU_PORT not specified in board file')
+
+    # pdu delay
+    if 'PDU_DELAY' in bvars:
+        pdu_delay = bvars['PDU_PORT']
+    else:
+        pdu_delay = '0'
+
+    cmd_map = {
+        'power-on' : 'on',
+        'power-off' : 'off',
+        'power-cycle' : 'reboot'
+    }
+    cmd = cmd_map[command]
+
+    query='&'.join(['hostname='+pdu_hostname, 'port='+pdu_port, 'delay='+pdu_delay])
+    url = 'http://%s:%s/power/control/%s?%s' % (pdudaemon_hostname, pdudaemon_port, cmd, query)
+    try:
+        resp = requests.get(url)
+    except requests.exceptions.ConnectionError:
+        error_out("Could not connect to PDUDaemon on %s:%s" % (
+            pdudaemon_hostname, pdudaemon_port))
+
+    if resp.status_code != 200:
+        error_out("pdudaemon did not accept the request")
+
+    return 0
+
+def do_ttc(command, bvars, board_name):
+    if 'TTC_TARGET' in bvars:
+        ttc_target = bvars['TTC_TARGET']
+    else:
+        ttc_target = board_name
+
+    cmd_map = {
+        'power-on' : 'on',
+        'power-off' : 'off',
+        'power-cycle' : 'reboot'
+    }
+    cmd = cmd_map[command]
+
+    subprocess.check_call('ttc %s %s' % (ttc_target, cmd), shell=True)
+
+    return 0
+
+def do_power_control(conf, options, command):
+    bmap = get_fuego_boards(conf)
+    board_names, options = get_board_arg(command, conf, options, "!m")
+
+    for board_name in board_names:
+        board = bmap[board_name]
+        bvars = get_board_vars(board, conf)
+        if 'BOARD_CONTROL' in bvars:
+            board_control = bvars['BOARD_CONTROL']
+        else:
+            error_out("BOARD_CONTROL is not defined for %s" % board_name)
+
+        if board_control in ["pdudaemon", "PDUDAEMON", "PDUDaemon", "PDUdaemon"]:
+            do_pdudaemon(command, bvars)
+        elif board_control in ["ttc", "TTC", "Ttc"]:
+            do_ttc(command, bvars, board_name)
+        else:
+            error_out("BOARD_CONTROL value %s is not supported" % board_control)
+    sys.exit(0)
+
 def main():
     # use global module names
     global re, time, copy2, subprocess, signal, fcntl, requests, json
@@ -5244,6 +5359,13 @@ def main():
         user_check(conf)
         do_add_view(conf, options)
 
+    if command in ["power-on", "power-off", "power-cycle"]:
+        user_check(conf)
+        try:
+            do_power_control(conf, options, command)
+        except Exception as e:
+            sys.exit(str(e) + '\n' + command_help[command][1])
+
     # all non-board commands have been handled
     #if command in board_mod_commands:
     #   check_reservation(bmap, command)
diff --git a/scripts/ftc_completion.sh b/scripts/ftc_completion.sh
index 984e003..67b1daf 100755
--- a/scripts/ftc_completion.sh
+++ b/scripts/ftc_completion.sh
@@ -55,6 +55,9 @@ _ftc()
         ["list-tests"]="-q"
         ["package-run"]="-o -f"
         ["package-test"]="-o -f"
+        ["power-on"]="-b"
+        ["power-off"]="-b"
+        ["power-cycle"]="-b"
         ["put-request"]="-R -s"
         ["put-run"]=""
         ["put-test"]=""
diff --git a/tests/Functional.fuego_ftc_test/ftc-tests b/tests/Functional.fuego_ftc_test/ftc-tests
index 4d38cbc..540fb7c 100644
--- a/tests/Functional.fuego_ftc_test/ftc-tests
+++ b/tests/Functional.fuego_ftc_test/ftc-tests
@@ -108,17 +108,10 @@ Test board variable manipulation
 	                    ZZFOO : "not here"
 	               board_path : /fuego-ro/boards/ftc-test.board
 	function_ov_board_control_reboot : function ov_board_control_reboot() {
-	                              if [ -z "$TTC_TARGET" ] ; then
-	                                TTC_TARGET="$NODE_NAME"
+	                              ftc power-cycle -b $NODE_NAME
+	                              if [ $? -ne 0 ]; then
+	                                abort_job "Error: could not power-cycle the board"
 	                              fi
-	                              case "$BOARD_CONTROL" in
-	                              "ttc")
-	                                $TTC $TTC_TARGET reboot
-	                                ;;
-	                              *)
-	                                abort_job "Error reason: unsupported BOARD_CONTROL ${BOARD_CONTROL}"
-	                                ;;
-	                              esac
 	                            }
 	                            
 	  function_ov_board_setup : function ov_board_setup () {
-- 
2.17.1

Re: [Fuego] [nirrognas][fuego-core][next 2/2] power: add ftc power control commands

[Tim.Bird]

This was applied to my 'next' branch.

I haven't tested the functionality of the commands, but it doesn't look to have 
broken anything.
 -- Tim


> -----Original Message-----
> From: Daniel Sangorrin
> 
> This adds support for controlling the power of a board and
> it is a step towards realizing a simple board provisioning
> subsystem.
> 
> Test functions should use shell wrappers on the shared
> shell library rather than calling ftc directly because of
> modularity (tests should be easy to use in other frameworks)
> and for sharing error handling code


Agreed.

> 
> [Note] I will add shell wrappers for ftc power-on/off if
> a test needs them.
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  overlays/base/base-board-test-vars.fuegoclass |  13 +-
>  overlays/base/base-board.fuegoclass           |  13 +-
>  scripts/ftc                                   | 122 ++++++++++++++++++
>  scripts/ftc_completion.sh                     |   3 +
>  tests/Functional.fuego_ftc_test/ftc-tests     |  13 +-
>  5 files changed, 134 insertions(+), 30 deletions(-)
> 
> diff --git a/overlays/base/base-board-test-vars.fuegoclass
> b/overlays/base/base-board-test-vars.fuegoclass
> index f834f47..31fb3b7 100644
> --- a/overlays/base/base-board-test-vars.fuegoclass
> +++ b/overlays/base/base-board-test-vars.fuegoclass
> @@ -196,17 +196,10 @@ function ov_transport_cmd() {
> 
>  # function to reboot the board
>  function ov_board_control_reboot() {
> -  if [ -z "$TTC_TARGET" ] ; then
> -    TTC_TARGET="$NODE_NAME"
> +  ftc power-cycle -b $NODE_NAME
> +  if [ $? -ne 0 ]; then
> +    abort_job "Error: could not power-cycle the board"
>    fi
> -  case "$BOARD_CONTROL" in
> -  "ttc")
> -    $TTC $TTC_TARGET reboot
> -    ;;
> -  *)
> -    abort_job "Error reason: unsupported BOARD_CONTROL
> ${BOARD_CONTROL}"
> -    ;;
> -  esac
>  }
> 
>  # the following definitions are used to test variable precedence
> diff --git a/overlays/base/base-board.fuegoclass b/overlays/base/base-
> board.fuegoclass
> index ae35464..0d07bec 100644
> --- a/overlays/base/base-board.fuegoclass
> +++ b/overlays/base/base-board.fuegoclass
> @@ -292,15 +292,8 @@ function ov_transport_cmd() {
> 
>  # function to reboot the board
>  function ov_board_control_reboot() {
> -  if [ -z "$TTC_TARGET" ] ; then
> -    TTC_TARGET="$NODE_NAME"
> +  ftc power-cycle -b $NODE_NAME
> +  if [ $? -ne 0 ]; then
> +    abort_job "Error: could not power-cycle the board"
>    fi
> -  case "$BOARD_CONTROL" in
> -  "ttc")
> -    $TTC $TTC_TARGET reboot
> -    ;;
> -  *)
> -    abort_job "Error reason: unsupported BOARD_CONTROL
> ${BOARD_CONTROL}"
> -    ;;
> -  esac
>  }
> diff --git a/scripts/ftc b/scripts/ftc
> index f4db937..ae43b4b 100755
> --- a/scripts/ftc
> +++ b/scripts/ftc
> @@ -425,6 +425,33 @@ A message and the exit code indicate if the resource
> was released.
> 
>     ex: ftc release-resource -b beaglebone"""),
> 
> +"power-on": ("Power on the selected board.",
> +    """Usage: ftc power-on -b <board1>[,<board2>...]
> +
> +This command can be used to power on a board. The board file must
> +contain a definition for BOARD_CONTROL (e.g., pdudaemon or ttc) and
> +the corresponding options (see README.pdu).
> +
> +Example: ftc power-on -b raspberrypi3"""),
> +
> +"power-off": ("Power off the selected board.",
> +    """Usage: ftc power-off -b <board1>[,<board2>...]
> +
> +This command can be used to power off a board. The board file must
> +contain a definition for BOARD_CONTROL (e.g., pdudaemon or ttc) and
> +the corresponding options (see README.pdu).
> +
> +Example: ftc power-off -b raspberrypi3"""),
> +
> +"power-cycle": ("Power cycle the selected board.",
> +    """Usage: ftc power-off -b <board1>[,<board2>...]
> +
> +This command can be used to power cycle (reboot) a board. The board file
> must
> +contain a definition for BOARD_CONTROL (e.g., pdudaemon or ttc) and
> +the corresponding options (see README.pdu).
> +
> +Example: ftc power-cycle -b raspberrypi3"""),
> +
>  "package-test": ("Package a test.",
>      """Usage: ftc package-test <test_name> [options]
> 
> @@ -4951,6 +4978,94 @@ def do_release_resource(conf, options):
> 
>      return 0
> 
> +def do_pdudaemon(command, bvars):
> +    # pdudaemon hostname
> +    if 'PDUDAEMON_HOSTNAME' in bvars:
> +        pdudaemon_hostname = bvars['PDUDAEMON_HOSTNAME']
> +    else:
> +        pdudaemon_hostname = 'localhost'
> +
> +    # pdudaemon port (optional, default 16421)
> +    if 'PDUDAEMON_PORT' in bvars:
> +        pdudaemon_port = bvars['PDUDAEMON_PORT']
> +    else:
> +        pdudaemon_port = '16421'
> +
> +    # pdu hostname
> +    if 'PDU_HOSTNAME' in bvars:
> +        pdu_hostname = bvars['PDU_HOSTNAME']
> +    else:
> +        error_out('PDU_HOSTNAME not specified in board file')
> +
> +    # pdu port
> +    if 'PDU_PORT' in bvars:
> +        pdu_port = bvars['PDU_PORT']
> +    else:
> +        error_out('PDU_PORT not specified in board file')
> +
> +    # pdu delay
> +    if 'PDU_DELAY' in bvars:
> +        pdu_delay = bvars['PDU_PORT']
> +    else:
> +        pdu_delay = '0'
> +
> +    cmd_map = {
> +        'power-on' : 'on',
> +        'power-off' : 'off',
> +        'power-cycle' : 'reboot'
> +    }
> +    cmd = cmd_map[command]
> +
> +    query='&'.join(['hostname='+pdu_hostname, 'port='+pdu_port,
> 'delay='+pdu_delay])
> +    url = 'http://%s:%s/power/control/%s?%s' % (pdudaemon_hostname,
> pdudaemon_port, cmd, query)
> +    try:
> +        resp = requests.get(url)
> +    except requests.exceptions.ConnectionError:
> +        error_out("Could not connect to PDUDaemon on %s:%s" % (
> +            pdudaemon_hostname, pdudaemon_port))
> +
> +    if resp.status_code != 200:
> +        error_out("pdudaemon did not accept the request")
> +
> +    return 0
> +
> +def do_ttc(command, bvars, board_name):
> +    if 'TTC_TARGET' in bvars:
> +        ttc_target = bvars['TTC_TARGET']
> +    else:
> +        ttc_target = board_name
> +
> +    cmd_map = {
> +        'power-on' : 'on',
> +        'power-off' : 'off',
> +        'power-cycle' : 'reboot'
> +    }
> +    cmd = cmd_map[command]
> +
> +    subprocess.check_call('ttc %s %s' % (ttc_target, cmd), shell=True)
> +
> +    return 0
> +
> +def do_power_control(conf, options, command):
> +    bmap = get_fuego_boards(conf)
> +    board_names, options = get_board_arg(command, conf, options, "!m")
> +
> +    for board_name in board_names:
> +        board = bmap[board_name]
> +        bvars = get_board_vars(board, conf)
> +        if 'BOARD_CONTROL' in bvars:
> +            board_control = bvars['BOARD_CONTROL']
> +        else:
> +            error_out("BOARD_CONTROL is not defined for %s" % board_name)
> +
> +        if board_control in ["pdudaemon", "PDUDAEMON", "PDUDaemon",
> "PDUdaemon"]:
> +            do_pdudaemon(command, bvars)
> +        elif board_control in ["ttc", "TTC", "Ttc"]:
> +            do_ttc(command, bvars, board_name)
> +        else:
> +            error_out("BOARD_CONTROL value %s is not supported" %
> board_control)
> +    sys.exit(0)
> +
>  def main():
>      # use global module names
>      global re, time, copy2, subprocess, signal, fcntl, requests, json
> @@ -5244,6 +5359,13 @@ def main():
>          user_check(conf)
>          do_add_view(conf, options)
> 
> +    if command in ["power-on", "power-off", "power-cycle"]:
> +        user_check(conf)
> +        try:
> +            do_power_control(conf, options, command)
> +        except Exception as e:
> +            sys.exit(str(e) + '\n' + command_help[command][1])
> +
>      # all non-board commands have been handled
>      #if command in board_mod_commands:
>      #   check_reservation(bmap, command)
> diff --git a/scripts/ftc_completion.sh b/scripts/ftc_completion.sh
> index 984e003..67b1daf 100755
> --- a/scripts/ftc_completion.sh
> +++ b/scripts/ftc_completion.sh
> @@ -55,6 +55,9 @@ _ftc()
>          ["list-tests"]="-q"
>          ["package-run"]="-o -f"
>          ["package-test"]="-o -f"
> +        ["power-on"]="-b"
> +        ["power-off"]="-b"
> +        ["power-cycle"]="-b"
>          ["put-request"]="-R -s"
>          ["put-run"]=""
>          ["put-test"]=""
> diff --git a/tests/Functional.fuego_ftc_test/ftc-tests
> b/tests/Functional.fuego_ftc_test/ftc-tests
> index 4d38cbc..540fb7c 100644
> --- a/tests/Functional.fuego_ftc_test/ftc-tests
> +++ b/tests/Functional.fuego_ftc_test/ftc-tests
> @@ -108,17 +108,10 @@ Test board variable manipulation
>  	                    ZZFOO : "not here"
>  	               board_path : /fuego-ro/boards/ftc-test.board
>  	function_ov_board_control_reboot : function
> ov_board_control_reboot() {
> -	                              if [ -z "$TTC_TARGET" ] ; then
> -	                                TTC_TARGET="$NODE_NAME"
> +	                              ftc power-cycle -b $NODE_NAME
> +	                              if [ $? -ne 0 ]; then
> +	                                abort_job "Error: could not power-cycle the board"
>  	                              fi
> -	                              case "$BOARD_CONTROL" in
> -	                              "ttc")
> -	                                $TTC $TTC_TARGET reboot
> -	                                ;;
> -	                              *)
> -	                                abort_job "Error reason: unsupported
> BOARD_CONTROL ${BOARD_CONTROL}"
> -	                                ;;
> -	                              esac
>  	                            }
> 
>  	  function_ov_board_setup : function ov_board_setup () {
> --
> 2.17.1
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [nirrognas][fuego-core][next 1/2] ftc: remove quotes from board and config variables

[Tim.Bird]

> -----Original Message-----
> From: Daniel Sangorrin
> 
> Although the configuration file is written without quotes,
> board files use them to assign values to variables.
> For example:
> BOARD_CONTROL="ttc"
> 
> When we parse a board file, we want to get the value of
> those variables as a string but without the quotes.
> print(bvars['BOARD_CONTROL'))
>  -> ttc instead of "ttc"
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
>  scripts/ftc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/ftc b/scripts/ftc
> index b81d511..f4db937 100755
> --- a/scripts/ftc
> +++ b/scripts/ftc
> @@ -1240,7 +1240,7 @@ def parse_shell_file(path, conf):
>              continue
> 
>          (name, value) = line.split('=', 1)
> -        value = value.strip()
> +        value = value.strip().replace('"', '')

I applied your patch, but it made me a bit uneasy, because it could theoretically
replace quotes other than those at the beginning and end of the string.

 I made the check for leading and trailing quotes
more explicit, and only remove them when present, and when they are paired.

>          name = name.strip()
>          var_map[name] = value


Here's the patch I came up with:
commit a1923c9c5a159cd9219748e4ba0c2dc5210b8d0f
Author: Tim Bird <tim.bird@sony.com>
Date:   Wed Jul 17 00:26:50 2019 -0700

    core: remove quotes using more strict method
    
    Change the way we remove quotes from a value in parse_shell_file()
    to only remove quotes at the beginning and end of the value,
    rather than anywhere in the value.  If a value had nested quotes inside,
    the previous method would incorrectly replace them.
    
    We only want to strip outside quotes.
    
    Signed-off-by: Tim Bird <tim.bird@sony.com>

diff --git a/scripts/ftc b/scripts/ftc
index 01c8e21..1588593 100755
--- a/scripts/ftc
+++ b/scripts/ftc
@@ -1267,7 +1267,9 @@ def parse_shell_file(path, conf):
             continue
 
         (name, value) = line.split('=', 1)
-        value = value.strip().replace('"', '')
+        value = value.strip()
+        if value.startswith('"') and value.endswith('"'):
+            value = value[1:-1]
         name = name.strip()
         var_map[name] = value
---

This is the fuegotest next branch.  Can you test it out and let me know if it works
with the data you were seeing a problem with?

Thanks,
 -- Tim