[PATCH 0/7] tools: More Python 3 fixes (part 1 of N)

[PATCH 0/7] tools: More Python 3 fixes (part 1 of N)

[Andrew Cooper]

Despite previous statements to the contrary, Xen still does not support Python 3.

Various notes are on https://gitlab.com/xen-project/xen/-/issues/114

The following scripts are installed, and given a python3 shebang, but are not
Py3 compatible:

  xencov_split
  xentrace_format
  xencons
  xenpvnetboot
  verify-stream-v2
  convert-legacy-stream

Most of these want converting, and should be reasonably easy, but I don't have
time right now.

This series deals with xencons (which is a not-invented-here telnet), various
bits of cleanup, and various fixes for shebangs.

The problematic remaining script is xenpvnetboot which, while looking
reasonably useful and also not PV specific, depends on a 3rd party module but
provides no dependency information at all.

Relatedly, and a critical/blocker for Xen 4.18.  Python 3.12, which will be
released ahead of the Xen 4.18 release, is deleteing distutils (recommending
setuputils as the replacement), so someone is going to have to rework all our
setup.py's to be compatible.

Andrew Cooper (7):
  tools/python: Drop pylintrc
  tools/misc: Drop xencons
  tools: Delete trailing whitespace in python scripts
  tools/pygrub: Factor out common setup.py parts
  tools: Use -s for python shebangs
  tools/python: Improve unit test handling
  tools/python: Drop shebangs from library files

 tools/Rules.mk                       |   2 +-
 tools/misc/Makefile                  |   2 -
 tools/misc/xencons                   |  92 --------
 tools/misc/xencov_split              |   1 -
 tools/pygrub/Makefile                |  11 +-
 tools/pygrub/src/ExtLinuxConf.py     |   2 +-
 tools/pygrub/src/GrubConf.py         |  30 +--
 tools/pygrub/src/fsimage/fsimage.c   |   8 +-
 tools/pygrub/src/pygrub              |  44 ++--
 tools/python/Makefile                |   4 +-
 tools/python/pylintrc                | 307 ---------------------------
 tools/python/xen/lowlevel/xc/xc.c    | 200 ++++++++---------
 tools/python/xen/lowlevel/xs/xs.c    |   4 +-
 tools/python/xen/migration/legacy.py |   1 -
 tools/python/xen/migration/libxc.py  |   1 -
 tools/python/xen/migration/libxl.py  |   1 -
 tools/python/xen/migration/public.py |   1 -
 tools/python/xen/migration/tests.py  |  15 +-
 tools/python/xen/migration/verify.py |   1 -
 tools/python/xen/migration/xl.py     |   1 -
 tools/python/xen/util.py             |   1 -
 tools/xenmon/xenmon.py               |  44 ++--
 22 files changed, 177 insertions(+), 596 deletions(-)
 delete mode 100755 tools/misc/xencons
 delete mode 100644 tools/python/pylintrc

-- 
2.30.2

[PATCH 1/7] tools/python: Drop pylintrc

[Andrew Cooper]

This was added in 2004 in c/s b7d4a69f0ccb5 and has never been referenced
since.  Given the the commit message of simply "Added .", it was quite
possibly a mistake in the first place.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/python/pylintrc | 307 ------------------------------------------
 1 file changed, 307 deletions(-)
 delete mode 100644 tools/python/pylintrc

diff --git a/tools/python/pylintrc b/tools/python/pylintrc
deleted file mode 100644
index efc4b0b3b2dd..000000000000
--- a/tools/python/pylintrc
+++ /dev/null
@@ -1,307 +0,0 @@
-# lint Python modules using external checkers.                            
-#                                                                                
-#     This is the main checker controling the other ones and the reports         
-#     generation. It is itself both a raw checker and an astng checker in order  
-#     to:                                                                        
-#     * handle message activation / deactivation at the module level             
-#     * handle some basic but necessary stats'data (number of classes, methods...)
-#                                                                                 
-# This checker also defines the following reports:                                    
-#   * R0001: Total errors / warnings                                              
-#   * R0002: % errors / warnings by module                                        
-#   * R0003: Messages                                                             
-#   * R0004: Global evaluation                                                    
-# 
-[MASTER]
-# Add <file or directory> to the black list. It should be a base name, not a
-# path. You may set this option multiple times.
-ignore=CVS
-
-# Pickle collected data for later comparisons.
-persistent=yes
-
-# Set the cache size for astng objects.
-cache-size=500
-
-
-
-[REPORTS]
-# Tells wether to display a full report or only the messages
-reports=yes
-
-# Use HTML as output format instead of text
-html=no
-
-# Use a parseable text output format, so your favorite text editor will be able
-# to jump to the line corresponding to a message.
-parseable=no
-
-# Colorizes text output using ansi escape codes
-color=no
-
-# Put messages in a separate file for each module / package specified on the
-# command line instead of printing them on stdout. Reports (if any) will be
-# written in a file name "pylint_global.[txt|html]".
-files-output=no
-
-# Python expression which should return a note less than 10 (10 is the highest
-# note).You have access to the variables errors warning, statement which
-# respectivly contain the number of errors / warnings messages and the total
-# number of statements analyzed. This is used by the global evaluation report
-# (R0004).
-evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
-
-# Add a comment according to your evaluation note. This is used by the global
-# evaluation report (R0004).
-comment=no
-
-# Include message's id in output
-include-ids=yes
-
-
-
-# checks for                                                              
-#     * unused variables / imports                                               
-#     * undefined variables                                                      
-#     * redefinition of variable from builtins or from an outer scope            
-#     * use of variable before assigment                                         
-#     
-[VARIABLES]
-# Enable / disable this checker
-enable-variables=yes
-
-# Tells wether we should check for unused import in __init__ files.
-init-import=no
-
-# List of variable names used for dummy variables (i.e. not used).
-dummy-variables=_,_1,_2,_3,_4,_5,dummy
-
-
-
-# checks for :                                                            
-#     * doc strings                                                              
-#     * modules / classes / functions / methods / arguments / variables name     
-#     * number of arguments, local variables, branchs, returns and statements in
-# functions, methods                                                       
-#     * required module attributes                                             
-#     * dangerous default values as arguments                                    
-#     * redefinition of function / method / class                                
-#     * uses of the global statement                                             
-#                                                                                 
-# This checker also defines the following reports:                                    
-#   * R0101: Statistics by type                                                   
-# 
-[BASIC]
-# Enable / disable this checker
-enable-basic=yes
-
-# Required attributes for module, separated by a comma
-required-attributes=
-
-# Regular expression which should only match functions or classes name which do
-# not require a docstring
-no-docstring-rgx=.*
-
-# Minimal length for module / class / function / method / argument / variable
-# names
-min-name-length=1
-
-# Regular expression which should only match correct module names
-module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
-
-# Regular expression which should only match correct class names
-class-rgx=[A-Z_][a-zA-Z0-9]+$
-
-# Regular expression which should only match correct function names
-function-rgx=[a-z_][A-Za-z0-9_]*$
-
-# Regular expression which should only match correct method names
-method-rgx=[a-z_][A-Za-z0-9_]*$
-
-# Regular expression which should only match correct argument names
-argument-rgx=[a-z_][A-Za-z0-9_]*$
-
-# Regular expression which should only match correct variable names
-variable-rgx=[a-z_][A-Za-z0-9_]*$
-
-# Good variable names which should always be accepted, separated by a comma
-good-names=i,j,k,ex,Run,_
-
-# Bad variable names which should always be refused, separated by a comma
-bad-names=foo,bar,baz,toto,tutu,tata
-
-# List of builtins function names that should not be used, separated by a comma
-bad-functions=apply,input
-
-
-
-# checks for sign of poor/misdesign:                                      
-#     * number of methods, attributes, local variables...                        
-#     * size, complexity of functions, methods                                   
-#     
-[DESIGN]
-# Enable / disable this checker
-enable-design=yes
-
-# Maximum number of arguments for function / method
-max-args=15
-
-# Maximum number of locals for function / method body
-max-locals=15
-
-# Maximum number of return / yield for function / method body
-max-returns=6
-
-# Maximum number of branch for function / method body
-max-branchs=12
-
-# Maximum number of statements in function / method body
-max-statements=50
-
-# Maximum number of parents for a class (see R0901).
-max-parents=7
-
-# Maximum number of attributes for a class (see R0902).
-max-attributes=7
-
-# Minimum number of public methods for a class (see R0903).
-min-public-methods=2
-
-# Maximum number of public methods for a class (see R0904).
-max-public-methods=20
-
-
-
-# checks for :                                                            
-#     * methods without self as first argument                                   
-#     * overriden methods signature                                              
-#     * access only to existant members via self                                 
-#     * attributes not defined in the __init__ method                            
-#     * supported interfaces implementation                                      
-#     * unreachable code                                                         
-#     
-[CLASSES]
-# Enable / disable this checker
-enable-classes=yes
-
-# List of interface methods to ignore, separated by a comma. This is used for
-# instance to not check methods defines in Zope's Interface base class.
-ignore-iface-methods=isImplementedBy,deferred,extends,names,namesAndDescriptions,queryDescriptionFor,getBases,getDescriptionFor,getDoc,getName,getTaggedValue,getTaggedValueTags,isEqualOrExtendedBy,setTaggedValue,isImplementedByInstancesOf,adaptWith,is_implemented_by
-
-# Tells wether missing members accessed in mixin class should be ignored. A
-# mixin class is detected if its name ends with "mixin" (case insensitive).
-ignore-mixin-members=yes
-
-
-
-# checks for                                                              
-#     * external modules dependencies                                            
-#     * relative / wildcard imports                                                         
-#     * cyclic imports                                                           
-#     * uses of deprecated modules
-#                                                                                 
-# This checker also defines the following reports:                                    
-#   * R0401: External dependencies                                                
-#   * R0402: Modules dependencies graph                                           
-# 
-[IMPORTS]
-# Enable / disable this checker
-enable-imports=no
-
-# Deprecated modules which should not be used, separated by a comma
-deprecated-modules=regsub,string,TERMIOS,Bastion,rexec
-
-# Create a graph of every (i.e. internal and external) dependencies in the given
-# file (report R0402 must not be disabled)
-import-graph=
-
-# Create a graph of external dependencies in the given file (report R0402 must
-# not be disabled)
-ext-import-graph=
-
-# Create a graph of internal dependencies in the given file (report R0402 must
-# not be disabled)
-int-import-graph=
-
-
-
-# checks for                                                              
-#     * excepts without exception filter                                         
-#     * string exceptions                                                        
-#     
-[EXCEPTIONS]
-# Enable / disable this checker
-enable-exceptions=yes
-
-
-
-# checks for :                                                            
-#     * unauthorized constructions                                               
-#     * strict indentation                                                       
-#     * line length                                                              
-#     * use of <> instead of !=
-#     
-[FORMAT]
-# Enable / disable this checker
-enable-format=no
-
-# Maximum number of characters on a single line.
-max-line-length=80
-
-# Maximum number of lines in a module
-max-module-lines=1000
-
-# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1 tab).
-indent-string='    '
-
-
-
-# does not check anything but gives some raw metrics :                    
-#     * total number of lines                                                    
-#     * total number of code lines                                               
-#     * total number of docstring lines                                          
-#     * total number of comments lines                                           
-#     * total number of empty lines                                              
-#                                                                                 
-# This checker also defines the following reports:                                    
-#   * R0701: Raw metrics                                                          
-# 
-[METRICS]
-# Enable / disable this checker
-enable-metrics=yes
-
-
-
-# checks for:                                                             
-#     * warning notes in the code like FIXME, XXX                                
-#     * PEP 263: source code with non ascii character but no encoding declaration
-#     
-[MISCELLANEOUS]
-# Enable / disable this checker
-enable-miscellaneous=yes
-
-# List of note tags to take in consideration, separated by a comma. Default to
-# FIXME, XXX, TODO
-notes=FIXME,XXX,TODO
-
-
-
-# checks for similarities and duplicated code. This computation may be
-#     memory / CPU intensive, so you should disable it if you experiments some
-#     problems.
-#                                                                                 
-# This checker also defines the following reports:                                    
-#   * R0801: Duplication                                                          
-# 
-[SIMILARITIES]
-# Enable / disable this checker
-enable-similarities=yes
-
-# Minimum lines number of a similarity.
-min-similarity-lines=4
-
-# Ignore comments when computing similarities.
-ignore-comments=yes
-
-
-
-- 
2.30.2

Re: [PATCH 1/7] tools/python: Drop pylintrc

[Anthony PERARD]

On Tue, Mar 14, 2023 at 02:15:14PM +0000, Andrew Cooper wrote:
> This was added in 2004 in c/s b7d4a69f0ccb5 and has never been referenced
> since.  Given the the commit message of simply "Added .", it was quite
> possibly a mistake in the first place.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Well, "pylintrc" seems to be a config file to be used by `pylint`, but
there's also no reference to it. So removing the file seems a good
option.

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

-- 
Anthony PERARD

Re: [PATCH 1/7] tools/python: Drop pylintrc

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 14185 bytes --]

On Tue, Mar 14, 2023 at 02:15:14PM +0000, Andrew Cooper wrote:
> This was added in 2004 in c/s b7d4a69f0ccb5 and has never been referenced
> since.  Given the the commit message of simply "Added .", it was quite
> possibly a mistake in the first place.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Ideally, we would have a pylint integrated into CI, but given frequency
of changes to that code, I don't think its worth it, so:

Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

> ---
> CC: Wei Liu <wl@xen.org>
> CC: Anthony PERARD <anthony.perard@citrix.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
> ---
>  tools/python/pylintrc | 307 ------------------------------------------
>  1 file changed, 307 deletions(-)
>  delete mode 100644 tools/python/pylintrc
> 
> diff --git a/tools/python/pylintrc b/tools/python/pylintrc
> deleted file mode 100644
> index efc4b0b3b2dd..000000000000
> --- a/tools/python/pylintrc
> +++ /dev/null
> @@ -1,307 +0,0 @@
> -# lint Python modules using external checkers.                            
> -#                                                                                
> -#     This is the main checker controling the other ones and the reports         
> -#     generation. It is itself both a raw checker and an astng checker in order  
> -#     to:                                                                        
> -#     * handle message activation / deactivation at the module level             
> -#     * handle some basic but necessary stats'data (number of classes, methods...)
> -#                                                                                 
> -# This checker also defines the following reports:                                    
> -#   * R0001: Total errors / warnings                                              
> -#   * R0002: % errors / warnings by module                                        
> -#   * R0003: Messages                                                             
> -#   * R0004: Global evaluation                                                    
> -# 
> -[MASTER]
> -# Add <file or directory> to the black list. It should be a base name, not a
> -# path. You may set this option multiple times.
> -ignore=CVS
> -
> -# Pickle collected data for later comparisons.
> -persistent=yes
> -
> -# Set the cache size for astng objects.
> -cache-size=500
> -
> -
> -
> -[REPORTS]
> -# Tells wether to display a full report or only the messages
> -reports=yes
> -
> -# Use HTML as output format instead of text
> -html=no
> -
> -# Use a parseable text output format, so your favorite text editor will be able
> -# to jump to the line corresponding to a message.
> -parseable=no
> -
> -# Colorizes text output using ansi escape codes
> -color=no
> -
> -# Put messages in a separate file for each module / package specified on the
> -# command line instead of printing them on stdout. Reports (if any) will be
> -# written in a file name "pylint_global.[txt|html]".
> -files-output=no
> -
> -# Python expression which should return a note less than 10 (10 is the highest
> -# note).You have access to the variables errors warning, statement which
> -# respectivly contain the number of errors / warnings messages and the total
> -# number of statements analyzed. This is used by the global evaluation report
> -# (R0004).
> -evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
> -
> -# Add a comment according to your evaluation note. This is used by the global
> -# evaluation report (R0004).
> -comment=no
> -
> -# Include message's id in output
> -include-ids=yes
> -
> -
> -
> -# checks for                                                              
> -#     * unused variables / imports                                               
> -#     * undefined variables                                                      
> -#     * redefinition of variable from builtins or from an outer scope            
> -#     * use of variable before assigment                                         
> -#     
> -[VARIABLES]
> -# Enable / disable this checker
> -enable-variables=yes
> -
> -# Tells wether we should check for unused import in __init__ files.
> -init-import=no
> -
> -# List of variable names used for dummy variables (i.e. not used).
> -dummy-variables=_,_1,_2,_3,_4,_5,dummy
> -
> -
> -
> -# checks for :                                                            
> -#     * doc strings                                                              
> -#     * modules / classes / functions / methods / arguments / variables name     
> -#     * number of arguments, local variables, branchs, returns and statements in
> -# functions, methods                                                       
> -#     * required module attributes                                             
> -#     * dangerous default values as arguments                                    
> -#     * redefinition of function / method / class                                
> -#     * uses of the global statement                                             
> -#                                                                                 
> -# This checker also defines the following reports:                                    
> -#   * R0101: Statistics by type                                                   
> -# 
> -[BASIC]
> -# Enable / disable this checker
> -enable-basic=yes
> -
> -# Required attributes for module, separated by a comma
> -required-attributes=
> -
> -# Regular expression which should only match functions or classes name which do
> -# not require a docstring
> -no-docstring-rgx=.*
> -
> -# Minimal length for module / class / function / method / argument / variable
> -# names
> -min-name-length=1
> -
> -# Regular expression which should only match correct module names
> -module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
> -
> -# Regular expression which should only match correct class names
> -class-rgx=[A-Z_][a-zA-Z0-9]+$
> -
> -# Regular expression which should only match correct function names
> -function-rgx=[a-z_][A-Za-z0-9_]*$
> -
> -# Regular expression which should only match correct method names
> -method-rgx=[a-z_][A-Za-z0-9_]*$
> -
> -# Regular expression which should only match correct argument names
> -argument-rgx=[a-z_][A-Za-z0-9_]*$
> -
> -# Regular expression which should only match correct variable names
> -variable-rgx=[a-z_][A-Za-z0-9_]*$
> -
> -# Good variable names which should always be accepted, separated by a comma
> -good-names=i,j,k,ex,Run,_
> -
> -# Bad variable names which should always be refused, separated by a comma
> -bad-names=foo,bar,baz,toto,tutu,tata
> -
> -# List of builtins function names that should not be used, separated by a comma
> -bad-functions=apply,input
> -
> -
> -
> -# checks for sign of poor/misdesign:                                      
> -#     * number of methods, attributes, local variables...                        
> -#     * size, complexity of functions, methods                                   
> -#     
> -[DESIGN]
> -# Enable / disable this checker
> -enable-design=yes
> -
> -# Maximum number of arguments for function / method
> -max-args=15
> -
> -# Maximum number of locals for function / method body
> -max-locals=15
> -
> -# Maximum number of return / yield for function / method body
> -max-returns=6
> -
> -# Maximum number of branch for function / method body
> -max-branchs=12
> -
> -# Maximum number of statements in function / method body
> -max-statements=50
> -
> -# Maximum number of parents for a class (see R0901).
> -max-parents=7
> -
> -# Maximum number of attributes for a class (see R0902).
> -max-attributes=7
> -
> -# Minimum number of public methods for a class (see R0903).
> -min-public-methods=2
> -
> -# Maximum number of public methods for a class (see R0904).
> -max-public-methods=20
> -
> -
> -
> -# checks for :                                                            
> -#     * methods without self as first argument                                   
> -#     * overriden methods signature                                              
> -#     * access only to existant members via self                                 
> -#     * attributes not defined in the __init__ method                            
> -#     * supported interfaces implementation                                      
> -#     * unreachable code                                                         
> -#     
> -[CLASSES]
> -# Enable / disable this checker
> -enable-classes=yes
> -
> -# List of interface methods to ignore, separated by a comma. This is used for
> -# instance to not check methods defines in Zope's Interface base class.
> -ignore-iface-methods=isImplementedBy,deferred,extends,names,namesAndDescriptions,queryDescriptionFor,getBases,getDescriptionFor,getDoc,getName,getTaggedValue,getTaggedValueTags,isEqualOrExtendedBy,setTaggedValue,isImplementedByInstancesOf,adaptWith,is_implemented_by
> -
> -# Tells wether missing members accessed in mixin class should be ignored. A
> -# mixin class is detected if its name ends with "mixin" (case insensitive).
> -ignore-mixin-members=yes
> -
> -
> -
> -# checks for                                                              
> -#     * external modules dependencies                                            
> -#     * relative / wildcard imports                                                         
> -#     * cyclic imports                                                           
> -#     * uses of deprecated modules
> -#                                                                                 
> -# This checker also defines the following reports:                                    
> -#   * R0401: External dependencies                                                
> -#   * R0402: Modules dependencies graph                                           
> -# 
> -[IMPORTS]
> -# Enable / disable this checker
> -enable-imports=no
> -
> -# Deprecated modules which should not be used, separated by a comma
> -deprecated-modules=regsub,string,TERMIOS,Bastion,rexec
> -
> -# Create a graph of every (i.e. internal and external) dependencies in the given
> -# file (report R0402 must not be disabled)
> -import-graph=
> -
> -# Create a graph of external dependencies in the given file (report R0402 must
> -# not be disabled)
> -ext-import-graph=
> -
> -# Create a graph of internal dependencies in the given file (report R0402 must
> -# not be disabled)
> -int-import-graph=
> -
> -
> -
> -# checks for                                                              
> -#     * excepts without exception filter                                         
> -#     * string exceptions                                                        
> -#     
> -[EXCEPTIONS]
> -# Enable / disable this checker
> -enable-exceptions=yes
> -
> -
> -
> -# checks for :                                                            
> -#     * unauthorized constructions                                               
> -#     * strict indentation                                                       
> -#     * line length                                                              
> -#     * use of <> instead of !=
> -#     
> -[FORMAT]
> -# Enable / disable this checker
> -enable-format=no
> -
> -# Maximum number of characters on a single line.
> -max-line-length=80
> -
> -# Maximum number of lines in a module
> -max-module-lines=1000
> -
> -# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1 tab).
> -indent-string='    '
> -
> -
> -
> -# does not check anything but gives some raw metrics :                    
> -#     * total number of lines                                                    
> -#     * total number of code lines                                               
> -#     * total number of docstring lines                                          
> -#     * total number of comments lines                                           
> -#     * total number of empty lines                                              
> -#                                                                                 
> -# This checker also defines the following reports:                                    
> -#   * R0701: Raw metrics                                                          
> -# 
> -[METRICS]
> -# Enable / disable this checker
> -enable-metrics=yes
> -
> -
> -
> -# checks for:                                                             
> -#     * warning notes in the code like FIXME, XXX                                
> -#     * PEP 263: source code with non ascii character but no encoding declaration
> -#     
> -[MISCELLANEOUS]
> -# Enable / disable this checker
> -enable-miscellaneous=yes
> -
> -# List of note tags to take in consideration, separated by a comma. Default to
> -# FIXME, XXX, TODO
> -notes=FIXME,XXX,TODO
> -
> -
> -
> -# checks for similarities and duplicated code. This computation may be
> -#     memory / CPU intensive, so you should disable it if you experiments some
> -#     problems.
> -#                                                                                 
> -# This checker also defines the following reports:                                    
> -#   * R0801: Duplication                                                          
> -# 
> -[SIMILARITIES]
> -# Enable / disable this checker
> -enable-similarities=yes
> -
> -# Minimum lines number of a similarity.
> -min-similarity-lines=4
> -
> -# Ignore comments when computing similarities.
> -ignore-comments=yes
> -
> -
> -
> -- 
> 2.30.2
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH 2/7] tools/misc: Drop xencons

[Andrew Cooper]

This is a python script which has it's shebang modified by be python3, but
was never converted to be python3 compatible.

The most recent reference I can find to this script (which isn't incidental
adjustments in the makefile) is from the Xen book, fileish 561e30b80402 which
says

  %% <snip>  Alternatively, if the
  %% Xen machine is connected to a serial-port server then we supply a
  %% dumb TCP terminal client, {\tt xencons}.

So this a not-invented-here version of telnet.  Delete it.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/misc/Makefile |  2 -
 tools/misc/xencons  | 92 ---------------------------------------------
 2 files changed, 94 deletions(-)
 delete mode 100755 tools/misc/xencons

diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index 1c6e1d6a0471..233a7948c050 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -12,7 +12,6 @@ CFLAGS += $(CFLAGS_libxenstore)
 # Everything to be installed in regular bin/
 INSTALL_BIN-$(CONFIG_X86)      += xen-cpuid
 INSTALL_BIN-$(CONFIG_X86)      += xen-detect
-INSTALL_BIN                    += xencons
 INSTALL_BIN                    += xencov_split
 INSTALL_BIN += $(INSTALL_BIN-y)
 
@@ -44,7 +43,6 @@ INSTALL_PRIVBIN                += xenpvnetboot
 TARGETS_ALL := $(INSTALL_BIN) $(INSTALL_SBIN) $(INSTALL_PRIVBIN)
 
 # Everything which only needs copying to install
-TARGETS_COPY += xencons
 TARGETS_COPY += xencov_split
 TARGETS_COPY += xenpvnetboot
 
diff --git a/tools/misc/xencons b/tools/misc/xencons
deleted file mode 100755
index 8bd3178eab4e..000000000000
--- a/tools/misc/xencons
+++ /dev/null
@@ -1,92 +0,0 @@
-#!/usr/bin/env python
-
-##############################################
-# Console client for Xen guest OSes
-# Copyright (c) 2004, K A Fraser
-##############################################
-
-import errno, os, signal, socket, struct, sys
-
-from termios import *
-# Indexes into termios.tcgetattr() list.
-IFLAG  = 0
-OFLAG  = 1
-CFLAG  = 2
-LFLAG  = 3
-ISPEED = 4
-OSPEED = 5
-CC     = 6
-
-def __child_death(signum, frame):
-    global stop
-    stop = True
-
-def __recv_from_sock(sock):
-    global stop
-    stop = False
-    while not stop:
-        try:
-            data = sock.recv(1024)
-        except socket.error, error:
-            if error[0] != errno.EINTR:
-                raise
-        else:
-            try:
-                os.write(1, data)
-            except os.error, error:
-                if error[0] != errno.EINTR:
-                    raise
-    os.wait()
-
-def __send_to_sock(sock):
-    while 1:
-        try:
-            data = os.read(0,1024)
-        except os.error, error:
-            if error[0] != errno.EINTR:
-                raise
-        else:
-            if ord(data[0]) == ord(']')-64:
-                break
-            try:
-                sock.send(data)
-            except socket.error, error:
-                if error[0] == errno.EPIPE:
-                    sys.exit(0)
-                if error[0] != errno.EINTR:
-                    raise
-    sys.exit(0)
-
-def connect(host,port):
-    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
-    sock.connect((host,port))
-
-    oattrs = tcgetattr(0)
-    nattrs = tcgetattr(0)
-    nattrs[IFLAG] = nattrs[IFLAG] & ~(BRKINT | ICRNL | INPCK | ISTRIP | IXON)
-    nattrs[OFLAG] = nattrs[OFLAG] & ~(OPOST)
-    nattrs[CFLAG] = nattrs[CFLAG] & ~(CSIZE | PARENB)
-    nattrs[CFLAG] = nattrs[CFLAG] | CS8
-    nattrs[LFLAG] = nattrs[LFLAG] & ~(ECHO | ICANON | IEXTEN | ISIG)
-    nattrs[CC][VMIN] = 1
-    nattrs[CC][VTIME] = 0
-
-    if os.fork():
-        signal.signal(signal.SIGCHLD, __child_death)
-        print "************ REMOTE CONSOLE: CTRL-] TO QUIT ********"
-        tcsetattr(0, TCSAFLUSH, nattrs)
-        try:
-            __recv_from_sock(sock)
-        finally:
-            tcsetattr(0, TCSAFLUSH, oattrs)
-            print
-            print "************ REMOTE CONSOLE EXITED *****************"
-    else:
-        signal.signal(signal.SIGPIPE, signal.SIG_IGN)
-        __send_to_sock(sock)
-
-if __name__ == '__main__':
-    if len(sys.argv) != 3:
-        print sys.argv[0] + " <host> <port>"
-        sys.exit(1)
-    connect(str(sys.argv[1]),int(sys.argv[2]))
-- 
2.30.2

Re: [PATCH 2/7] tools/misc: Drop xencons

[Anthony PERARD]

On Tue, Mar 14, 2023 at 02:15:15PM +0000, Andrew Cooper wrote:
> This is a python script which has it's shebang modified by be python3, but
> was never converted to be python3 compatible.

Not quite, that's the original shebang ;-)

> The most recent reference I can find to this script (which isn't incidental
> adjustments in the makefile) is from the Xen book, fileish 561e30b80402 which
> says
> 
>   %% <snip>  Alternatively, if the
>   %% Xen machine is connected to a serial-port server then we supply a
>   %% dumb TCP terminal client, {\tt xencons}.
> 
> So this a not-invented-here version of telnet.  Delete it.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

Re: [PATCH 2/7] tools/misc: Drop xencons

[Andrew Cooper]

On 16/03/2023 11:33 am, Anthony PERARD wrote:
> On Tue, Mar 14, 2023 at 02:15:15PM +0000, Andrew Cooper wrote:
>> This is a python script which has it's shebang modified by be python3, but
>> was never converted to be python3 compatible.
> Not quite, that's the original shebang ;-)

What I meant was that INSTALL_PYTHON_PROG turns it into a python3 shebang.

I'll see if I can think of some slightly clearer wording.

>
>> The most recent reference I can find to this script (which isn't incidental
>> adjustments in the makefile) is from the Xen book, fileish 561e30b80402 which
>> says
>>
>>   %% <snip>  Alternatively, if the
>>   %% Xen machine is connected to a serial-port server then we supply a
>>   %% dumb TCP terminal client, {\tt xencons}.
>>
>> So this a not-invented-here version of telnet.  Delete it.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks.

~Andrew

Re: [PATCH 2/7] tools/misc: Drop xencons

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

On Tue, Mar 14, 2023 at 02:15:15PM +0000, Andrew Cooper wrote:
> This is a python script which has it's shebang modified by be python3, but
> was never converted to be python3 compatible.
> 
> The most recent reference I can find to this script (which isn't incidental
> adjustments in the makefile) is from the Xen book, fileish 561e30b80402 which
> says
> 
>   %% <snip>  Alternatively, if the
>   %% Xen machine is connected to a serial-port server then we supply a
>   %% dumb TCP terminal client, {\tt xencons}.
> 
> So this a not-invented-here version of telnet.  Delete it.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Not sure if necessary, but in any case:

Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH 3/7] tools: Delete trailing whitespace in python scripts

[Andrew Cooper]

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/misc/xencov_split            |   1 -
 tools/pygrub/src/ExtLinuxConf.py   |   2 +-
 tools/pygrub/src/GrubConf.py       |  30 ++---
 tools/pygrub/src/fsimage/fsimage.c |   8 +-
 tools/pygrub/src/pygrub            |  44 +++----
 tools/python/xen/lowlevel/xc/xc.c  | 200 ++++++++++++++---------------
 tools/python/xen/lowlevel/xs/xs.c  |   4 +-
 tools/xenmon/xenmon.py             |  44 +++----
 8 files changed, 166 insertions(+), 167 deletions(-)

diff --git a/tools/misc/xencov_split b/tools/misc/xencov_split
index 5771f6cfc285..e4f68ebb6eda 100755
--- a/tools/misc/xencov_split
+++ b/tools/misc/xencov_split
@@ -94,4 +94,3 @@ if __name__ == "__main__":
         sys.exit(1)
     except KeyboardInterrupt:
         sys.exit(1)
-
diff --git a/tools/pygrub/src/ExtLinuxConf.py b/tools/pygrub/src/ExtLinuxConf.py
index 9fd635b9cf7a..4e990a9304b6 100644
--- a/tools/pygrub/src/ExtLinuxConf.py
+++ b/tools/pygrub/src/ExtLinuxConf.py
@@ -207,7 +207,7 @@ class ExtLinuxConfigFile(object):
                  "f1": None,
                  "f2": None,
                  }
-        
+
 if __name__ == "__main__":
     if len(sys.argv) < 2:
         raise RuntimeError("Need a configuration file to read")
diff --git a/tools/pygrub/src/GrubConf.py b/tools/pygrub/src/GrubConf.py
index 73f1bbed2f54..580c9628ca64 100644
--- a/tools/pygrub/src/GrubConf.py
+++ b/tools/pygrub/src/GrubConf.py
@@ -95,7 +95,7 @@ class _GrubImage(object):
         self.title = title.strip()
 
     def __repr__(self):
-        return ("title: %s\n" 
+        return ("title: %s\n"
                 "  root: %s\n"
                 "  kernel: %s\n"
                 "  args: %s\n"
@@ -140,7 +140,7 @@ class _GrubImage(object):
 class GrubImage(_GrubImage):
     def __init__(self, title, lines):
         _GrubImage.__init__(self, title, lines)
-    
+
     def set_from_line(self, line, replace = None):
         (com, arg) = grub_exact_split(line, 2)
 
@@ -265,7 +265,7 @@ class _GrubConfigFile(object):
 class GrubConfigFile(_GrubConfigFile):
     def __init__(self, fn = None):
         _GrubConfigFile.__init__(self,fn)
-        
+
     def new_image(self, title, lines):
         return GrubImage(title, lines)
 
@@ -297,7 +297,7 @@ class GrubConfigFile(_GrubConfigFile):
                 img = []
                 title = l[6:]
                 continue
-                
+
             if img is not None:
                 img.append(l)
                 continue
@@ -310,7 +310,7 @@ class GrubConfigFile(_GrubConfigFile):
                     logging.info("Ignored directive %s" %(com,))
             else:
                 logging.warning("Unknown directive %s" %(com,))
-                
+
         if img:
             self.add_image(GrubImage(title, img))
 
@@ -322,7 +322,7 @@ def grub2_handle_set(arg):
     com="set:" + com
     m = re.match("([\"\'])(.*)\\1", arg)
     if m is not None:
-        arg=m.group(2) 
+        arg=m.group(2)
     return (com,arg)
 
 class Grub2Image(_GrubImage):
@@ -334,7 +334,7 @@ class Grub2Image(_GrubImage):
 
         if com == "set":
             (com,arg) = grub2_handle_set(arg)
-            
+
         if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
@@ -351,7 +351,7 @@ class Grub2Image(_GrubImage):
         else:
             self.lines.pop(replace)
             self.lines.insert(replace, line)
-                
+
     commands = {'set:root': 'root',
                 'linux': 'kernel',
                 'linux16': 'kernel',
@@ -360,14 +360,14 @@ class Grub2Image(_GrubImage):
                 'echo': None,
                 'insmod': None,
                 'search': None}
-    
+
 class Grub2ConfigFile(_GrubConfigFile):
     def __init__(self, fn = None):
         _GrubConfigFile.__init__(self, fn)
-       
+
     def new_image(self, title, lines):
         return Grub2Image(title, lines)
- 
+
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
@@ -431,10 +431,10 @@ class Grub2ConfigFile(_GrubConfigFile):
                 continue
 
             (com, arg) = grub_exact_split(l, 2)
-        
+
             if com == "set":
                 (com,arg) = grub2_handle_set(arg)
-                
+
             if com in self.commands:
                 if self.commands[com] is not None:
                     arg_strip = arg.strip()
@@ -448,7 +448,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                 pass
             else:
                 logging.warning("Unknown directive %s" %(com,))
-            
+
         if img is not None:
             raise RuntimeError("syntax error: end of file with open menuentry(%d %s)" % (len(img),img))
 
@@ -466,7 +466,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                 'if': None,
                 'fi': None,
                 }
-        
+
 if __name__ == "__main__":
     if len(sys.argv) < 3:
         raise RuntimeError('Need a grub version ("grub" or "grub2") and a grub.conf or grub.cfg to read')
diff --git a/tools/pygrub/src/fsimage/fsimage.c b/tools/pygrub/src/fsimage/fsimage.c
index 2ebbbe35df92..fdcfa1a3c040 100644
--- a/tools/pygrub/src/fsimage/fsimage.c
+++ b/tools/pygrub/src/fsimage/fsimage.c
@@ -31,7 +31,7 @@ typedef struct fsimage_fs {
 	fsi_t *fs;
 } fsimage_fs_t;
 
-typedef struct fsimage_file { 
+typedef struct fsimage_file {
 	PyObject_HEAD
 	fsimage_fs_t *fs;
 	fsi_file_t *file;
@@ -47,7 +47,7 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 	ssize_t bytesread = 0;
 	PyObject * buffer;
 
-	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|iL", kwlist, 
+	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|iL", kwlist,
 	    &size, &offset))
 		return (NULL);
 
@@ -62,7 +62,7 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 
 	if (buffer == NULL)
 		return (NULL);
- 
+
 	while (1) {
 		int err;
 		void *buf =
@@ -255,7 +255,7 @@ fsimage_open(PyObject *o, PyObject *args, PyObject *kwargs)
 	uint64_t offset = 0;
 	fsimage_fs_t *fs;
 
-	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s|Ls", kwlist, 
+	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s|Ls", kwlist,
 	    &name, &offset, &options))
 		return (NULL);
 
diff --git a/tools/pygrub/src/pygrub b/tools/pygrub/src/pygrub
index ce7ab0eb8cf3..a759d90ade5e 100755
--- a/tools/pygrub/src/pygrub
+++ b/tools/pygrub/src/pygrub
@@ -46,7 +46,7 @@ def enable_cursor(ison):
         val = 2
     else:
         val = 0
-        
+
     try:
         curses.curs_set(val)
     except _curses.error:
@@ -114,7 +114,7 @@ FDISK_PART_GPT=0xee
 def get_partition_offsets(file):
     image_type = identify_disk_image(file)
     if image_type == DISK_TYPE_RAW:
-        # No MBR: assume whole disk filesystem, which is like a 
+        # No MBR: assume whole disk filesystem, which is like a
         # single partition starting at 0
         return [0]
     elif image_type == DISK_TYPE_HYBRIDISO:
@@ -135,7 +135,7 @@ def get_partition_offsets(file):
         partbuf = buf[poff:poff+16]
         offset  = struct.unpack("<L", partbuf[8:12])[0] * SECTOR_SIZE
         type    = struct.unpack("<B", partbuf[4:5])[0]
-        
+
         # offset == 0 implies this partition is not enabled
         if offset == 0:
             continue
@@ -171,7 +171,7 @@ class GrubLineEditor(curses.textpad.Textbox):
         screen.noutrefresh()
         win = curses.newwin(1, 74, startx, starty + 2)
         curses.textpad.Textbox.__init__(self, win)
-        
+
         self.line = list(line)
         self.pos = len(line)
         self.cancelled = False
@@ -237,7 +237,7 @@ class GrubLineEditor(curses.textpad.Textbox):
         if self.cancelled:
             return None
         return string.join(self.line, "")
-        
+
 
 class Grub:
     ENTRY_WIN_LINES = 8
@@ -261,7 +261,7 @@ class Grub:
             self.entry_win = curses.newwin(Grub.ENTRY_WIN_LINES + 2, 74, 2, 1)
             self.text_win = curses.newwin(10, 70, 12, 5)
             curses.def_prog_mode()
-        
+
         curses.reset_prog_mode()
         self.screen.erase()
 
@@ -279,7 +279,7 @@ class Grub:
             self.start_image = self.selected_image
         if self.selected_image < self.start_image:
             self.start_image = self.selected_image
-        
+
         for y in range(self.start_image, len(self.cf.images)):
             i = self.cf.images[y]
             if y > self.start_image + maxy:
@@ -329,7 +329,7 @@ class Grub:
                 l = img.lines[idx].expandtabs().ljust(70)
                 if len(l) > 70:
                     l = l[:69] + ">"
-                    
+
                 self.entry_win.addstr(idp, 2, l)
                 if idx == curline:
                     self.entry_win.attroff(curses.A_REVERSE)
@@ -367,7 +367,7 @@ class Grub:
                 self.command_line_mode()
                 if self.isdone:
                     return
-                
+
             # bound at the top and bottom
             if curline < 0:
                 curline = 0
@@ -400,7 +400,7 @@ class Grub:
         lines = []
         while 1:
             t = GrubLineEditor(self.screen, y, 2)
-            enable_cursor(True)            
+            enable_cursor(True)
             ret = t.edit()
             if ret:
                 if ret in ("quit", "return"):
@@ -410,7 +410,7 @@ class Grub:
                     lines.append(ret)
                     continue
 
-                # if we got boot, then we want to boot the entered image 
+                # if we got boot, then we want to boot the entered image
                 img = self.cf.new_image("entered", lines)
                 self.cf.add_image(img)
                 self.selected_image = len(self.cf.images) - 1
@@ -423,7 +423,7 @@ class Grub:
     def read_config(self, fn, fs = None):
         """Read the given file to parse the config.  If fs = None, then
         we're being given a raw config file rather than a disk image."""
-        
+
         if not os.access(fn, os.R_OK):
             raise RuntimeError("Unable to access %s" %(fn,))
 
@@ -499,7 +499,7 @@ class Grub:
         while not self.isdone:
             self.run_main(timeout)
             timeout = -1
-            
+
         return self.selected_image
 
     def run_main(self, timeout = -1):
@@ -529,7 +529,7 @@ class Grub:
         self.start_image = 0
         while (timeout == -1 or mytime < int(timeout)):
             draw()
-            if timeout != -1 and mytime != -1: 
+            if timeout != -1 and mytime != -1:
                 self.screen.addstr(20, 5, "Will boot selected entry in %2d seconds"
                                    %(int(timeout) - mytime))
             else:
@@ -600,7 +600,7 @@ class Grub:
                 self.selected_image = 0
             elif self.selected_image >= len(self.cf.images):
                 self.selected_image = len(self.cf.images) - 1
-        
+
 def get_entry_idx(cf, entry):
     # first, see if the given entry is numeric
     try:
@@ -697,10 +697,10 @@ def sniff_solaris(fs, cfg):
     # Unpleasant. Typically we'll have 'root=foo -k' or 'root=foo /kernel -k',
     # and we need to maintain Xen properties (root= and ip=) and the kernel
     # before any user args.
-    
+
     xenargs = ""
     userargs = ""
-    
+
     if not cfg["args"]:
         cfg["args"] = cfg["kernel"]
     else:
@@ -712,7 +712,7 @@ def sniff_solaris(fs, cfg):
         cfg["args"] = xenargs + " " + cfg["kernel"] + " " + userargs
 
     return cfg
- 
+
 def sniff_netware(fs, cfg):
     if not fs.file_exists("/nwserver/xnloader.sys"):
         return cfg
@@ -729,7 +729,7 @@ def format_sxp(kernel, ramdisk, args):
     if args:
         s += "(args %s)" % repr(args)
     return s
-                
+
 def format_simple(kernel, ramdisk, args, sep):
     for check in (kernel, ramdisk, args):
         if check is not None and sep in check:
@@ -744,7 +744,7 @@ def format_simple(kernel, ramdisk, args, sep):
 
 if __name__ == "__main__":
     sel = None
-    
+
     def usage():
         print("Usage: %s [-q|--quiet] [-i|--interactive] [-l|--list-entries] [-n|--not-really] [--output=] [--kernel=] [--ramdisk=] [--args=] [--entry=] [--output-directory=] [--output-format=sxp|simple|simple0] [--offset=] <image>" %(sys.argv[0],), file=sys.stderr)
 
@@ -783,7 +783,7 @@ if __name__ == "__main__":
         opts, args = getopt.gnu_getopt(sys.argv[1:], 'qilnh::',
                                    ["quiet", "interactive", "list-entries", "not-really", "help",
                                     "output=", "output-format=", "output-directory=", "offset=",
-                                    "entry=", "kernel=", 
+                                    "entry=", "kernel=",
                                     "ramdisk=", "args=", "isconfig", "debug"])
     except getopt.GetoptError:
         usage()
@@ -967,4 +967,4 @@ if __name__ == "__main__":
         os.write(fd, ostring)
     else:
         os.write(fd, ostring.encode())
-    
+
diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index cfb2734a992b..e3a0c13bac31 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -1,6 +1,6 @@
 /******************************************************************************
  * Xc.c
- * 
+ *
  * Copyright (c) 2003-2004, K A Fraser (University of Cambridge)
  */
 
@@ -108,7 +108,7 @@ static PyObject *pyxc_domain_dumpcore(XcObject *self, PyObject *args)
 
     if ( xc_domain_dumpcore(self->xc_handle, dom, corefile) != 0 )
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -143,7 +143,7 @@ static PyObject *pyxc_domain_create(XcObject *self,
         return NULL;
     if ( pyhandle != NULL )
     {
-        if ( !PyList_Check(pyhandle) || 
+        if ( !PyList_Check(pyhandle) ||
              (PyList_Size(pyhandle) != sizeof(xen_domain_handle_t)) )
             goto out_exception;
 
@@ -191,7 +191,7 @@ static PyObject *pyxc_domain_max_vcpus(XcObject *self, PyObject *args)
 
     if (xc_domain_max_vcpus(self->xc_handle, dom, max) != 0)
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -226,7 +226,7 @@ static PyObject *pyxc_domain_shutdown(XcObject *self, PyObject *args)
 
     if ( xc_domain_shutdown(self->xc_handle, dom, reason) != 0 )
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -258,7 +258,7 @@ static PyObject *pyxc_vcpu_setaffinity(XcObject *self,
 
     static char *kwd_list[] = { "domid", "vcpu", "cpumap", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "i|iO", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "i|iO", kwd_list,
                                       &dom, &vcpu, &cpulist) )
         return NULL;
 
@@ -272,7 +272,7 @@ static PyObject *pyxc_vcpu_setaffinity(XcObject *self,
 
     if ( (cpulist != NULL) && PyList_Check(cpulist) )
     {
-        for ( i = 0; i < PyList_Size(cpulist); i++ ) 
+        for ( i = 0; i < PyList_Size(cpulist); i++ )
         {
             long cpu = PyLongOrInt_AsLong(PyList_GetItem(cpulist, i));
             if ( cpu < 0 || cpu >= nr_cpus )
@@ -285,7 +285,7 @@ static PyObject *pyxc_vcpu_setaffinity(XcObject *self,
             cpumap[cpu / 8] |= 1 << (cpu % 8);
         }
     }
-  
+
     if ( xc_vcpu_setaffinity(self->xc_handle, dom, vcpu, cpumap,
                              NULL, XEN_VCPUAFFINITY_HARD) != 0 )
     {
@@ -293,7 +293,7 @@ static PyObject *pyxc_vcpu_setaffinity(XcObject *self,
         return pyxc_error_to_exception(self->xc_handle);
     }
     Py_INCREF(zero);
-    free(cpumap); 
+    free(cpumap);
     return zero;
 }
 
@@ -307,7 +307,7 @@ static PyObject *pyxc_domain_sethandle(XcObject *self, PyObject *args)
     if (!PyArg_ParseTuple(args, "iO", &dom, &pyhandle))
         return NULL;
 
-    if ( !PyList_Check(pyhandle) || 
+    if ( !PyList_Check(pyhandle) ||
          (PyList_Size(pyhandle) != sizeof(xen_domain_handle_t)) )
     {
         goto out_exception;
@@ -323,7 +323,7 @@ static PyObject *pyxc_domain_sethandle(XcObject *self, PyObject *args)
 
     if (xc_domain_sethandle(self->xc_handle, dom, handle) < 0)
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 
@@ -345,7 +345,7 @@ static PyObject *pyxc_domain_getinfo(XcObject *self,
     xc_dominfo_t *info;
 
     static char *kwd_list[] = { "first_dom", "max_doms", NULL };
-    
+
     if ( !PyArg_ParseTupleAndKeywords(args, kwds, "|ii", kwd_list,
                                       &first_dom, &max_doms) )
         return NULL;
@@ -418,7 +418,7 @@ static PyObject *pyxc_vcpu_getinfo(XcObject *self,
     int nr_cpus;
 
     static char *kwd_list[] = { "domid", "vcpu", NULL };
-    
+
     if ( !PyArg_ParseTupleAndKeywords(args, kwds, "i|i", kwd_list,
                                       &dom, &vcpu) )
         return NULL;
@@ -473,7 +473,7 @@ static PyObject *pyxc_hvm_param_get(XcObject *self,
     int param;
     uint64_t value;
 
-    static char *kwd_list[] = { "domid", "param", NULL }; 
+    static char *kwd_list[] = { "domid", "param", NULL };
     if ( !PyArg_ParseTupleAndKeywords(args, kwds, "ii", kwd_list,
                                       &dom, &param) )
         return NULL;
@@ -493,7 +493,7 @@ static PyObject *pyxc_hvm_param_set(XcObject *self,
     int param;
     uint64_t value;
 
-    static char *kwd_list[] = { "domid", "param", "value", NULL }; 
+    static char *kwd_list[] = { "domid", "param", "value", NULL };
     if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iiL", kwd_list,
                                       &dom, &param, &value) )
         return NULL;
@@ -663,7 +663,7 @@ static PyObject *pyxc_get_device_group(XcObject *self,
 
     if ( rc < 0 )
     {
-        free(sdev_array); 
+        free(sdev_array);
         return pyxc_error_to_exception(self->xc_handle);
     }
 
@@ -786,7 +786,7 @@ static PyObject *pyxc_physdev_pci_access_modify(XcObject *self,
 
     static char *kwd_list[] = { "domid", "bus", "dev", "func", "enable", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iiiii", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iiiii", kwd_list,
                                       &dom, &bus, &dev, &func, &enable) )
         return NULL;
 
@@ -901,7 +901,7 @@ static PyObject *pyxc_physinfo(XcObject *self)
                             "nr_nodes",         pinfo.nr_nodes,
                             "threads_per_core", pinfo.threads_per_core,
                             "cores_per_socket", pinfo.cores_per_socket,
-                            "nr_cpus",          pinfo.nr_cpus, 
+                            "nr_cpus",          pinfo.nr_cpus,
                             "total_memory",     pages_to_kib(pinfo.total_pages),
                             "free_memory",      pages_to_kib(pinfo.free_pages),
                             "scrub_memory",     pages_to_kib(pinfo.scrub_pages),
@@ -1191,13 +1191,13 @@ static PyObject *pyxc_shadow_control(PyObject *self,
 
     static char *kwd_list[] = { "dom", "op", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "i|i", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "i|i", kwd_list,
                                       &dom, &op) )
         return NULL;
-    
+
     if ( xc_shadow_control(xc->xc_handle, dom, op, NULL, 0) < 0 )
         return pyxc_error_to_exception(xc->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -1214,26 +1214,26 @@ static PyObject *pyxc_shadow_mem_control(PyObject *self,
 
     static char *kwd_list[] = { "dom", "mb", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "i|i", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "i|i", kwd_list,
                                       &dom, &mbarg) )
         return NULL;
-    
-    if ( mbarg < 0 ) 
+
+    if ( mbarg < 0 )
         op = XEN_DOMCTL_SHADOW_OP_GET_ALLOCATION;
-    else 
+    else
     {
         mb = mbarg;
         op = XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION;
     }
     if ( xc_shadow_control(xc->xc_handle, dom, op, &mb, 0) < 0 )
         return pyxc_error_to_exception(xc->xc_handle);
-    
+
     mbarg = mb;
     return Py_BuildValue("i", mbarg);
 }
 
 static PyObject *pyxc_sched_id_get(XcObject *self) {
-    
+
     int sched_id;
     if (xc_sched_id(self->xc_handle, &sched_id) != 0)
         return PyErr_SetFromErrno(xc_error_obj);
@@ -1251,10 +1251,10 @@ static PyObject *pyxc_sched_credit_domain_set(XcObject *self,
     static char *kwd_list[] = { "domid", "weight", "cap", NULL };
     static char kwd_type[] = "I|HH";
     struct xen_domctl_sched_credit sdom;
-    
+
     weight = 0;
     cap = (uint16_t)~0U;
-    if( !PyArg_ParseTupleAndKeywords(args, kwds, kwd_type, kwd_list, 
+    if( !PyArg_ParseTupleAndKeywords(args, kwds, kwd_type, kwd_list,
                                      &domid, &weight, &cap) )
         return NULL;
 
@@ -1272,10 +1272,10 @@ static PyObject *pyxc_sched_credit_domain_get(XcObject *self, PyObject *args)
 {
     uint32_t domid;
     struct xen_domctl_sched_credit sdom;
-    
+
     if( !PyArg_ParseTuple(args, "I", &domid) )
         return NULL;
-    
+
     if ( xc_sched_credit_domain_get(self->xc_handle, domid, &sdom) != 0 )
         return pyxc_error_to_exception(self->xc_handle);
 
@@ -1336,7 +1336,7 @@ static PyObject *pyxc_domain_setmaxmem(XcObject *self, PyObject *args)
 
     if (xc_domain_setmaxmem(self->xc_handle, dom, maxmem_kb) != 0)
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -1349,12 +1349,12 @@ static PyObject *pyxc_domain_set_target_mem(XcObject *self, PyObject *args)
     if (!PyArg_ParseTuple(args, "ii", &dom, &mem_kb))
         return NULL;
 
-    mem_pages = mem_kb / 4; 
+    mem_pages = mem_kb / 4;
 
     if (xc_domain_set_pod_target(self->xc_handle, dom, mem_pages,
 				 NULL, NULL, NULL) != 0)
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -1369,7 +1369,7 @@ static PyObject *pyxc_domain_set_memmap_limit(XcObject *self, PyObject *args)
 
     if ( xc_domain_set_memmap_limit(self->xc_handle, dom, maplimit_kb) != 0 )
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -1383,7 +1383,7 @@ static PyObject *pyxc_domain_ioport_permission(XcObject *self,
 
     static char *kwd_list[] = { "domid", "first_port", "nr_ports", "allow_access", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iiii", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iiii", kwd_list,
                                       &dom, &first_port, &nr_ports, &allow_access) )
         return NULL;
 
@@ -1406,7 +1406,7 @@ static PyObject *pyxc_domain_irq_permission(PyObject *self,
 
     static char *kwd_list[] = { "domid", "pirq", "allow_access", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iii", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iii", kwd_list,
                                       &dom, &pirq, &allow_access) )
         return NULL;
 
@@ -1429,7 +1429,7 @@ static PyObject *pyxc_domain_iomem_permission(PyObject *self,
 
     static char *kwd_list[] = { "domid", "first_pfn", "nr_pfns", "allow_access", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "illi", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "illi", kwd_list,
                                       &dom, &first_pfn, &nr_pfns, &allow_access) )
         return NULL;
 
@@ -1480,7 +1480,7 @@ static PyObject *pyxc_domain_send_trigger(XcObject *self,
 
     static char *kwd_list[] = { "domid", "trigger", "vcpu", NULL };
 
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "ii|i", kwd_list, 
+    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "ii|i", kwd_list,
                                       &dom, &trigger, &vcpu) )
         return NULL;
 
@@ -1534,7 +1534,7 @@ static PyObject *pyxc_dom_set_memshr(XcObject *self, PyObject *args)
 
     if (xc_memshr_control(self->xc_handle, dom, enable) != 0)
         return pyxc_error_to_exception(self->xc_handle);
-    
+
     Py_INCREF(zero);
     return zero;
 }
@@ -1758,11 +1758,11 @@ static PyObject *pyflask_sid_to_context(PyObject *self, PyObject *args,
     if (!xc_handle) {
         return PyErr_SetFromErrno(xc_error_obj);
     }
-    
+
     ret = xc_flask_sid_to_context(xc_handle, sid, ctx, ctx_len);
-    
+
     xc_interface_close(xc_handle);
-    
+
     if ( ret != 0 ) {
         errno = -ret;
         return PyErr_SetFromErrno(xc_error_obj);
@@ -1779,7 +1779,7 @@ static PyObject *pyflask_load(PyObject *self, PyObject *args, PyObject *kwds)
     int ret;
 
     static char *kwd_list[] = { "policy", NULL };
-  
+
     if( !PyArg_ParseTupleAndKeywords(args, kwds, "s#", kwd_list, &policy, &len) )
         return NULL;
 
@@ -1809,11 +1809,11 @@ static PyObject *pyflask_getenforce(PyObject *self)
     if (!xc_handle) {
         return PyErr_SetFromErrno(xc_error_obj);
     }
-    
+
     ret = xc_flask_getenforce(xc_handle);
-    
+
     xc_interface_close(xc_handle);
-    
+
     if ( ret < 0 ) {
         errno = -ret;
         return PyErr_SetFromErrno(xc_error_obj);
@@ -1839,11 +1839,11 @@ static PyObject *pyflask_setenforce(PyObject *self, PyObject *args,
     if (!xc_handle) {
         return PyErr_SetFromErrno(xc_error_obj);
     }
-    
+
     ret = xc_flask_setenforce(xc_handle, mode);
-    
+
     xc_interface_close(xc_handle);
-    
+
     if ( ret != 0 ) {
         errno = -ret;
         return PyErr_SetFromErrno(xc_error_obj);
@@ -1861,7 +1861,7 @@ static PyObject *pyflask_access(PyObject *self, PyObject *args,
     uint32_t req, allowed, decided, auditallow, auditdeny, seqno;
     int ret;
 
-    static char *kwd_list[] = { "src_context", "tar_context", 
+    static char *kwd_list[] = { "src_context", "tar_context",
                                 "tar_class", "req_permissions",
                                 "decided", "auditallow","auditdeny",
                                 "seqno", NULL };
@@ -1875,10 +1875,10 @@ static PyObject *pyflask_access(PyObject *self, PyObject *args,
     if (!xc_handle) {
         return PyErr_SetFromErrno(xc_error_obj);
     }
-    
+
     ret = xc_flask_access(xc_handle, scon, tcon, tclass, req, &allowed, &decided,
                         &auditallow, &auditdeny, &seqno);
-        
+
     xc_interface_close(xc_handle);
 
     if ( ret != 0 ) {
@@ -1890,14 +1890,14 @@ static PyObject *pyflask_access(PyObject *self, PyObject *args,
 }
 
 static PyMethodDef pyxc_methods[] = {
-    { "domain_create", 
-      (PyCFunction)pyxc_domain_create, 
+    { "domain_create",
+      (PyCFunction)pyxc_domain_create,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Create a new domain.\n"
       " dom    [int, 0]:        Domain identifier to use (allocated if zero).\n"
       "Returns: [int] new domain identifier; -1 on error.\n" },
 
-    { "domain_max_vcpus", 
+    { "domain_max_vcpus",
       (PyCFunction)pyxc_domain_max_vcpus,
       METH_VARARGS, "\n"
       "Set the maximum number of VCPUs a domain may create.\n"
@@ -1905,43 +1905,43 @@ static PyMethodDef pyxc_methods[] = {
       " max     [int, 0]:      New maximum number of VCPUs in domain.\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_dumpcore", 
-      (PyCFunction)pyxc_domain_dumpcore, 
+    { "domain_dumpcore",
+      (PyCFunction)pyxc_domain_dumpcore,
       METH_VARARGS, "\n"
       "Dump core of a domain.\n"
       " dom [int]: Identifier of domain to dump core of.\n"
       " corefile [string]: Name of corefile to be created.\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_pause", 
-      (PyCFunction)pyxc_domain_pause, 
+    { "domain_pause",
+      (PyCFunction)pyxc_domain_pause,
       METH_VARARGS, "\n"
       "Temporarily pause execution of a domain.\n"
       " dom [int]: Identifier of domain to be paused.\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_unpause", 
-      (PyCFunction)pyxc_domain_unpause, 
+    { "domain_unpause",
+      (PyCFunction)pyxc_domain_unpause,
       METH_VARARGS, "\n"
       "(Re)start execution of a domain.\n"
       " dom [int]: Identifier of domain to be unpaused.\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_destroy", 
-      (PyCFunction)pyxc_domain_destroy, 
+    { "domain_destroy",
+      (PyCFunction)pyxc_domain_destroy,
       METH_VARARGS, "\n"
       "Destroy a domain.\n"
       " dom [int]:    Identifier of domain to be destroyed.\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_destroy_hook", 
-      (PyCFunction)pyxc_domain_destroy_hook, 
+    { "domain_destroy_hook",
+      (PyCFunction)pyxc_domain_destroy_hook,
       METH_VARARGS, "\n"
       "Add a hook for arch stuff before destroy a domain.\n"
       " dom [int]:    Identifier of domain to be destroyed.\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_resume", 
+    { "domain_resume",
       (PyCFunction)pyxc_domain_resume,
       METH_VARARGS, "\n"
       "Resume execution of a suspended domain.\n"
@@ -1949,7 +1949,7 @@ static PyMethodDef pyxc_methods[] = {
       " fast [int]: Use cooperative resume.\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_shutdown", 
+    { "domain_shutdown",
       (PyCFunction)pyxc_domain_shutdown,
       METH_VARARGS, "\n"
       "Shutdown a domain.\n"
@@ -1957,8 +1957,8 @@ static PyMethodDef pyxc_methods[] = {
       " reason     [int, 0]:      Reason for shutdown.\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "vcpu_setaffinity", 
-      (PyCFunction)pyxc_vcpu_setaffinity, 
+    { "vcpu_setaffinity",
+      (PyCFunction)pyxc_vcpu_setaffinity,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Pin a VCPU to a specified set CPUs.\n"
       " dom [int]:     Identifier of domain to which VCPU belongs.\n"
@@ -1966,7 +1966,7 @@ static PyMethodDef pyxc_methods[] = {
       " cpumap [list, []]: list of usable CPUs.\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_sethandle", 
+    { "domain_sethandle",
       (PyCFunction)pyxc_domain_sethandle,
       METH_VARARGS, "\n"
       "Set domain's opaque handle.\n"
@@ -1974,8 +1974,8 @@ static PyMethodDef pyxc_methods[] = {
       " handle [list of 16 ints]: New opaque handle.\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_getinfo", 
-      (PyCFunction)pyxc_domain_getinfo, 
+    { "domain_getinfo",
+      (PyCFunction)pyxc_domain_getinfo,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Get information regarding a set of domains, in increasing id order.\n"
       " first_dom [int, 0]:    First domain to retrieve info about.\n"
@@ -2000,8 +2000,8 @@ static PyMethodDef pyxc_methods[] = {
       "reason why it shut itself down.\n"
       " cpupool  [int]   Id of cpupool domain is bound to.\n" },
 
-    { "vcpu_getinfo", 
-      (PyCFunction)pyxc_vcpu_getinfo, 
+    { "vcpu_getinfo",
+      (PyCFunction)pyxc_vcpu_getinfo,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Get information regarding a VCPU.\n"
       " dom  [int]:    Domain to retrieve info about.\n"
@@ -2025,7 +2025,7 @@ static PyMethodDef pyxc_methods[] = {
       " xenstore_domid [int]: \n"
       "Returns: None on success. Raises exception on error.\n" },
 
-    { "hvm_get_param", 
+    { "hvm_get_param",
       (PyCFunction)pyxc_hvm_param_get,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "get a parameter of HVM guest OS.\n"
@@ -2033,7 +2033,7 @@ static PyMethodDef pyxc_methods[] = {
       " param   [int]:      No. of HVM param.\n"
       "Returns: [long] value of the param.\n" },
 
-    { "hvm_set_param", 
+    { "hvm_set_param",
       (PyCFunction)pyxc_hvm_param_set,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "set a parameter of HVM guest OS.\n"
@@ -2076,12 +2076,12 @@ static PyMethodDef pyxc_methods[] = {
        " dom     [int]:      Domain to deassign device from.\n"
        " pci_str [str]:      PCI devices.\n"
        "Returns: [int] 0 on success, or device bdf that can't be deassigned.\n" },
-  
+
     { "sched_id_get",
       (PyCFunction)pyxc_sched_id_get,
       METH_NOARGS, "\n"
       "Get the current scheduler type in use.\n"
-      "Returns: [int] sched_id.\n" },    
+      "Returns: [int] sched_id.\n" },
 
     { "sched_credit_domain_set",
       (PyCFunction)pyxc_sched_credit_domain_set,
@@ -2119,7 +2119,7 @@ static PyMethodDef pyxc_methods[] = {
       "Returns:   [dict]\n"
       " weight    [short]: domain's scheduling weight\n"},
 
-    { "evtchn_alloc_unbound", 
+    { "evtchn_alloc_unbound",
       (PyCFunction)pyxc_evtchn_alloc_unbound,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Allocate an unbound port that will await a remote connection.\n"
@@ -2127,7 +2127,7 @@ static PyMethodDef pyxc_methods[] = {
       " remote_dom [int]: Remote domain to accept connections from.\n\n"
       "Returns: [int] Unbound event-channel port.\n" },
 
-    { "evtchn_reset", 
+    { "evtchn_reset",
       (PyCFunction)pyxc_evtchn_reset,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Reset all connections.\n"
@@ -2152,9 +2152,9 @@ static PyMethodDef pyxc_methods[] = {
       " func   [int]: PCI function\n"
       " enable [int]: Non-zero means enable access; else disable access\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
- 
-    { "readconsolering", 
-      (PyCFunction)pyxc_readconsolering, 
+
+    { "readconsolering",
+      (PyCFunction)pyxc_readconsolering,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Read Xen's console ring.\n"
       " clear [int, 0]: Bool - clear the ring after reading from it?\n\n"
@@ -2202,40 +2202,40 @@ static PyMethodDef pyxc_methods[] = {
       "Returns [str]: Xen buildid"
       "        [None]: on failure.\n" },
 
-    { "shadow_control", 
-      (PyCFunction)pyxc_shadow_control, 
+    { "shadow_control",
+      (PyCFunction)pyxc_shadow_control,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Set parameter for shadow pagetable interface\n"
       " dom [int]:   Identifier of domain.\n"
       " op [int, 0]: operation\n\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "shadow_mem_control", 
-      (PyCFunction)pyxc_shadow_mem_control, 
+    { "shadow_mem_control",
+      (PyCFunction)pyxc_shadow_mem_control,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Set or read shadow pagetable memory use\n"
       " dom [int]:   Identifier of domain.\n"
       " mb [int, -1]: MB of shadow memory this domain should have.\n\n"
       "Returns: [int] MB of shadow memory in use by this domain.\n" },
 
-    { "domain_setmaxmem", 
-      (PyCFunction)pyxc_domain_setmaxmem, 
+    { "domain_setmaxmem",
+      (PyCFunction)pyxc_domain_setmaxmem,
       METH_VARARGS, "\n"
       "Set a domain's memory limit\n"
       " dom [int]: Identifier of domain.\n"
       " maxmem_kb [int]: .\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_set_target_mem", 
-      (PyCFunction)pyxc_domain_set_target_mem, 
+    { "domain_set_target_mem",
+      (PyCFunction)pyxc_domain_set_target_mem,
       METH_VARARGS, "\n"
       "Set a domain's memory target\n"
       " dom [int]: Identifier of domain.\n"
       " mem_kb [int]: .\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
-    { "domain_set_memmap_limit", 
-      (PyCFunction)pyxc_domain_set_memmap_limit, 
+    { "domain_set_memmap_limit",
+      (PyCFunction)pyxc_domain_set_memmap_limit,
       METH_VARARGS, "\n"
       "Set a domain's physical memory mapping limit\n"
       " dom [int]: Identifier of domain.\n"
@@ -2309,7 +2309,7 @@ static PyMethodDef pyxc_methods[] = {
       "Inject debug keys into Xen.\n"
       " keys    [str]: String of keys to inject.\n" },
 
-    { "dom_set_memshr", 
+    { "dom_set_memshr",
       (PyCFunction)pyxc_dom_set_memshr,
       METH_VARARGS, "\n"
       "Enable/disable memory sharing for the domain.\n"
@@ -2391,20 +2391,20 @@ static PyMethodDef pyxc_methods[] = {
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Loads a policy into the hypervisor.\n"
       " policy [str]: policy to be load\n"
-      "Returns: [int]: 0 on success; -1 on failure.\n" }, 
-      
+      "Returns: [int]: 0 on success; -1 on failure.\n" },
+
     { "flask_getenforce",
       (PyCFunction)pyflask_getenforce,
       METH_NOARGS, "\n"
       "Returns the current mode of the Flask XSM module.\n"
-      "Returns: [int]: 0 for permissive; 1 for enforcing; -1 on failure.\n" }, 
+      "Returns: [int]: 0 for permissive; 1 for enforcing; -1 on failure.\n" },
 
     { "flask_setenforce",
       (PyCFunction)pyflask_setenforce,
       METH_VARARGS | METH_KEYWORDS, "\n"
       "Modifies the current mode for the Flask XSM module.\n"
       " mode [int]: mode to change to\n"
-      "Returns: [int]: 0 on success; -1 on failure.\n" }, 
+      "Returns: [int]: 0 on success; -1 on failure.\n" },
 
     { "flask_access",
       (PyCFunction)pyflask_access,
@@ -2423,7 +2423,7 @@ static PyMethodDef pyxc_methods[] = {
       " auditdeny [int] permissions set to audit on deny\n"
       " seqno [int] not used\n"
       "Returns: [int]: 0 on all permission granted; -1 if any permissions are \
-       denied\n" }, 
+       denied\n" },
 
     { NULL, NULL, 0, NULL }
 };
diff --git a/tools/python/xen/lowlevel/xs/xs.c b/tools/python/xen/lowlevel/xs/xs.c
index 3ba5a8b893d9..87f1187bb116 100644
--- a/tools/python/xen/lowlevel/xs/xs.c
+++ b/tools/python/xen/lowlevel/xs/xs.c
@@ -1,4 +1,4 @@
-/* 
+/*
  * Python interface to the Xen Store Daemon.
  *
  * This library is free software; you can redistribute it and/or
@@ -859,7 +859,7 @@ static void remove_watch(XsHandle *self, PyObject *token)
  * Parse transaction and path arguments from the given args and kwds,
  * convert the given self value to an xs_handle, and return all three by
  * reference.
- * 
+ *
  * @return 1 on success, in which case *xh, *th, and *path are valid, or 0 on
  * failure.
  */
diff --git a/tools/xenmon/xenmon.py b/tools/xenmon/xenmon.py
index 977ada688771..1b357aed5127 100644
--- a/tools/xenmon/xenmon.py
+++ b/tools/xenmon/xenmon.py
@@ -13,12 +13,12 @@
 #   This program is free software; you can redistribute it and/or modify
 #   it under the terms of the GNU General Public License as published by
 #   the Free Software Foundation; under version 2 of the License.
-# 
+#
 #   This program is distributed in the hope that it will be useful,
 #   but WITHOUT ANY WARRANTY; without even the implied warranty of
 #   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 #   GNU General Public License for more details.
-# 
+#
 #   You should have received a copy of the GNU General Public License
 #   along with this program; If not, see <http://www.gnu.org/licenses/>.
 #####################################################################
@@ -79,7 +79,7 @@ def setup_cmdline_parser():
     parser.add_option("-p", "--prefix", dest="prefix",
                       default = "log", help="prefix to use for output files")
     parser.add_option("-t", "--time", dest="duration",
-            action="store", type="int", default=10, 
+            action="store", type="int", default=10,
             help="stop logging to file after this much time has elapsed (in seconds). set to 0 to keep logging indefinitely")
     parser.add_option("-i", "--interval", dest="interval",
             action="store", type="int", default=1000,
@@ -179,7 +179,7 @@ class DomainInfo(object):
         return [total/(float(passed)/10**9), avg]
 
     def stats(self, passed):
-        return [self.gotten_stats(passed), self.allocated_stats(passed), self.blocked_stats(passed), 
+        return [self.gotten_stats(passed), self.allocated_stats(passed), self.blocked_stats(passed),
                 self.waited_stats(passed), self.ec_stats(passed), self.io_stats(passed)]
 
 # report values over desired interval
@@ -187,13 +187,13 @@ def summarize(startat, endat, duration, samples):
     dominfos = {}
     for i in range(0, NDOMAINS):
         dominfos[i] = DomainInfo()
-        
+
     passed = 1              # to prevent zero division
     curid = startat
     numbuckets = 0
     lost_samples = []
     ffp_samples = []
-    
+
     while passed < duration:
         for i in range(0, NDOMAINS):
             if dom_in_use[i]:
@@ -203,7 +203,7 @@ def summarize(startat, endat, duration, samples):
                 dominfos[i].blocked_sum += samples[curid][3*NDOMAINS + i]
                 dominfos[i].exec_count += samples[curid][4*NDOMAINS + i]
                 dominfos[i].iocount_sum += samples[curid][5*NDOMAINS + i]
-    
+
         passed += samples[curid][6*NDOMAINS]
         lost_samples.append(samples[curid][6*NDOMAINS + 2])
         ffp_samples.append(samples[curid][6*NDOMAINS + 3])
@@ -271,7 +271,7 @@ def show_livestats(cpu):
     cpu_10sec_usage = 0.0
     heartbeat = 1
     global dom_in_use, options
-    
+
     # mmap the (the first chunk of the) file
     shmf = open(SHM_FILE, "r+")
     shm = mmap.mmap(shmf.fileno(), QOS_DATA_SIZE)
@@ -284,7 +284,7 @@ def show_livestats(cpu):
     stdscr.keypad(1)
     stdscr.timeout(1000)
     [maxy, maxx] = stdscr.getmaxyx()
-    
+
     # display in a loop
     while True:
 
@@ -342,7 +342,7 @@ def show_livestats(cpu):
             cpuidx = cpuidx + 1
 
         # calculate starting and ending datapoints; never look at "next" since
-        # it represents live data that may be in transition. 
+        # it represents live data that may be in transition.
         startat = next - 1
         if next + 10 < NSAMPLES:
             endat = next + 10
@@ -374,7 +374,7 @@ def show_livestats(cpu):
 
             if h1[dom][0][1] > 0 or domain_id[dom] == IDLE_DOMAIN:
                 # display gotten
-                row += 1 
+                row += 1
                 col = 2
                 display_domain_id(stdscr, row, col, domain_id[dom])
                 col += 4
@@ -396,7 +396,7 @@ def show_livestats(cpu):
 
                 if dom != IDLE_DOMAIN:
                     cpu_1sec_usage = cpu_1sec_usage + h1[dom][0][1]
-    
+
                 # display allocated
                 if options.allocated:
                     row += 1
@@ -454,7 +454,7 @@ def show_livestats(cpu):
                     row += 1
                     col = 2
                     display_domain_id(stdscr, row, col, domain_id[dom])
-                    
+
                     col += 28
                     display(stdscr, row, col, "%d/s" % h2[dom][4])
                     col += 42
@@ -490,14 +490,14 @@ def show_livestats(cpu):
         display(stdscr, row, 1, star)
         display(stdscr, row, 2, TOTALS % (total_h2_cpu, total_h1_cpu))
         row += 1
-#        display(stdscr, row, 2, 
-#                "\tFFP: %d (Min: %d, Max: %d)\t\t\tFFP: %d (Min: %d, Max %d)" % 
+#        display(stdscr, row, 2,
+#                "\tFFP: %d (Min: %d, Max: %d)\t\t\tFFP: %d (Min: %d, Max %d)" %
 #                (math.ceil(f2[1]), f2[0], f2[2], math.ceil(f1[1]), f1[0], f1[2]), _c.A_BOLD)
 
         if l1[1] > 1 :
             row += 1
-            display(stdscr, row, 2, 
-                    "\tRecords lost: %d (Min: %d, Max: %d)\t\t\tRecords lost: %d (Min: %d, Max %d)" % 
+            display(stdscr, row, 2,
+                    "\tRecords lost: %d (Min: %d, Max: %d)\t\t\tRecords lost: %d (Min: %d, Max %d)" %
                     (math.ceil(l2[1]), l2[0], l2[2], math.ceil(l1[1]), l1[0], l1[2]), _c.A_BOLD)
 
         # grab a char from tty input; exit if interrupt hit
@@ -505,11 +505,11 @@ def show_livestats(cpu):
             c = stdscr.getch()
         except:
             break
-        
+
         # q = quit
         if c == ord('q'):
             break
-    
+
         # c = cycle to a new cpu of interest
         if c == ord('c'):
             cpu = (cpu + 1) % ncpu
@@ -560,7 +560,7 @@ class Delayed(object):
     def close(self):
         if  self.opened:
             self.file.close()
-            
+
 
 def writelog():
     global options
@@ -639,7 +639,7 @@ def writelog():
                                      h1[dom][1],
                                      h1[dom][2][0], h1[dom][2][1], h1[dom][2][2],
                                      h1[dom][3][0], h1[dom][3][1], h1[dom][3][2],
-                                     h1[dom][4], 
+                                     h1[dom][4],
                                      h1[dom][5][0], h1[dom][5][1]))
                     outfiles[dom].flush()
             curr = time.time()
@@ -698,7 +698,7 @@ def main():
        options.mspersample > options.duration * 1000:
         parser.error("option --ms_per_sample: too large (> %d ms)" %
                      (options.duration * 1000))
-    
+
     start_xenbaked()
     if options.live:
         show_livestats(options.cpu)
-- 
2.30.2

Re: [PATCH 3/7] tools: Delete trailing whitespace in python scripts

[Anthony PERARD]

On Tue, Mar 14, 2023 at 02:15:16PM +0000, Andrew Cooper wrote:
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

Re: [PATCH 3/7] tools: Delete trailing whitespace in python scripts

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 301 bytes --]

On Tue, Mar 14, 2023 at 02:15:16PM +0000, Andrew Cooper wrote:
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH 4/7] tools/pygrub: Factor out common setup.py parts

[Andrew Cooper]

... to mirror the tools/python side in c/s 2b8314a3c354.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/pygrub/Makefile | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
index 37b214621492..29ad0513212f 100644
--- a/tools/pygrub/Makefile
+++ b/tools/pygrub/Makefile
@@ -6,19 +6,20 @@ PY_CFLAGS = $(CFLAGS) $(PY_NOOPT_CFLAGS)
 PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS)
 INSTALL_LOG = build/installed_files.txt
 
+setup.py = CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" \
+           $(PYTHON) setup.py
+
 .PHONY: all
 all: build
 .PHONY: build
 build:
-	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py build
+	$(setup.py) build
 
 .PHONY: install
 install: all
 	$(INSTALL_DIR) $(DESTDIR)/$(bindir)
-	CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" \
-		LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) setup.py install \
-		--record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
-		 --root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force
+	$(setup.py) install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
+		--root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force
 	set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \
 	             "`readlink -f $(DESTDIR)/$(bindir)`" != \
 	             "`readlink -f $(LIBEXEC_BIN)`" ]; then \
-- 
2.30.2

Re: [PATCH 4/7] tools/pygrub: Factor out common setup.py parts

[Anthony PERARD]

On Tue, Mar 14, 2023 at 02:15:17PM +0000, Andrew Cooper wrote:
> ... to mirror the tools/python side in c/s 2b8314a3c354.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

[PATCH 5/7] tools: Use -s for python shebangs

[Andrew Cooper]

This is mandated by the Fedora packaging guidelines because it is a security
vulnerability otherwise in suid scripts.  It's a very good idea generally,
because it prevents the users local python environment interfering from system
packaged scripts.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/Rules.mk        | 2 +-
 tools/pygrub/Makefile | 2 +-
 tools/python/Makefile | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/Rules.mk b/tools/Rules.mk
index 6e135387bd7e..18cf83f5be83 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -179,7 +179,7 @@ CFLAGS += $(CFLAGS-y)
 CFLAGS += $(EXTRA_CFLAGS_XEN_TOOLS)
 
 INSTALL_PYTHON_PROG = \
-	$(XEN_ROOT)/tools/python/install-wrap "$(PYTHON_PATH)" $(INSTALL_PROG)
+	$(XEN_ROOT)/tools/python/install-wrap "$(PYTHON_PATH) -s" $(INSTALL_PROG)
 
 %.opic: %.c
 	$(CC) $(CPPFLAGS) -DPIC $(CFLAGS) $(CFLAGS_$*.opic) -fPIC -c -o $@ $< $(APPEND_CFLAGS)
diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
index 29ad0513212f..04b3995cc0f6 100644
--- a/tools/pygrub/Makefile
+++ b/tools/pygrub/Makefile
@@ -7,7 +7,7 @@ PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS)
 INSTALL_LOG = build/installed_files.txt
 
 setup.py = CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" \
-           $(PYTHON) setup.py
+           $(PYTHON) setup.py --executable="$(PYTHON_PATH) -s"
 
 .PHONY: all
 all: build
diff --git a/tools/python/Makefile b/tools/python/Makefile
index cc764236478a..511e7deae409 100644
--- a/tools/python/Makefile
+++ b/tools/python/Makefile
@@ -12,7 +12,7 @@ setup.py = CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLA
            SHLIB_libxenctrl="$(SHLIB_libxenctrl)" \
            SHLIB_libxenguest="$(SHLIB_libxenguest)" \
            SHLIB_libxenstore="$(SHLIB_libxenstore)" \
-           $(PYTHON) setup.py
+           $(PYTHON) setup.py --executable="$(PYTHON_PATH) -s"
 
 .PHONY: build
 build:
-- 
2.30.2

Re: [PATCH 5/7] tools: Use -s for python shebangs

[Andrew Cooper]

On 14/03/2023 2:15 pm, Andrew Cooper wrote:
> diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
> index 29ad0513212f..04b3995cc0f6 100644
> --- a/tools/pygrub/Makefile
> +++ b/tools/pygrub/Makefile
> @@ -7,7 +7,7 @@ PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS)
>  INSTALL_LOG = build/installed_files.txt
>  
>  setup.py = CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" \
> -           $(PYTHON) setup.py
> +           $(PYTHON) setup.py --executable="$(PYTHON_PATH) -s"

/sigh

CI has gone a wall of red to this.  Apparently --executable is only
known by setuputils, not distutils.

Which is a problem because it means the main pygrub executable won't get
a corrected shebang, as it doesn't pass through install-wrap.

In the short term, I could fix that by moving the main pygrub binary out
of setup.py and do it manually.  Thoughts?

~Andrew

Re: [PATCH 5/7] tools: Use -s for python shebangs

[Anthony PERARD]

On Tue, Mar 14, 2023 at 02:50:48PM +0000, Andrew Cooper wrote:
> On 14/03/2023 2:15 pm, Andrew Cooper wrote:
> > diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
> > index 29ad0513212f..04b3995cc0f6 100644
> > --- a/tools/pygrub/Makefile
> > +++ b/tools/pygrub/Makefile
> > @@ -7,7 +7,7 @@ PY_LDFLAGS = $(SHLIB_LDFLAGS) $(APPEND_LDFLAGS)
> >  INSTALL_LOG = build/installed_files.txt
> >  
> >  setup.py = CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDSHARED="$(CC)" LDFLAGS="$(PY_LDFLAGS)" \
> > -           $(PYTHON) setup.py
> > +           $(PYTHON) setup.py --executable="$(PYTHON_PATH) -s"
> 
> /sigh
> 
> CI has gone a wall of red to this.  Apparently --executable is only
> known by setuputils, not distutils.
> 
> Which is a problem because it means the main pygrub executable won't get
> a corrected shebang, as it doesn't pass through install-wrap.
> 
> In the short term, I could fix that by moving the main pygrub binary out
> of setup.py and do it manually.  Thoughts?

Or do the fixing in "setup.py", https://stackoverflow.com/a/17099342 :-)
But that probably more work that necessary.
Doing the binary installation in the Makefile sounds fine.

Thanks,

-- 
Anthony PERARD

[PATCH v2 5/7] tools: Use -s for python shebangs

[Andrew Cooper]

This is mandated by the Fedora packaging guidelines because it is a security
vulnerability otherwise in suid scripts.  While Xen doesn't have suid scripts,
it's a very good idea generally, because it prevents the users local python
environment interfering from system packaged scripts.

pygrub is the odd-script-out, being installed by distutils rather than
manually with INSTALL_PYTHON_PROG.  distutils has no nice way of editing the
shebang, so arrange to use INSTALL_PYTHON_PROG on pygrub too.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>

v2:
 * Remove accidental setuputils dependency.
---
 tools/Rules.mk        | 2 +-
 tools/pygrub/Makefile | 4 +++-
 tools/pygrub/setup.py | 1 -
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/tools/Rules.mk b/tools/Rules.mk
index 6e135387bd7e..18cf83f5be83 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -179,7 +179,7 @@ CFLAGS += $(CFLAGS-y)
 CFLAGS += $(EXTRA_CFLAGS_XEN_TOOLS)
 
 INSTALL_PYTHON_PROG = \
-	$(XEN_ROOT)/tools/python/install-wrap "$(PYTHON_PATH)" $(INSTALL_PROG)
+	$(XEN_ROOT)/tools/python/install-wrap "$(PYTHON_PATH) -s" $(INSTALL_PROG)
 
 %.opic: %.c
 	$(CC) $(CPPFLAGS) -DPIC $(CFLAGS) $(CFLAGS_$*.opic) -fPIC -c -o $@ $< $(APPEND_CFLAGS)
diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
index 29ad0513212f..4963bc89c6ed 100644
--- a/tools/pygrub/Makefile
+++ b/tools/pygrub/Makefile
@@ -18,8 +18,10 @@ build:
 .PHONY: install
 install: all
 	$(INSTALL_DIR) $(DESTDIR)/$(bindir)
+	$(INSTALL_DIR) $(DESTDIR)/$(LIBEXEC_BIN)
 	$(setup.py) install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
-		--root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force
+		--root="$(DESTDIR)" --force
+	$(INSTALL_PYTHON_PROG) src/pygrub $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
 	set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \
 	             "`readlink -f $(DESTDIR)/$(bindir)`" != \
 	             "`readlink -f $(LIBEXEC_BIN)`" ]; then \
diff --git a/tools/pygrub/setup.py b/tools/pygrub/setup.py
index 0e4e3d02d372..502aa4df2dae 100644
--- a/tools/pygrub/setup.py
+++ b/tools/pygrub/setup.py
@@ -23,7 +23,6 @@ setup(name='pygrub',
       author_email='katzj@redhat.com',
       license='GPL',
       package_dir={'grub': 'src', 'fsimage': 'src'},
-      scripts = ["src/pygrub"],
       packages=pkgs,
       ext_modules = [ xenfsimage ]
       )
-- 
2.30.2

Re: [PATCH v2 5/7] tools: Use -s for python shebangs

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 3045 bytes --]

On Thu, Mar 16, 2023 at 07:37:44PM +0000, Andrew Cooper wrote:
> This is mandated by the Fedora packaging guidelines because it is a security
> vulnerability otherwise in suid scripts.  While Xen doesn't have suid scripts,
> it's a very good idea generally, because it prevents the users local python
> environment interfering from system packaged scripts.
> 
> pygrub is the odd-script-out, being installed by distutils rather than
> manually with INSTALL_PYTHON_PROG.  distutils has no nice way of editing the
> shebang, so arrange to use INSTALL_PYTHON_PROG on pygrub too.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Wei Liu <wl@xen.org>
> CC: Anthony PERARD <anthony.perard@citrix.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
> 
> v2:
>  * Remove accidental setuputils dependency.

... and tools/python/setup.py doesn't install any scripts, so it isn't
relevant there.

Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

> ---
>  tools/Rules.mk        | 2 +-
>  tools/pygrub/Makefile | 4 +++-
>  tools/pygrub/setup.py | 1 -
>  3 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/Rules.mk b/tools/Rules.mk
> index 6e135387bd7e..18cf83f5be83 100644
> --- a/tools/Rules.mk
> +++ b/tools/Rules.mk
> @@ -179,7 +179,7 @@ CFLAGS += $(CFLAGS-y)
>  CFLAGS += $(EXTRA_CFLAGS_XEN_TOOLS)
>  
>  INSTALL_PYTHON_PROG = \
> -	$(XEN_ROOT)/tools/python/install-wrap "$(PYTHON_PATH)" $(INSTALL_PROG)
> +	$(XEN_ROOT)/tools/python/install-wrap "$(PYTHON_PATH) -s" $(INSTALL_PROG)
>  
>  %.opic: %.c
>  	$(CC) $(CPPFLAGS) -DPIC $(CFLAGS) $(CFLAGS_$*.opic) -fPIC -c -o $@ $< $(APPEND_CFLAGS)
> diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
> index 29ad0513212f..4963bc89c6ed 100644
> --- a/tools/pygrub/Makefile
> +++ b/tools/pygrub/Makefile
> @@ -18,8 +18,10 @@ build:
>  .PHONY: install
>  install: all
>  	$(INSTALL_DIR) $(DESTDIR)/$(bindir)
> +	$(INSTALL_DIR) $(DESTDIR)/$(LIBEXEC_BIN)
>  	$(setup.py) install --record $(INSTALL_LOG) $(PYTHON_PREFIX_ARG) \
> -		--root="$(DESTDIR)" --install-scripts=$(LIBEXEC_BIN) --force
> +		--root="$(DESTDIR)" --force
> +	$(INSTALL_PYTHON_PROG) src/pygrub $(DESTDIR)/$(LIBEXEC_BIN)/pygrub
>  	set -e; if [ $(bindir) != $(LIBEXEC_BIN) -a \
>  	             "`readlink -f $(DESTDIR)/$(bindir)`" != \
>  	             "`readlink -f $(LIBEXEC_BIN)`" ]; then \
> diff --git a/tools/pygrub/setup.py b/tools/pygrub/setup.py
> index 0e4e3d02d372..502aa4df2dae 100644
> --- a/tools/pygrub/setup.py
> +++ b/tools/pygrub/setup.py
> @@ -23,7 +23,6 @@ setup(name='pygrub',
>        author_email='katzj@redhat.com',
>        license='GPL',
>        package_dir={'grub': 'src', 'fsimage': 'src'},
> -      scripts = ["src/pygrub"],
>        packages=pkgs,
>        ext_modules = [ xenfsimage ]
>        )
> -- 
> 2.30.2
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH 6/7] tools/python: Improve unit test handling

[Andrew Cooper]

 * Add X86_{CPUID,MSR}_POLICY_FORMAT checks which were missed previously.
 * Drop test_suite().  It hasn't been necessary since the Py2.3 era.
 * Drop the __main__ logic.  This can't be used without manually adjusting the
   include path, and `make test` knows how to do the right thing.
 * For `make test`, use `-v` to see which tests have been discovered and run.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/python/Makefile               |  2 +-
 tools/python/xen/migration/tests.py | 14 ++------------
 2 files changed, 3 insertions(+), 13 deletions(-)

diff --git a/tools/python/Makefile b/tools/python/Makefile
index 511e7deae409..697299bf2802 100644
--- a/tools/python/Makefile
+++ b/tools/python/Makefile
@@ -36,7 +36,7 @@ uninstall:
 
 .PHONY: test
 test:
-	LD_LIBRARY_PATH=$$(readlink -f ../libs/ctrl):$$(readlink -f ../xenstore) $(PYTHON) -m unittest discover
+	LD_LIBRARY_PATH=$$(readlink -f ../libs/ctrl):$$(readlink -f ../xenstore) $(PYTHON) -m unittest discover -v
 
 .PHONY: clean
 clean:
diff --git a/tools/python/xen/migration/tests.py b/tools/python/xen/migration/tests.py
index ff2768946bb5..f22e2c2b7cf0 100644
--- a/tools/python/xen/migration/tests.py
+++ b/tools/python/xen/migration/tests.py
@@ -26,6 +26,8 @@ class TestLibxc(unittest.TestCase):
                          (libxc.X86_TSC_INFO_FORMAT, 24),
                          (libxc.HVM_PARAMS_ENTRY_FORMAT, 16),
                          (libxc.HVM_PARAMS_FORMAT, 8),
+                         (libxc.X86_CPUID_POLICY_FORMAT, 24),
+                         (libxc.X86_MSR_POLICY_FORMAT, 16),
                          ):
             self.assertEqual(calcsize(fmt), sz)
 
@@ -40,15 +42,3 @@ class TestLibxl(unittest.TestCase):
                          (libxl.EMULATOR_HEADER_FORMAT, 8),
                          ):
             self.assertEqual(calcsize(fmt), sz)
-
-
-def test_suite():
-    suite = unittest.TestSuite()
-
-    suite.addTest(unittest.makeSuite(TestLibxc))
-    suite.addTest(unittest.makeSuite(TestLibxl))
-
-    return suite
-
-if __name__ == "__main__":
-    unittest.main()
-- 
2.30.2

Re: [PATCH 6/7] tools/python: Improve unit test handling

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 2703 bytes --]

On Tue, Mar 14, 2023 at 02:15:19PM +0000, Andrew Cooper wrote:
>  * Add X86_{CPUID,MSR}_POLICY_FORMAT checks which were missed previously.
>  * Drop test_suite().  It hasn't been necessary since the Py2.3 era.
>  * Drop the __main__ logic.  This can't be used without manually adjusting the
>    include path, and `make test` knows how to do the right thing.
>  * For `make test`, use `-v` to see which tests have been discovered and run.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

> ---
> CC: Wei Liu <wl@xen.org>
> CC: Anthony PERARD <anthony.perard@citrix.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
> ---
>  tools/python/Makefile               |  2 +-
>  tools/python/xen/migration/tests.py | 14 ++------------
>  2 files changed, 3 insertions(+), 13 deletions(-)
> 
> diff --git a/tools/python/Makefile b/tools/python/Makefile
> index 511e7deae409..697299bf2802 100644
> --- a/tools/python/Makefile
> +++ b/tools/python/Makefile
> @@ -36,7 +36,7 @@ uninstall:
>  
>  .PHONY: test
>  test:
> -	LD_LIBRARY_PATH=$$(readlink -f ../libs/ctrl):$$(readlink -f ../xenstore) $(PYTHON) -m unittest discover
> +	LD_LIBRARY_PATH=$$(readlink -f ../libs/ctrl):$$(readlink -f ../xenstore) $(PYTHON) -m unittest discover -v
>  
>  .PHONY: clean
>  clean:
> diff --git a/tools/python/xen/migration/tests.py b/tools/python/xen/migration/tests.py
> index ff2768946bb5..f22e2c2b7cf0 100644
> --- a/tools/python/xen/migration/tests.py
> +++ b/tools/python/xen/migration/tests.py
> @@ -26,6 +26,8 @@ class TestLibxc(unittest.TestCase):
>                           (libxc.X86_TSC_INFO_FORMAT, 24),
>                           (libxc.HVM_PARAMS_ENTRY_FORMAT, 16),
>                           (libxc.HVM_PARAMS_FORMAT, 8),
> +                         (libxc.X86_CPUID_POLICY_FORMAT, 24),
> +                         (libxc.X86_MSR_POLICY_FORMAT, 16),
>                           ):
>              self.assertEqual(calcsize(fmt), sz)
>  
> @@ -40,15 +42,3 @@ class TestLibxl(unittest.TestCase):
>                           (libxl.EMULATOR_HEADER_FORMAT, 8),
>                           ):
>              self.assertEqual(calcsize(fmt), sz)
> -
> -
> -def test_suite():
> -    suite = unittest.TestSuite()
> -
> -    suite.addTest(unittest.makeSuite(TestLibxc))
> -    suite.addTest(unittest.makeSuite(TestLibxl))
> -
> -    return suite
> -
> -if __name__ == "__main__":
> -    unittest.main()
> -- 
> 2.30.2
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH 7/7] tools/python: Drop shebangs from library files

[Andrew Cooper]

These aren't runable scripts, so shouldn't have shebangs.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
---
 tools/python/xen/migration/legacy.py | 1 -
 tools/python/xen/migration/libxc.py  | 1 -
 tools/python/xen/migration/libxl.py  | 1 -
 tools/python/xen/migration/public.py | 1 -
 tools/python/xen/migration/tests.py  | 1 -
 tools/python/xen/migration/verify.py | 1 -
 tools/python/xen/migration/xl.py     | 1 -
 tools/python/xen/util.py             | 1 -
 8 files changed, 8 deletions(-)

diff --git a/tools/python/xen/migration/legacy.py b/tools/python/xen/migration/legacy.py
index 6456d6157ce3..e196ca876095 100644
--- a/tools/python/xen/migration/legacy.py
+++ b/tools/python/xen/migration/legacy.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """
diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index 9881f5ced4ea..e52e632cb106 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """
diff --git a/tools/python/xen/migration/libxl.py b/tools/python/xen/migration/libxl.py
index 5c4d4fe0631b..5dcb50fe0207 100644
--- a/tools/python/xen/migration/libxl.py
+++ b/tools/python/xen/migration/libxl.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """
diff --git a/tools/python/xen/migration/public.py b/tools/python/xen/migration/public.py
index fab2f84587b7..23183ef67db8 100644
--- a/tools/python/xen/migration/public.py
+++ b/tools/python/xen/migration/public.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """
diff --git a/tools/python/xen/migration/tests.py b/tools/python/xen/migration/tests.py
index f22e2c2b7cf0..fcf94b0bb264 100644
--- a/tools/python/xen/migration/tests.py
+++ b/tools/python/xen/migration/tests.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """
diff --git a/tools/python/xen/migration/verify.py b/tools/python/xen/migration/verify.py
index 1e38f4a3c01e..b847c4bd220f 100644
--- a/tools/python/xen/migration/verify.py
+++ b/tools/python/xen/migration/verify.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """
diff --git a/tools/python/xen/migration/xl.py b/tools/python/xen/migration/xl.py
index 978e744dfd95..139d496654df 100644
--- a/tools/python/xen/migration/xl.py
+++ b/tools/python/xen/migration/xl.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """
diff --git a/tools/python/xen/util.py b/tools/python/xen/util.py
index a11358eefa13..47ceb5bd21fe 100644
--- a/tools/python/xen/util.py
+++ b/tools/python/xen/util.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 import os
-- 
2.30.2

Re: [PATCH 7/7] tools/python: Drop shebangs from library files

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 3423 bytes --]

On Tue, Mar 14, 2023 at 02:15:20PM +0000, Andrew Cooper wrote:
> These aren't runable scripts, so shouldn't have shebangs.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

> ---
> CC: Wei Liu <wl@xen.org>
> CC: Anthony PERARD <anthony.perard@citrix.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Bernhard Kaindl <bernhard.kaindl@citrix.com>
> ---
>  tools/python/xen/migration/legacy.py | 1 -
>  tools/python/xen/migration/libxc.py  | 1 -
>  tools/python/xen/migration/libxl.py  | 1 -
>  tools/python/xen/migration/public.py | 1 -
>  tools/python/xen/migration/tests.py  | 1 -
>  tools/python/xen/migration/verify.py | 1 -
>  tools/python/xen/migration/xl.py     | 1 -
>  tools/python/xen/util.py             | 1 -
>  8 files changed, 8 deletions(-)
> 
> diff --git a/tools/python/xen/migration/legacy.py b/tools/python/xen/migration/legacy.py
> index 6456d6157ce3..e196ca876095 100644
> --- a/tools/python/xen/migration/legacy.py
> +++ b/tools/python/xen/migration/legacy.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  """
> diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
> index 9881f5ced4ea..e52e632cb106 100644
> --- a/tools/python/xen/migration/libxc.py
> +++ b/tools/python/xen/migration/libxc.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  """
> diff --git a/tools/python/xen/migration/libxl.py b/tools/python/xen/migration/libxl.py
> index 5c4d4fe0631b..5dcb50fe0207 100644
> --- a/tools/python/xen/migration/libxl.py
> +++ b/tools/python/xen/migration/libxl.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  """
> diff --git a/tools/python/xen/migration/public.py b/tools/python/xen/migration/public.py
> index fab2f84587b7..23183ef67db8 100644
> --- a/tools/python/xen/migration/public.py
> +++ b/tools/python/xen/migration/public.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  """
> diff --git a/tools/python/xen/migration/tests.py b/tools/python/xen/migration/tests.py
> index f22e2c2b7cf0..fcf94b0bb264 100644
> --- a/tools/python/xen/migration/tests.py
> +++ b/tools/python/xen/migration/tests.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  """
> diff --git a/tools/python/xen/migration/verify.py b/tools/python/xen/migration/verify.py
> index 1e38f4a3c01e..b847c4bd220f 100644
> --- a/tools/python/xen/migration/verify.py
> +++ b/tools/python/xen/migration/verify.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  """
> diff --git a/tools/python/xen/migration/xl.py b/tools/python/xen/migration/xl.py
> index 978e744dfd95..139d496654df 100644
> --- a/tools/python/xen/migration/xl.py
> +++ b/tools/python/xen/migration/xl.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  """
> diff --git a/tools/python/xen/util.py b/tools/python/xen/util.py
> index a11358eefa13..47ceb5bd21fe 100644
> --- a/tools/python/xen/util.py
> +++ b/tools/python/xen/util.py
> @@ -1,4 +1,3 @@
> -#!/usr/bin/env python
>  # -*- coding: utf-8 -*-
>  
>  import os
> -- 
> 2.30.2
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

RE: [PATCH 0/7] tools: More Python 3 fixes (part 1 of N)

[Henry Wang]

Hi Andrew,

> -----Original Message-----
> From: Andrew Cooper <andrew.cooper3@citrix.com>
> Subject: [PATCH 0/7] tools: More Python 3 fixes (part 1 of N)
> 
> Relatedly, and a critical/blocker for Xen 4.18.  Python 3.12, which will be
> released ahead of the Xen 4.18 release, is deleteing distutils (recommending
> setuputils as the replacement), so someone is going to have to rework all our
> setup.py's to be compatible.

Thanks for noting this, I've taken a note of this issue in my 4.18 release blocker list.

Kind regards,
Henry