[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Gustavo Zacarias]

Switch to distribute fork since it's maintained and fixes a few issues
while bumping to version 0.6.36.
There's work upstream in re-merging with setuptools which is now
supposedly maintained, see
http://mail.python.org/pipermail/distutils-sig/2013-March/020126.html

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
 .../python-setuptools-add-executable.patch         | 50 ++++++++++++++++++++++
 package/python-setuptools/python-setuptools.mk     |  8 ++--
 2 files changed, 54 insertions(+), 4 deletions(-)
 create mode 100644 package/python-setuptools/python-setuptools-add-executable.patch

diff --git a/package/python-setuptools/python-setuptools-add-executable.patch b/package/python-setuptools/python-setuptools-add-executable.patch
new file mode 100644
index 0000000..a14a3ed
--- /dev/null
+++ b/package/python-setuptools/python-setuptools-add-executable.patch
@@ -0,0 +1,50 @@
+Add a new --executable option to distribute so that we can
+force the shebang line in installed python scripts.
+
+Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
+
+diff -Nura distribute-0.6.36.orig/setuptools/command/install.py distribute-0.6.36/setuptools/command/install.py
+--- distribute-0.6.36.orig/setuptools/command/install.py	2013-04-13 09:46:23.160823598 -0300
++++ distribute-0.6.36/setuptools/command/install.py	2013-04-13 11:17:28.418841986 -0300
+@@ -6,6 +6,7 @@
+     """Use easy_install to install the package, w/dependencies"""
+
+     user_options = _install.user_options + [
++        ('executable=', 'e', "specify final destination interpreter path"),
+         ('old-and-unmanageable', None, "Try not to use this!"),
+         ('single-version-externally-managed', None,
+             "used by system package builders to create 'flat' eggs"),
+@@ -21,6 +22,7 @@
+
+     def initialize_options(self):
+         _install.initialize_options(self)
++	self.executable = None
+         self.old_and_unmanageable = None
+         self.single_version_externally_managed = None
+         self.no_compile = None  # make DISTUTILS_DEBUG work right!
+diff -Nura distribute-0.6.36.orig/setuptools/command/install_scripts.py distribute-0.6.36/setuptools/command/install_scripts.py
+--- distribute-0.6.36.orig/setuptools/command/install_scripts.py	2013-04-13 09:46:23.160823598 -0300
++++ distribute-0.6.36/setuptools/command/install_scripts.py	2013-04-13 11:24:32.305416400 -0300
+@@ -10,6 +10,13 @@
+     def initialize_options(self):
+         _install_scripts.initialize_options(self)
+         self.no_ep = False
++	self.executable = None
++
++    def finalize_options(self):
++        _install_scripts.finalize_options(self)
++        self.set_undefined_options('install',
++	    ('executable','executable')
++	)
+
+     def run(self):
+         from setuptools.command.easy_install import get_script_args
+@@ -31,6 +38,8 @@
+         )
+         bs_cmd = self.get_finalized_command('build_scripts')
+         executable = getattr(bs_cmd,'executable',sys_executable)
++	if self.executable is not None:
++	    executable = self.executable
+         is_wininst = getattr(
+             self.get_finalized_command("bdist_wininst"), '_is_running', False
+         )
diff --git a/package/python-setuptools/python-setuptools.mk b/package/python-setuptools/python-setuptools.mk
index 25fddb9..64f6ded 100644
--- a/package/python-setuptools/python-setuptools.mk
+++ b/package/python-setuptools/python-setuptools.mk
@@ -4,9 +4,9 @@
 #
 #############################################################
 
-PYTHON_SETUPTOOLS_VERSION = 0.6c11
-PYTHON_SETUPTOOLS_SOURCE  = setuptools-$(PYTHON_SETUPTOOLS_VERSION).tar.gz
-PYTHON_SETUPTOOLS_SITE    = http://pypi.python.org/packages/source/s/setuptools
+PYTHON_SETUPTOOLS_VERSION = 0.6.36
+PYTHON_SETUPTOOLS_SOURCE  = distribute-$(PYTHON_SETUPTOOLS_VERSION).tar.gz
+PYTHON_SETUPTOOLS_SITE    = http://pypi.python.org/packages/source/d/distribute
 PYTHON_SETUPTOOLS_DEPENDENCIES = python
 
 define HOST_PYTHON_SETUPTOOLS_BUILD_CMDS
@@ -24,7 +24,7 @@ endef
 
 define PYTHON_SETUPTOOLS_INSTALL_TARGET_CMDS
 	(cd $(@D); PYTHONPATH="$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages"\
-	$(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
+	$(HOST_DIR)/usr/bin/python setup.py install --executable=/usr/bin/python --single-version-externally-managed --root=/ --prefix=$(TARGET_DIR)/usr)
 endef
 
 $(eval $(generic-package))
-- 
1.8.1.5

[Buildroot] [PATCH 2/3] python-bottle: bump version and fix shebang

[Gustavo Zacarias]

Bump to version 0.11.6
Add the --executable shebang fixup option to the build phase so that the
script ends with a proper #!/usr/bin/python interpeter line set.

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
 package/python-bottle/python-bottle.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/python-bottle/python-bottle.mk b/package/python-bottle/python-bottle.mk
index 370b31b..9df359f 100644
--- a/package/python-bottle/python-bottle.mk
+++ b/package/python-bottle/python-bottle.mk
@@ -4,7 +4,7 @@
 #
 #############################################################
 
-PYTHON_BOTTLE_VERSION = 0.11.4
+PYTHON_BOTTLE_VERSION = 0.11.6
 PYTHON_BOTTLE_SOURCE = bottle-$(PYTHON_BOTTLE_VERSION).tar.gz
 PYTHON_BOTTLE_SITE = http://pypi.python.org/packages/source/b/bottle
 PYTHON_BOTTLE_DEPENDENCIES = python
@@ -12,7 +12,7 @@ PYTHON_BOTTLE_LICENSE = MIT
 # README.rst refers to the file "LICENSE" but it's not included
 
 define PYTHON_BOTTLE_BUILD_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
+	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build --executable=/usr/bin/python)
 endef
 
 define PYTHON_BOTTLE_INSTALL_TARGET_CMDS
-- 
1.8.1.5

[Buildroot] [PATCH 2/3] python-bottle: bump version and fix shebang

[Peter Korsgaard]

>>>>> "Gustavo" == Gustavo Zacarias <gustavo@zacarias.com.ar> writes:

 Gustavo> Bump to version 0.11.6

 Gustavo> Add the --executable shebang fixup option to the build phase
 Gustavo> so that the script ends with a proper #!/usr/bin/python
 Gustavo> interpeter line set.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 3/3] python-serial: fix shebang

[Gustavo Zacarias]

Add the --executable shebang fixup option to the build phase so that the
script ends with a proper #!/usr/bin/python interpeter line set.

Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
 package/python-serial/python-serial.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/python-serial/python-serial.mk b/package/python-serial/python-serial.mk
index fb6b31c..4bfc355 100644
--- a/package/python-serial/python-serial.mk
+++ b/package/python-serial/python-serial.mk
@@ -6,12 +6,12 @@
 
 PYTHON_SERIAL_VERSION = 2.6
 PYTHON_SERIAL_SOURCE  = pyserial-$(PYTHON_SERIAL_VERSION).tar.gz
-PYTHON_SERIAL_SITE    = http://pypi.python.org/packages/source/p/pyserial/
+PYTHON_SERIAL_SITE    = http://pypi.python.org/packages/source/p/pyserial
 
 PYTHON_SERIAL_DEPENDENCIES = python
 
 define PYTHON_SERIAL_BUILD_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
+	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build --executable=/usr/bin/python)
 endef
 
 define PYTHON_SERIAL_INSTALL_TARGET_CMDS
-- 
1.8.1.5

[Buildroot] [PATCH 3/3] python-serial: fix shebang

[Peter Korsgaard]

>>>>> "Gustavo" == Gustavo Zacarias <gustavo@zacarias.com.ar> writes:

 Gustavo> Add the --executable shebang fixup option to the build phase
 Gustavo> so that the script ends with a proper #!/usr/bin/python
 Gustavo> interpeter line set.

Committed, thanks.

I see we have the same issue with usr/bin/smtpd.py from python itself,
but I'm not sure how to fix that (besides running sed after
installation).

Any idea?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 3/3] python-serial: fix shebang

[Gustavo Zacarias]

On 05/06/2013 04:35 AM, Peter Korsgaard wrote:

> I see we have the same issue with usr/bin/smtpd.py from python itself,
> but I'm not sure how to fix that (besides running sed after
> installation).
> 
> Any idea?

Some (all?) distributions just remove it, it's a smtp proxy, kind of a
sample program, not something we want lingering on the target filesystem
IMHO.
Regards.

[Buildroot] [PATCH 3/3] python-serial: fix shebang

[Peter Korsgaard]

>>>>> "Gustavo" == Gustavo Zacarias <gustavo@zacarias.com.ar> writes:

 Gustavo> On 05/06/2013 04:35 AM, Peter Korsgaard wrote:
 >> I see we have the same issue with usr/bin/smtpd.py from python itself,
 >> but I'm not sure how to fix that (besides running sed after
 >> installation).
 >> 
 >> Any idea?

 Gustavo> Some (all?) distributions just remove it, it's a smtp proxy,
 Gustavo> kind of a sample program, not something we want lingering on
 Gustavo> the target filesystem

Ok, thanks. Care to send a patch doing that?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Arnout Vandecappelle]

  Hi Gustavo,

  I usually don't bother reviewing your patches, but now I found something:

On 13/04/13 18:56, Gustavo Zacarias wrote:
> Switch to distribute fork since it's maintained and fixes a few issues
> while bumping to version 0.6.36.
> There's work upstream in re-merging with setuptools which is now
> supposedly maintained, see
> http://mail.python.org/pipermail/distutils-sig/2013-March/020126.html
>
> Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
> ---
>   .../python-setuptools-add-executable.patch         | 50 ++++++++++++++++++++++
>   package/python-setuptools/python-setuptools.mk     |  8 ++--
>   2 files changed, 54 insertions(+), 4 deletions(-)
>   create mode 100644 package/python-setuptools/python-setuptools-add-executable.patch
>
> diff --git a/package/python-setuptools/python-setuptools-add-executable.patch b/package/python-setuptools/python-setuptools-add-executable.patch
> new file mode 100644
> index 0000000..a14a3ed
> --- /dev/null
> +++ b/package/python-setuptools/python-setuptools-add-executable.patch
> @@ -0,0 +1,50 @@
> +Add a new --executable option to distribute so that we can
> +force the shebang line in installed python scripts.
> +
> +Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
> +
> +diff -Nura distribute-0.6.36.orig/setuptools/command/install.py distribute-0.6.36/setuptools/command/install.py
> +--- distribute-0.6.36.orig/setuptools/command/install.py	2013-04-13 09:46:23.160823598 -0300
> ++++ distribute-0.6.36/setuptools/command/install.py	2013-04-13 11:17:28.418841986 -0300
> +@@ -6,6 +6,7 @@
> +     """Use easy_install to install the package, w/dependencies"""
> +
> +     user_options = _install.user_options + [
> ++        ('executable=', 'e', "specify final destination interpreter path"),
> +         ('old-and-unmanageable', None, "Try not to use this!"),
> +         ('single-version-externally-managed', None,
> +             "used by system package builders to create 'flat' eggs"),
> +@@ -21,6 +22,7 @@
> +
> +     def initialize_options(self):
> +         _install.initialize_options(self)
> ++	self.executable = None

  Mixing tabs with spaces... Also a few other instances in the rest of 
this patch.

> +         self.old_and_unmanageable = None
> +         self.single_version_externally_managed = None
> +         self.no_compile = None  # make DISTUTILS_DEBUG work right!
> +diff -Nura distribute-0.6.36.orig/setuptools/command/install_scripts.py distribute-0.6.36/setuptools/command/install_scripts.py
> +--- distribute-0.6.36.orig/setuptools/command/install_scripts.py	2013-04-13 09:46:23.160823598 -0300
> ++++ distribute-0.6.36/setuptools/command/install_scripts.py	2013-04-13 11:24:32.305416400 -0300
> +@@ -10,6 +10,13 @@
> +     def initialize_options(self):
> +         _install_scripts.initialize_options(self)
> +         self.no_ep = False
> ++	self.executable = None
> ++
> ++    def finalize_options(self):
> ++        _install_scripts.finalize_options(self)
> ++        self.set_undefined_options('install',
> ++	    ('executable','executable')
> ++	)
> +
> +     def run(self):
> +         from setuptools.command.easy_install import get_script_args
> +@@ -31,6 +38,8 @@
> +         )
> +         bs_cmd = self.get_finalized_command('build_scripts')
> +         executable = getattr(bs_cmd,'executable',sys_executable)
> ++	if self.executable is not None:
> ++	    executable = self.executable
> +         is_wininst = getattr(
> +             self.get_finalized_command("bdist_wininst"), '_is_running', False
> +         )
> diff --git a/package/python-setuptools/python-setuptools.mk b/package/python-setuptools/python-setuptools.mk
> index 25fddb9..64f6ded 100644
> --- a/package/python-setuptools/python-setuptools.mk
> +++ b/package/python-setuptools/python-setuptools.mk
> @@ -4,9 +4,9 @@
>   #
>   #############################################################
>
> -PYTHON_SETUPTOOLS_VERSION = 0.6c11
> -PYTHON_SETUPTOOLS_SOURCE  = setuptools-$(PYTHON_SETUPTOOLS_VERSION).tar.gz
> -PYTHON_SETUPTOOLS_SITE    = http://pypi.python.org/packages/source/s/setuptools
> +PYTHON_SETUPTOOLS_VERSION = 0.6.36
> +PYTHON_SETUPTOOLS_SOURCE  = distribute-$(PYTHON_SETUPTOOLS_VERSION).tar.gz

  I would add a comment here, e.g.

# "distribute" is a fork of the unmaintained setuptools package. There
# are plans to re-merge it into setuptools; if this happens, we can
# switch back to http://pypi.python.org/packages/source/s/setuptools

> +PYTHON_SETUPTOOLS_SITE    = http://pypi.python.org/packages/source/d/distribute
>   PYTHON_SETUPTOOLS_DEPENDENCIES = python
>
>   define HOST_PYTHON_SETUPTOOLS_BUILD_CMDS
> @@ -24,7 +24,7 @@ endef
>
>   define PYTHON_SETUPTOOLS_INSTALL_TARGET_CMDS
>   	(cd $(@D); PYTHONPATH="$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages"\
> -	$(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
> +	$(HOST_DIR)/usr/bin/python setup.py install --executable=/usr/bin/python --single-version-externally-managed --root=/ --prefix=$(TARGET_DIR)/usr)

  Please split this long line. Also, maybe it's worth explaining why we 
want --single-version-externally-managed?


  Regards,
  Arnout

>   endef
>
>   $(eval $(generic-package))
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Gustavo Zacarias]

On 04/16/2013 03:27 AM, Arnout Vandecappelle wrote:

>  Mixing tabs with spaces... Also a few other instances in the rest of
> this patch.

That's probably my vim settings doing something evil, i don't have a
special case for python, i'll fix it.

>  I would add a comment here, e.g.
> 
> # "distribute" is a fork of the unmaintained setuptools package. There
> # are plans to re-merge it into setuptools; if this happens, we can
> # switch back to http://pypi.python.org/packages/source/s/setuptools

Fair enough.

>  Please split this long line. Also, maybe it's worth explaining why we
> want --single-version-externally-managed?

That's for the trick to work, when python builds an egg (zip file) in
the build phase not only are we wasting time doing double decompression
later on the target but we are also finishing up the package (kind of a
mini-install phase) into the egg.
That makes the trick fail (probably solvable by doing --execute in the
build itself, didn't try since it's not the way IMHO).
I'll add a note since it's required for other python packages that do
use setuptools/distribution.
V2 when Thomas confirms it solves his issues with lava-test, did it for
me in my tests.
Regards.

[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Peter Korsgaard]

>>>>> "Gustavo" == Gustavo Zacarias <gustavo@zacarias.com.ar> writes:

Hi,

 >> Please split this long line. Also, maybe it's worth explaining why we
 >> want --single-version-externally-managed?

 Gustavo> That's for the trick to work, when python builds an egg (zip
 Gustavo> file) in the build phase not only are we wasting time doing
 Gustavo> double decompression later on the target but we are also
 Gustavo> finishing up the package (kind of a mini-install phase) into
 Gustavo> the egg.

 Gustavo> That makes the trick fail (probably solvable by doing
 Gustavo> --execute in the build itself, didn't try since it's not the
 Gustavo> way IMHO).  I'll add a note since it's required for other
 Gustavo> python packages that do use setuptools/distribution.  V2 when
 Gustavo> Thomas confirms it solves his issues with lava-test, did it
 Gustavo> for me in my tests.

Did this ever get tested? Thomas, could you send your ack / Gustavo,
could you send a v2 fixing the issues pointed out by Arnout?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Gustavo Zacarias]

On 05/06/2013 04:34 AM, Peter Korsgaard wrote:

> Did this ever get tested? Thomas, could you send your ack / Gustavo,
> could you send a v2 fixing the issues pointed out by Arnout?

It's partially solved for lava-test since it's pulling other
dependencies by itself - that should probably be solved via new
buildroot python packages since it's doing so because they're not found,
and it's not nice when a package starts downloading stuff at
build/install time.
Yes, a V2 is forthcoming addressing Arnout's comments and a few more
fixes/improvements.
You shouldn't have committed the fixes to python-bottle and
python-serial since they both depend on the patch to this package,
they'll probably fail now.
Regards.

[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Peter Korsgaard]

>>>>> "Gustavo" == Gustavo Zacarias <gustavo@zacarias.com.ar> writes:

 Gustavo> On 05/06/2013 04:34 AM, Peter Korsgaard wrote:
 >> Did this ever get tested? Thomas, could you send your ack / Gustavo,
 >> could you send a v2 fixing the issues pointed out by Arnout?

 Gustavo> It's partially solved for lava-test since it's pulling other
 Gustavo> dependencies by itself - that should probably be solved via new
 Gustavo> buildroot python packages since it's doing so because they're not found,
 Gustavo> and it's not nice when a package starts downloading stuff at
 Gustavo> build/install time.
 Gustavo> Yes, a V2 is forthcoming addressing Arnout's comments and a few more
 Gustavo> fixes/improvements.

Ok, great.

 Gustavo> You shouldn't have committed the fixes to python-bottle and
 Gustavo> python-serial since they both depend on the patch to this package,
 Gustavo> they'll probably fail now.

Hmm, odd - They built fine here:

>>> python-bottle 0.11.6 Building
(cd /home/peko/source/buildroot/output/build/python-bottle-0.11.6; /home/peko/source/buildroot/output/host/usr/bin/python setup.py build --executable=/usr/bin/python)
running build
running build_py
creating build
creating build/lib
copying bottle.py -> build/lib
running build_scripts
creating build/scripts-2.7
copying and adjusting bottle.py -> build/scripts-2.7
changing mode of build/scripts-2.7/bottle.py from 644 to 755

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 1/3] python-setuptools: forkbump and add workaround for shebang

[Gustavo Zacarias]

On 05/06/2013 08:30 AM, Peter Korsgaard wrote:
> Hmm, odd - They built fine here:

My bad, those are python distutils-based which has the --executable
functionality builtin (yes, there's more than one way of
building/installing modules in python, yes, it hurts).
Regards.