Re: [RFC] libselinux: Use Python distutils to install SELinux python bindings

[Nicolas Iooss <nicolas.iooss@m4x.org>]

Hi,

Thanks for your patch. Some comments below.

On Fri, Jun 7, 2019 at 5:35 PM Petr Lautrbach <plautrba@redhat.com> wrote:
>
> Follow officially documented way how to build C extension modules using
> distutils - https://docs.python.org/3.8/extending/building.html#building
>
> Fixes:
>
> - https://bugzilla.redhat.com/show_bug.cgi?id=1715589 - selinux python module
> fails to load when it's built using SWIG-4.0:
>
> >>> import selinux
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib64/python3.7/site-packages/selinux/__init__.py", line 13, in <module>
>     from . import _selinux
> ImportError: cannot import name '_selinux' from 'selinux' (/usr/lib64/python3.7/site-packages/selinux/__init__.py)
> >>>
>
> SWIG-4.0 changed (again?) its behavior so that it uses: from . import _selinux
> which looks for _selinux module in the same directory as where __init__.py is -
> $(PYLIBDIR)/site-packages/selinux. But _selinux module is installed into
> $(PYLIBDIR)/site-packages/ since a9604c30a5e2f ("libselinux: Change the location
> of _selinux.so").
>
> On the other hand, the module can't be loaded when it's built using older SWIG.
> I'd use a symlink from $(PYLIBDIR)/site-packages/selinux to
> $(PYLIBDIR)/site-packages/ in order to fix that

I agree. For information, Debian sid still uses SWIG 3.0 (cf.
https://packages.debian.org/sid/swig), and this patch indeed breaks
things there ("make install-pywrap" succeeds but "import selinux" in
python fails to find module _selinux). Adding the symlink for _selinux
native module can be done with:

--- a/libselinux/src/Makefile
+++ b/libselinux/src/Makefile
@@ -175,6 +175,7 @@ install: all
 install-pywrap: pywrap
        $(PYTHON) setup.py install --prefix=$(PREFIX) `test -n
"$(DESTDIR)" && echo --root $(DESTDIR)`
        install -m 644 selinux.py $(DESTDIR)$(PYTHONLIBDIR)/selinux/__init__.py
+       ln -sf selinux/_selinux$(PYCEXT)
$(DESTDIR)$(PYTHONLIBDIR)/_selinux$(PYCEXT)

 install-rubywrap: rubywrap
        test -d $(DESTDIR)$(RUBYINSTALL) || install -m 755 -d
$(DESTDIR)$(RUBYINSTALL)

>
> - https://bugzilla.redhat.com/show_bug.cgi?id=1715756 - audit2why python module
> fails to build with Python 3.8
>
> - https://bugzilla.redhat.com/show_bug.cgi?id=1709576 - selinux python module
> doesn't provide any Python metadata
>
> Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
> ---
>  libselinux/src/.gitignore |  2 +-
>  libselinux/src/Makefile   | 37 ++++++++-----------------------------
>  libselinux/src/setup.py   | 24 ++++++++++++++++++++++++
>  3 files changed, 33 insertions(+), 30 deletions(-)
>  create mode 100644 libselinux/src/setup.py
>
> diff --git a/libselinux/src/.gitignore b/libselinux/src/.gitignore
> index 4dcc3b3b..428afe5a 100644
> --- a/libselinux/src/.gitignore
> +++ b/libselinux/src/.gitignore
> @@ -1,4 +1,4 @@
>  selinux.py
> -selinuxswig_wrap.c
> +selinuxswig_python_wrap.c
>  selinuxswig_python_exception.i
>  selinuxswig_ruby_wrap.c
> diff --git a/libselinux/src/Makefile b/libselinux/src/Makefile
> index e9ed0383..826c830c 100644
> --- a/libselinux/src/Makefile
> +++ b/libselinux/src/Makefile
> @@ -36,7 +36,7 @@ TARGET=libselinux.so
>  LIBPC=libselinux.pc
>  SWIGIF= selinuxswig_python.i selinuxswig_python_exception.i
>  SWIGRUBYIF= selinuxswig_ruby.i
> -SWIGCOUT= selinuxswig_wrap.c
> +SWIGCOUT= selinuxswig_python_wrap.c
>  SWIGPYOUT= selinux.py
>  SWIGRUBYCOUT= selinuxswig_ruby_wrap.c
>  SWIGLOBJ:= $(patsubst %.c,$(PYPREFIX)%.lo,$(SWIGCOUT))
> @@ -55,7 +55,7 @@ ifeq ($(LIBSEPOLA),)
>          LDLIBS_LIBSEPOLA := -l:libsepol.a
>  endif
>
> -GENERATED=$(SWIGCOUT) $(SWIGRUBYCOUT) selinuxswig_python_exception.i
> +GENERATED=$(SWIGCOUT) $(SWIGRUBYCOUT) $(SWIGCOUT) selinuxswig_python_exception.i
>  SRCS= $(filter-out $(GENERATED) audit2why.c, $(sort $(wildcard *.c)))
>
>  MAX_STACK_SIZE=32768
> @@ -125,25 +125,18 @@ DISABLE_FLAGS+= -DNO_ANDROID_BACKEND
>  SRCS:= $(filter-out label_backends_android.c, $(SRCS))
>  endif
>
> -SWIG = swig -Wall -python -o $(SWIGCOUT) -outdir ./ $(DISABLE_FLAGS)
> -
>  SWIGRUBY = swig -Wall -ruby -o $(SWIGRUBYCOUT) -outdir ./ $(DISABLE_FLAGS)
>
>  all: $(LIBA) $(LIBSO) $(LIBPC)
>
> -pywrap: all $(SWIGFILES) $(AUDIT2WHYSO)
> +pywrap: all selinuxswig_python_exception.i
> +       CFLAGS="$(SWIG_CFLAGS)" $(PYTHON) setup.py build_ext -I $(DESTDIR)$(INCLUDEDIR) -L $(DESTDIR)$(LIBDIR)
>
>  rubywrap: all $(SWIGRUBYSO)
>
> -$(SWIGLOBJ): $(SWIGCOUT)
> -       $(CC) $(CFLAGS) $(SWIG_CFLAGS) $(PYINC) -fPIC -DSHARED -c -o $@ $<
> -
>  $(SWIGRUBYLOBJ): $(SWIGRUBYCOUT)
>         $(CC) $(CFLAGS) $(SWIG_CFLAGS) $(RUBYINC) -fPIC -DSHARED -c -o $@ $<
>
> -$(SWIGSO): $(SWIGLOBJ)
> -       $(CC) $(CFLAGS) $(LDFLAGS) -L. -shared -o $@ $< -lselinux $(PYLIBS)
> -
>  $(SWIGRUBYSO): $(SWIGRUBYLOBJ)
>         $(CC) $(CFLAGS) $(LDFLAGS) -L. -shared -o $@ $^ -lselinux $(RUBYLIBS)
>
> @@ -161,29 +154,15 @@ $(LIBPC): $(LIBPC).in ../VERSION
>  selinuxswig_python_exception.i: ../include/selinux/selinux.h
>         bash -e exception.sh > $@ || (rm -f $@ ; false)
>
> -$(AUDIT2WHYLOBJ): audit2why.c
> -       $(CC) $(filter-out -Werror, $(CFLAGS)) $(PYINC) -fPIC -DSHARED -c -o $@ $<
> -
> -$(AUDIT2WHYSO): $(AUDIT2WHYLOBJ) $(LIBSEPOLA)
> -       $(CC) $(CFLAGS) $(LDFLAGS) -L. -shared -o $@ $^ -lselinux $(LDLIBS_LIBSEPOLA) $(PYLIBS) -Wl,-soname,audit2why.so,--version-script=audit2why.map,-z,defs
> -
>  %.o:  %.c policy.h
>         $(CC) $(CFLAGS) $(TLSFLAGS) -c -o $@ $<
>
>  %.lo:  %.c policy.h
>         $(CC) $(CFLAGS) -fPIC -DSHARED -c -o $@ $<
>
> -$(SWIGCOUT): $(SWIGIF)
> -       $(SWIG) $<
> -
> -$(SWIGPYOUT): $(SWIGCOUT)
> -
>  $(SWIGRUBYCOUT): $(SWIGRUBYIF)
>         $(SWIGRUBY) $<
>
> -swigify: $(SWIGIF)
> -       $(SWIG) $<
> -
>  install: all
>         test -d $(DESTDIR)$(LIBDIR) || install -m 755 -d $(DESTDIR)$(LIBDIR)
>         install -m 644 $(LIBA) $(DESTDIR)$(LIBDIR)
> @@ -194,10 +173,8 @@ install: all
>         ln -sf --relative $(DESTDIR)$(SHLIBDIR)/$(LIBSO) $(DESTDIR)$(LIBDIR)/$(TARGET)
>
>  install-pywrap: pywrap
> -       test -d $(DESTDIR)$(PYTHONLIBDIR)/selinux || install -m 755 -d $(DESTDIR)$(PYTHONLIBDIR)/selinux
> -       install -m 755 $(SWIGSO) $(DESTDIR)$(PYTHONLIBDIR)/_selinux$(PYCEXT)
> -       install -m 755 $(AUDIT2WHYSO) $(DESTDIR)$(PYTHONLIBDIR)/selinux/audit2why$(PYCEXT)
> -       install -m 644 $(SWIGPYOUT) $(DESTDIR)$(PYTHONLIBDIR)/selinux/__init__.py
> +       $(PYTHON) setup.py install --prefix=$(PREFIX) `test -n "$(DESTDIR)" && echo --root $(DESTDIR)`
> +       install -m 644 selinux.py $(DESTDIR)$(PYTHONLIBDIR)/selinux/__init__.py
>
>  install-rubywrap: rubywrap
>         test -d $(DESTDIR)$(RUBYINSTALL) || install -m 755 -d $(DESTDIR)$(RUBYINSTALL)
> @@ -208,6 +185,8 @@ relabel:
>
>  clean-pywrap:
>         -rm -f $(SWIGLOBJ) $(SWIGSO) $(AUDIT2WHYLOBJ) $(AUDIT2WHYSO)
> +       $(PYTHON) setup.py clean
> +       -rm -rf build *~ \#* *pyc .#*
>
>  clean-rubywrap:
>         -rm -f $(SWIGRUBYLOBJ) $(SWIGRUBYSO)
> diff --git a/libselinux/src/setup.py b/libselinux/src/setup.py
> new file mode 100644
> index 00000000..b12e7869
> --- /dev/null
> +++ b/libselinux/src/setup.py
> @@ -0,0 +1,24 @@
> +#!/usr/bin/python3
> +
> +from distutils.core import Extension, setup
> +
> +setup(
> +    name="selinux",
> +    version="2.9",
> +    description="SELinux python 3 bindings",
> +    author="SELinux Project",
> +    author_email="selinux@vger.kernel.org",
> +    ext_modules=[
> +        Extension('selinux._selinux',
> +                  sources=['selinuxswig_python.i'],
> +                  include_dirs=['../include'],
> +                  library_dirs=['.'],
> +                  libraries=['selinux']),
> +        Extension('selinux.audit2why',
> +                  sources=['audit2why.c'],
> +                  include_dirs=['../include'],
> +                  library_dirs=['.'],
> +                  libraries=['selinux'],
> +                  extra_link_args=['-l:libsepol.a'])
> +    ],
> +)
> --
> 2.22.0.rc3

When building audit2why extension, the version script "audit2why.map"
is no longer used. This might cause libsepol's symbols to "leak" to
the global symbol namespace used by the linker when audit2why
extension gets loaded (I have not tested if this really happens and I
may have missed some details).
Could you use extra_link_args=['-l:libsepol.a',
'-Wl,--version-script=audit2why.map'] and test if this works?

With the symlink and the version script, I managed to build the
extension on Arch Linux and Debian 9 and performed some minimal tests.

Thanks,
Nicolas