build-sys: minor issues

build-sys: minor issues

[Ruediger Meier]

Hi,

I have two minor issues with our build-sys:

1. ./configure --enable-all-programs
also triggers --enable-deprecated-last. Would be nice to exclude this 
deprecated stuff somehow.

2. make dist
changes files in source dir, which makes the git clone dirty. Would be 
nice if we could avoid this somehow.

cu,
Rudi

Re: build-sys: minor issues

[Ruediger Meier]

On Monday 02 June 2014, Ruediger Meier wrote:
> I have two minor issues with our build-sys:

Another issue:

3. --enable-gtk-doc does not work in out-of-tree build:

$ mkdir build 
$ cd build
$ ../configure --enable-gtk-doc 
$ make V=1
[...]
gtkdoc-mkhtml --path="/home/rudi/devel/util-linux/build/libmount/docs:/home/rudi/devel/util-linux/build/libmount/docs/xml:/home/rudi/devel/util-linux/build/../libmount/docs" 
\
                 \
                libmount \
                /home/rudi/devel/util-linux/build/../libmount/docs/libmount-docs.xml
/home/rudi/devel/util-linux/build/libmount/docs/version.xml:1: parser 
error : Start tag expected, '<' not found
2.24.900-7afc22
^
/home/rudi/devel/util-linux/build/../libmount/docs/libmount-docs.xml:59: 
element include: XInclude error : could not 
load /home/rudi/devel/util-linux/libmount/docs/xml/version.xml, and no 
fallback was found
make[2]: *** [html-build.stamp] Error 6
make[2]: Leaving directory 
`/home/rudi/devel/util-linux/build/libmount/docs'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/rudi/devel/util-linux/build'
make: *** [all] Error 2


BTW maybe we could consider about adding these docs to dist tarball to 
not bother users with gtk-doc deps? (But I know then it's even more 
complicated to rebuild cleanly in builddir when the existing one in 
dist srcdir is outdated.)

cu,
Rudi

Re: build-sys: minor issues

[Karel Zak]

On Mon, Jun 02, 2014 at 12:56:16AM +0200, Ruediger Meier wrote:
> 1. ./configure --enable-all-programs
> also triggers --enable-deprecated-last. Would be nice to exclude this 
> deprecated stuff somehow.

 fixed, thanks for your report.

> 2. make dist
> changes files in source dir, which makes the git clone dirty. Would be 
> nice if we could avoid this somehow.

 What files? I see po/ stuff modification and it's more political
 issue than technical. The reason is that I update po/ only once
 before release (to avoid many many huge commits with irrelevant
 changes).

    Karel


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: build-sys: minor issues

[Karel Zak]

On Mon, Jun 02, 2014 at 02:33:38AM +0200, Ruediger Meier wrote:
> On Monday 02 June 2014, Ruediger Meier wrote:
> > I have two minor issues with our build-sys:
> 
> Another issue:
> 
> 3. --enable-gtk-doc does not work in out-of-tree build:

 I'll try to fix it (it worked time ago if I good remember). Frankly,
 the way how is gtk-doc designed to work with autotools sucks...

 BTW, our TODO:

   * use something better than gtk-doc for libmount and libblkid (doxyden?)

  ;-)

> BTW maybe we could consider about adding these docs to dist tarball to 
> not bother users with gtk-doc deps? (But I know then it's even more 

 Hmm... not sure, I guess the number of people who care about libs
 docs is pretty small and for mainstream distros is the dependence on
 gtk-doc no issue.

> complicated to rebuild cleanly in builddir when the existing one in 
> dist srcdir is outdated.)

    Karel


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: build-sys: minor issues

[Ruediger Meier]

On Monday 02 June 2014, Karel Zak wrote:
> On Mon, Jun 02, 2014 at 12:56:16AM +0200, Ruediger Meier wrote:
> > 1. ./configure --enable-all-programs
> > also triggers --enable-deprecated-last. Would be nice to exclude
> > this deprecated stuff somehow.
>
>  fixed, thanks for your report.

Thanks! All the recent build-sys tweaks were really helpful!

> > 2. make dist
> > changes files in source dir, which makes the git clone dirty. Would
> > be nice if we could avoid this somehow.
>
>  What files? I see po/ stuff modification and it's more political
>  issue than technical. The reason is that I update po/ only once
>  before release (to avoid many many huge commits with irrelevant
>  changes).

I have to admit that I'm not really familar with this translations.
But IMO generally it's not nice that "make dist" pollutes the git
clone.

I see 3 possibilities how to avoid this.

1. Don't track these po files in git at all. Always generate in
   builddir. Put them into dist only.
2. Don't update them with "make dist". Add new target "make update-po".
3. Keep them in srcdir but update to builddir. Add a script
   "commit-po-updates" to manulally move them from builddir to
   srcdir from time to time. 

My use case is a cronjob like this
  git pull origin
  ./autogen.sh && ./configure && make dist
  "push-tarball-to-build-host" 

Because of the current git pollution I have to do always "git 
reset --hard" before "autogen.sh" to not have "-dirty" suffixed version 
numbers.

cu,
Rudi 

Re: build-sys: minor issues

[Ruediger Meier]

On Monday 02 June 2014, Karel Zak wrote:
> On Mon, Jun 02, 2014 at 02:33:38AM +0200, Ruediger Meier wrote:
> > On Monday 02 June 2014, Ruediger Meier wrote:
> > > I have two minor issues with our build-sys:
> >
> > Another issue:
> >
> > 3. --enable-gtk-doc does not work in out-of-tree build:
>
>  I'll try to fix it (it worked time ago if I good remember). Frankly,
>  the way how is gtk-doc designed to work with autotools sucks...

Thanks.

>  BTW, our TODO:
>
>    * use something better than gtk-doc for libmount and libblkid
> (doxyden?)
>
>   ;-)

So maybe we could re-think the point below when doxygen epoch will come.

> > BTW maybe we could consider about adding these docs to dist tarball
> > to not bother users with gtk-doc deps?
>
>  Hmm... not sure, I guess the number of people who care about libs
>  docs is pretty small and for mainstream distros is the dependence on
>  gtk-doc no issue.

I know it's debatable. Just in ideal world the lib interfaces (and docs) 
are fixed for a certain release and do not depend on build host. This 
is shown more clearly if you distribute the fixed documentations along 
with the sources. It's like projects with help2man-man pages often 
distribute these man pages too.

Another minor advantage is that users would be able to compare the docs 
across serveral util-linux versions without need to build all the docs. 
(Might be difficult if an old util-linux build-sys does not work 
anymore on recent host).

cu,
Rudi

Re: build-sys: minor issues

[Karel Zak]

On Mon, Jun 02, 2014 at 02:33:38AM +0200, Ruediger Meier wrote:
> On Monday 02 June 2014, Ruediger Meier wrote:
> > I have two minor issues with our build-sys:
> 
> Another issue:
> 
> 3. --enable-gtk-doc does not work in out-of-tree build:
> 
> $ mkdir build 
> $ cd build
> $ ../configure --enable-gtk-doc 
> $ make V=1
> [...]
> gtkdoc-mkhtml --path="/home/rudi/devel/util-linux/build/libmount/docs:/home/rudi/devel/util-linux/build/libmount/docs/xml:/home/rudi/devel/util-linux/build/../libmount/docs" 
> \
>                  \
>                 libmount \
>                 /home/rudi/devel/util-linux/build/../libmount/docs/libmount-docs.xml
> /home/rudi/devel/util-linux/build/libmount/docs/version.xml:1: parser 
> error : Start tag expected, '<' not found
> 2.24.900-7afc22
> ^
> /home/rudi/devel/util-linux/build/../libmount/docs/libmount-docs.xml:59: 
> element include: XInclude error : could not 
> load /home/rudi/devel/util-linux/libmount/docs/xml/version.xml, and no 
> fallback was found

 Fixed.

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: build-sys: minor issues

[Karel Zak]

On Mon, Jun 02, 2014 at 12:33:23PM +0200, Ruediger Meier wrote:
> On Monday 02 June 2014, Karel Zak wrote:
> > On Mon, Jun 02, 2014 at 12:56:16AM +0200, Ruediger Meier wrote:
> > > 1. ./configure --enable-all-programs
> > > also triggers --enable-deprecated-last. Would be nice to exclude
> > > this deprecated stuff somehow.
> >
> >  fixed, thanks for your report.
> 
> Thanks! All the recent build-sys tweaks were really helpful!
> 
> > > 2. make dist
> > > changes files in source dir, which makes the git clone dirty. Would
> > > be nice if we could avoid this somehow.
> >
> >  What files? I see po/ stuff modification and it's more political
> >  issue than technical. The reason is that I update po/ only once
> >  before release (to avoid many many huge commits with irrelevant
> >  changes).
> 
> I have to admit that I'm not really familar with this translations.
> But IMO generally it's not nice that "make dist" pollutes the git
> clone.

 I see two issues with po/Makefile:

 * "make" generates po/*.gmo files into srcdir by

     cd $(srcdir) && rm -f $${lang}.gmo && $(GMSGFMT) ...

 * "make dist" updates po/*.po files in srcdir by

     test "$(srcdir)" = . && cdcmd="" || cdcmd="cd $(srcdir) && "; \
     echo "$${cdcmd}$(MSGMERGE_UPDATE) $(MSGMERGE_OPTIONS) 


 The problem is that we don't control this Makefile, it's created by 
 autopoint and maintained by gettext upstream. 
 
 Yes, it's mistake to modify $srcdir.

> I see 3 possibilities how to avoid this.
> 
> 1. Don't track these po files in git at all. Always generate in
>    builddir. Put them into dist only.

 No, I'm sure we want to have .po files in the git, it's part of the package.

> 2. Don't update them with "make dist". Add new target "make update-po".

 Well, I use "make distcheck" to generate the final release tarballs
 and it would be nice to keep it usable :-)

> 3. Keep them in srcdir but update to builddir. Add a script
>    "commit-po-updates" to manulally move them from builddir to
>    srcdir from time to time. 
> 
> My use case is a cronjob like this
>   git pull origin
>   ./autogen.sh && ./configure && make dist
>   "push-tarball-to-build-host" 
> 
> Because of the current git pollution I have to do always "git 
> reset --hard" before "autogen.sh" to not have "-dirty" suffixed version 
> numbers.

 What about to improve ./tools/git-version-gen to not use -dirty suffix when
 only po/*.po stuff is modified? 
 
 Note that git-version-gen is our script, and it just for developers,
 no problem to change it.
 
    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: build-sys: minor issues

[Ruediger Meier]

On Monday 02 June 2014, Ruediger Meier wrote:
> I have two minor issues with our build-sys:

Another issue:

4. --disable-all-programs disables pylibmount forever

--with-python   does not turn it on again
--enable-pylibmount  does not exist

or do I miss something?

Re: build-sys: minor issues

[Benno Schulenberg]

On Tue, Jun 3, 2014, at 13:27, Karel Zak wrote:
> On Mon, Jun 02, 2014 at 12:33:23PM +0200, Ruediger Meier wrote:
> > 1. Don't track these po files in git at all. Always generate in
> >    builddir. Put them into dist only.
> 
>  No, I'm sure we want to have .po files in the git, it's part of the package.

Well, I think it's nice to have copies of those files in many repos around the
globe, but it's not really needed.  And thus several GNU packages (coreutils,
grep, tar) have chosen to drop the PO files from their version control.  The
files get fetched from the TP at release time.

> > 2. Don't update them with "make dist". Add new target "make update-po".
> 
>  Well, I use "make distcheck" to generate the final release tarballs
>  and it would be nice to keep it usable :-)

Easy: add another target, "make release", that does an update-po and
then distcheck.  :)

Benno

-- 
http://www.fastmail.fm - Send your email first class

Re: build-sys: minor issues

[Karel Zak]

On Tue, Jun 03, 2014 at 09:46:14PM +0200, Benno Schulenberg wrote:
> 
> On Tue, Jun 3, 2014, at 13:27, Karel Zak wrote:
> > On Mon, Jun 02, 2014 at 12:33:23PM +0200, Ruediger Meier wrote:
> > > 1. Don't track these po files in git at all. Always generate in
> > >    builddir. Put them into dist only.
> > 
> >  No, I'm sure we want to have .po files in the git, it's part of the package.
> 
> Well, I think it's nice to have copies of those files in many repos around the
> globe, but it's not really needed.  And thus several GNU packages (coreutils,
> grep, tar) have chosen to drop the PO files from their version control.  The
> files get fetched from the TP at release time.

 I don't like it, there is many situations when people create their
 private "release", run "make install" etc. -- the dependence on TP
 sucks in this case.

> > > 2. Don't update them with "make dist". Add new target "make update-po".
> > 
> >  Well, I use "make distcheck" to generate the final release tarballs
> >  and it would be nice to keep it usable :-)
> 
> Easy: add another target, "make release", that does an update-po and
> then distcheck.  :)

 I don't think it is so easy, we don't control po/Makefile, it's
 generated by gettext (autopoint) stuff and it's difficult to override
 the generated targets.
 
 It would be possible to remove "po" from automake SUBDIRS and call
 the targets by "make -C po" from our top level Makefile, but it's
 also not so easy because automake does not provide hooks or -local
 targets for all things (for example there is no dist-local) etc.


 Maybe the best way would be to maintain our po/Makemodule.am and do
 not use the autopoint junk at all. Then we can:
 
  * keeps things in $builddir
  * update po stuff when it's really necessary
  * fix the problem with MKDIR_P
  * do not depend on ugly gettext versioning
   (AM_GNU_GETTEXT_VERSION) etc.  
  * make po/ Makefile non-recursive

 ... but it's to late for v2.25.

 Frankly, all the problem all the time we have with autotools are
 problems with gettext. The autoconf, automake and libtools are just
 fine, but gettext is probably from another universe.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: build-sys: minor issues

[Karel Zak]

On Tue, Jun 03, 2014 at 03:09:24PM +0200, Ruediger Meier wrote:
> On Monday 02 June 2014, Ruediger Meier wrote:
> > I have two minor issues with our build-sys:
> 
> Another issue:
> 
> 4. --disable-all-programs disables pylibmount forever
> 
> --with-python   does not turn it on again
> --enable-pylibmount  does not exist

 OK, I have added --enable-pylibmount, so you have to use:

 ./configure --disable-all-programs \
             --enable-pylibmount \
             --enable-libmount \
             --enable-libblkid

 to compile the binding.


> or do I miss something?

note that we don't have --enable-* for all utils. It's probably
overkill for small utils or utils expected everywhere.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: build-sys: minor issues

[Ruediger Meier]

On Tuesday 03 June 2014, Karel Zak wrote:
> On Mon, Jun 02, 2014 at 02:33:38AM +0200, Ruediger Meier wrote:
> >
> > Another issue:
> >
> > 3. --enable-gtk-doc does not work in out-of-tree build:
> > [...]

>
>  Fixed.

Thanks, it works.

For testing I've run it several times on travis farm from within 
distcheck. Unfortunately docs still fail randomly in 1 of 10 runs.

$ make distcheck
[...]
make[1]: Entering directory 
`/home/travis/build/rudimeier/util-linux/libblkid/docs'
make[1]: Circular scan-build.stamp <- libblkid-sections.txt dependency 
dropped.
  GEN    scan-build.stamp
[...]
Making all in libblkid/docs
make[3]: Entering directory 
`/home/travis/build/rudimeier/util-linux/util-linux-2.24.913-183e/_build/libblkid/docs'
gtk-doc: Preparing build
cp: cannot create regular file `./libblkid-sections.txt': Permission 
denied
make[3]: *** [scan-build.stamp] Error 1

Note that this "Permission denied" also happens in successful runs.

Here full logs of good and bad run:
https://s3.amazonaws.com/archive.travis-ci.org/jobs/26722236/log.txt
https://s3.amazonaws.com/archive.travis-ci.org/jobs/26722239/log.txt


Don't know whether this happens only out-of-tree or because of parallel 
build. "Permission denied" in distcheck always smells like writing to 
srcdir. Maybe you have already an idea what it could be or I would just 
add debug messages and watch it.

cu,
Rudi

Re: build-sys: minor issues

[Ruediger Meier]

On Wednesday 04 June 2014, Karel Zak wrote:
> On Tue, Jun 03, 2014 at 03:09:24PM +0200, Ruediger Meier wrote:
> > On Monday 02 June 2014, Ruediger Meier wrote:
> > > I have two minor issues with our build-sys:
> >
> > Another issue:
> >
> > 4. --disable-all-programs disables pylibmount forever
> >
> > --with-python   does not turn it on again
> > --enable-pylibmount  does not exist
>
>  OK, I have added --enable-pylibmount, so you have to use:
>
>  ./configure --disable-all-programs \
>              --enable-pylibmount \
>              --enable-libmount \
>              --enable-libblkid
>
>  to compile the binding.

Thanks!

> > or do I miss something?
>
> note that we don't have --enable-* for all utils. It's probably
> overkill for small utils or utils expected everywhere.

Ah, I thought you've had added --enable-* more generically for 
everything lately.

But no problem in doubt there is still
$ ./configure --enable-all-programs
$ make xyz

cu,
Rudi