[Cocci] Replacing suffix rules in make scripts?

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

Hello,

I have achieved a bit of progress with one experiment for build script adjustments.
I would like to try further changes out. Unfortunately, I stumbled on an error
message like the following from the program ?GNU Make 4.2.1-1.7? then.

?
/usr/bin/ocamlopt.opt ?
File "cocci.ml", line 1:
Error: Could not find the .cmi file for interface cocci.mli.
make[3]: *** [Makefile:533: cocci.cmx] Error 2
?


It seems that the following code is relevant in this use case.

?
.ml.cmo:
	$(OCAMLC_CMD) -c $<
.mli.cmi:
	$(OCAMLC_CMD) -c $<
.ml.cmx:
	$(OCAMLOPT_CMD) -c $<
?


Other documentation provides this information:
??
Suffix rules cannot have any prerequisites of their own.
??


So I get the impression that an other build approach will be safer there.
How do you think about to add a dependency specification so that interface
descriptions will be always compiled before the corresponding OCaml code?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> So I get the impression that an other build approach will be safer there.
> How do you think about to add a dependency specification so that interface
> descriptions will be always compiled before the corresponding OCaml code?

Hello,

I have achieved another bit of progress with my attempt to replace suffix rules
by pattern rules in some make scripts.
I stumbled on the next software development challenge as follows.


elfring at Sonne:~/Projekte/Coccinelle/20160205> LANG=C make V=1
?
make[6]: Entering directory '/home/elfring/Projekte/Coccinelle/20160205/globals'
/usr/bin/ocamlopt.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.ml
File "iteration.ml", line 36, characters 4-28:
Error: Unbound module Flag
make[6]: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:12: iteration.cmx] Error 2
?


* Do I overlook an ?obvious? software dependency once more in this use case?

* How should the relevant details be resolved here?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[Julia Lawall]

On Wed, 14 Jun 2017, SF Markus Elfring wrote:

> > So I get the impression that an other build approach will be safer there.
> > How do you think about to add a dependency specification so that interface
> > descriptions will be always compiled before the corresponding OCaml code?
>
> Hello,
>
> I have achieved another bit of progress with my attempt to replace suffix rules
> by pattern rules in some make scripts.
> I stumbled on the next software development challenge as follows.
>
>
> elfring at Sonne:~/Projekte/Coccinelle/20160205> LANG=C make V=1
> ?
> make[6]: Entering directory '/home/elfring/Projekte/Coccinelle/20160205/globals'
> /usr/bin/ocamlopt.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.ml
> File "iteration.ml", line 36, characters 4-28:
> Error: Unbound module Flag
> make[6]: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:12: iteration.cmx] Error 2
> ?
>
>
> * Do I overlook an ?obvious? software dependency once more in this use case?
>
> * How should the relevant details be resolved here?

If you have changed things, then perhaps you have caused the problem?

julia

>
> Regards,
> Markus
> _______________________________________________
> Cocci mailing list
> Cocci at systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> If you have changed things, then perhaps you have caused the problem?

This is also another possibility in principle.

It seems that I wonder again why some source files should be compiled without
interface description files there.


An other script variant showed the following result.


elfring at Sonne:~/Projekte/Coccinelle/20160205/globals> LANG=C make iteration.cmx V=1
My test command:
/usr/bin/ocamlopt.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml
File "flag.ml", line 24, characters 17-36:
Error: Unbound module Commands
make: *** [Makefile:62: flag.cmx] Error 2


Can it be that the corresponding dependency specifications were incomplete anyhow
in this make script for the direct use at such a place?

Would you like to help any more to get these dependencies more consistent?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> If you have changed things, then perhaps you have caused the problem?

I have found an approach for the adjustment of affected build scripts
where I can improve the involved dependency specifications in a few
subdirectories on my own to some degree.
But I guess that a directory like ?parsing_cocci? provides bigger development
challenges to make the dependencies safer there.
I am curious on how a dependency for a software interface like ?Visitor_ast0_types?
should be expressed for so many source files in a consistent way.

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[Julia Lawall]

On Thu, 15 Jun 2017, SF Markus Elfring wrote:

> > If you have changed things, then perhaps you have caused the problem?
>
> I have found an approach for the adjustment of affected build scripts
> where I can improve the involved dependency specifications in a few
> subdirectories on my own to some degree.
> But I guess that a directory like ?parsing_cocci? provides bigger development
> challenges to make the dependencies safer there.
> I am curious on how a dependency for a software interface like ?Visitor_ast0_types?
> should be expressed for so many source files in a consistent way.

What problem are you trying to solve?  Why do the dependency
specifications need improving?

julia

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

>> I am curious on how a dependency for a software interface like ?Visitor_ast0_types?
>> should be expressed for so many source files in a consistent way.
> 
> What problem are you trying to solve?

I would like to achieve something more for the topics which were also registered
in the issue tracker.


> Why do the dependency specifications need improving?

Examples:

elfring at Sonne:~/Projekte/Coccinelle/20160205/parsing_cocci> LANG=C make V=1
/usr/bin/ocamldep *.mli *.ml > .depend
/usr/bin/ocamlc.opt ?
File "_none_", line 1:
Error: Cannot find file flag_parsing_cocci.cmo
make: *** [Makefile:67: cocci_parser.cma] Error 2


elfring at Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make V=1
/usr/bin/ocamlc.opt ?
File "includes.mli", line 38, characters 46-60:
Error: Unbound module Ast_c
make: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:6: includes.cmi] Error 2


These messages indicate reasons why I am also fiddling with the build scripts
for this software again.

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[Julia Lawall]

On Fri, 16 Jun 2017, SF Markus Elfring wrote:

> >> I am curious on how a dependency for a software interface like ?Visitor_ast0_types?
> >> should be expressed for so many source files in a consistent way.
> >
> > What problem are you trying to solve?
>
> I would like to achieve something more for the topics which were also registered
> in the issue tracker.
>
>
> > Why do the dependency specifications need improving?
>
> Examples:
>
> elfring at Sonne:~/Projekte/Coccinelle/20160205/parsing_cocci> LANG=C make V=1
> /usr/bin/ocamldep *.mli *.ml > .depend
> /usr/bin/ocamlc.opt ?
> File "_none_", line 1:
> Error: Cannot find file flag_parsing_cocci.cmo
> make: *** [Makefile:67: cocci_parser.cma] Error 2
>
>
> elfring at Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make V=1
> /usr/bin/ocamlc.opt ?
> File "includes.mli", line 38, characters 46-60:
> Error: Unbound module Ast_c
> make: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:6: includes.cmi] Error 2
>
>
> These messages indicate reasons why I am also fiddling with the build scripts
> for this software again.

These command lines contain ... so I can't see what is being done.  But I
have never seen such a message, and no one else has ever reported such a
message.

julia

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

>> These messages indicate reasons why I am also fiddling with the build scripts
>> for this software again.
> 
> These command lines contain ... so I can't see what is being done.

I find the passed concrete parameters which I intentionally left out for
the really desired clarification not so interesting at the moment.

If the indicated software build dependencies would completely work as expected,
there were no questionable error messages about missing generated files.


> But I have never seen such a message, and no one else has ever reported
> such a message.

I am trying to update special implementation details in evolving build scripts.

I wonder then about corresponding ?side effects?. How much can interested
software developers help here?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> OK, I don't undertand what command you are doing to get the problem,

I showed how I could stumble on open issues.


> so I can't reproduce it, so I can't determine whether it is a real issue.

There are chances for the reproduction if you would also dare to fiddle
more with the build scripts (and the Autotools build system generally).
Which software combinations would you like to try out further there?


> If you want feedback, you have to give the complete command.

The mentioned command examples reference generated files.


> The OCaml makefile specifies an order in which the files should be
> compiled in the project.

I agree that the make scripts of the Coccinelle software contain also
some dependency specifications. Unfortunately, I observe that they are not
completely effective to avoid unwanted error messages.


> If you are sidestepping that ordering, then you are not doing things
> in the intended manner.

We got different views about corresponding software evolution.


> There is only an intent to support running make or make opt in the
> subdirectories or at the root.

I find this kind of restriction unnecessary.


> Perhaps Thierry will put a new make file that has some other properties
> (I don't know which ones) in the future.

I am curious on your ideas for this software area.


> But there is no intent to support anything else at the moment.

I hope that the desired support will also evolve together with solutions
for the remaining open issues.


> Only Thierry would do somehting about the build system,

How are the chances that additional software developers will care about
any more update candidates in this software?


> and if his existing work makes your efforts useless,

Would you like to share any related information for your plans?


> then you are just wasting your time.

This is also one of the usual risk in free software development, isn't it?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

>> I showed how I could stumble on open issues.
> 
> No you didn't.

I try then once more to help you to interpret the mentioned
error message examples.


The command ?ocamldep *.mli *.ml > .depend? was performed for
relevant subdirectories.
Unfortunately, this approach seems to be insufficient for the desired
safe integration of dependency information so far.


> I don't know what are the arguments to the ocaml compiler.

If you think in terms of the build system, you would know that this tool
needs input files.
Unfortunately, the current build scripts did not trigger the generation
of the desired prerequisites for the command examples at the end.


> What file are you trying to compile?

The attempted data processing steps can not succeed when required
files were not built before.
If these dependencies will be fixed, then we can worry if an error message
like ?Unbound module Ast_c? can occur because of other reasons.

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> The command ?ocamldep *.mli *.ml > .depend? was performed for
> relevant subdirectories.
> Unfortunately, this approach seems to be insufficient for the desired
> safe integration of dependency information so far.

How do you think about to work any more with dedicated files
for each source file by pattern rules like it is described in the manual
section ?Generating prerequisites automatically??
https://www.gnu.org/software/make/manual/html_node/Automatic-Prerequisites.html

Would it make sense to split the data from the command ?ocamldep? into
smaller units?


Do I stumble on unexpected software behaviour by the tool ?GNU Make 4.2.1-1.7?
on my openSUSE Tumbleweed system?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

>> Do I stumble on unexpected software behaviour by the tool ?GNU Make 4.2.1-1.7?
>> on my openSUSE Tumbleweed system?
> 
> Since you don't the complete command line, I have no idea whether the
> behavior is expected or not.

Can you get the impression from the mentioned error message examples
that the desired software build prerequisites failed anyhow?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

>> Can you get the impression from the mentioned error message examples
>> that the desired software build prerequisites failed anyhow?
> 
> Something fails in your situation, but without knowing the arguments to
> the compiler, I don't know what.

You could know also from the reduced information.


Is it really easier for you to see the following data again?

elfring at Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make V=1
/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I  ../parsing_cocci  -c includes.mli
File "includes.mli", line 38, characters 46-60:
Error: Unbound module Ast_c
make: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:6: includes.cmi] Error 2


Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> The command ?ocamldep *.mli *.ml > .depend? was performed for
> relevant subdirectories.
> Unfortunately, this approach seems to be insufficient for the desired
> safe integration of dependency information so far.

I have got another idea. The adjustment results were more promising for my
build script changes in other subdirectories.
Small make functions construct extra rules there.

If the used make rules from the ocamldep call do not produce the desired effects
for checks of prerequisites so far, I could try to convert the provided data
to an other style of dependency specification. Such generated make rules would
finally contain one useful command in each recipe.

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> OK, I don't undertand what command you are doing to get the problem,

Did we achieve a better common understanding in the meantime?


> so I can't reproduce it, so I can't determine whether it is a real issue.

* Are you really keen to achieve this also in your own test environments?

* How much does my build difficulty fit into your development attention
  and software change capacity?


> The OCaml makefile specifies an order in which the files should be
> compiled in the project.  If you are sidestepping that ordering, then you
> are not doing things in the intended manner.

I am just curious to fix the dependency checking (without unexpected
side effects) for build prerequisites somehow in the involved software components.

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> The OCaml makefile specifies an order in which the files should be
> compiled in the project.  If you are sidestepping that ordering, then you
> are not doing things in the intended manner.

I hope that some details were clarified to some degree in the meantime.


> There is only an intent to support running make or make opt in the
> subdirectories or at the root.  Perhaps Thierry will put a new make file
> that has some other properties (I don't know which ones) in the future.
> But there is no intent to support anything else at the moment.

I suggest to take also another look around the discussed build scripts.

Example:
https://github.com/coccinelle/coccinelle/blob/cd539cd12ad0674d5a8f4c33f6c21ae00aec8e57/globals/Makefile#L50


Now I wonder about the following test result for my current ?developer rules?.


elfring at Sonne:~/Projekte/Coccinelle/20160205/globals> make clean && LANG=C make --no-builtin-rules V=1 && make clean && LANG=C make --no-builtin-rules -j4 V=1
rm -f ?
/usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -a -o globals.cma config.cmo flag.cmo regexp_pcre.cmo regexp.cmo iteration.cmo
rm -f ?
/usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml
?
/usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.mli
make: *** No rule to make target 'flag.cmi', needed by 'iteration.cmo'.  Stop.
?


Why do I get such an error message for a parallel software build attempt
when the specified dependencies worked as expected for the non-parallel
build try before?

Do you get any more ideas on how to explain such a test surprise?

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> Why do I get such an error message for a parallel software build attempt
> when the specified dependencies worked as expected for the non-parallel
> build try before?

How do you think about the corresponding update suggestion
?Improve some dependency specifications in make rules??
https://github.com/elfring/Coccinelle-20160205/commit/9597598779f3604272621f6d42ead79eaafad2a8

Regards,
Markus

[Cocci] Replacing suffix rules in make scripts?

[SF Markus Elfring]

> So I get the impression that an other build approach will be safer there.
> How do you think about to add a dependency specification so that interface
> descriptions will be always compiled before the corresponding OCaml code?

I showed change possibilities which can improve the software situation
to some degree.

* Improve several dependency specifications in make rules
  https://github.com/coccinelle/coccinelle/pull/108

* Fix specifications for build dependencies between shared components
  https://github.com/coccinelle/coccinelle/issues/110

* Support references for the top-level source file directory in build scripts
  https://github.com/coccinelle/coccinelle/issues/104


Which ideas would you like to integrate from the available development approaches?

Regards,
Markus