Improving documentation for programming interfaces

Improving documentation for programming interfaces

[Markus Elfring]

Hello,

Linux supports some programming interfaces. Several functions are provided
as usual. Their application documentation is an ongoing development challenge.

Now I would like to clarify possibilities for the specification of desired
information together with data types besides properties which are handled by
the programming language “C” so far.
It seems that no customised attributes are supported at the moment.
Thus I imagine to specify helpful annotations as macros.

Example:
Some functions allocate resources to which a pointer (or handle) is returned.
I would find it nice then if such a pointer would contain also the background
information by which functions the resource should usually be released.

Can it become easier to determine such data?

Regards,
Markus

Re: Improving documentation for programming interfaces

[Theodore Y. Ts'o]

On Fri, Dec 20, 2019 at 02:30:10PM +0100, Markus Elfring wrote:
> Linux supports some programming interfaces. Several functions are provided
> as usual. Their application documentation is an ongoing development challenge.
> 
> Now I would like to clarify possibilities for the specification of desired
> information together with data types besides properties which are handled by
> the programming language “C” so far.
> It seems that no customised attributes are supported at the moment.
> Thus I imagine to specify helpful annotations as macros.
> 
> Example:
> Some functions allocate resources to which a pointer (or handle) is returned.
> I would find it nice then if such a pointer would contain also the background
> information by which functions the resource should usually be released.
> 
> Can it become easier to determine such data?

Markus,

It's unclear to me what you are requesting/proposing?  Can you be a
bit more concrete?

						- Ted

Re: Improving documentation for programming interfaces

[Markus Elfring]

>> Some functions allocate resources to which a pointer (or handle) is returned.
>> I would find it nice then if such a pointer would contain also the background
>> information by which functions the resource should usually be released.
>>
>> Can it become easier to determine such data?
…
> It's unclear to me what you are requesting/proposing?

I suggest to clarify combinations for object construction and proper resource release.


> Can you be a bit more concrete?

Further examples:
* kmalloc ⇒ kfree
* kobject_create ⇒ kobject_put
* device_register ⇒ put_device

Can preprocessor macros help to express any more relationships for similar function pairs?

Regards,
Markus

Re: Improving documentation for programming interfaces

[Theodore Y. Ts'o]

On Fri, Dec 20, 2019 at 05:23:27PM +0100, Markus Elfring wrote:
> >> Some functions allocate resources to which a pointer (or handle) is returned.
> >> I would find it nice then if such a pointer would contain also the background
> >> information by which functions the resource should usually be released.
> >>
> >> Can it become easier to determine such data?
> …
> > It's unclear to me what you are requesting/proposing?
> 
> I suggest to clarify combinations for object construction and proper resource release.
> 
> > Can you be a bit more concrete?
> 
> Further examples:
> * kmalloc ⇒ kfree
> * kobject_create ⇒ kobject_put
> * device_register ⇒ put_device
> 
> Can preprocessor macros help to express any more relationships for similar function pairs?

Sorry, this is still not making sense.  You said "a pointer would
contain also the background information by which the resource should
usually be released".  Huh?  There's no room for a pointer to also
store context of whether it was allocated using kmalloc, or malloc,
etc.

Did you have some concrete idea of how a preprocessor macros could be
used to perform what appears to be completely impractical?

And how would that information be used by the kernel?  And for what
benefit?  And can you show that the benefits will be worth the costs?

	      	      	   	- Ted

Re: Improving documentation for programming interfaces

[Markus Elfring]

>> Further examples:
>> * kmalloc ⇒ kfree
>> * kobject_create ⇒ kobject_put
>> * device_register ⇒ put_device
>>
>> Can preprocessor macros help to express any more relationships for similar function pairs?
>
> Sorry, this is still not making sense.

I suggest to reconsider such a view.


> …  You said "a pointer would
> contain also the background information by which the resource should
> usually be released".  Huh?  There's no room for a pointer to also
> store context of whether it was allocated using kmalloc, or malloc, etc.

There are metadata (software documentation) to consider in the source code.

You get usually informed by comments for some allocation functions
which is the corresponding resource release function.
Such comments can vary. Thus I would appreciate to work with these data
in a more structured format.


> Did you have some concrete idea of how a preprocessor macros could be
> used to perform what appears to be completely impractical?

A macro can be chosen to which function names can be passed.

Example:
TRIGGER_RELEASE_AFTER(kfree, kmalloc, kcalloc, kzalloc, kmemdup)

Appropriate data type definitions could eventually be generated
by related macros.


> And how would that information be used by the kernel?

Documentation purposes.


> And for what benefit?

Consistent representation of relationships in a structured way.


> And can you show that the benefits will be worth the costs?

I propose to improve possibilities also for the support of better
source code analysis.

Regards,
Markus

Re: Improving documentation for programming interfaces

[Enrico Weigelt, metux IT consult]

On 20.12.19 16:19, Theodore Y. Ts'o wrote:

Hi folks,

> On Fri, Dec 20, 2019 at 02:30:10PM +0100, Markus Elfring wrote:
>> Linux supports some programming interfaces. Several functions are provided
>> as usual. Their application documentation is an ongoing development challenge.
>>
>> Now I would like to clarify possibilities for the specification of desired
>> information together with data types besides properties which are handled by
>> the programming language “C” so far.

@Markus:

hmm, maybe we could add some kinda-OOP-style metadata into the type
documentation ? Or maybe extend doxygen to crossref types vs functions
operating on them.

>> It seems that no customised attributes are supported at the moment.
>> Thus I imagine to specify helpful annotations as macros.

Do you mean _attribute__(...) or comments ?

<snip>

> It's unclear to me what you are requesting/proposing?  Can you be a
> bit more concrete?

@Ted:

I guess he's thinking about some kind of meta-language for expressing
common things we know from oop-world, like ctors, dtors, getters, etc.


Maybe some doxygen experts here, who could tell what we already could
extract from existing sources ?


For start, I'd like to propose a few rules:

* consistent naming of 'release' functions (AFAIK, many of them are
  already named <foo>_put()).
* for each non-trivial (non-private) object/struct, there should be
  a corresponding release function (even if it's just an alias to
  kfree()
* consistent nameing of list-type structs, so generic macros can
  be used on the struct itself (instead just a container list header
  struct)



--mtx

-- 
---
Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
GPG/PGP-Schlüssel zu.
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287

Re: Improving documentation for programming interfaces

[Markus Elfring]

> hmm, maybe we could add some kinda-OOP-style metadata into the type documentation ?

I hope so.


> Or maybe extend doxygen to crossref types vs functions operating on them.

An advanced cross-reference variant would be nice (besides the “Elixir”).


>>> It seems that no customised attributes are supported at the moment.
>>> Thus I imagine to specify helpful annotations as macros.
>
> Do you mean _attribute__(...) or comments ?

I propose to encode helpful information into macro calls as needed
for the C programming language.


> I guess he's thinking about some kind of meta-language for expressing
> common things we know from oop-world, like ctors, dtors, getters, etc.

This feedback fits also to my imaginations for a better taxonomy.

Regards,
Markus

Re: Improving documentation for programming interfaces

[Theodore Y. Ts'o]

On Wed, Jan 08, 2020 at 01:40:45PM +0100, Markus Elfring wrote:
> 
> I propose to encode helpful information into macro calls as needed
> for the C programming language.

Perhaps it would be useful to for you to express what you are
proposing in the form of a patch?  That way people can see how
disruptive such changes might be, and how hard it might be to maintain
them.  That's on the "cost" side of the equation.

On the "benefits" side of the equation, is there are ways in which it
will directly benefit the kernel developers who will need to review
the patches and review the annotations, that can be demonstrated
immediately?  Not in some abstract way, or "when I my research work is
completed", but a very concrete way that will be obvious to those of
us who are still not completely convinced?

If the costs are very small, then the requirement for demonstrating
great benefits will also be small.  But if the costs are large (e.g.,
widespread renaming of huge numbers of functions, adding a lot of
extra complexity into code paths, use of silly/stupid names such as
"TransactionAwarePersistenceManagerFactoryProxy"), then need to show
benefits commensurate with such costs will also be great.

	 	      	   	      - Ted

Re: Improving documentation for programming interfaces

[Markus Elfring]

>> I propose to encode helpful information into macro calls as needed
>> for the C programming language.
>
> Perhaps it would be useful to for you to express what you are
> proposing in the form of a patch?

I hope that some of our communication difficulties on this topic
can be reduced also by other means.


> On the "benefits" side of the equation, is there are ways in which it
> will directly benefit the kernel developers who will need to review
> the patches and review the annotations, that can be demonstrated immediately?

Do you find a macro like “__must_check” useful enough to increase
the development of further annotations?


> If the costs are very small, then the requirement for demonstrating
> great benefits will also be small.

Would you like to reconsider these concerns according to better
software documentation at all?

Regards,
Markus

Re: Improving documentation for programming interfaces

[Markus Elfring]

>> I propose to encode helpful information into macro calls as needed
>> for the C programming language.
…
> On the "benefits" side of the equation, is there are ways in which it
> will directly benefit the kernel developers who will need to review
> the patches and review the annotations, that can be demonstrated
> immediately?  Not in some abstract way, or "when I my research work is
> completed", but a very concrete way that will be obvious to those of
> us who are still not completely convinced?

Would any contributors like to clarify further possibilities around
the application of source code annotation languages?

Regards,
Markus