Re: [for-next][PATCH 12/26] Documentation: bootconfig: Add a doc for extended boot config

Re: [for-next][PATCH 12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

I wonder about a few details in the added text.


…
> +++ b/Documentation/admin-guide/bootconfig.rst
…
> +C onfig File Limitation

How do you think about to omit a space character at the beginning
of this line?


> +Currently the maximum config size size is 32KB …

Would you like to avoid a word duplication here?


> +Note: this is not the number of entries but nodes, an entry must consume
> +more than 2 nodes (a key-word and a value). …

I find the relevance of the term “nodes” unclear at the moment.


Could an other wording be nicer than the abbreviation “a doc for … config”
in the commit subject?

Regards,
Markus

Re: [for-next][PATCH 12/26] Documentation: bootconfig: Add a doc for extended boot config

[Masami Hiramatsu]

Hi,

On Thu, 20 Feb 2020 10:10:20 +0100
Markus Elfring <Markus.Elfring@web.de> wrote:

> I wonder about a few details in the added text.
> 
> 
> …
> > +++ b/Documentation/admin-guide/bootconfig.rst
> …
> > +C onfig File Limitation
> 
> How do you think about to omit a space character at the beginning
> of this line?

That was my mistake. I used restructured text extension for vim
which collapsed all sections and use "space" key to expand.
Accidentally, I run into edit mode and hit "space" to expand it.
(it actually expanded but also put a space there and I missed it...)

Anyway, it has been fixed (pointed by Rundy)

> > +Currently the maximum config size size is 32KB …
> 
> Would you like to avoid a word duplication here?

Oops, still exist. Thanks!


> > +Note: this is not the number of entries but nodes, an entry must consume
> > +more than 2 nodes (a key-word and a value). …
> 
> I find the relevance of the term “nodes” unclear at the moment.

Indeed, "node" is not well defined. What about this?
---
Each key consists of words separated by dot, and value also consists of
values separated by comma. Here, each word and each value is generally
called a "node".
---

> 
> Could an other wording be nicer than the abbreviation “a doc for … config”
> in the commit subject?

OK, I'll try next time. 

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [for-next][PATCH 12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

>>> +Currently the maximum config size size is 32KB …
>>
>> Would you like to avoid a word duplication here?
>
> Oops, still exist.

Is there a need to separate the number from the following unit?


> Indeed, "node" is not well defined. What about this?
> ---
> Each key consists of words separated by dot, and value also consists of
> values separated by comma. Here, each word and each value is generally
> called a "node".

I have got still understanding difficulties with such an interpretation.

* Do other contributors find an other word also more appropriate for this use case?

* How will the influence evolve for naming these items?

* Is each element just a string (according to specific rules)?


>> Could an other wording be nicer than the abbreviation “a doc for … config”
>> in the commit subject?
>
> OK, I'll try next time.

Will words like “descriptions”and “configuration”be helpful?

Regards,
Markus

Re: [for-next][PATCH 12/26] Documentation: bootconfig: Add a doc for extended boot config

[Masami Hiramatsu]

On Thu, 20 Feb 2020 16:00:23 +0100
Markus Elfring <Markus.Elfring@web.de> wrote:

> >>> +Currently the maximum config size size is 32KB …
> >>
> >> Would you like to avoid a word duplication here?
> >
> > Oops, still exist.
> 
> Is there a need to separate the number from the following unit?

Sorry, I couldn't understand what you pointed here. Would you mean the
number of file size and nodes?

> > Indeed, "node" is not well defined. What about this?
> > ---
> > Each key consists of words separated by dot, and value also consists of
> > values separated by comma. Here, each word and each value is generally
> > called a "node".
> 
> I have got still understanding difficulties with such an interpretation.
> 
> * Do other contributors find an other word also more appropriate for this use case?

No.

> * How will the influence evolve for naming these items?

Node is used in its API, but from the user's point of view, I think it
is OK to use "key-word" and "value".
Also, since it is hard to count those numbers by manual, I think user
can depend on tools/bootconfig which shows the number of node in the
configuration file now.

> * Is each element just a string (according to specific rules)?

Yes.

> >> Could an other wording be nicer than the abbreviation “a doc for … config”
> >> in the commit subject?
> >
> > OK, I'll try next time.
> 
> Will words like “descriptions”and “configuration”be helpful?

Like "descriptions of ..." ?

Thank you,

> 
> Regards,
> Markus


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

>> Is there a need to separate the number from the following unit?
>
> Sorry, I couldn't understand what you pointed here.

Can the specification “… size is 32 KiB …”be more appropriate
(besides a small wording adjustment)?


> Like "descriptions of ..." ?

I got another idea also for the provided documentation format.
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/Documentation/admin-guide/bootconfig.rst?id=bee46b309a13ca158c99c325d0408fb2f0db207f#n18

* Will a file format description become helpful in the way of
  an extended Backus–Naur form?

* How will data processing evolve around the added structures?

Regards,
Markus

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Masami Hiramatsu]

On Fri, 21 Feb 2020 17:43:32 +0100
Markus Elfring <Markus.Elfring@web.de> wrote:

> >> Is there a need to separate the number from the following unit?
> >
> > Sorry, I couldn't understand what you pointed here.
> 
> Can the specification “… size is 32 KiB …”be more appropriate
> (besides a small wording adjustment)?

OK, I'll update as so :)

> > Like "descriptions of ..." ?
> 
> I got another idea also for the provided documentation format.
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/Documentation/admin-guide/bootconfig.rst?id=bee46b309a13ca158c99c325d0408fb2f0db207f#n18
> 
> * Will a file format description become helpful in the way of
>   an extended Backus–Naur form?

Good suggestion! Let me try to write an EBNF section.
I think EBNF can logically explain the format, but not intuitive
- we need some examples.

> * How will data processing evolve around the added structures?

OK, I'll add some more API (and usage) differences from the legacy
command line.

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

>> * Will a file format description become helpful in the way of
>>   an extended Backus–Naur form?
>
> Good suggestion! Let me try to write an EBNF section.

Is there a need to provide two format descriptions as separate files
(so that they can help more for different software users)?

* RST
* EBNF


Will it matter to adjust another wording?

-/proc/bootconfig is a user-space interface of the boot config.
+The file “/proc/bootconfig” is an user-space interface to the configuration
+of system boot parameters.

Regards,
Markus

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Masami Hiramatsu]

On Sat, 22 Feb 2020 10:48:28 +0100
Markus Elfring <Markus.Elfring@web.de> wrote:

> >> * Will a file format description become helpful in the way of
> >>   an extended Backus–Naur form?
> >
> > Good suggestion! Let me try to write an EBNF section.
> 
> Is there a need to provide two format descriptions as separate files
> (so that they can help more for different software users)?
> 
> * RST
> * EBNF

Hmm, since RST is enough flexible, we can write it as a section.
Then user can copy & paste if they need it.

> 
> 
> Will it matter to adjust another wording?
> 
> -/proc/bootconfig is a user-space interface of the boot config.
> +The file “/proc/bootconfig” is an user-space interface to the configuration
> +of system boot parameters.

OK.

Thank you,


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

>> Is there a need to provide two format descriptions as separate files
>> (so that they can help more for different software users)?
>>
>> * RST
>> * EBNF
>
> Hmm, since RST is enough flexible, we can write it as a section.

I guess that there are further design options to consider.


> Then user can copy & paste if they need it.

I imagine that it can be more convenient to refer to an EBNF file directly
if an other software developer would like to generate customised parsers
based on available information.

Regards,
Markus

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Masami Hiramatsu]

Hi Markus,

On Sat, 22 Feb 2020 17:15:57 +0100
Markus Elfring <Markus.Elfring@web.de> wrote:

> >> Is there a need to provide two format descriptions as separate files
> >> (so that they can help more for different software users)?
> >>
> >> * RST
> >> * EBNF
> >
> > Hmm, since RST is enough flexible, we can write it as a section.
> 
> I guess that there are further design options to consider.
> 
> 
> > Then user can copy & paste if they need it.
> 
> I imagine that it can be more convenient to refer to an EBNF file directly
> if an other software developer would like to generate customised parsers
> based on available information.

OK, I'll try to make a split EBNF file and include it.

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

> OK, I'll try to make a split EBNF file and include it.

Thanks for your positive feedback.


How do you think about to clarify any additional software design options
around involved data structures?

Regards,
Markus

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Masami Hiramatsu]

Hi,

On Mon, 24 Feb 2020 11:00:31 +0100
Markus Elfring <Markus.Elfring@web.de> wrote:

> > OK, I'll try to make a split EBNF file and include it.
> 
> Thanks for your positive feedback.
> 
> 
> How do you think about to clarify any additional software design options
> around involved data structures?

Sorry, what would you mean the "involved data structures" here?
Would you mean the usage of APIs or when to use bootconfig or command line?

Thank you,

> 
> Regards,
> Markus


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

>> How do you think about to clarify any additional software design options
>> around involved data structures?
>
> Sorry, what would you mean the "involved data structures" here?
> Would you mean the usage of APIs or when to use bootconfig or command line?

Additional system boot parameters can be managed also by a single file.
The file format will trigger specific parsing efforts.

* Linux functions are provided for the handling of available information.

* I imagine that another software library will become helpful
  besides the file interface.
  Will further bindings evolve for various programming languages?

* How do you think about to work with a higher level file system?
  Would you like to increase the granularity for corresponding
  data accesses (by user-space processes)?

Regards,
Markus

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Masami Hiramatsu]

On Tue, 25 Feb 2020 08:56:41 +0100
Markus Elfring <Markus.Elfring@web.de> wrote:

> >> How do you think about to clarify any additional software design options
> >> around involved data structures?
> >
> > Sorry, what would you mean the "involved data structures" here?
> > Would you mean the usage of APIs or when to use bootconfig or command line?
> 
> Additional system boot parameters can be managed also by a single file.
> The file format will trigger specific parsing efforts.

Maybe. If someone is interested in expanding their command, (e.g. vim)
they can use EBNF to understand syntax, or directly reuse lib/bootconfig.c
which provides a compact parser. :)

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [for-next][12/26] Documentation: bootconfig: Add a doc for extended boot config

[Markus Elfring]

>> The file format will trigger specific parsing efforts.
>
> Maybe. If someone is interested in expanding their command, (e.g. vim)
> they can use EBNF to understand syntax,

I hope so.


> or directly reuse lib/bootconfig.c which provides a compact parser. :)

I see some development challenges according to this design direction.
This software component is using programming interfaces from
the Linux kernel, isn't it?
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/lib/bootconfig.c?id=c99b17ac03994525092fd66bed14b4a0c82f0b4d#n9

I guess that other approaches should be considered for the desired
software reuse in this system configuration area.

Is there a need to map key and value combinations directly to files
(and directories) for a more convenient data processing by user-space processes?

Regards,
Markus

[for-next][PATCH 12/26] Documentation: bootconfig: Add a doc for extended boot config

[Steven Rostedt]

From: Masami Hiramatsu <mhiramat@kernel.org>

Add a documentation for extended boot config under
admin-guide, since it is including the syntax of boot config.

Link: http://lkml.kernel.org/r/157867230658.17873.9309879174829924324.stgit@devnote2

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 Documentation/admin-guide/bootconfig.rst | 184 +++++++++++++++++++++++
 Documentation/admin-guide/index.rst      |   1 +
 MAINTAINERS                              |   1 +
 3 files changed, 186 insertions(+)
 create mode 100644 Documentation/admin-guide/bootconfig.rst

diff --git a/Documentation/admin-guide/bootconfig.rst b/Documentation/admin-guide/bootconfig.rst
new file mode 100644
index 000000000000..f7475df2a718
--- /dev/null
+++ b/Documentation/admin-guide/bootconfig.rst
@@ -0,0 +1,184 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==================
+Boot Configuration
+==================
+
+:Author: Masami Hiramatsu <mhiramat@kernel.org>
+
+Overview
+========
+
+The boot configuration is expanding current kernel cmdline to support
+additional key-value data when boot the kernel in an efficient way.
+This allows adoministrators to pass a structured-Key config file.
+
+Config File Syntax
+==================
+
+The boot config syntax is a simple structured key-value. Each key consists
+of dot-connected-words, and key and value are connected by "=". The value
+has to be terminated by semi-colon (``;``) or newline (``\n``).
+For array value, array entries are separated by comma (``,``). ::
+
+KEY[.WORD[...]] = VALUE[, VALUE2[...]][;]
+
+Each key word must contain only alphabets, numbers, dash (``-``) or underscore
+(``_``). And each value only contains printable characters or spaces except
+for delimiters such as semi-colon (``;``), new-line (``\n``), comma (``,``),
+hash (``#``) and closing brace (``}``).
+
+If you want to use those delimiters in a value, you can use either double-
+quotes (``"VALUE"``) or single-quotes (``'VALUE'``) to quote it. Note that
+you can not escape these quotes.
+
+There can be a key which doesn't have value or has an empty value. Those keys
+are used for checking the key exists or not (like a boolean).
+
+Key-Value Syntax
+----------------
+
+The boot config file syntax allows user to merge partially same word keys
+by brace. For example::
+
+ foo.bar.baz = value1
+ foo.bar.qux.quux = value2
+
+These can be written also in::
+
+ foo.bar {
+    baz = value1
+    qux.quux = value2
+ }
+
+Or more shorter, written as following::
+
+ foo.bar { baz = value1; qux.quux = value2 }
+
+In both styles, same key words are automatically merged when parsing it
+at boot time. So you can append similar trees or key-values.
+
+Comments
+--------
+
+The config syntax accepts shell-script style comments. The comments start
+with hash ("#") until newline ("\n") will be ignored.
+
+::
+
+ # comment line
+ foo = value # value is set to foo.
+ bar = 1, # 1st element
+       2, # 2nd element
+       3  # 3rd element
+
+This is parsed as below::
+
+ foo = value
+ bar = 1, 2, 3
+
+Note that you can not put a comment between value and delimiter(``,`` or
+``;``). This means following config has a syntax error ::
+
+ key = 1 # comment
+       ,2
+
+
+/proc/bootconfig
+================
+
+/proc/bootconfig is a user-space interface of the boot config.
+Unlike /proc/cmdline, this file shows the key-value style list.
+Each key-value pair is shown in each line with following style::
+
+ KEY[.WORDS...] = "[VALUE]"[,"VALUE2"...]
+
+
+Boot Kernel With a Boot Config
+==============================
+
+Since the boot configuration file is loaded with initrd, it will be added
+to the end of the initrd (initramfs) image file. The Linux kernel decodes
+the last part of the initrd image in memory to get the boot configuration
+data.
+Because of this "piggyback" method, there is no need to change or
+update the boot loader and the kernel image itself.
+
+To do this operation, Linux kernel provides "bootconfig" command under
+tools/bootconfig, which allows admin to apply or delete the config file
+to/from initrd image. You can build it by follwoing command::
+
+ # make -C tools/bootconfig
+
+To add your boot config file to initrd image, run bootconfig as below
+(Old data is removed automatically if exists)::
+
+ # tools/bootconfig/bootconfig -a your-config /boot/initrd.img-X.Y.Z
+
+To remove the config from the image, you can use -d option as below::
+
+ # tools/bootconfig/bootconfig -d /boot/initrd.img-X.Y.Z
+
+
+C onfig File Limitation
+======================
+
+Currently the maximum config size size is 32KB and the total key-words (not
+key-value entries) must be under 1024 nodes.
+Note: this is not the number of entries but nodes, an entry must consume
+more than 2 nodes (a key-word and a value). So theoretically, it will be
+up to 512 key-value pairs. If keys contains 3 words in average, it can
+contain 256 key-value pairs. In most cases, the number of config items
+will be under 100 entries and smaller than 8KB, so it would be enough.
+If the node number exceeds 1024, parser returns an error even if the file
+size is smaller than 32KB.
+Anyway, since bootconfig command verifies it when appending a boot config
+to initrd image, user can notice it before boot.
+
+
+Bootconfig APIs
+===============
+
+User can query or loop on key-value pairs, also it is possible to find
+a root (prefix) key node and find key-values under that node.
+
+If you have a key string, you can query the value directly with the key
+using xbc_find_value(). If you want to know what keys exist in the SKC
+tree, you can use xbc_for_each_key_value() to iterate key-value pairs.
+Note that you need to use xbc_array_for_each_value() for accessing
+each arraies value, e.g.::
+
+ vnode = NULL;
+ xbc_find_value("key.word", &vnode);
+ if (vnode && xbc_node_is_array(vnode))
+    xbc_array_for_each_value(vnode, value) {
+      printk("%s ", value);
+    }
+
+If you want to focus on keys which has a prefix string, you can use
+xbc_find_node() to find a node which prefix key words, and iterate
+keys under the prefix node with xbc_node_for_each_key_value().
+
+But the most typical usage is to get the named value under prefix
+or get the named array under prefix as below::
+
+ root = xbc_find_node("key.prefix");
+ value = xbc_node_find_value(root, "option", &vnode);
+ ...
+ xbc_node_for_each_array_value(root, "array-option", value, anode) {
+    ...
+ }
+
+This accesses a value of "key.prefix.option" and an array of
+"key.prefix.array-option".
+
+Locking is not needed, since after initialized, the config becomes readonly.
+All data and keys must be copied if you need to modify it.
+
+
+Functions and structures
+========================
+
+.. kernel-doc:: include/linux/bootconfig.h
+.. kernel-doc:: lib/bootconfig.c
+
diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
index 4405b7485312..9e0f1e3fd152 100644
--- a/Documentation/admin-guide/index.rst
+++ b/Documentation/admin-guide/index.rst
@@ -64,6 +64,7 @@ configure specific aspects of kernel behavior to your liking.
    binderfs
    binfmt-misc
    blockdev/index
+   bootconfig
    braille-console
    btmrvl
    cgroup-v1/index
diff --git a/MAINTAINERS b/MAINTAINERS
index 903e8a7ed0bf..47873f2e6696 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15777,6 +15777,7 @@ F:	lib/bootconfig.c
 F:	fs/proc/bootconfig.c
 F:	include/linux/bootconfig.h
 F:	tools/bootconfig/*
+F:	Documentation/admin-guide/bootconfig.rst
 
 SUN3/3X
 M:	Sam Creasey <sammy@sammy.net>
-- 
2.24.1