[PATCH v9 0/7] env: Allow environment in text files

* [PATCH v9 0/7] env: Allow environment in text files
@ 2021-10-19 22:44 Simon Glass
  2021-10-19 22:44 ` [PATCH v9 1/7] sandbox: Drop distro_boot Simon Glass
                   ` (7 more replies)
  0 siblings, 8 replies; 38+ messages in thread
From: Simon Glass @ 2021-10-19 22:44 UTC (permalink / raw)
  To: U-Boot Mailing List
  Cc: Marek Behún, Rasmus Villemoes, Heinrich Schuchardt,
	Tom Rini, Wolfgang Denk, Simon Glass, Joe Hershberger

One barrier to completing the 7-year-long Kconfig migration is that
the default environment is implemented using ad-hoc CONFIG options.
At present U-Boot environment variables, and thus scripts, are defined
by CONFIG_EXTRA_ENV_SETTINGS.

It is not really feasible to move the environment to Kconfig as it is
hundreds of lines of text in some cases.

Even considering the current situation, it is painful to add large
amounts of text to the config-header file and dealing with quoting and
newlines is harder than it should be. It would be better if we could just
type the script into a text file and have it included by U-Boot.

This is already supported by the CONFIG_USE_DEFAULT_ENV_FILE feature. but
that does not support use of CONFIG options or comments, so is best suited
for use by other build systems wanting to define the U-Boot environment.

Add a feature that brings in a .env file associated with the board
config, if present. To use it, create a file board/<vendor>/<board>.env or
use CONFIG_ENV_SOURCE_FILE to set a filename.

The environment variables should be of the form "var=value". Values can
extend to multiple lines. This series converts the existing environment
documentation to rST and updates it to explain how to use this.

Note: this series was originally sent eight years ago:

https://patchwork.ozlabs.org/project/uboot/patch/1382763695-2849-4-git-send-email-sjg@chromium.org/

It has been updated to work with Kconfig, etc. Some review comments in
that patch were infeasible so I have not addressed them. I would like
this series to be considered independently, on its merits.

Rather than deal with the complexity of rewriting the distro-boot
script, this is disabled for sandbox. The forthcoming bootmethod approach
should provide the same functionality without needing the complex
scripting in the environment.

Migration needs more thought, although it can be done later. It may be
possible to do migrate automatically, using buildman to extract the
built-in environmnent from the ELF file.

This would produce a pretty ugly conversion though, since it would drop
all the intermediate variables used to create the environment.

Better would be to parse the config.h file, figure out the components of
CONFIG_EXTRA_ENV_SETTINGS then output these as separate pieces in the
file. It is not clear how easy that would be, nor whether the result would
be very pretty. Also the __stringify() macro needs to be handled somehow.

This series is available at u-boot-dm/env-working

Comments welcome.

Changes in v9:
- Drop mention of other strange characters
- Clarify that the + restriction is on the variable name not its value
- Add some tests for the script
- Deal with leading tabs
- Squash indentation down to one space
- Convert newlines within strings to spaces, which seems more consistent
- Handle appending an empty string to an empty var
- Fix blank line between tags
- Fix typo in commit message
- More bikeshedding on env_get_autostart()
- Fix '<vendor><board>' in cover letter
- Use env_get_yesno() in env_get_autostart() and update docs

Changes in v8:
- Update commit message to avoid mentioning the 'env' subdirectory
- Update commit message to mention the + restriction, etc.
- Overwrite the env file each time, to avoid incremental-build problems
- Fix ambiguity about what is ignored
- Go into more detail about the change of behaviour with autostart

Changes in v7:
- Use 'env' basename instead of 'environment' for intermediate output files
- Show a message indicating the source text file being used
- Give an error if CONFIG_EXTRA_ENV_SETTINGS is also defined
- Use CONFIG_ENV_SOURCE_FILE instead of rules to specify the text-file name
- Make board.env the default name if CONFIG_ENV_SOURCE_FILE is empty
- Rewrite the documentation
- Drop the use of common.env
- Update awk script to output the whole CONFIG string, or just a comment
- Add new patch to explain the relationship with DEFAULT_ENV_FILE
- A few more tweaks
- Update the cover letter

Changes in v6:
- Move all updates to a separate patch
- Combine the two env2string.awk patches into one
- Move all updates to a separate patch
- More updates and improvements
- Add new patch to tidy up use of autostart env var

Changes in v5:
- Minor updates as suggested by Wolfgang
- Explain how to include the common.env file
- Explain why variables starting with _ , and / are not supported
- Expand the definition of how to declare an environment variable
- Explain what happens to empty variables
- Update maintainer
- Move use of += to this patch
- Explain that environment variables may not end in +
- Minor updates as suggested by Wolfgang

Changes in v4:
- Add new patch to move environment documentation to rST
- Move this from being part of configuring U-Boot to part of building it
- Don't put the environment in autoconf.mk as it is not needed
- Add documentation in rST format instead of README
- Drop mention of import/export
- Update awk script to ignore blank lines, as generated by clang
- Add documentation in rST format instead of README
- Add new patch to move environment documentation to rST

Changes in v3:
- Adjust Makefile to generate the .inc and .h files in separate fules
- Add more detail in the README about the format of .env files
- Improve the comment about " in the awk script
- Correctly terminate environment files with \n
- Define __UBOOT_CONFIG__ when collecting environment files
- Add new patch to use a text-based environment for sandbox

Changes in v2:
- Move .env file from include/configs to board/
- Use awk script to process environment since it is much easier on the brain
- Add information and updated example script to README
- Add dependency rule so that the environment is rebuilt when it changes
- Add separate patch to enable C preprocessor for environment files
- Enable var+=value form to simplify composing variables in multiple steps

Simon Glass (7):
  sandbox: Drop distro_boot
  doc: Move environment documentation to rST
  env: Allow U-Boot scripts to be placed in a .env file
  sandbox: Use a text-based environment
  doc: Mention CONFIG_DEFAULT_ENV_FILE
  doc: Improve environment documentation
  bootm: Tidy up use of autostart env var

 MAINTAINERS               |   7 +
 Makefile                  |  66 ++++-
 README                    | 328 -------------------------
 board/sandbox/sandbox.env |  25 ++
 cmd/bootm.c               |   4 +-
 cmd/elf.c                 |   3 +-
 common/bootm_os.c         |   5 +-
 config.mk                 |   2 +
 doc/usage/environment.rst | 490 ++++++++++++++++++++++++++++++++++++++
 doc/usage/index.rst       |   1 +
 env/Kconfig               |  18 ++
 env/common.c              |   5 +
 env/embedded.c            |   1 +
 include/configs/sandbox.h |  40 ----
 include/env.h             |   7 +
 include/env_default.h     |  11 +
 scripts/env2string.awk    |  71 ++++++
 test/py/tests/test_env.py |  93 ++++++++
 18 files changed, 799 insertions(+), 378 deletions(-)
 create mode 100644 board/sandbox/sandbox.env
 create mode 100644 doc/usage/environment.rst
 create mode 100644 scripts/env2string.awk

-- 
2.33.0.1079.g6e70778dc9-goog

^ permalink raw reply	[flat|nested] 38+ messages in thread