terminal-colors.d(5) clarifications needed

terminal-colors.d(5) clarifications needed

[Lubomir Rintel]

Hi everyone,

we (the NetworkManager crowd) are currently pondering addition of
terminal-colors.d(5) support to nmcli, our command line utility.

We intend to be consistent with the specification and ideally implement
it the same way as the existing implementations. There's several
details I feel should be cleared up before my work [1] could be
merged. 

[1] https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?
h=lr/nmcli-colors&id=db7479c

Here's my notes:

1.) It is unclear whether multiple .schem files should override each
    other or be merged. We use the overriding behavior -- take the
    highest priority one and ignore the rest.

2.) Is "name.schem" or "@term.schem" "more specific"? We assume it's
    "name.schem".

3.) What is the "Color names" paragraph for? How was the set chosen?
    (Why is "white" not present?) Can it be used in place of
    an ansi-sequence and translated somehow? We just ignore it.

4.) Of what use would "Escape sequences" in ansi-sequence be? We don't
    accept that.

5.) We don't implement TERMINAL_COLORS_DEBUG because it's unclear what
    should it do.

Perhaps you could help me getting things done right, comment on the
points above, and maybe figure out how can the manual be improved for
future implementers?

Thanks,
Lubo

Re: terminal-colors.d(5) clarifications needed

[Karel Zak]

On Mon, Apr 23, 2018 at 04:07:39PM +0200, Lubomir Rintel wrote:
> 1.) It is unclear whether multiple .schem files should override each
>     other or be merged. We use the overriding behavior -- take the
>     highest priority one and ignore the rest.

>From the man page:

If there are more files that match for a utility, then the file with
the more specific  filename  wins.   For  example,  the  filename
"@xterm.scheme"  has  less  priority  than "dmesg@xterm.scheme".  The
lowest priority are those files without a utility name and terminal
identifier (e.g. "disable").

https://github.com/karelzak/util-linux/blob/master/lib/colors.c#L77

 * The terminal-colors.d/ evaluation is based on "scores":
 *
 *  filename                    score
 *  ---------------------------------------
 *  type                        1
 *  @termname.type              10 + 1
 *  utilname.type               20 + 1
 *  utilname@termname.type      20 + 10 + 1
 *
 * the match with higher score wins. The score is per type.

The "type" is "disable", "enable" or "scheme".

> 2.) Is "name.schem" or "@term.schem" "more specific"? We assume it's
>     "name.schem".

Yes.

> 3.) What is the "Color names" paragraph for? How was the set chosen?
>     (Why is "white" not present?) Can it be used in place of
>     an ansi-sequence and translated somehow? We just ignore it.

It's list of human readable aliases for colors. Not sure why we do not
have "white" :-)

https://github.com/karelzak/util-linux/blob/master/include/color-names.h
https://github.com/karelzak/util-linux/blob/master/lib/color-names.c#L23


> 4.) Of what use would "Escape sequences" in ansi-sequence be? We don't
>     accept that.

This is extension based on coreutils. IMHO it's not important.

> 5.) We don't implement TERMINAL_COLORS_DEBUG because it's unclear what
>     should it do.

Yes, this is unnecessary.

> Perhaps you could help me getting things done right, comment on the
> points above, and maybe figure out how can the manual be improved for
> future implementers?

I'll improve the man page. Thanks for your feedback.

Maybe we can convert our util-linux terminal-colors.d implementation
to simple shared library. Would be something like this acceptable for
NetworkManager? 

The interface is pretty simple (for example fdisk):

  colors_init(colormode, "fdisk");

  ...
  color_scheme_enable("welcome", UL_COLOR_GREEN);
  fdisk_info(cxt, _("Welcome to fdisk (%s)."), PACKAGE_STRING);
  color_disable();


where "colormode" is NEVER, ALWAYS, AUTO. The UL_COLOR_GREEN is
the default color.

It would be pretty simple to convert it to stared library; just use
less generic prefix (than color_), add control struct to avoid global
variables.


Note that the most important feature of terminal-colors.d is to
provide a way how to disable/enable colors for all system or specific
terminal. We need a way how to keep terminal usable for people who
cannot use colors. The "scheme" stuff is the next level :-)

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: terminal-colors.d(5) clarifications needed

[Lubomir Rintel]

On Tue, 2018-04-24 at 09:59 +0200, Karel Zak wrote:
> On Mon, Apr 23, 2018 at 04:07:39PM +0200, Lubomir Rintel wrote:
> > 1.) It is unclear whether multiple .schem files should override
> > each
> >     other or be merged. We use the overriding behavior -- take the
> >     highest priority one and ignore the rest.
> 
> From the man page:
> 
> If there are more files that match for a utility, then the file with
> the more specific  filename  wins.   For  example,  the  filename
> "@xterm.scheme"  has  less  priority  than "dmesg@xterm.scheme".  The
> lowest priority are those files without a utility name and terminal
> identifier (e.g. "disable").
> 
> https://github.com/karelzak/util-linux/blob/master/lib/colors.c#L77
> 
>  * The terminal-colors.d/ evaluation is based on "scores":
>  *
>  *  filename                    score
>  *  ---------------------------------------
>  *  type                        1
>  *  @termname.type              10 + 1
>  *  utilname.type               20 + 1
>  *  utilname@termname.type      20 + 10 + 1
>  *
>  * the match with higher score wins. The score is per type.
> 
> The "type" is "disable", "enable" or "scheme".

Some misunderstanding here. Perhaps I was not too clear -- I'll try to
rephrase. Suppose I have both "utilname.type" and
"utilname@termname.type":

utilname.type:

error 1;1
warning 2;2

utilname@termname.type:

warning 3;3

What is the effective result?

Is it (overriding behavior):

warning 3;3

Or (merging behavior):

error 1;1
warning 3;3

We assume "overriding", but the documentation is not clear enough to
me.

> > 3.) What is the "Color names" paragraph for? How was the set
> > chosen?
> >     (Why is "white" not present?) Can it be used in place of
> >     an ansi-sequence and translated somehow? We just ignore it.
> 
> It's list of human readable aliases for colors. Not sure why we do
> not
> have "white" :-)
> 
> https://github.com/karelzak/util-linux/blob/master/include/color-
> names.h

Now, what's special about brown? :)

> https://github.com/karelzak/util-linux/blob/master/lib/color-
> names.c#L23
> 
>
> > 4.) Of what use would "Escape sequences" in ansi-sequence be? We
> > don't
> >     accept that.
> 
> This is extension based on coreutils. IMHO it's not important.

I it correct to say that only ANSI terminals are actually supported?
Are there even terminals that use coloring in a way that's incompatible
with ANSI escape sequences?

I'm asking because the fact that colors are represented as color codes
without the leading escape suggests that either the application somehow
figures out the correct escape (does termcap/terminfo know that?) or
just slaps in ESC [ assuming an ANSI terminal.

You also seem to just translate the "Color names" to ANSI escapes.

If it's indeed true, then I guess you could just drop the "Escape
sequences" paragraph, because it doesn't make any sense on ANSI
terminals. Or does it?

> > 5.) We don't implement TERMINAL_COLORS_DEBUG because it's unclear
> > what
> >     should it do.
> 
> Yes, this is unnecessary.
> 
> > Perhaps you could help me getting things done right, comment on the
> > points above, and maybe figure out how can the manual be improved
> > for
> > future implementers?
> 
> I'll improve the man page. Thanks for your feedback.

Feel free to Cc me with the patch, I'd be happy to provide my pair of
eyes to see if things make sense to me.

> Maybe we can convert our util-linux terminal-colors.d implementation
> to simple shared library. Would be something like this acceptable for
> NetworkManager? 
> 
> The interface is pretty simple (for example fdisk):
> 
>   colors_init(colormode, "fdisk");
> 
>   ...
>   color_scheme_enable("welcome", UL_COLOR_GREEN);
>   fdisk_info(cxt, _("Welcome to fdisk (%s)."), PACKAGE_STRING);
>   color_disable();
> 
> 
> where "colormode" is NEVER, ALWAYS, AUTO. The UL_COLOR_GREEN is
> the default color.
> 
> It would be pretty simple to convert it to stared library; just use
> less generic prefix (than color_), add control struct to avoid global
> variables.

Well, perhaps not. We tend to be super careful about adding
dependencies. I suspect we'd need to provide a way to disable the
functionality because some users just slap in busybox in place of util-
linux. Given how minimal our terminal-colors.d implementation it it's
probably not worth doing.

If you care about terminal-colors.d being more ubiquitous while also
sharing the implementation as much as possible then an effort would be
best spent making Glib use it. We, and everyone else, would inherit the
functionality and since recently Glib uses coloring in its logging
machinery.

> Note that the most important feature of terminal-colors.d is to
> provide a way how to disable/enable colors for all system or specific
> terminal. We need a way how to keep terminal usable for people who
> cannot use colors. The "scheme" stuff is the next level :-)

Yes, that's precisely why we're doing this too :)

> 
>     Karel

Thanks for the clarifications. It's much appreciated.

Lubo

Re: terminal-colors.d(5) clarifications needed

[Karel Zak]

On Wed, Apr 25, 2018 at 09:26:52AM +0200, Lubomir Rintel wrote:
> Some misunderstanding here. Perhaps I was not too clear -- I'll try to
> rephrase. Suppose I have both "utilname.type" and
> "utilname@termname.type":
> 
> utilname.type:
> 
> error 1;1
> warning 2;2
> 
> utilname@termname.type:
> 
> warning 3;3
> 
> What is the effective result?
> 
> Is it (overriding behavior):
> 
> warning 3;3

This is the result.

> Or (merging behavior):
> 
> error 1;1
> warning 3;3

Only one file is open according to score calculated from the file name.
The winner is used for everything, no merging at all.

> > > 4.) Of what use would "Escape sequences" in ansi-sequence be? We
> > > don't
> > >     accept that.
> > 
> > This is extension based on coreutils. IMHO it's not important.
> 
> I it correct to say that only ANSI terminals are actually supported?

Yes.

> Are there even terminals that use coloring in a way that's incompatible
> with ANSI escape sequences?

We use only 

    ncolors = tigetnum("colors");

and if ncolors <= 2 then all the stuff is disabled for the terminal.

> I'm asking because the fact that colors are represented as color codes
> without the leading escape suggests that either the application somehow

 12;34

is internally converted to "\033[12;34m". It's only user friendly way
how to write the codes to the file. The result is always \033[...

> figures out the correct escape (does termcap/terminfo know that?) or
> just slaps in ESC [ assuming an ANSI terminal.

Frankly, it would be possible to be more smart with the codes (like
coretils, see below), but I have doubts it's important enough... we do
not have any bug report in last 5 years, so I don't care ;-)

> You also seem to just translate the "Color names" to ANSI escapes.

Yes.

> If it's indeed true, then I guess you could just drop the "Escape
> sequences" paragraph, because it doesn't make any sense on ANSI
> terminals. Or does it?

It allows to specify control chars

# echo "example \n" > /etc/terminal-colors.d/foo.scheme
# ./test_colors --name foo --color-scheme example

Hello World!
# 

... not sure how usable it's, but it was suggested to be more
compatible with coreutils dir_colors, so we support it.


Note that dir_colors from coreutils also supports non-ANSI as they
have LEFTCODE and RIGHTCODE setting to overwrite begin and end of the
sequence, so you can manually set something else than "\033[" and "m".

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com