[U-Boot] [PATCH 0/8] video/console: Fix various DM_VIDEO console issues

[U-Boot] [PATCH 0/8] video/console: Fix various DM_VIDEO console issues

[Andre Przywara]

The graphical console (vidconsole-uclass.c) for DM_VIDEO based display
drivers has several issues:
- Many ANSI sequences are not handled properly.
- The character set used is assumed to be the original IBM PC code
  page 437, even though the UEFI code expect the terminal to display
  UTF-8 encoded characters. Truetype fonts expect ISO8859-1.
- The USB keyboard does not handle arrow keys correctly.
- There is no visible cursor displayed.
- Truetype fonts only work in 16-bit screen modes.
- The maximum font width supported is 8 pixels.
- The "bmp" command to display bitmaps only works in certain video modes.

This leads to a very poor user experience, up to a point where
applications become unusable (Grub/EFI and U-Boot's bootmenu come to mind).

This patch set aims to fix the most important of those issues (the first
three). I have further patches to address the rest, but they are not so
nice or have issues, so I will send them later.

Patch 1/8 is a bug fix (for ASCII characters > 127). Patch 2-4 extend the
ANSI sequence handling. Patch 5 and 6 fix the character encoding for
the bitmap fonts. The USB keyboard learns about proper arrow key handling
in patch 7, while patch 8 fixes a minor Kconfig omission for sunxi.

After this series I can use both the bootmenu and Grub/EFI properly. Tested
on Sandbox and an Pine64-LTS board.

Cheers,
Andre.

Andre Przywara (8):
  video/console: Fix DM_VIDEO font glyph array indexing
  video/console: Implement reverse video ANSI sequence for DM_VIDEO
  video/console: Implement relative cursor movement ANSI handling
  video/console: Implement ANSI clear line command
  video/console: Factor out actual character output
  video/console: Convert UTF-8 codes to CP437 code points
  usb: kbd: Properly translate up/down arrow keys
  sunxi: allow boards to de-select SYS_WHITE_ON_BLACK font scheme

 common/usb_kbd.c                  |  24 +++++-
 drivers/video/Kconfig             |   2 +-
 drivers/video/Makefile            |   1 +
 drivers/video/console_normal.c    |   3 +-
 drivers/video/console_rotate.c    |   7 +-
 drivers/video/utf8_cp437.c        | 170 ++++++++++++++++++++++++++++++++++++++
 drivers/video/vidconsole-uclass.c | 113 +++++++++++++++++++++----
 drivers/video/video-uclass.c      |   1 +
 include/configs/sunxi-common.h    |   1 -
 include/video.h                   |   2 +
 include/video_console.h           |   9 ++
 11 files changed, 310 insertions(+), 23 deletions(-)
 create mode 100644 drivers/video/utf8_cp437.c

-- 
2.14.5

[U-Boot] [PATCH 1/8] video/console: Fix DM_VIDEO font glyph array indexing

[Andre Przywara]

When the character to be printed on a DM_VIDEO console is from the
"extended ASCII" range (0x80 - 0xff), it will be treated as a negative
number, as it's declared as a signed char. This leads to negative array
indicies into the glyph bitmap array, and random garbled characters.

Cast the character to an unsigned type to make the index always positive
and avoid an out-of-bounds access.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 drivers/video/console_normal.c | 3 ++-
 drivers/video/console_rotate.c | 7 ++++---
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/video/console_normal.c b/drivers/video/console_normal.c
index 2cfa510d5f..511589eaff 100644
--- a/drivers/video/console_normal.c
+++ b/drivers/video/console_normal.c
@@ -84,7 +84,8 @@ static int console_normal_putc_xy(struct udevice *dev, uint x_frac, uint y,
 		return -EAGAIN;
 
 	for (row = 0; row < VIDEO_FONT_HEIGHT; row++) {
-		uchar bits = video_fontdata[ch * VIDEO_FONT_HEIGHT + row];
+		unsigned int idx = (uint8_t)ch * VIDEO_FONT_HEIGHT + row;
+		uchar bits = video_fontdata[idx];
 
 		switch (vid_priv->bpix) {
 #ifdef CONFIG_VIDEO_BPP8
diff --git a/drivers/video/console_rotate.c b/drivers/video/console_rotate.c
index f076570335..e5ebaa03fa 100644
--- a/drivers/video/console_rotate.c
+++ b/drivers/video/console_rotate.c
@@ -90,7 +90,7 @@ static int console_putc_xy_1(struct udevice *dev, uint x_frac, uint y, char ch)
 	int i, col;
 	int mask = 0x80;
 	void *line;
-	uchar *pfont = video_fontdata + ch * VIDEO_FONT_HEIGHT;
+	uchar *pfont = video_fontdata + (uint8_t)ch * VIDEO_FONT_HEIGHT;
 
 	line = vid_priv->fb + (VID_TO_PIXEL(x_frac) + 1) *
 			vid_priv->line_length - (y + 1) * pbytes;
@@ -222,7 +222,8 @@ static int console_putc_xy_2(struct udevice *dev, uint x_frac, uint y, char ch)
 			VIDEO_FONT_WIDTH - 1) * VNBYTES(vid_priv->bpix);
 
 	for (row = 0; row < VIDEO_FONT_HEIGHT; row++) {
-		uchar bits = video_fontdata[ch * VIDEO_FONT_HEIGHT + row];
+		unsigned int idx = (uint8_t)ch * VIDEO_FONT_HEIGHT + row;
+		uchar bits = video_fontdata[idx];
 
 		switch (vid_priv->bpix) {
 #ifdef CONFIG_VIDEO_BPP8
@@ -348,7 +349,7 @@ static int console_putc_xy_3(struct udevice *dev, uint x_frac, uint y, char ch)
 	void *line = vid_priv->fb +
 		(vid_priv->ysize - VID_TO_PIXEL(x_frac) - 1) *
 		vid_priv->line_length + y * pbytes;
-	uchar *pfont = video_fontdata + ch * VIDEO_FONT_HEIGHT;
+	uchar *pfont = video_fontdata + (uint8_t)ch * VIDEO_FONT_HEIGHT;
 
 	if (x_frac + VID_TO_POS(vc_priv->x_charsize) > vc_priv->xsize_frac)
 		return -EAGAIN;
-- 
2.14.5

[U-Boot] [PATCH 1/8] video/console: Fix DM_VIDEO font glyph array indexing

[Simon Glass]

Hi Andre,

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> When the character to be printed on a DM_VIDEO console is from the
> "extended ASCII" range (0x80 - 0xff), it will be treated as a negative
> number, as it's declared as a signed char. This leads to negative array
> indicies into the glyph bitmap array, and random garbled characters.
>
> Cast the character to an unsigned type to make the index always positive
> and avoid an out-of-bounds access.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/console_normal.c | 3 ++-
>  drivers/video/console_rotate.c | 7 ++++---
>  2 files changed, 6 insertions(+), 4 deletions(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

Should use u8 instead of uint8_t.

It might make sense to adjust one of the tests to output such a character.

Regards,
Simon

[U-Boot] [PATCH 1/8] video/console: Fix DM_VIDEO font glyph array indexing

[Anatolij Gustschin]

On Sat, 23 Mar 2019 01:29:55 +0000
Andre Przywara andre.przywara at arm.com wrote:

> When the character to be printed on a DM_VIDEO console is from the
> "extended ASCII" range (0x80 - 0xff), it will be treated as a negative
> number, as it's declared as a signed char. This leads to negative array
> indicies into the glyph bitmap array, and random garbled characters.
> 
> Cast the character to an unsigned type to make the index always positive
> and avoid an out-of-bounds access.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/console_normal.c | 3 ++-
>  drivers/video/console_rotate.c | 7 ++++---
>  2 files changed, 6 insertions(+), 4 deletions(-)

Applied (with s/uint8_t/u8/) to u-boot-video/master, thanks!

--
Anatolij

[U-Boot] [PATCH 2/8] video/console: Implement reverse video ANSI sequence for DM_VIDEO

[Andre Przywara]

The video console for DM_VIDEO compliant drivers only understands a very
small number of ANSI sequences. First and foremost it misses the "reverse
video" command, which is used by our own bootmenu command to highlight
the selected entry.

To avoid forcing people to use their imagination when using the
bootmenu, let's just implement the rather simple reverse effect. We need
to store the background colour index for that, so that we can
recalculate both the foreground and background colour pixel values.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 drivers/video/vidconsole-uclass.c | 11 ++++++++++-
 drivers/video/video-uclass.c      |  1 +
 include/video.h                   |  2 ++
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c
index 2ca19d4049..87f43c2030 100644
--- a/drivers/video/vidconsole-uclass.c
+++ b/drivers/video/vidconsole-uclass.c
@@ -360,6 +360,13 @@ static void vidconsole_escape_char(struct udevice *dev, char ch)
 				vid_priv->colour_fg = vid_console_color(
 						vid_priv, vid_priv->fg_col_idx);
 				break;
+			case 7:
+				/* reverse video */
+				vid_priv->colour_fg = vid_console_color(
+						vid_priv, vid_priv->bg_col_idx);
+				vid_priv->colour_bg = vid_console_color(
+						vid_priv, vid_priv->fg_col_idx);
+				break;
 			case 30 ... 37:
 				/* foreground color */
 				vid_priv->fg_col_idx &= ~7;
@@ -369,8 +376,10 @@ static void vidconsole_escape_char(struct udevice *dev, char ch)
 				break;
 			case 40 ... 47:
 				/* background color */
+				vid_priv->bg_col_idx &= ~7;
+				vid_priv->bg_col_idx |= val - 40;
 				vid_priv->colour_bg = vid_console_color(
-							vid_priv, val - 40);
+						vid_priv, vid_priv->bg_col_idx);
 				break;
 			default:
 				/* ignore unsupported SGR parameter */
diff --git a/drivers/video/video-uclass.c b/drivers/video/video-uclass.c
index f307cf243b..14aac88d6d 100644
--- a/drivers/video/video-uclass.c
+++ b/drivers/video/video-uclass.c
@@ -136,6 +136,7 @@ void video_set_default_colors(struct udevice *dev, bool invert)
 		back = temp;
 	}
 	priv->fg_col_idx = fore;
+	priv->bg_col_idx = back;
 	priv->colour_fg = vid_console_color(priv, fore);
 	priv->colour_bg = vid_console_color(priv, back);
 }
diff --git a/include/video.h b/include/video.h
index 1d57b48b17..485071d072 100644
--- a/include/video.h
+++ b/include/video.h
@@ -70,6 +70,7 @@ enum video_log2_bpp {
  *		the LCD is updated
  * @cmap:	Colour map for 8-bit-per-pixel displays
  * @fg_col_idx:	Foreground color code (bit 3 = bold, bit 0-2 = color)
+ * @bg_col_idx:	Background color code (bit 3 = bold, bit 0-2 = color)
  */
 struct video_priv {
 	/* Things set up by the driver: */
@@ -92,6 +93,7 @@ struct video_priv {
 	bool flush_dcache;
 	ushort *cmap;
 	u8 fg_col_idx;
+	u8 bg_col_idx;
 };
 
 /* Placeholder - there are no video operations at present */
-- 
2.14.5

[U-Boot] [PATCH 2/8] video/console: Implement reverse video ANSI sequence for DM_VIDEO

[Simon Glass]

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> The video console for DM_VIDEO compliant drivers only understands a very
> small number of ANSI sequences. First and foremost it misses the "reverse
> video" command, which is used by our own bootmenu command to highlight
> the selected entry.
>
> To avoid forcing people to use their imagination when using the
> bootmenu, let's just implement the rather simple reverse effect. We need
> to store the background colour index for that, so that we can
> recalculate both the foreground and background colour pixel values.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 11 ++++++++++-
>  drivers/video/video-uclass.c      |  1 +
>  include/video.h                   |  2 ++
>  3 files changed, 13 insertions(+), 1 deletion(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH 2/8] video/console: Implement reverse video ANSI sequence for DM_VIDEO

[Anatolij Gustschin]

On Sat, 23 Mar 2019 01:29:56 +0000
Andre Przywara andre.przywara at arm.com wrote:

> The video console for DM_VIDEO compliant drivers only understands a very
> small number of ANSI sequences. First and foremost it misses the "reverse
> video" command, which is used by our own bootmenu command to highlight
> the selected entry.
> 
> To avoid forcing people to use their imagination when using the
> bootmenu, let's just implement the rather simple reverse effect. We need
> to store the background colour index for that, so that we can
> recalculate both the foreground and background colour pixel values.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 11 ++++++++++-
>  drivers/video/video-uclass.c      |  1 +
>  include/video.h                   |  2 ++
>  3 files changed, 13 insertions(+), 1 deletion(-)

Applied to u-boot-video/master, thanks!

--
Anatolij

[U-Boot] [PATCH v2 2/8] video/console: Implement reverse video ANSI sequence for DM_VIDEO

[Anatolij Gustschin]

From: Andre Przywara <andre.przywara@arm.com>

The video console for DM_VIDEO compliant drivers only understands a very
small number of ANSI sequences. First and foremost it misses the "reverse
video" command, which is used by our own bootmenu command to highlight
the selected entry.

To avoid forcing people to use their imagination when using the
bootmenu, let's just implement the rather simple reverse effect. We need
to store the background colour index for that, so that we can
recalculate both the foreground and background colour pixel values.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
[agust: merged BG color escape seq change to fix "ut dm video_ansi" test]
Signed-off-by: Anatolij Gustschin <agust@denx.de>
---
Changes in v2:
 - Fix to render standard bg colors (as used to be before v1 2/8 patch).
   Catched by "dm video_ansi" test.

 drivers/video/vidconsole-uclass.c | 13 +++++++++++--
 drivers/video/video-uclass.c      |  1 +
 include/video.h                   |  2 ++
 3 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c
index 2ca19d4049..3ff65f3232 100644
--- a/drivers/video/vidconsole-uclass.c
+++ b/drivers/video/vidconsole-uclass.c
@@ -360,6 +360,13 @@ static void vidconsole_escape_char(struct udevice *dev, char ch)
 				vid_priv->colour_fg = vid_console_color(
 						vid_priv, vid_priv->fg_col_idx);
 				break;
+			case 7:
+				/* reverse video */
+				vid_priv->colour_fg = vid_console_color(
+						vid_priv, vid_priv->bg_col_idx);
+				vid_priv->colour_bg = vid_console_color(
+						vid_priv, vid_priv->fg_col_idx);
+				break;
 			case 30 ... 37:
 				/* foreground color */
 				vid_priv->fg_col_idx &= ~7;
@@ -368,9 +375,11 @@ static void vidconsole_escape_char(struct udevice *dev, char ch)
 						vid_priv, vid_priv->fg_col_idx);
 				break;
 			case 40 ... 47:
-				/* background color */
+				/* background color, also mask the bold bit */
+				vid_priv->bg_col_idx &= ~0xf;
+				vid_priv->bg_col_idx |= val - 40;
 				vid_priv->colour_bg = vid_console_color(
-							vid_priv, val - 40);
+						vid_priv, vid_priv->bg_col_idx);
 				break;
 			default:
 				/* ignore unsupported SGR parameter */
diff --git a/drivers/video/video-uclass.c b/drivers/video/video-uclass.c
index f307cf243b..14aac88d6d 100644
--- a/drivers/video/video-uclass.c
+++ b/drivers/video/video-uclass.c
@@ -136,6 +136,7 @@ void video_set_default_colors(struct udevice *dev, bool invert)
 		back = temp;
 	}
 	priv->fg_col_idx = fore;
+	priv->bg_col_idx = back;
 	priv->colour_fg = vid_console_color(priv, fore);
 	priv->colour_bg = vid_console_color(priv, back);
 }
diff --git a/include/video.h b/include/video.h
index 1d57b48b17..485071d072 100644
--- a/include/video.h
+++ b/include/video.h
@@ -70,6 +70,7 @@ enum video_log2_bpp {
  *		the LCD is updated
  * @cmap:	Colour map for 8-bit-per-pixel displays
  * @fg_col_idx:	Foreground color code (bit 3 = bold, bit 0-2 = color)
+ * @bg_col_idx:	Background color code (bit 3 = bold, bit 0-2 = color)
  */
 struct video_priv {
 	/* Things set up by the driver: */
@@ -92,6 +93,7 @@ struct video_priv {
 	bool flush_dcache;
 	ushort *cmap;
 	u8 fg_col_idx;
+	u8 bg_col_idx;
 };
 
 /* Placeholder - there are no video operations at present */
-- 
2.17.1

[U-Boot] [PATCH v2 2/8] video/console: Implement reverse video ANSI sequence for DM_VIDEO

[Anatolij Gustschin]

On Sun, 14 Apr 2019 15:05:36 +0200
Anatolij Gustschin agust at denx.de wrote:
...
> Changes in v2:
>  - Fix to render standard bg colors (as used to be before v1 2/8 patch).
>    Catched by "dm video_ansi" test.
> 
>  drivers/video/vidconsole-uclass.c | 13 +++++++++++--
>  drivers/video/video-uclass.c      |  1 +
>  include/video.h                   |  2 ++
>  3 files changed, 14 insertions(+), 2 deletions(-)

Applied to u-boot-video/master, thanks!

--
Anatolij

[U-Boot] [PATCH 3/8] video/console: Implement relative cursor movement ANSI handling

[Andre Przywara]

The ANSI terminal escapce sequence standard defines relative cursor
movement commands (ESC [ A-F). So far the DM_VIDEO console code was
ignoring them.

Interpret those sequences and move the cursor by the requested amount of
rows or columns in the right direction. This brings the code on par with
the legacy video console driver (cfb_console).

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 drivers/video/vidconsole-uclass.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c
index 87f43c2030..cbd63f0ce8 100644
--- a/drivers/video/vidconsole-uclass.c
+++ b/drivers/video/vidconsole-uclass.c
@@ -259,6 +259,43 @@ static void vidconsole_escape_char(struct udevice *dev, char ch)
 	priv->escape = 0;
 
 	switch (ch) {
+	case 'A':
+	case 'B':
+	case 'C':
+	case 'D':
+	case 'E':
+	case 'F': {
+		int row, col, num;
+		char *s = priv->escape_buf;
+
+		/*
+		 * Cursor up/down: [%dA, [%dB, [%dE, [%dF
+		 * Cursor left/right: [%dD, [%dC
+		 */
+		s++;    /* [ */
+		s = parsenum(s, &num);
+		if (num == 0)			/* No digit in sequence ... */
+			num = 1;		/* ... means "move by 1". */
+
+		get_cursor_position(priv, &row, &col);
+		if (ch == 'A' || ch == 'F')
+			row -= num;
+		if (ch == 'C')
+			col += num;
+		if (ch == 'D')
+			col -= num;
+		if (ch == 'B' || ch == 'E')
+			row += num;
+		if (ch == 'E' || ch == 'F')
+			col = 0;
+		if (col < 0)
+			col = 0;
+		if (row < 0)
+			row = 0;
+		/* Right and bottom overflows are handled in the callee. */
+		set_cursor_position(priv, row, col);
+		break;
+	}
 	case 'H':
 	case 'f': {
 		int row, col;
-- 
2.14.5

[U-Boot] [PATCH 3/8] video/console: Implement relative cursor movement ANSI handling

[Simon Glass]

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> The ANSI terminal escapce sequence standard defines relative cursor
> movement commands (ESC [ A-F). So far the DM_VIDEO console code was
> ignoring them.
>
> Interpret those sequences and move the cursor by the requested amount of
> rows or columns in the right direction. This brings the code on par with
> the legacy video console driver (cfb_console).
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 37 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 37 insertions(+)

Reviewed-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH 3/8] video/console: Implement relative cursor movement ANSI handling

[Anatolij Gustschin]

On Sat, 23 Mar 2019 01:29:57 +0000
Andre Przywara andre.przywara at arm.com wrote:

> The ANSI terminal escapce sequence standard defines relative cursor
> movement commands (ESC [ A-F). So far the DM_VIDEO console code was
> ignoring them.
> 
> Interpret those sequences and move the cursor by the requested amount of
> rows or columns in the right direction. This brings the code on par with
> the legacy video console driver (cfb_console).
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 37 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 37 insertions(+)

Applied to u-boot-video/master, thanks!

--
Anatolij

[U-Boot] [PATCH 3/8] video/console: Implement relative cursor movement ANSI handling

[Anatolij Gustschin]

On Tue, 9 Apr 2019 23:05:11 +0200
Anatolij Gustschin agust at denx.de wrote:
...
> >  drivers/video/vidconsole-uclass.c | 37 +++++++++++++++++++++++++++++++++++++
> >  1 file changed, 37 insertions(+)  
> 
> Applied to u-boot-video/master, thanks!

I've dropped all applied patches of this series now, some of them
introduced dm video_ansi test error [1]. Please fix. Thanks!

[1] https://travis-ci.org/vdsao/u-boot-video/jobs/517944146#L1081

--
Anatolij

[U-Boot] [PATCH 3/8] video/console: Implement relative cursor movement ANSI handling

[André Przywara]

On 11/04/2019 13:09, Anatolij Gustschin wrote:

Hi Anatolij,

thanks for the heads up!

> On Tue, 9 Apr 2019 23:05:11 +0200
> Anatolij Gustschin agust at denx.de wrote:
> ...
>>>  drivers/video/vidconsole-uclass.c | 37 +++++++++++++++++++++++++++++++++++++
>>>  1 file changed, 37 insertions(+)  
>>
>> Applied to u-boot-video/master, thanks!
> 
> I've dropped all applied patches of this series now, some of them
> introduced dm video_ansi test error [1]. Please fix. Thanks!

Hmh, good one. Didn't find an easy way to get to the bottom of this
within the ut test system, so I copied the ANSI sequences out and
replayed them with a custom command, inspecting the (sandbox) screen
manually. Is there a canonical way to trace down those issues?

Anyway, the fix for patch 2/8 is rather simple (see below), do you want
to fix this up in your tree? Or shall I sent a v2?
The head of my tree passes the video_ansi test now.

(This patch won't apply cleanly (blame Thunderbird), but I think you get
the idea...)
--- a/drivers/video/vidconsole-uclass.c
+++ b/drivers/video/vidconsole-uclass.c
@@ -464,7 +464,7 @@ static void vidconsole_escape_char(struct udevice
*dev, char ch)
 			break;
 		case 40 ... 47:
-			/* background color */
-			vid_priv->bg_col_idx &= ~7;
+			/* background color, also mask the bold bit */
+			vid_priv->bg_col_idx &= ~0xf;
 			vid_priv->bg_col_idx |= val - 40;
 			vid_priv->colour_bg = vid_console_color(
 					vid_priv, vid_priv->bg_col_idx);

Cheers,
Andre.

[U-Boot] [PATCH 3/8] video/console: Implement relative cursor movement ANSI handling

[Anatolij Gustschin]

Hi André,

On Sat, 13 Apr 2019 22:40:03 +0100
André Przywara andre.przywara at arm.com wrote:
...
> > I've dropped all applied patches of this series now, some of them
> > introduced dm video_ansi test error [1]. Please fix. Thanks!  
> 
> Hmh, good one. Didn't find an easy way to get to the bottom of this
> within the ut test system, so I copied the ANSI sequences out and
> replayed them with a custom command, inspecting the (sandbox) screen
> manually. Is there a canonical way to trace down those issues?

You can build sandbox defconfig an run U-Boot with "--show_lcd"
option, then run the "ut dm video_ansi" command to inspect the
rendered characters and colors, e.g.:

$ make sandbox_defconfig
$ make
$ ./u-boot -d arch/sandbox/dts/test.dtb -l

U-Boot 2019.04-00326-g7b64a70a3a (Apr 14 2019 - 14:37:44 +0200)

Model: sandbox
DRAM:  128 MiB
MMC:   mmc2: 2 (SD), mmc1: 1 (SD), mmc0: 0 (SD)
In:    serial
Out:   vidconsole
Err:   vidconsole
Model: sandbox
SCSI:  
Net:   eth0: eth at 10002000, eth5: eth at 10003000, eth3: sbe5, eth1: eth at 10004000
Hit any key to stop autoboot:  0 
=> ut dm video_ansi
Test: dm_test_video_ansi: video.c
Failures: 0

In the "U-Boot" window you will see the output of the video_ansi test.

Without your fix for masking bit 3 in the bg index, the high-intensity
background colors will be used, you will see brighter BG colors. The test
expects results for standard background colors (indexes 0-7).
 
> Anyway, the fix for patch 2/8 is rather simple (see below), do you want
> to fix this up in your tree? Or shall I sent a v2?

Thanks! I've merged your fix for patch 2/8, no need to resend. Build-
testing now.

--
Anatolij

[U-Boot] [PATCH 3/8] video/console: Implement relative cursor movement ANSI handling

[André Przywara]

On 14/04/2019 13:54, Anatolij Gustschin wrote:
> Hi André,
> 
> On Sat, 13 Apr 2019 22:40:03 +0100
> André Przywara andre.przywara at arm.com wrote:
> ...
>>> I've dropped all applied patches of this series now, some of them
>>> introduced dm video_ansi test error [1]. Please fix. Thanks!  
>>
>> Hmh, good one. Didn't find an easy way to get to the bottom of this
>> within the ut test system, so I copied the ANSI sequences out and
>> replayed them with a custom command, inspecting the (sandbox) screen
>> manually. Is there a canonical way to trace down those issues?
> 
> You can build sandbox defconfig an run U-Boot with "--show_lcd"
> option, then run the "ut dm video_ansi" command to inspect the
> rendered characters and colors, e.g.:
> 
> $ make sandbox_defconfig
> $ make
> $ ./u-boot -d arch/sandbox/dts/test.dtb -l

Ah, I did everything exactly as you, except I was using "-D"
(sandbox64.dtb), not test.dtb. With sandbox64.dtb I could run the test
and see the test summary report, but didn't see the actual graphical output.

> U-Boot 2019.04-00326-g7b64a70a3a (Apr 14 2019 - 14:37:44 +0200)
> 
> Model: sandbox
> DRAM:  128 MiB
> MMC:   mmc2: 2 (SD), mmc1: 1 (SD), mmc0: 0 (SD)
> In:    serial
> Out:   vidconsole
> Err:   vidconsole
> Model: sandbox
> SCSI:  
> Net:   eth0: eth at 10002000, eth5: eth at 10003000, eth3: sbe5, eth1: eth at 10004000
> Hit any key to stop autoboot:  0 
> => ut dm video_ansi
> Test: dm_test_video_ansi: video.c
> Failures: 0
> 
> In the "U-Boot" window you will see the output of the video_ansi test.
> 
> Without your fix for masking bit 3 in the bg index, the high-intensity
> background colors will be used, you will see brighter BG colors. The test
> expects results for standard background colors (indexes 0-7).

Yes, that's what I figured after sending the same escape sequences as
the test from a hacked "cls" command. But it took me actually a
screenshot to spot the difference ;-)

>> Anyway, the fix for patch 2/8 is rather simple (see below), do you want
>> to fix this up in your tree? Or shall I sent a v2?
> 
> Thanks! I've merged your fix for patch 2/8, no need to resend. Build-
> testing now.

Great, thanks a lot!

Cheers,
Andre.

[U-Boot] [PATCH 4/8] video/console: Implement ANSI clear line command

[Andre Przywara]

There is a standard ANSI terminal escape sequence to clear a whole line
of text. So far the DM_VIDEO console was missing this code.

Detect the sequence and use vidconsole_set_row with the background
colour to fix this omission.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 drivers/video/vidconsole-uclass.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c
index cbd63f0ce8..7d914ed5ca 100644
--- a/drivers/video/vidconsole-uclass.c
+++ b/drivers/video/vidconsole-uclass.c
@@ -346,6 +346,25 @@ static void vidconsole_escape_char(struct udevice *dev, char ch)
 		}
 		break;
 	}
+	case 'K': {
+		struct video_priv *vid_priv = dev_get_uclass_priv(dev->parent);
+		int mode;
+
+		/*
+		 * Clear (parts of) current line
+		 *   [0K       - clear line to end
+		 *   [2K       - clear entire line
+		 */
+		parsenum(priv->escape_buf + 1, &mode);
+
+		if (mode == 2) {
+			int row, col;
+
+			get_cursor_position(priv, &row, &col);
+			vidconsole_set_row(dev, row, vid_priv->colour_bg);
+		}
+		break;
+	}
 	case 'm': {
 		struct video_priv *vid_priv = dev_get_uclass_priv(dev->parent);
 		char *s = priv->escape_buf;
-- 
2.14.5

[U-Boot] [PATCH 4/8] video/console: Implement ANSI clear line command

[Simon Glass]

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> There is a standard ANSI terminal escape sequence to clear a whole line
> of text. So far the DM_VIDEO console was missing this code.
>
> Detect the sequence and use vidconsole_set_row with the background
> colour to fix this omission.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
>

Reviewed-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH 4/8] video/console: Implement ANSI clear line command

[Anatolij Gustschin]

On Sat, 23 Mar 2019 01:29:58 +0000
Andre Przywara andre.przywara at arm.com wrote:

> There is a standard ANSI terminal escape sequence to clear a whole line
> of text. So far the DM_VIDEO console was missing this code.
> 
> Detect the sequence and use vidconsole_set_row with the background
> colour to fix this omission.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)

Applied to u-boot-video/master, thanks!

--
Anatolij

[U-Boot] [PATCH 5/8] video/console: Factor out actual character output

[Andre Przywara]

In preparation for doing character set translations, factor out the
actual glyph display functionality into a separate function.
This will be used in a subsequent patch.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 drivers/video/vidconsole-uclass.c | 42 +++++++++++++++++++++++++--------------
 1 file changed, 27 insertions(+), 15 deletions(-)

diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c
index 7d914ed5ca..e16567029a 100644
--- a/drivers/video/vidconsole-uclass.c
+++ b/drivers/video/vidconsole-uclass.c
@@ -457,6 +457,32 @@ error:
 	priv->escape = 0;
 }
 
+/* Put that actual character on the screen (using the CP437 code page). */
+static int vidconsole_output_glyph(struct udevice *dev, char ch)
+{
+	struct vidconsole_priv *priv = dev_get_uclass_priv(dev);
+	int ret;
+
+	/*
+	 * Failure of this function normally indicates an unsupported
+	 * colour depth. Check this and return an error to help with
+	 * diagnosis.
+	 */
+	ret = vidconsole_putc_xy(dev, priv->xcur_frac, priv->ycur, ch);
+	if (ret == -EAGAIN) {
+		vidconsole_newline(dev);
+		ret = vidconsole_putc_xy(dev, priv->xcur_frac, priv->ycur, ch);
+	}
+	if (ret < 0)
+		return ret;
+	priv->xcur_frac += ret;
+	priv->last_ch = ch;
+	if (priv->xcur_frac >= priv->xsize_frac)
+		vidconsole_newline(dev);
+
+	return 0;
+}
+
 int vidconsole_put_char(struct udevice *dev, char ch)
 {
 	struct vidconsole_priv *priv = dev_get_uclass_priv(dev);
@@ -494,23 +520,9 @@ int vidconsole_put_char(struct udevice *dev, char ch)
 		priv->last_ch = 0;
 		break;
 	default:
-		/*
-		 * Failure of this function normally indicates an unsupported
-		 * colour depth. Check this and return an error to help with
-		 * diagnosis.
-		 */
-		ret = vidconsole_putc_xy(dev, priv->xcur_frac, priv->ycur, ch);
-		if (ret == -EAGAIN) {
-			vidconsole_newline(dev);
-			ret = vidconsole_putc_xy(dev, priv->xcur_frac,
-						 priv->ycur, ch);
-		}
+		ret = vidconsole_output_glyph(dev, ch);
 		if (ret < 0)
 			return ret;
-		priv->xcur_frac += ret;
-		priv->last_ch = ch;
-		if (priv->xcur_frac >= priv->xsize_frac)
-			vidconsole_newline(dev);
 		break;
 	}
 
-- 
2.14.5

[U-Boot] [PATCH 5/8] video/console: Factor out actual character output

[Simon Glass]

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> In preparation for doing character set translations, factor out the
> actual glyph display functionality into a separate function.
> This will be used in a subsequent patch.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 42 +++++++++++++++++++++++++--------------
>  1 file changed, 27 insertions(+), 15 deletions(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH 5/8] video/console: Factor out actual character output

[Anatolij Gustschin]

On Sat, 23 Mar 2019 01:29:59 +0000
Andre Przywara andre.przywara at arm.com wrote:

> In preparation for doing character set translations, factor out the
> actual glyph display functionality into a separate function.
> This will be used in a subsequent patch.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/vidconsole-uclass.c | 42 +++++++++++++++++++++++++--------------
>  1 file changed, 27 insertions(+), 15 deletions(-)

Applied to u-boot-video/master, thanks!

--
Anatolij

[U-Boot] [PATCH 6/8] video/console: Convert UTF-8 codes to CP437 code points

[Andre Przywara]

The character set used by U-Boot's built-in fonts is the old "code
page 437" (from the original IBM PC).
However people would probably expect UTF-8 on a terminal these days, the
UEFI code definitely does.

Provide a conversion routine to convert a UTF-8 byte stream into a CP437
character code. This uses a combination of arrays and switch/case
statements to provide an efficient way of translating the large Unicode
character range to the 8 bits used for CP437.

This fixes UEFI display on the DM_VIDEO console, which were garbled for
any non-ASCII characters, for instance for the block graphic characters
used by Grub to display the menu.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 drivers/video/Makefile            |   1 +
 drivers/video/utf8_cp437.c        | 170 ++++++++++++++++++++++++++++++++++++++
 drivers/video/vidconsole-uclass.c |   8 +-
 include/video_console.h           |   9 ++
 4 files changed, 186 insertions(+), 2 deletions(-)
 create mode 100644 drivers/video/utf8_cp437.c

diff --git a/drivers/video/Makefile b/drivers/video/Makefile
index 671f037c35..8decf407bb 100644
--- a/drivers/video/Makefile
+++ b/drivers/video/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_CONSOLE_TRUETYPE) += console_truetype.o fonts/
 obj-$(CONFIG_DISPLAY) += display-uclass.o
 obj-$(CONFIG_DM_VIDEO) += backlight-uclass.o
 obj-$(CONFIG_DM_VIDEO) += panel-uclass.o simple_panel.o
+obj-$(CONFIG_DM_VIDEO) += utf8_cp437.o
 obj-$(CONFIG_DM_VIDEO) += video-uclass.o vidconsole-uclass.o
 obj-$(CONFIG_DM_VIDEO) += video_bmp.o
 endif
diff --git a/drivers/video/utf8_cp437.c b/drivers/video/utf8_cp437.c
new file mode 100644
index 0000000000..983da39406
--- /dev/null
+++ b/drivers/video/utf8_cp437.c
@@ -0,0 +1,170 @@
+/*
+ * Convert UTF-8 bytes into a code page 437 character.
+ * Based on the table in the Code_page_437 Wikipedia page.
+ */
+
+#include <stdint.h>
+
+static uint8_t code_points_00a0[] = {
+	255, 173, 155, 156,   7, 157,   7,  21,
+	  7,   7, 166, 174, 170,   7,   7,   7,
+	248, 241, 253,   7,   7, 230,  20, 250,
+	  7,   7, 167, 175, 172, 171,   7, 168,
+	  7,   7,   7,   7, 142, 143, 146, 128,
+	  7, 144,   7,   7,   7,   7,   7,   7,
+	  7, 165,   7,   7,   7,   7, 153,   7,
+	  7,   7,   7,   7, 154,   7,   7, 225,
+	133, 160, 131,   7, 132, 134, 145, 135,
+	138, 130, 136, 137, 141, 161, 140, 139,
+	  7, 164, 149, 162, 147,   7, 148, 246,
+	  7, 151, 163, 150, 129,   7,   7, 152,
+};
+
+static uint8_t code_points_2550[] = {
+	205, 186, 213, 214, 201, 184, 183, 187,
+	212, 211, 200, 190, 189, 188, 198, 199,
+	204, 181, 182, 185, 209, 210, 203, 207,
+	208, 202, 216, 215, 206
+};
+
+static uint8_t utf8_convert_11bit(uint16_t code)
+{
+	switch (code) {
+	case 0x0192: return 159;
+	case 0x0393: return 226;
+	case 0x0398: return 233;
+	case 0x03A3: return 228;
+	case 0x03A6: return 232;
+	case 0x03A9: return 234;
+	case 0x03B1: return 224;
+	case 0x03B4: return 235;
+	case 0x03B5: return 238;
+	case 0x03C0: return 227;
+	case 0x03C3: return 229;
+	case 0x03C4: return 231;
+	case 0x03C6: return 237;
+	}
+
+	return 0;
+};
+
+static uint8_t utf8_convert_2xxx(uint16_t code)
+{
+	switch (code) {
+	case 0x2022: return 7;
+	case 0x203C: return 19;
+	case 0x207F: return 252;
+	case 0x20A7: return 158;
+	case 0x2190: return 27;
+	case 0x2191: return 24;
+	case 0x2192: return 26;
+	case 0x2193: return 25;
+	case 0x2194: return 29;
+	case 0x2195: return 18;
+	case 0x21A8: return 23;
+	case 0x2219: return 249;
+	case 0x221A: return 251;
+	case 0x221E: return 236;
+	case 0x221F: return 28;
+	case 0x2229: return 239;
+	case 0x2248: return 247;
+	case 0x2261: return 240;
+	case 0x2264: return 243;
+	case 0x2265: return 242;
+	case 0x2310: return 169;
+	case 0x2320: return 244;
+	case 0x2321: return 245;
+	case 0x2500: return 196;
+	case 0x2502: return 179;
+	case 0x250C: return 218;
+	case 0x2510: return 191;
+	case 0x2514: return 192;
+	case 0x2518: return 217;
+	case 0x251C: return 195;
+	case 0x2524: return 180;
+	case 0x252C: return 194;
+	case 0x2534: return 193;
+	case 0x253C: return 197;
+	case 0x2580: return 223;
+	case 0x2584: return 220;
+	case 0x2588: return 219;
+	case 0x258C: return 221;
+	case 0x2590: return 222;
+	case 0x2591: return 176;
+	case 0x2592: return 177;
+	case 0x2593: return 178;
+	case 0x25A0: return 254;
+	case 0x25AC: return 22;
+	case 0x25B2: return 30;
+	case 0x25BA: return 16;
+	case 0x25BC: return 31;
+	case 0x25C4: return 17;
+	case 0x25CB: return 9;
+	case 0x25D8: return 8;
+	case 0x25D9: return 10;
+	case 0x263A: return 1;
+	case 0x263B: return 2;
+	case 0x263C: return 15;
+	case 0x2640: return 12;
+	case 0x2642: return 11;
+	case 0x2660: return 6;
+	case 0x2663: return 5;
+	case 0x2665: return 3;
+	case 0x2666: return 4;
+	case 0x266A: return 13;
+	case 0x266B: return 14;
+	}
+
+	return 0;
+}
+
+uint8_t convert_uc16_to_cp437(uint16_t code)
+{
+	if (code < 0x7f)		// ASCII
+		return code;
+	if (code < 0xa0)		// high control characters
+		return code;
+	if (code < 0x100)		// international characters
+		return code_points_00a0[code - 0xa0];
+	if (code < 0x800)
+		return utf8_convert_11bit(code);
+	if (code >= 0x2550 && code < 0x256d)	// block graphics
+		return code_points_2550[code - 0x2550];
+
+	return utf8_convert_2xxx(code);
+}
+
+uint8_t convert_utf8_to_cp437(uint8_t c, uint32_t *esc)
+{
+	int shift;
+	uint32_t ucp;
+
+	if (c < 127)			// ASCII
+		return c;
+	if (c == 127)
+		return 8;		// DEL (?)
+
+	switch (c & 0xf0) {
+	case 0xc0: case 0xd0:		// two bytes sequence
+		*esc = (1U << 24) | ((c & 0x1f) << 6);
+		return 0;
+	case 0xe0:			// three bytes sequence
+		*esc = (2U << 24) | ((c & 0x0f) << 12);
+		return 0;
+	case 0xf0:			// four bytes sequence
+		*esc = (3U << 24) | ((c & 0x07) << 18);
+		return 0;
+	case 0x80: case 0x90: case 0xa0: case 0xb0:	// continuation
+		shift = (*esc >> 24) - 1;
+		ucp = *esc & 0xffffff;
+		if (shift) {
+			*esc = (shift << 24) | ucp | (c & 0x3f) << (shift * 6);
+			return 0;
+		}
+		*esc = 0;
+
+		return convert_uc16_to_cp437(ucp | (c & 0x3f));
+	}
+
+	return 0;
+}
diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c
index e16567029a..275c6c05c8 100644
--- a/drivers/video/vidconsole-uclass.c
+++ b/drivers/video/vidconsole-uclass.c
@@ -457,7 +457,7 @@ error:
 	priv->escape = 0;
 }
 
-/* Put that actual character on the screen (using the CP437 code page). */
+/* Put that actual character on the screen (using the font native code page). */
 static int vidconsole_output_glyph(struct udevice *dev, char ch)
 {
 	struct vidconsole_priv *priv = dev_get_uclass_priv(dev);
@@ -486,6 +486,7 @@ static int vidconsole_output_glyph(struct udevice *dev, char ch)
 int vidconsole_put_char(struct udevice *dev, char ch)
 {
 	struct vidconsole_priv *priv = dev_get_uclass_priv(dev);
+	uint8_t glyph_idx;
 	int ret;
 
 	if (priv->escape) {
@@ -520,7 +521,10 @@ int vidconsole_put_char(struct udevice *dev, char ch)
 		priv->last_ch = 0;
 		break;
 	default:
-		ret = vidconsole_output_glyph(dev, ch);
+		glyph_idx = convert_utf8_to_cp437(ch, &priv->ucs);
+		if (glyph_idx == 0)	/* UTF-8 continuation */
+			return 0;
+		ret = vidconsole_output_glyph(dev, glyph_idx);
 		if (ret < 0)
 			return ret;
 		break;
diff --git a/include/video_console.h b/include/video_console.h
index 52a41ac200..07e5fd0226 100644
--- a/include/video_console.h
+++ b/include/video_console.h
@@ -81,6 +81,7 @@ struct vidconsole_priv {
 	int escape_len;
 	int row_saved;
 	int col_saved;
+	u32 ucs;
 	char escape_buf[32];
 };
 
@@ -240,6 +241,14 @@ void vidconsole_position_cursor(struct udevice *dev, unsigned col,
  */
 u32 vid_console_color(struct video_priv *priv, unsigned int idx);
 
+/*
+ * Convert an UTF-8 byte into the corresponding character in the CP437
+ * code page. Returns 0 if that character is part of a multi-byte sequence.
+ * for which *esc holds the state of. Repeatedly feed in more bytes until
+ * the return value is not 0 anymore.
+ */
+uint8_t convert_utf8_to_cp437(uint8_t c, uint32_t *esc);
+
 #endif
 
 #endif
-- 
2.14.5

[U-Boot] [PATCH 6/8] video/console: Convert UTF-8 codes to CP437 code points

[Simon Glass]

Hi Andre,

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> The character set used by U-Boot's built-in fonts is the old "code
> page 437" (from the original IBM PC).
> However people would probably expect UTF-8 on a terminal these days, the
> UEFI code definitely does.
>
> Provide a conversion routine to convert a UTF-8 byte stream into a CP437
> character code. This uses a combination of arrays and switch/case
> statements to provide an efficient way of translating the large Unicode
> character range to the 8 bits used for CP437.
>
> This fixes UEFI display on the DM_VIDEO console, which were garbled for
> any non-ASCII characters, for instance for the block graphic characters
> used by Grub to display the menu.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/Makefile            |   1 +
>  drivers/video/utf8_cp437.c        | 170 ++++++++++++++++++++++++++++++++++++++
>  drivers/video/vidconsole-uclass.c |   8 +-
>  include/video_console.h           |   9 ++
>  4 files changed, 186 insertions(+), 2 deletions(-)
>  create mode 100644 drivers/video/utf8_cp437.c

OMG unicode comes to U-Boot. This might be the beginning of the end.

Can we make this a Kconfig option to avoid increasing code size? We
can imply it when EFI is enabled.

Regards,
Simon

[U-Boot] [PATCH 6/8] video/console: Convert UTF-8 codes to CP437 code points

[Alexander Graf]

On 31.03.19 04:18, Simon Glass wrote:
> Hi Andre,
>
> On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>> The character set used by U-Boot's built-in fonts is the old "code
>> page 437" (from the original IBM PC).
>> However people would probably expect UTF-8 on a terminal these days, the
>> UEFI code definitely does.
>>
>> Provide a conversion routine to convert a UTF-8 byte stream into a CP437
>> character code. This uses a combination of arrays and switch/case
>> statements to provide an efficient way of translating the large Unicode
>> character range to the 8 bits used for CP437.
>>
>> This fixes UEFI display on the DM_VIDEO console, which were garbled for
>> any non-ASCII characters, for instance for the block graphic characters
>> used by Grub to display the menu.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>>  drivers/video/Makefile            |   1 +
>>  drivers/video/utf8_cp437.c        | 170 ++++++++++++++++++++++++++++++++++++++
>>  drivers/video/vidconsole-uclass.c |   8 +-
>>  include/video_console.h           |   9 ++
>>  4 files changed, 186 insertions(+), 2 deletions(-)
>>  create mode 100644 drivers/video/utf8_cp437.c
> OMG unicode comes to U-Boot. This might be the beginning of the end.
>
> Can we make this a Kconfig option to avoid increasing code size? We
> can imply it when EFI is enabled.


This looks vaguely familiar. Take a look at include/cp437.h. We even
have a Kconfig option for it already :).

Alex


>
> Regards,
> Simon

[U-Boot] [PATCH 6/8] video/console: Convert UTF-8 codes to CP437 code points

[André Przywara]

On 31/03/2019 19:28, Alexander Graf wrote:

Hi Simon, Alex,

> On 31.03.19 04:18, Simon Glass wrote:
>> Hi Andre,
>>
>> On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>>> The character set used by U-Boot's built-in fonts is the old "code
>>> page 437" (from the original IBM PC).
>>> However people would probably expect UTF-8 on a terminal these days, the
>>> UEFI code definitely does.
>>>
>>> Provide a conversion routine to convert a UTF-8 byte stream into a CP437
>>> character code. This uses a combination of arrays and switch/case
>>> statements to provide an efficient way of translating the large Unicode
>>> character range to the 8 bits used for CP437.
>>>
>>> This fixes UEFI display on the DM_VIDEO console, which were garbled for
>>> any non-ASCII characters, for instance for the block graphic characters
>>> used by Grub to display the menu.
>>>
>>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>>> ---
>>>  drivers/video/Makefile            |   1 +
>>>  drivers/video/utf8_cp437.c        | 170 ++++++++++++++++++++++++++++++++++++++
>>>  drivers/video/vidconsole-uclass.c |   8 +-
>>>  include/video_console.h           |   9 ++
>>>  4 files changed, 186 insertions(+), 2 deletions(-)
>>>  create mode 100644 drivers/video/utf8_cp437.c
>> OMG unicode comes to U-Boot. This might be the beginning of the end.

Well, while I can understand reservations against the (complexity of)
Unicode, but it isn't too bad after all:
- We don't blow everything up to 16 bits, instead just use UTF-8 here,
which is a quite clever way of keeping things 8 bits mostly.
- There is some Unicode in U-Boot already, namely in the UEFI code. Here
the UCS-2 encoding (fixed 16 bits) used in UEFI get converted into
UTF-8. This works nicely these days because on serial lines there is
probably an UTF-8 capable terminal emulator on the other end.
- The Truetype font console already uses a Unicode-to-glyph-ID mapping.
It's just not in affect because it expects an int, but we only have a
signed char as in input, so effectively limit everything to 7-bit ASCII.

>> Can we make this a Kconfig option to avoid increasing code size? We
>> can imply it when EFI is enabled.

Sure, sounds easy enough. Just be aware that the current situation is
somewhat broken, since Truetype is somewhat ISO8859-1/ASCII, bitmap
fonts use CP437, but serial terminal (emulators) use probably UTF-8
these days. So the option would be to switch between (7-bit) ASCII and
Unicode? Or between "current mess" and Unicode?

Actually I found a better way to fix both bitmap and Truetype fonts in a
joint effort, a so I will send a different patch later on, considering a
Kconfig option.

> This looks vaguely familiar. Take a look at include/cp437.h. We even
> have a Kconfig option for it already :).

But this is a) for converting FAT file name entries, and b) is from
CP437 to Unicode, but we need it the other way round (incoming
UCS-2/UTF-8 to CP437 glyphs). I briefly considered a reverse lookup
scheme, but found this switch-case/array-look-up combination more
elegant, given that we need to do this on every character.

Cheers,
Andre.

[U-Boot] [PATCH 7/8] usb: kbd: Properly translate up/down arrow keys

[Andre Przywara]

So far arrows key pressed on an USB keyboard got translated to some
low ASCII control sequences (Ctrl+N, Ctrl+P). Some programs understand
these codes, but the standard for those keys is to use ANSI control
sequences for cursor movement (ESC [ A).
Our own boot menu is a victim of this, currently we cannot change the
selection with an USB keyboard due to this.

Since we already implement a queue for USB key codes, we can just insert
the three character ANSI sequence into the key buffer. This fixes the
bootmenu, and is more universal for other users (UEFI) as well.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 common/usb_kbd.c | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/common/usb_kbd.c b/common/usb_kbd.c
index 020f0d4117..cc99c6be07 100644
--- a/common/usb_kbd.c
+++ b/common/usb_kbd.c
@@ -145,6 +145,12 @@ static void usb_kbd_put_queue(struct usb_kbd_pdata *data, char c)
 	data->usb_kbd_buffer[data->usb_in_pointer] = c;
 }
 
+static void usb_kbd_put_sequence(struct usb_kbd_pdata *data, char *s)
+{
+	for (; *s; s++)
+		usb_kbd_put_queue(data, *s);
+}
+
 /*
  * Set the LEDs. Since this is used in the irq routine, the control job is
  * issued with a timeout of 0. This means, that the job is queued without
@@ -235,9 +241,25 @@ static int usb_kbd_translate(struct usb_kbd_pdata *data, unsigned char scancode,
 	}
 
 	/* Report keycode if any */
-	if (keycode) {
+	if (keycode)
 		debug("%c", keycode);
+
+	switch (keycode) {
+	case 0x0e:					/* Down arrow key */
+		usb_kbd_put_sequence(data, "\e[B");
+		break;
+	case 0x10:					/* Up arrow key */
+		usb_kbd_put_sequence(data, "\e[A");
+		break;
+	case 0x06:					/* Right arrow key */
+		usb_kbd_put_sequence(data, "\e[C");
+		break;
+	case 0x02:					/* Left arrow key */
+		usb_kbd_put_sequence(data, "\e[D");
+		break;
+	default:
 		usb_kbd_put_queue(data, keycode);
+		break;
 	}
 
 	return 0;
-- 
2.14.5

[U-Boot] [PATCH 7/8] usb: kbd: Properly translate up/down arrow keys

[Simon Glass]

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> So far arrows key pressed on an USB keyboard got translated to some
> low ASCII control sequences (Ctrl+N, Ctrl+P). Some programs understand
> these codes, but the standard for those keys is to use ANSI control

Which standard?

> sequences for cursor movement (ESC [ A).
> Our own boot menu is a victim of this, currently we cannot change the
> selection with an USB keyboard due to this.
>
> Since we already implement a queue for USB key codes, we can just insert
> the three character ANSI sequence into the key buffer. This fixes the
> bootmenu, and is more universal for other users (UEFI) as well.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  common/usb_kbd.c | 24 +++++++++++++++++++++++-
>  1 file changed, 23 insertions(+), 1 deletion(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH 7/8] usb: kbd: Properly translate up/down arrow keys

[André Przywara]

On 30/03/2019 21:18, Simon Glass wrote:
> On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:

Hi Simon,

many thanks for the review of all those patches, much appreciated!

>> So far arrows key pressed on an USB keyboard got translated to some
>> low ASCII control sequences (Ctrl+N, Ctrl+P). Some programs understand
>> these codes, but the standard for those keys is to use ANSI control
> 
> Which standard?

The only real standard for encoding arrow keys seems to be "ANSI
terminal" escape sequences, I think ECMA-48 is the official name(?)
Also since our very own U-Boot boot menu requires this ...

Cheers,
Andre

>> sequences for cursor movement (ESC [ A).
>> Our own boot menu is a victim of this, currently we cannot change the
>> selection with an USB keyboard due to this.
>>
>> Since we already implement a queue for USB key codes, we can just insert
>> the three character ANSI sequence into the key buffer. This fixes the
>> bootmenu, and is more universal for other users (UEFI) as well.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>>  common/usb_kbd.c | 24 +++++++++++++++++++++++-
>>  1 file changed, 23 insertions(+), 1 deletion(-)
> 
> Reviewed-by: Simon Glass <sjg@chromium.org>
> 

[U-Boot] [PATCH 7/8] usb: kbd: Properly translate up/down arrow keys

[Simon Glass]

Hi André,

On Sun, 31 Mar 2019 at 18:04, André Przywara <andre.przywara@arm.com> wrote:
>
> On 30/03/2019 21:18, Simon Glass wrote:
> > On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> Hi Simon,
>
> many thanks for the review of all those patches, much appreciated!

You're welcome, your patches are very easy to review.

>
> >> So far arrows key pressed on an USB keyboard got translated to some
> >> low ASCII control sequences (Ctrl+N, Ctrl+P). Some programs understand
> >> these codes, but the standard for those keys is to use ANSI control
> >
> > Which standard?
>
> The only real standard for encoding arrow keys seems to be "ANSI
> terminal" escape sequences, I think ECMA-48 is the official name(?)
> Also since our very own U-Boot boot menu requires this ...

OK I see, thanks.

Regards,
Simon

[..]

[U-Boot] [PATCH 7/8] usb: kbd: Properly translate up/down arrow keys

[Anatolij Gustschin]

On Sat, 23 Mar 2019 01:30:01 +0000
Andre Przywara andre.przywara at arm.com wrote:

> So far arrows key pressed on an USB keyboard got translated to some
> low ASCII control sequences (Ctrl+N, Ctrl+P). Some programs understand
> these codes, but the standard for those keys is to use ANSI control
> sequences for cursor movement (ESC [ A).
> Our own boot menu is a victim of this, currently we cannot change the
> selection with an USB keyboard due to this.
> 
> Since we already implement a queue for USB key codes, we can just insert
> the three character ANSI sequence into the key buffer. This fixes the
> bootmenu, and is more universal for other users (UEFI) as well.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  common/usb_kbd.c | 24 +++++++++++++++++++++++-
>  1 file changed, 23 insertions(+), 1 deletion(-)

Applied to u-boot-video/master, thanks!

--
Anatolij

[U-Boot] [PATCH 8/8] sunxi: allow boards to de-select SYS_WHITE_ON_BLACK font scheme

[Andre Przywara]

In the sunxi-common.h config header we unconditionally define
CONFIG_SYS_WHITE_ON_BLACK, although it's actually a Kconfig option which
could be individually selected by a user.
Remove this #define from the header and let it default to "y" on sunxi
boards (like we do for other platforms).

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 drivers/video/Kconfig          | 2 +-
 include/configs/sunxi-common.h | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig
index 2eac4b6381..43412873c9 100644
--- a/drivers/video/Kconfig
+++ b/drivers/video/Kconfig
@@ -120,7 +120,7 @@ config CONSOLE_TRUETYPE_SIZE
 
 config SYS_WHITE_ON_BLACK
 	bool "Display console as white on a black background"
-	default y if ARCH_AT91 || ARCH_EXYNOS || ARCH_ROCKCHIP || TEGRA || X86
+	default y if ARCH_AT91 || ARCH_EXYNOS || ARCH_ROCKCHIP || TEGRA || X86 || ARCH_SUNXI
 	help
 	 Normally the display is black on a white background, Enable this
 	 option to invert this, i.e. white on a black background. This can be
diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi-common.h
index b01d1c3c84..ee18260be6 100644
--- a/include/configs/sunxi-common.h
+++ b/include/configs/sunxi-common.h
@@ -449,7 +449,6 @@ extern int soft_i2c_gpio_scl;
 	"stdout=serial,vga\0" \
 	"stderr=serial,vga\0"
 #elif CONFIG_DM_VIDEO
-#define CONFIG_SYS_WHITE_ON_BLACK
 #define CONSOLE_STDOUT_SETTINGS \
 	"stdout=serial,vidconsole\0" \
 	"stderr=serial,vidconsole\0"
-- 
2.14.5

[U-Boot] [PATCH 8/8] sunxi: allow boards to de-select SYS_WHITE_ON_BLACK font scheme

[Simon Glass]

On Fri, 22 Mar 2019 at 19:32, Andre Przywara <andre.przywara@arm.com> wrote:
>
> In the sunxi-common.h config header we unconditionally define
> CONFIG_SYS_WHITE_ON_BLACK, although it's actually a Kconfig option which
> could be individually selected by a user.
> Remove this #define from the header and let it default to "y" on sunxi
> boards (like we do for other platforms).
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/Kconfig          | 2 +-
>  include/configs/sunxi-common.h | 1 -
>  2 files changed, 1 insertion(+), 2 deletions(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

[U-Boot] [PATCH 8/8] sunxi: allow boards to de-select SYS_WHITE_ON_BLACK font scheme

[Anatolij Gustschin]

On Sat, 23 Mar 2019 01:30:02 +0000
Andre Przywara andre.przywara at arm.com wrote:

> In the sunxi-common.h config header we unconditionally define
> CONFIG_SYS_WHITE_ON_BLACK, although it's actually a Kconfig option which
> could be individually selected by a user.
> Remove this #define from the header and let it default to "y" on sunxi
> boards (like we do for other platforms).
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  drivers/video/Kconfig          | 2 +-
>  include/configs/sunxi-common.h | 1 -
>  2 files changed, 1 insertion(+), 2 deletions(-)

Applied to u-boot-video/master, thanks!

--
Anatolij