I can't apply this.
git am says:
error: cannot convert from Y to UTF-8
fatal: could not parse patch
Key is When git send-email asks for UTF-8 encoding for the patch then do not enter “y” but just press
Enter
Thanks,
Anuj
On Wed, 2021-12-29 at 01:13 -0800, pgowda wrote:
> Upstream-Status: Backport
> [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad
> 4905595e843d5665a5]
> Upstream-Status: Backport
> [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bd5e882cf6e0def3dd1bc1
> 06075d59a303fe0d1e]
> Upstream-Status: Backport
> [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=51c500269bf53749b10780
> 7d84271385fad35628]
> Upstream-Status: Backport
> [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=1a7f2c0774129750fdf73e
> 9f1b78f0ce983c9ab3]
> Upstream-Status: Backport
> [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bef32d4a28595e933f24fe
> f378cf052a30b674a7]
>
> Signed-off-by: pgowda <pgowda.cve@gmail.com>
> ---
> meta/recipes-devtools/gcc/gcc-10.2.inc | 5 +
> .../gcc/gcc/0001-CVE-2021-42574.patch | 2906
> +++++++++++++++++
> .../gcc/gcc/0002-CVE-2021-42574.patch | 2270 +++++++++++++
> .../gcc/gcc/0003-CVE-2021-42574.patch | 1724 ++++++++++
> .../gcc/gcc/0004-CVE-2021-42574.patch | 138 +
> .../gcc/gcc/0005-CVE-2021-42574.patch | 575 ++++
> 6 files changed, 7618 insertions(+)
> create mode 100644 meta/recipes-devtools/gcc/gcc/0001-CVE-2021-
> 42574.patch
> create mode 100644 meta/recipes-devtools/gcc/gcc/0002-CVE-2021-
> 42574.patch
> create mode 100644 meta/recipes-devtools/gcc/gcc/0003-CVE-2021-
> 42574.patch
> create mode 100644 meta/recipes-devtools/gcc/gcc/0004-CVE-2021-
> 42574.patch
> create mode 100644 meta/recipes-devtools/gcc/gcc/0005-CVE-2021-
> 42574.patch
>
> diff --git a/meta/recipes-devtools/gcc/gcc-10.2.inc b/meta/recipes-
> devtools/gcc/gcc-10.2.inc
> index 5626bf20f0..fd7f16d7bf 100644
> --- a/meta/recipes-devtools/gcc/gcc-10.2.inc
> +++ b/meta/recipes-devtools/gcc/gcc-10.2.inc
> @@ -74,6 +74,11 @@ SRC_URI = "\
> file://0002-CVE-2021-35465.patch \
> file://0003-CVE-2021-35465.patch \
> file://0004-CVE-2021-35465.patch \
> + file://0001-CVE-2021-42574.patch \
> + file://0002-CVE-2021-42574.patch \
> + file://0003-CVE-2021-42574.patch \
> + file://0004-CVE-2021-42574.patch \
> + file://0005-CVE-2021-42574.patch \
> "
> SRC_URI[sha256sum] =
> "b8dd4368bb9c7f0b98188317ee0254dd8cc99d1e3a18d0ff146c855fe16c1d8c"
>
> diff --git a/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch
> b/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch
> new file mode 100644
> index 0000000000..e0f4f7d32f
> --- /dev/null
> +++ b/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch
> @@ -0,0 +1,2906 @@
> +From 004bb936d6d5f177af26ad4905595e843d5665a5 Mon Sep 17 00:00:00
> 2001
> +From: Lewis Hyatt <lhyatt@gmail.com>
> +Date: Tue, 14 Jul 2020 12:05:56 -0400
> +Subject: [PATCH] diagnostics: Support conversion of tabs to spaces
> [PR49973]
> + [PR86904]
> +
> +Supports conversion of tabs to spaces when outputting diagnostics.
> Also
> +adds -fdiagnostics-column-unit and -fdiagnostics-column-origin
> options to
> +control how the column number is output, thereby resolving the two
> PRs.
> +
> +gcc/c-family/ChangeLog:
> +
> + PR other/86904
> + * c-indentation.c (should_warn_for_misleading_indentation):
> Get
> + global tabstop from the new source.
> + * c-opts.c (c_common_handle_option): Remove handling of -
> ftabstop, which
> + is now a common option.
> + * c.opt: Likewise.
> +
> +gcc/ChangeLog:
> +
> + PR preprocessor/49973
> + PR other/86904
> + * common.opt: Handle -ftabstop here instead of in c-family
> + options. Add -fdiagnostics-column-unit= and
> + -fdiagnostics-column-origin= options.
> + * opts.c (common_handle_option): Handle the new options.
> + * diagnostic-format-json.cc (json_from_expanded_location):
> Add
> + diagnostic_context argument. Use it to convert column
> numbers as per
> + the new options.
> + (json_from_location_range): Likewise.
> + (json_from_fixit_hint): Likewise.
> + (json_end_diagnostic): Pass the new context argument to
> helper
> + functions above. Add "column-origin" field to the output.
> + (test_unknown_location): Add the new context argument to
> calls to
> + helper functions.
> + (test_bad_endpoints): Likewise.
> + * diagnostic-show-locus.c
> + (exploc_with_display_col::exploc_with_display_col): Support
> + tabstop parameter.
> + (layout_point::layout_point): Make use of class
> + exploc_with_display_col.
> + (layout_range::layout_range): Likewise.
> + (struct line_bounds): Clarify that the units are now always
> + display columns. Rename members accordingly. Add
> constructor.
> + (layout::print_source_line): Add support for tab expansion.
> + (make_range): Adapt to class layout_range changes.
> + (layout::maybe_add_location_range): Likewise.
> + (layout::layout): Adapt to class exploc_with_display_col
> changes.
> + (layout::calculate_x_offset_display): Support tabstop
> parameter.
> + (layout::print_annotation_line): Adapt to struct line_bounds
> changes.
> + (layout::print_line): Likewise.
> + (line_label::line_label): Add diagnostic_context argument.
> + (get_affected_range): Likewise.
> + (get_printed_columns): Likewise.
> + (layout::print_any_labels): Adapt to struct line_label
> changes.
> + (class correction): Add m_tabstop member.
> + (correction::correction): Add tabstop argument.
> + (correction::compute_display_cols): Use m_tabstop.
> + (class line_corrections): Add m_context member.
> + (line_corrections::line_corrections): Add diagnostic_context
> argument.
> + (line_corrections::add_hint): Use m_context to handle
> tabstops.
> + (layout::print_trailing_fixits): Adapt to class
> line_corrections
> + changes.
> + (test_layout_x_offset_display_utf8): Support tabstop
> parameter.
> + (test_layout_x_offset_display_tab): New selftest.
> + (test_one_liner_colorized_utf8): Likewise.
> + (test_tab_expansion): Likewise.
> + (test_diagnostic_show_locus_one_liner_utf8): Call the new
> tests.
> + (diagnostic_show_locus_c_tests): Likewise.
> + (test_overlapped_fixit_printing): Adapt to helper class and
> + function changes.
> + (test_overlapped_fixit_printing_utf8): Likewise.
> + (test_overlapped_fixit_printing_2): Likewise.
> + * diagnostic.h (enum diagnostics_column_unit): New enum.
> + (struct diagnostic_context): Add members for the new options.
> + (diagnostic_converted_column): Declare.
> + (json_from_expanded_location): Add new context argument.
> + * diagnostic.c (diagnostic_initialize): Initialize new
> members.
> + (diagnostic_converted_column): New function.
> + (maybe_line_and_column): Be willing to output a column of 0.
> + (diagnostic_get_location_text): Convert column number as per
> the new
> + options.
> + (diagnostic_report_current_module): Likewise.
> + (assert_location_text): Add origin and column_unit arguments
> for
> + testing the new functionality.
> + (test_diagnostic_get_location_text): Test the new
> functionality.
> + * doc/invoke.texi: Document the new options and behavior.
> + * input.h (location_compute_display_column): Add tabstop
> argument.
> + * input.c (location_compute_display_column): Likewise.
> + (test_cpp_utf8): Add selftests for tab expansion.
> + * tree-diagnostic-path.cc (default_tree_make_json_for_path):
> Pass the
> + new context argument to json_from_expanded_location().
> +
> +libcpp/ChangeLog:
> +
> + PR preprocessor/49973
> + PR other/86904
> + * include/cpplib.h (struct cpp_options): Removed support for
> -ftabstop,
> + which is now handled by diagnostic_context.
> + (class cpp_display_width_computation): New class.
> + (cpp_byte_column_to_display_column): Add optional tabstop
> argument.
> + (cpp_display_width): Likewise.
> + (cpp_display_column_to_byte_column): Likewise.
> + * charset.c
> + (cpp_display_width_computation::cpp_display_width_computation
> ): New
> + function.
> + (cpp_display_width_computation::advance_display_cols):
> Likewise.
> + (compute_next_display_width): Removed and implemented this
> + functionality in a new function...
> + (cpp_display_width_computation::process_next_codepoint):
> ...here.
> + (cpp_byte_column_to_display_column): Added tabstop argument.
> + Reimplemented in terms of class
> cpp_display_width_computation.
> + (cpp_display_column_to_byte_column): Likewise.
> + * init.c (cpp_create_reader): Remove handling of -ftabstop,
> which is now
> + handled by diagnostic_context.
> +
> +gcc/testsuite/ChangeLog:
> +
> + PR preprocessor/49973
> + PR other/86904
> + * c-c++-common/Wmisleading-indentation-3.c: Adjust expected
> output
> + for new defaults.
> + * c-c++-common/Wmisleading-indentation.c: Likewise.
> + * c-c++-common/diagnostic-format-json-1.c: Likewise.
> + * c-c++-common/diagnostic-format-json-2.c: Likewise.
> + * c-c++-common/diagnostic-format-json-3.c: Likewise.
> + * c-c++-common/diagnostic-format-json-4.c: Likewise.
> + * c-c++-common/diagnostic-format-json-5.c: Likewise.
> + * c-c++-common/missing-close-symbol.c: Likewise.
> + * g++.dg/diagnostic/bad-binary-ops.C: Likewise.
> + * g++.dg/parse/error4.C: Likewise.
> + * g++.old-deja/g++.brendan/crash11.C: Likewise.
> + * g++.old-deja/g++.pt/overload2.C: Likewise.
> + * g++.old-deja/g++.robertl/eb109.C: Likewise.
> + * gcc.dg/analyzer/malloc-paths-9.c: Likewise.
> + * gcc.dg/bad-binary-ops.c: Likewise.
> + * gcc.dg/format/branch-1.c: Likewise.
> + * gcc.dg/format/pr79210.c: Likewise.
> + * gcc.dg/plugin/diagnostic-test-expressions-1.c: Likewise.
> + * gcc.dg/plugin/diagnostic-test-string-literals-1.c:
> Likewise.
> + * gcc.dg/redecl-4.c: Likewise.
> + * gfortran.dg/diagnostic-format-json-1.F90: Likewise.
> + * gfortran.dg/diagnostic-format-json-2.F90: Likewise.
> + * gfortran.dg/diagnostic-format-json-3.F90: Likewise.
> + * go.dg/arrayclear.go: Add a comment explaining why adding a
> + comment was necessary to work around a dejagnu bug.
> + * c-c++-common/diagnostic-units-1.c: New test.
> + * c-c++-common/diagnostic-units-2.c: New test.
> + * c-c++-common/diagnostic-units-3.c: New test.
> + * c-c++-common/diagnostic-units-4.c: New test.
> + * c-c++-common/diagnostic-units-5.c: New test.
> + * c-c++-common/diagnostic-units-6.c: New test.
> + * c-c++-common/diagnostic-units-7.c: New test.
> + * c-c++-common/diagnostic-units-8.c: New test.
> +
> +CVE: CVE-2021-42574
> +Upstream-Status: Backport
> [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad
> 4905595e843d5665a5]
> +Signed-off-by: Pgowda <pgowda.cve@gmail.com>
> +---
> + gcc/c-family/c-indentation.c | 5 +-
> + gcc/c-family/c-opts.c | 6 -
> + gcc/c-family/c.opt | 4 -
> + gcc/common.opt | 21 +
> + gcc/diagnostic-format-json.cc | 55 +-
> + gcc/diagnostic-show-locus.c | 504 +++++++++++++--
> ---
> + gcc/diagnostic.c | 113 +++-
> + gcc/diagnostic.h | 28 +-
> + gcc/doc/invoke.texi | 68 ++-
> + gcc/input.c | 72 ++-
> + gcc/input.h | 4 +-
> + gcc/opts.c | 14 +
> + .../c-c++-common/Wmisleading-indentation-3.c | 12 +-
> + .../c-c++-common/Wmisleading-indentation.c | 6 +-
> + .../c-c++-common/diagnostic-format-json-1.c | 5 +
> + .../c-c++-common/diagnostic-format-json-2.c | 5 +
> + .../c-c++-common/diagnostic-format-json-3.c | 5 +
> + .../c-c++-common/diagnostic-format-json-4.c | 9 +
> + .../c-c++-common/diagnostic-format-json-5.c | 9 +
> + .../c-c++-common/diagnostic-units-1.c | 28 +
> + .../c-c++-common/diagnostic-units-2.c | 28 +
> + .../c-c++-common/diagnostic-units-3.c | 28 +
> + .../c-c++-common/diagnostic-units-4.c | 28 +
> + .../c-c++-common/diagnostic-units-5.c | 28 +
> + .../c-c++-common/diagnostic-units-6.c | 28 +
> + .../c-c++-common/diagnostic-units-7.c | 28 +
> + .../c-c++-common/diagnostic-units-8.c | 28 +
> + .../c-c++-common/missing-close-symbol.c | 6 +-
> + .../g++.dg/diagnostic/bad-binary-ops.C | 8 +-
> + gcc/testsuite/g++.dg/parse/error4.C | 2 +-
> + .../g++.old-deja/g++.brendan/crash11.C | 4 +-
> + gcc/testsuite/g++.old-deja/g++.pt/overload2.C | 2 +-
> + .../g++.old-deja/g++.robertl/eb109.C | 4 +-
> + .../gcc.dg/analyzer/malloc-paths-9.c | 2 +-
> + gcc/testsuite/gcc.dg/bad-binary-ops.c | 8 +-
> + gcc/testsuite/gcc.dg/format/branch-1.c | 2 +-
> + gcc/testsuite/gcc.dg/format/pr79210.c | 2 +-
> + .../plugin/diagnostic-test-expressions-1.c | 16 +-
> + .../diagnostic-test-string-literals-1.c | 4 +-
> + gcc/testsuite/gcc.dg/redecl-4.c | 2 +-
> + .../gfortran.dg/diagnostic-format-json-1.F90 | 5 +
> + .../gfortran.dg/diagnostic-format-json-2.F90 | 5 +
> + .../gfortran.dg/diagnostic-format-json-3.F90 | 5 +
> + gcc/testsuite/go.dg/arrayclear.go | 3 +
> + gcc/tree-diagnostic-path.cc | 5 +-
> + libcpp/charset.c | 98 ++--
> + libcpp/include/cpplib.h | 40 +-
> + libcpp/init.c | 1 -
> + 48 files changed, 1106 insertions(+), 287 deletions(-)
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-1.c
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-2.c
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-3.c
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-4.c
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-5.c
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-6.c
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-7.c
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-8.c
> +
> +diff --git a/gcc/c-family/c-indentation.c b/gcc/c-family/c-
> indentation.c
> +--- a/gcc/c-family/c-indentation.c 2020-07-22 23:35:17.296384022
> -0700
> ++++ b/gcc/c-family/c-indentation.c 2021-12-25 01:20:53.475636694
> -0800
> +@@ -24,8 +24,7 @@ along with GCC; see the file COPYING3.
> + #include "c-common.h"
> + #include "c-indentation.h"
> + #include "selftest.h"
> +-
> +-extern cpp_options *cpp_opts;
> ++#include "diagnostic.h"
> +
> + /* Round up VIS_COLUMN to nearest tab stop. */
> +
> +@@ -294,7 +293,7 @@ should_warn_for_misleading_indentation (
> + expanded_location next_stmt_exploc = expand_location
> (next_stmt_loc);
> + expanded_location guard_exploc = expand_location (guard_loc);
> +
> +- const unsigned int tab_width = cpp_opts->tabstop;
> ++ const unsigned int tab_width = global_dc->tabstop;
> +
> + /* They must be in the same file. */
> + if (next_stmt_exploc.file != body_exploc.file)
> +diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
> +--- a/gcc/c-family/c.opt 2021-12-24 20:23:42.816809230 -0800
> ++++ b/gcc/c-family/c.opt 2021-12-25 01:20:53.475636694 -0800
> +@@ -1876,10 +1876,6 @@ Enum(strong_eval_order) String(some) Val
> + EnumValue
> + Enum(strong_eval_order) String(all) Value(2)
> +
> +-ftabstop=
> +-C ObjC C++ ObjC++ Joined RejectNegative UInteger
> +--ftabstop=<number> Distance between tab stops for column
> reporting.
> +-
> + ftemplate-backtrace-limit=
> + C++ ObjC++ Joined RejectNegative UInteger
> Var(template_backtrace_limit) Init(10)
> + Set the maximum number of template instantiation notes for a single
> warning or error.
> +diff --git a/gcc/c-family/c-opts.c b/gcc/c-family/c-opts.c
> +--- a/gcc/c-family/c-opts.c 2021-12-24 20:23:44.824774786 -0800
> ++++ b/gcc/c-family/c-opts.c 2021-12-25 01:20:53.475636694 -0800
> +@@ -504,12 +504,6 @@ c_common_handle_option (size_t scode, co
> + cpp_opts->track_macro_expansion = 2;
> + break;
> +
> +- case OPT_ftabstop_:
> +- /* It is documented that we silently ignore silly values. */
> +- if (value >= 1 && value <= 100)
> +- cpp_opts->tabstop = value;
> +- break;
> +-
> + case OPT_fexec_charset_:
> + cpp_opts->narrow_charset = arg;
> + break;
> +diff --git a/gcc/common.opt b/gcc/common.opt
> +--- a/gcc/common.opt 2021-12-24 20:23:42.480814993 -0800
> ++++ b/gcc/common.opt 2021-12-25 01:20:53.475636694 -0800
> +@@ -1325,6 +1325,14 @@ Enum(diagnostic_url_rule) String(always)
> + EnumValue
> + Enum(diagnostic_url_rule) String(auto) Value(DIAGNOSTICS_URL_AUTO)
> +
> ++fdiagnostics-column-unit=
> ++Common Joined RejectNegative Enum(diagnostics_column_unit)
> ++-fdiagnostics-column-unit=[display|byte] Select whether column
> numbers are output as display columns (default) or raw bytes.
> ++
> ++fdiagnostics-column-origin=
> ++Common Joined RejectNegative UInteger
> ++-fdiagnostics-column-origin=<number> Set the number of the first
> column. The default is 1-based as per GNU style, but some utilities
> may expect 0-based, for example.
> ++
> + fdiagnostics-format=
> + Common Joined RejectNegative Enum(diagnostics_output_format)
> + -fdiagnostics-format=[text|json] Select output format.
> +@@ -1334,6 +1342,15 @@ SourceInclude
> + diagnostic.h
> +
> + Enum
> ++Name(diagnostics_column_unit) Type(int)
> ++
> ++EnumValue
> ++Enum(diagnostics_column_unit) String(display)
> Value(DIAGNOSTICS_COLUMN_UNIT_DISPLAY)
> ++
> ++EnumValue
> ++Enum(diagnostics_column_unit) String(byte)
> Value(DIAGNOSTICS_COLUMN_UNIT_BYTE)
> ++
> ++Enum
> + Name(diagnostics_output_format) Type(int)
> +
> + EnumValue
> +@@ -1362,6 +1379,10 @@ fdiagnostics-path-format=
> + Common Joined RejectNegative Var(flag_diagnostics_path_format)
> Enum(diagnostic_path_format) Init(DPF_INLINE_EVENTS)
> + Specify how to print any control-flow path associated with a
> diagnostic.
> +
> ++ftabstop=
> ++Common Joined RejectNegative UInteger
> ++-ftabstop=<number> Distance between tab stops for column
> reporting.
> ++
> + Enum
> + Name(diagnostic_path_format) Type(int)
> +
> +diff --git a/gcc/diagnostic.c b/gcc/diagnostic.c
> +--- a/gcc/diagnostic.c 2020-07-22 23:35:17.556386887 -0700
> ++++ b/gcc/diagnostic.c 2021-12-25 01:23:41.300841207 -0800
> +@@ -38,6 +38,7 @@ along with GCC; see the file COPYING3.
> + #include "selftest.h"
> + #include "selftest-diagnostic.h"
> + #include "opts.h"
> ++#include "cpplib.h"
> +
> + #ifdef HAVE_TERMIOS_H
> + # include <termios.h>
> +@@ -219,6 +220,9 @@ diagnostic_initialize (diagnostic_contex
> + context->min_margin_width = 0;
> + context->show_ruler_p = false;
> + context->parseable_fixits_p = false;
> ++ context->column_unit = DIAGNOSTICS_COLUMN_UNIT_DISPLAY;
> ++ context->column_origin = 1;
> ++ context->tabstop = 8;
> + context->edit_context_ptr = NULL;
> + context->diagnostic_group_nesting_depth = 0;
> + context->diagnostic_group_emission_count = 0;
> +@@ -353,8 +357,51 @@ diagnostic_get_color_for_kind (diagnosti
> + return diagnostic_kind_color[kind];
> + }
> +
> ++/* Given an expanded_location, convert the column (which is in 1-
> based bytes)
> ++ to the requested units, without converting the origin.
> ++ Return -1 if the column is invalid (<= 0). */
> ++
> ++static int
> ++convert_column_unit (enum diagnostics_column_unit column_unit,
> ++ int tabstop,
> ++ expanded_location s)
> ++{
> ++ if (s.column <= 0)
> ++ return -1;
> ++
> ++ switch (column_unit)
> ++ {
> ++ default:
> ++ gcc_unreachable ();
> ++
> ++ case DIAGNOSTICS_COLUMN_UNIT_DISPLAY:
> ++ {
> ++ cpp_char_column_policy policy (tabstop, cpp_wcwidth);
> ++ return location_compute_display_column (s, policy);
> ++ }
> ++
> ++ case DIAGNOSTICS_COLUMN_UNIT_BYTE:
> ++ return s.column;
> ++ }
> ++}
> ++
> ++/* Given an expanded_location, convert the column (which is in 1-
> based bytes)
> ++ to the requested units and origin. Return -1 if the column is
> ++ invalid (<= 0). */
> ++int
> ++diagnostic_converted_column (diagnostic_context *context,
> expanded_location s)
> ++{
> ++ int one_based_col
> ++ = convert_column_unit (context->column_unit, context->tabstop,
> s);
> ++ if (one_based_col <= 0)
> ++ return -1;
> ++ return one_based_col + (context->column_origin - 1);
> ++}
> ++
> + /* Return a formatted line and column ':%line:%column'. Elided if
> +- zero. The result is a statically allocated buffer. */
> ++ line == 0 or col < 0. (A column of 0 may be valid due to the
> ++ -fdiagnostics-column-origin option.)
> ++ The result is a statically allocated buffer. */
> +
> + static const char *
> + maybe_line_and_column (int line, int col)
> +@@ -363,8 +410,9 @@ maybe_line_and_column (int line, int col
> +
> + if (line)
> + {
> +- size_t l = snprintf (result, sizeof (result),
> +- col ? ":%d:%d" : ":%d", line, col);
> ++ size_t l
> ++ = snprintf (result, sizeof (result),
> ++ col >= 0 ? ":%d:%d" : ":%d", line, col);
> + gcc_checking_assert (l < sizeof (result));
> + }
> + else
> +@@ -383,8 +431,14 @@ diagnostic_get_location_text (diagnostic
> + const char *locus_cs = colorize_start (pp_show_color (pp),
> "locus");
> + const char *locus_ce = colorize_stop (pp_show_color (pp));
> + const char *file = s.file ? s.file : progname;
> +- int line = strcmp (file, N_("<built-in>")) ? s.line : 0;
> +- int col = context->show_column ? s.column : 0;
> ++ int line = 0;
> ++ int col = -1;
> ++ if (strcmp (file, N_("<built-in>")))
> ++ {
> ++ line = s.line;
> ++ if (context->show_column)
> ++ col = diagnostic_converted_column (context, s);
> ++ }
> +
> + const char *line_col = maybe_line_and_column (line, col);
> + return build_message_string ("%s%s%s:%s", locus_cs, file,
> +@@ -650,14 +704,20 @@ diagnostic_report_current_module (diagno
> + if (! MAIN_FILE_P (map))
> + {
> + bool first = true;
> ++ expanded_location s = {};
> + do
> + {
> + where = linemap_included_from (map);
> + map = linemap_included_from_linemap (line_table, map);
> +- const char *line_col
> +- = maybe_line_and_column (SOURCE_LINE (map, where),
> +- first && context-
> >show_column
> +- ? SOURCE_COLUMN (map, where)
> : 0);
> ++ s.file = LINEMAP_FILE (map);
> ++ s.line = SOURCE_LINE (map, where);
> ++ int col = -1;
> ++ if (first && context->show_column)
> ++ {
> ++ s.column = SOURCE_COLUMN (map, where);
> ++ col = diagnostic_converted_column (context, s);
> ++ }
> ++ const char *line_col = maybe_line_and_column (s.line,
> col);
> + static const char *const msgs[] =
> + {
> + N_("In file included from"),
> +@@ -666,7 +726,7 @@ diagnostic_report_current_module (diagno
> + unsigned index = !first;
> + pp_verbatim (context->printer, "%s%s %r%s%s%R",
> + first ? "" : ",\n", _(msgs[index]),
> +- "locus", LINEMAP_FILE (map), line_col);
> ++ "locus", s.file, line_col);
> + first = false;
> + }
> + while (! MAIN_FILE_P (map));
> +@@ -2042,10 +2102,15 @@ test_print_parseable_fixits_replace ()
> + static void
> + assert_location_text (const char *expected_loc_text,
> + const char *filename, int line, int column,
> +- bool show_column)
> ++ bool show_column,
> ++ int origin = 1,
> ++ enum diagnostics_column_unit column_unit
> ++ = DIAGNOSTICS_COLUMN_UNIT_BYTE)
> + {
> + test_diagnostic_context dc;
> + dc.show_column = show_column;
> ++ dc.column_unit = column_unit;
> ++ dc.column_origin = origin;
> +
> + expanded_location xloc;
> + xloc.file = filename;
> +@@ -2069,7 +2134,10 @@ test_diagnostic_get_location_text ()
> + assert_location_text ("PROGNAME:", NULL, 0, 0, true);
> + assert_location_text ("<built-in>:", "<built-in>", 42, 10, true);
> + assert_location_text ("foo.c:42:10:", "foo.c", 42, 10, true);
> +- assert_location_text ("foo.c:42:", "foo.c", 42, 0, true);
> ++ assert_location_text ("foo.c:42:9:", "foo.c", 42, 10, true, 0);
> ++ assert_location_text ("foo.c:42:1010:", "foo.c", 42, 10, true,
> 1001);
> ++ for (int origin = 0; origin != 2; ++origin)
> ++ assert_location_text ("foo.c:42:", "foo.c", 42, 0, true,
> origin);
> + assert_location_text ("foo.c:", "foo.c", 0, 10, true);
> + assert_location_text ("foo.c:42:", "foo.c", 42, 10, false);
> + assert_location_text ("foo.c:", "foo.c", 0, 10, false);
> +@@ -2077,6 +2145,41 @@ test_diagnostic_get_location_text ()
> + maybe_line_and_column (INT_MAX, INT_MAX);
> + maybe_line_and_column (INT_MIN, INT_MIN);
> +
> ++ {
> ++ /* In order to test display columns vs byte columns, we need to
> create a
> ++ file for location_get_source_line() to read. */
> ++
> ++ const char *const content = "smile \xf0\x9f\x98\x82\n";
> ++ const int line_bytes = strlen (content) - 1;
> ++ const int def_tabstop = 8;
> ++ const int display_width = cpp_display_width (content,
> line_bytes,
> ++ def_tabstop);
> ++ ASSERT_EQ (line_bytes - 2, display_width);
> ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
> ++ const char *const fname = tmp.get_filename ();
> ++ const int buf_len = strlen (fname) + 16;
> ++ char *const expected = XNEWVEC (char, buf_len);
> ++
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, line_bytes);
> ++ assert_location_text (expected, fname, 1, line_bytes, true,
> ++ 1, DIAGNOSTICS_COLUMN_UNIT_BYTE);
> ++
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, line_bytes -
> 1);
> ++ assert_location_text (expected, fname, 1, line_bytes, true,
> ++ 0, DIAGNOSTICS_COLUMN_UNIT_BYTE);
> ++
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, display_width);
> ++ assert_location_text (expected, fname, 1, line_bytes, true,
> ++ 1, DIAGNOSTICS_COLUMN_UNIT_DISPLAY);
> ++
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, display_width -
> 1);
> ++ assert_location_text (expected, fname, 1, line_bytes, true,
> ++ 0, DIAGNOSTICS_COLUMN_UNIT_DISPLAY);
> ++
> ++ XDELETEVEC (expected);
> ++ }
> ++
> ++
> + progname = old_progname;
> + }
> +
> +diff --git a/gcc/diagnostic-format-json.cc b/gcc/diagnostic-format-
> json.cc
> +--- a/gcc/diagnostic-format-json.cc 2020-07-22 23:35:17.556386887
> -0700
> ++++ b/gcc/diagnostic-format-json.cc 2021-12-25 01:20:53.475636694
> -0800
> +@@ -23,6 +23,7 @@ along with GCC; see the file COPYING3.
> + #include "system.h"
> + #include "coretypes.h"
> + #include "diagnostic.h"
> ++#include "selftest-diagnostic.h"
> + #include "diagnostic-metadata.h"
> + #include "json.h"
> + #include "selftest.h"
> +@@ -43,21 +44,43 @@ static json::array *cur_children_array;
> + /* Generate a JSON object for LOC. */
> +
> + json::value *
> +-json_from_expanded_location (location_t loc)
> ++json_from_expanded_location (diagnostic_context *context,
> location_t loc)
> + {
> + expanded_location exploc = expand_location (loc);
> + json::object *result = new json::object ();
> + if (exploc.file)
> + result->set ("file", new json::string (exploc.file));
> + result->set ("line", new json::integer_number (exploc.line));
> +- result->set ("column", new json::integer_number (exploc.column));
> ++
> ++ const enum diagnostics_column_unit orig_unit = context-
> >column_unit;
> ++ struct
> ++ {
> ++ const char *name;
> ++ enum diagnostics_column_unit unit;
> ++ } column_fields[] = {
> ++ {"display-column", DIAGNOSTICS_COLUMN_UNIT_DISPLAY},
> ++ {"byte-column", DIAGNOSTICS_COLUMN_UNIT_BYTE}
> ++ };
> ++ int the_column = INT_MIN;
> ++ for (int i = 0; i != sizeof column_fields / sizeof
> (*column_fields); ++i)
> ++ {
> ++ context->column_unit = column_fields[i].unit;
> ++ const int col = diagnostic_converted_column (context,
> exploc);
> ++ result->set (column_fields[i].name, new json::integer_number
> (col));
> ++ if (column_fields[i].unit == orig_unit)
> ++ the_column = col;
> ++ }
> ++ gcc_assert (the_column != INT_MIN);
> ++ result->set ("column", new json::integer_number (the_column));
> ++ context->column_unit = orig_unit;
> + return result;
> + }
> +
> + /* Generate a JSON object for LOC_RANGE. */
> +
> + static json::object *
> +-json_from_location_range (const location_range *loc_range, unsigned
> range_idx)
> ++json_from_location_range (diagnostic_context *context,
> ++ const location_range *loc_range, unsigned
> range_idx)
> + {
> + location_t caret_loc = get_pure_location (loc_range->m_loc);
> +
> +@@ -68,13 +91,13 @@ json_from_location_range (const location
> + location_t finish_loc = get_finish (loc_range->m_loc);
> +
> + json::object *result = new json::object ();
> +- result->set ("caret", json_from_expanded_location (caret_loc));
> ++ result->set ("caret", json_from_expanded_location (context,
> caret_loc));
> + if (start_loc != caret_loc
> + && start_loc != UNKNOWN_LOCATION)
> +- result->set ("start", json_from_expanded_location (start_loc));
> ++ result->set ("start", json_from_expanded_location (context,
> start_loc));
> + if (finish_loc != caret_loc
> + && finish_loc != UNKNOWN_LOCATION)
> +- result->set ("finish", json_from_expanded_location
> (finish_loc));
> ++ result->set ("finish", json_from_expanded_location (context,
> finish_loc));
> +
> + if (loc_range->m_label)
> + {
> +@@ -91,14 +114,14 @@ json_from_location_range (const location
> + /* Generate a JSON object for HINT. */
> +
> + static json::object *
> +-json_from_fixit_hint (const fixit_hint *hint)
> ++json_from_fixit_hint (diagnostic_context *context, const fixit_hint
> *hint)
> + {
> + json::object *fixit_obj = new json::object ();
> +
> + location_t start_loc = hint->get_start_loc ();
> +- fixit_obj->set ("start", json_from_expanded_location
> (start_loc));
> ++ fixit_obj->set ("start", json_from_expanded_location (context,
> start_loc));
> + location_t next_loc = hint->get_next_loc ();
> +- fixit_obj->set ("next", json_from_expanded_location (next_loc));
> ++ fixit_obj->set ("next", json_from_expanded_location (context,
> next_loc));
> + fixit_obj->set ("string", new json::string (hint->get_string
> ()));
> +
> + return fixit_obj;
> +@@ -190,11 +213,13 @@ json_end_diagnostic (diagnostic_context
> + else
> + {
> + /* Otherwise, make diag_obj be the top-level object within
> the group;
> +- add a "children" array. */
> ++ add a "children" array and record the column origin. */
> + toplevel_array->append (diag_obj);
> + cur_group = diag_obj;
> + cur_children_array = new json::array ();
> + diag_obj->set ("children", cur_children_array);
> ++ diag_obj->set ("column-origin",
> ++ new json::integer_number (context-
> >column_origin));
> + }
> +
> + const rich_location *richloc = diagnostic->richloc;
> +@@ -205,7 +230,7 @@ json_end_diagnostic (diagnostic_context
> + for (unsigned int i = 0; i < richloc->get_num_locations (); i++)
> + {
> + const location_range *loc_range = richloc->get_range (i);
> +- json::object *loc_obj = json_from_location_range (loc_range,
> i);
> ++ json::object *loc_obj = json_from_location_range (context,
> loc_range, i);
> + if (loc_obj)
> + loc_array->append (loc_obj);
> + }
> +@@ -217,7 +242,7 @@ json_end_diagnostic (diagnostic_context
> + for (unsigned int i = 0; i < richloc->get_num_fixit_hints ();
> i++)
> + {
> + const fixit_hint *hint = richloc->get_fixit_hint (i);
> +- json::object *fixit_obj = json_from_fixit_hint (hint);
> ++ json::object *fixit_obj = json_from_fixit_hint (context,
> hint);
> + fixit_array->append (fixit_obj);
> + }
> + }
> +@@ -320,7 +345,8 @@ namespace selftest {
> + static void
> + test_unknown_location ()
> + {
> +- delete json_from_expanded_location (UNKNOWN_LOCATION);
> ++ test_diagnostic_context dc;
> ++ delete json_from_expanded_location (&dc, UNKNOWN_LOCATION);
> + }
> +
> + /* Verify that we gracefully handle attempts to serialize bad
> +@@ -338,7 +364,8 @@ test_bad_endpoints ()
> + loc_range.m_range_display_kind = SHOW_RANGE_WITH_CARET;
> + loc_range.m_label = NULL;
> +
> +- json::object *obj = json_from_location_range (&loc_range, 0);
> ++ test_diagnostic_context dc;
> ++ json::object *obj = json_from_location_range (&dc, &loc_range,
> 0);
> + /* We should have a "caret" value, but no "start" or "finish"
> values. */
> + ASSERT_TRUE (obj != NULL);
> + ASSERT_TRUE (obj->get ("caret") != NULL);
> +diff --git a/gcc/diagnostic.h b/gcc/diagnostic.h
> +--- a/gcc/diagnostic.h 2020-07-22 23:35:17.556386887 -0700
> ++++ b/gcc/diagnostic.h 2021-12-25 01:20:53.479636627 -0800
> +@@ -24,6 +24,20 @@ along with GCC; see the file COPYING3.
> + #include "pretty-print.h"
> + #include "diagnostic-core.h"
> +
> ++/* An enum for controlling what units to use for the column number
> ++ when diagnostics are output, used by the -fdiagnostics-column-
> unit option.
> ++ Tabs will be expanded or not according to the value of -
> ftabstop. The origin
> ++ (default 1) is controlled by -fdiagnostics-column-origin. */
> ++
> ++enum diagnostics_column_unit
> ++{
> ++ /* The default from GCC 11 onwards: display columns. */
> ++ DIAGNOSTICS_COLUMN_UNIT_DISPLAY,
> ++
> ++ /* The behavior in GCC 10 and earlier: simple bytes. */
> ++ DIAGNOSTICS_COLUMN_UNIT_BYTE
> ++};
> ++
> + /* Enum for overriding the standard output format. */
> +
> + enum diagnostics_output_format
> +@@ -280,6 +294,15 @@ struct diagnostic_context
> + rest of the diagnostic. */
> + bool parseable_fixits_p;
> +
> ++ /* What units to use when outputting the column number. */
> ++ enum diagnostics_column_unit column_unit;
> ++
> ++ /* The origin for the column number (1-based or 0-based
> typically). */
> ++ int column_origin;
> ++
> ++ /* The size of the tabstop for tab expansion. */
> ++ int tabstop;
> ++
> + /* If non-NULL, an edit_context to which fix-it hints should be
> + applied, for generating patches. */
> + edit_context *edit_context_ptr;
> +@@ -458,6 +481,8 @@ diagnostic_same_line (const diagnostic_c
> + }
> +
> + extern const char *diagnostic_get_color_for_kind (diagnostic_t
> kind);
> ++extern int diagnostic_converted_column (diagnostic_context
> *context,
> ++ expanded_location s);
> +
> + /* Pure text formatting support functions. */
> + extern char *file_name_as_prefix (diagnostic_context *, const char
> *);
> +@@ -470,6 +495,7 @@ extern void diagnostic_output_format_ini
> + /* Compute the number of digits in the decimal representation of an
> integer. */
> + extern int num_digits (int);
> +
> +-extern json::value *json_from_expanded_location (location_t loc);
> ++extern json::value *json_from_expanded_location (diagnostic_context
> *context,
> ++ location_t loc);
> +
> + #endif /* ! GCC_DIAGNOSTIC_H */
> +diff --git a/gcc/diagnostic-show-locus.c b/gcc/diagnostic-show-
> locus.c
> +--- a/gcc/diagnostic-show-locus.c 2020-07-22 23:35:17.556386887
> -0700
> ++++ b/gcc/diagnostic-show-locus.c 2021-12-25 01:20:53.479636627
> -0800
> +@@ -175,9 +175,10 @@ enum column_unit {
> + class exploc_with_display_col : public expanded_location
> + {
> + public:
> +- exploc_with_display_col (const expanded_location &exploc)
> ++ exploc_with_display_col (const expanded_location &exploc, int
> tabstop)
> + : expanded_location (exploc),
> +- m_display_col (location_compute_display_column (exploc)) {}
> ++ m_display_col (location_compute_display_column (exploc,
> tabstop))
> ++ {}
> +
> + int m_display_col;
> + };
> +@@ -189,11 +190,11 @@ class exploc_with_display_col : public e
> + class layout_point
> + {
> + public:
> +- layout_point (const expanded_location &exploc)
> ++ layout_point (const exploc_with_display_col &exploc)
> + : m_line (exploc.line)
> + {
> + m_columns[CU_BYTES] = exploc.column;
> +- m_columns[CU_DISPLAY_COLS] = location_compute_display_column
> (exploc);
> ++ m_columns[CU_DISPLAY_COLS] = exploc.m_display_col;
> + }
> +
> + linenum_type m_line;
> +@@ -205,10 +206,10 @@ class layout_point
> + class layout_range
> + {
> + public:
> +- layout_range (const expanded_location *start_exploc,
> +- const expanded_location *finish_exploc,
> ++ layout_range (const exploc_with_display_col &start_exploc,
> ++ const exploc_with_display_col &finish_exploc,
> + enum range_display_kind range_display_kind,
> +- const expanded_location *caret_exploc,
> ++ const exploc_with_display_col &caret_exploc,
> + unsigned original_idx,
> + const range_label *label);
> +
> +@@ -226,22 +227,18 @@ class layout_range
> +
> + /* A struct for use by layout::print_source_line for telling
> + layout::print_annotation_line the extents of the source line
> that
> +- it printed, so that underlines can be clipped appropriately. */
> ++ it printed, so that underlines can be clipped appropriately.
> Units
> ++ are 1-based display columns. */
> +
> + struct line_bounds
> + {
> +- int m_first_non_ws;
> +- int m_last_non_ws;
> ++ int m_first_non_ws_disp_col;
> ++ int m_last_non_ws_disp_col;
> +
> +- void convert_to_display_cols (char_span line)
> ++ line_bounds ()
> + {
> +- m_first_non_ws = cpp_byte_column_to_display_column
> (line.get_buffer (),
> +- line.length
> (),
> +-
> m_first_non_ws)
> ;
> +-
> +- m_last_non_ws = cpp_byte_column_to_display_column
> (line.get_buffer (),
> +- line.length
> (),
> +-
> m_last_non_ws);
> ++ m_first_non_ws_disp_col = INT_MAX;
> ++ m_last_non_ws_disp_col = 0;
> + }
> + };
> +
> +@@ -351,8 +348,8 @@ class layout
> + private:
> + bool will_show_line_p (linenum_type row) const;
> + void print_leading_fixits (linenum_type row);
> +- void print_source_line (linenum_type row, const char *line, int
> line_bytes,
> +- line_bounds *lbounds_out);
> ++ line_bounds print_source_line (linenum_type row, const char
> *line,
> ++ int line_bytes);
> + bool should_print_annotation_line_p (linenum_type row) const;
> + void start_annotation_line (char margin_char = ' ') const;
> + void print_annotation_line (linenum_type row, const line_bounds
> lbounds);
> +@@ -513,16 +510,16 @@ colorizer::get_color_by_name (const char
> + Initialize various layout_point fields from expanded_location
> + equivalents; we've already filtered on file. */
> +
> +-layout_range::layout_range (const expanded_location *start_exploc,
> +- const expanded_location *finish_exploc,
> ++layout_range::layout_range (const exploc_with_display_col
> &start_exploc,
> ++ const exploc_with_display_col
> &finish_exploc,
> + enum range_display_kind
> range_display_kind,
> +- const expanded_location *caret_exploc,
> ++ const exploc_with_display_col
> &caret_exploc,
> + unsigned original_idx,
> + const range_label *label)
> +-: m_start (*start_exploc),
> +- m_finish (*finish_exploc),
> ++: m_start (start_exploc),
> ++ m_finish (finish_exploc),
> + m_range_display_kind (range_display_kind),
> +- m_caret (*caret_exploc),
> ++ m_caret (caret_exploc),
> + m_original_idx (original_idx),
> + m_label (label)
> + {
> +@@ -646,6 +643,9 @@ layout_range::intersects_line_p (linenum
> +
> + #if CHECKING_P
> +
> ++/* Default for when we don't care what the tab expansion is set
> to. */
> ++static const int def_tabstop = 8;
> ++
> + /* Create some expanded locations for testing layout_range. The
> filename
> + member of the explocs is set to the empty string. This member
> will only be
> + inspected by the calls to location_compute_display_column() made
> from the
> +@@ -662,8 +662,11 @@ make_range (int start_line, int start_co
> + = {"", start_line, start_col, NULL, false};
> + const expanded_location finish_exploc
> + = {"", end_line, end_col, NULL, false};
> +- return layout_range (&start_exploc, &finish_exploc,
> SHOW_RANGE_WITHOUT_CARET,
> +- &start_exploc, 0, NULL);
> ++ return layout_range (exploc_with_display_col (start_exploc,
> def_tabstop),
> ++ exploc_with_display_col (finish_exploc,
> def_tabstop),
> ++ SHOW_RANGE_WITHOUT_CARET,
> ++ exploc_with_display_col (start_exploc,
> def_tabstop),
> ++ 0, NULL);
> + }
> +
> + /* Selftests for layout_range::contains_point and
> +@@ -964,7 +967,7 @@ layout::layout (diagnostic_context * con
> + : m_context (context),
> + m_pp (context->printer),
> + m_primary_loc (richloc->get_range (0)->m_loc),
> +- m_exploc (richloc->get_expanded_location (0)),
> ++ m_exploc (richloc->get_expanded_location (0), context->tabstop),
> + m_colorizer (context, diagnostic_kind),
> + m_colorize_source_p (context->colorize_source_p),
> + m_show_labels_p (context->show_labels_p),
> +@@ -1060,7 +1063,10 @@ layout::maybe_add_location_range (const
> +
> + /* Everything is now known to be in the correct source file,
> + but it may require further sanitization. */
> +- layout_range ri (&start, &finish, loc_range-
> >m_range_display_kind, &caret,
> ++ layout_range ri (exploc_with_display_col (start, m_context-
> >tabstop),
> ++ exploc_with_display_col (finish, m_context-
> >tabstop),
> ++ loc_range->m_range_display_kind,
> ++ exploc_with_display_col (caret, m_context-
> >tabstop),
> + original_idx, loc_range->m_label);
> +
> + /* If we have a range that finishes before it starts (perhaps
> +@@ -1394,7 +1400,7 @@ layout::calculate_x_offset_display ()
> + = get_line_bytes_without_trailing_whitespace (line.get_buffer
> (),
> + line.length ());
> + int eol_display_column
> +- = cpp_display_width (line.get_buffer (), line_bytes);
> ++ = cpp_display_width (line.get_buffer (), line_bytes, m_context-
> >tabstop);
> + if (caret_display_column > eol_display_column
> + || !caret_display_column)
> + {
> +@@ -1445,16 +1451,13 @@ layout::calculate_x_offset_display ()
> + }
> +
> + /* Print line ROW of source code, potentially colorized at any
> ranges, and
> +- populate *LBOUNDS_OUT.
> +- LINE is the source line (not necessarily 0-terminated) and
> LINE_BYTES
> +- is its length in bytes.
> +- This function deals only with byte offsets, not display columns,
> so
> +- m_x_offset_display must be converted from display to byte
> units. In
> +- particular, LINE_BYTES and LBOUNDS_OUT are in bytes. */
> ++ return the line bounds. LINE is the source line (not
> necessarily
> ++ 0-terminated) and LINE_BYTES is its length in bytes. In order
> to handle both
> ++ colorization and tab expansion, this function tracks the line
> position in
> ++ both byte and display column units. */
> +
> +-void
> +-layout::print_source_line (linenum_type row, const char *line, int
> line_bytes,
> +- line_bounds *lbounds_out)
> ++line_bounds
> ++layout::print_source_line (linenum_type row, const char *line, int
> line_bytes)
> + {
> + m_colorizer.set_normal_text ();
> +
> +@@ -1469,30 +1472,29 @@ layout::print_source_line (linenum_type
> + else
> + pp_space (m_pp);
> +
> +- /* We will stop printing the source line at any trailing
> whitespace, and start
> +- printing it as per m_x_offset_display. */
> ++ /* We will stop printing the source line at any trailing
> whitespace. */
> + line_bytes = get_line_bytes_without_trailing_whitespace (line,
> +
> line_bytes);
> +- int x_offset_bytes = 0;
> +- if (m_x_offset_display)
> +- {
> +- x_offset_bytes = cpp_display_column_to_byte_column (line,
> line_bytes,
> +-
> m_x_offset_display);
> +- /* In case the leading portion of the line that will be
> skipped over ends
> +- with a character with wcwidth > 1, then it is possible we
> skipped too
> +- much, so account for that by padding with spaces. */
> +- const int overage
> +- = cpp_byte_column_to_display_column (line, line_bytes,
> x_offset_bytes)
> +- - m_x_offset_display;
> +- for (int column = 0; column < overage; ++column)
> +- pp_space (m_pp);
> +- line += x_offset_bytes;
> +- }
> +
> +- /* Print the line. */
> +- int first_non_ws = INT_MAX;
> +- int last_non_ws = 0;
> +- for (int col_byte = 1 + x_offset_bytes; col_byte <= line_bytes;
> col_byte++)
> ++ /* This object helps to keep track of which display column we are
> at, which is
> ++ necessary for computing the line bounds in display units, for
> doing
> ++ tab expansion, and for implementing m_x_offset_display. */
> ++ cpp_display_width_computation dw (line, line_bytes, m_context-
> >tabstop);
> ++
> ++ /* Skip the first m_x_offset_display display columns. In case
> the leading
> ++ portion that will be skipped ends with a character with
> wcwidth > 1, then
> ++ it is possible we skipped too much, so account for that by
> padding with
> ++ spaces. Note that this does the right thing too in case a tab
> was the last
> ++ character to be skipped over; the tab is effectively replaced
> by the
> ++ correct number of trailing spaces needed to offset by the
> desired number of
> ++ display columns. */
> ++ for (int skipped_display_cols = dw.advance_display_cols
> (m_x_offset_display);
> ++ skipped_display_cols > m_x_offset_display; --
> skipped_display_cols)
> ++ pp_space (m_pp);
> ++
> ++ /* Print the line and compute the line_bounds. */
> ++ line_bounds lbounds;
> ++ while (!dw.done ())
> + {
> + /* Assuming colorization is enabled for the caret and
> underline
> + characters, we may also colorize the associated characters
> +@@ -1510,7 +1512,8 @@ layout::print_source_line (linenum_type
> + {
> + bool in_range_p;
> + point_state state;
> +- in_range_p = get_state_at_point (row, col_byte,
> ++ const int start_byte_col = dw.bytes_processed () + 1;
> ++ in_range_p = get_state_at_point (row, start_byte_col,
> + 0, INT_MAX,
> + CU_BYTES,
> + &state);
> +@@ -1519,22 +1522,44 @@ layout::print_source_line (linenum_type
> + else
> + m_colorizer.set_normal_text ();
> + }
> +- char c = *line;
> +- if (c == '\0' || c == '\t' || c == '\r')
> +- c = ' ';
> +- if (c != ' ')
> ++
> ++ /* Get the display width of the next character to be output,
> expanding
> ++ tabs and replacing some control bytes with spaces as
> necessary. */
> ++ const char *c = dw.next_byte ();
> ++ const int start_disp_col = dw.display_cols_processed () + 1;
> ++ const int this_display_width = dw.process_next_codepoint ();
> ++ if (*c == '\t')
> ++ {
> ++ /* The returned display width is the number of spaces into
> which the
> ++ tab should be expanded. */
> ++ for (int i = 0; i != this_display_width; ++i)
> ++ pp_space (m_pp);
> ++ continue;
> ++ }
> ++ if (*c == '\0' || *c == '\r')
> + {
> +- last_non_ws = col_byte;
> +- if (first_non_ws == INT_MAX)
> +- first_non_ws = col_byte;
> ++ /* cpp_wcwidth() promises to return 1 for all control
> bytes, and we
> ++ want to output these as a single space too, so this case
> is
> ++ actually the same as the '\t' case. */
> ++ gcc_assert (this_display_width == 1);
> ++ pp_space (m_pp);
> ++ continue;
> + }
> +- pp_character (m_pp, c);
> +- line++;
> ++
> ++ /* We have a (possibly multibyte) character to output; update
> the line
> ++ bounds if it is not whitespace. */
> ++ if (*c != ' ')
> ++ {
> ++ lbounds.m_last_non_ws_disp_col = dw.display_cols_processed
> ();
> ++ if (lbounds.m_first_non_ws_disp_col == INT_MAX)
> ++ lbounds.m_first_non_ws_disp_col = start_disp_col;
> ++ }
> ++
> ++ /* Output the character. */
> ++ while (c != dw.next_byte ()) pp_character (m_pp, *c++);
> + }
> + print_newline ();
> +-
> +- lbounds_out->m_first_non_ws = first_non_ws;
> +- lbounds_out->m_last_non_ws = last_non_ws;
> ++ return lbounds;
> + }
> +
> + /* Determine if we should print an annotation line for ROW.
> +@@ -1576,14 +1601,13 @@ layout::start_annotation_line (char marg
> + }
> +
> + /* Print a line consisting of the caret/underlines for the given
> +- source line. This function works with display columns, rather
> than byte
> +- counts; in particular, LBOUNDS should be in display column
> units. */
> ++ source line. */
> +
> + void
> + layout::print_annotation_line (linenum_type row, const line_bounds
> lbounds)
> + {
> + int x_bound = get_x_bound_for_row (row, m_exploc.m_display_col,
> +- lbounds.m_last_non_ws);
> ++ lbounds.m_last_non_ws_disp_col);
> +
> + start_annotation_line ();
> + pp_space (m_pp);
> +@@ -1593,8 +1617,8 @@ layout::print_annotation_line (linenum_t
> + bool in_range_p;
> + point_state state;
> + in_range_p = get_state_at_point (row, column,
> +- lbounds.m_first_non_ws,
> +- lbounds.m_last_non_ws,
> ++
> lbounds.m_first_non_ws_disp_col,
> ++
> lbounds.m_last_non_ws_disp_col,
> + CU_DISPLAY_COLS,
> + &state);
> + if (in_range_p)
> +@@ -1631,12 +1655,14 @@ layout::print_annotation_line (linenum_t
> + class line_label
> + {
> + public:
> +- line_label (int state_idx, int column, label_text text)
> ++ line_label (diagnostic_context *context, int state_idx, int
> column,
> ++ label_text text)
> + : m_state_idx (state_idx), m_column (column),
> + m_text (text), m_label_line (0), m_has_vbar (true)
> + {
> + const int bytes = strlen (text.m_buffer);
> +- m_display_width = cpp_display_width (text.m_buffer, bytes);
> ++ m_display_width
> ++ = cpp_display_width (text.m_buffer, bytes, context->tabstop);
> + }
> +
> + /* Sorting is primarily by column, then by state index. */
> +@@ -1696,7 +1722,7 @@ layout::print_any_labels (linenum_type r
> + if (text.m_buffer == NULL)
> + continue;
> +
> +- labels.safe_push (line_label (i, disp_col, text));
> ++ labels.safe_push (line_label (m_context, i, disp_col, text));
> + }
> + }
> +
> +@@ -1976,7 +2002,8 @@ public:
> +
> + /* Get the range of bytes or display columns that HINT would
> affect. */
> + static column_range
> +-get_affected_range (const fixit_hint *hint, enum column_unit
> col_unit)
> ++get_affected_range (diagnostic_context *context,
> ++ const fixit_hint *hint, enum column_unit
> col_unit)
> + {
> + expanded_location exploc_start = expand_location (hint-
> >get_start_loc ());
> + expanded_location exploc_finish = expand_location (hint-
> >get_next_loc ());
> +@@ -1986,11 +2013,13 @@ get_affected_range (const fixit_hint *hi
> + int finish_column;
> + if (col_unit == CU_DISPLAY_COLS)
> + {
> +- start_column = location_compute_display_column
> (exploc_start);
> ++ start_column
> ++ = location_compute_display_column (exploc_start, context-
> >tabstop);
> + if (hint->insertion_p ())
> + finish_column = start_column - 1;
> + else
> +- finish_column = location_compute_display_column
> (exploc_finish);
> ++ finish_column
> ++ = location_compute_display_column (exploc_finish, context-
> >tabstop);
> + }
> + else
> + {
> +@@ -2003,12 +2032,12 @@ get_affected_range (const fixit_hint *hi
> + /* Get the range of display columns that would be printed for
> HINT. */
> +
> + static column_range
> +-get_printed_columns (const fixit_hint *hint)
> ++get_printed_columns (diagnostic_context *context, const fixit_hint
> *hint)
> + {
> + expanded_location exploc = expand_location (hint->get_start_loc
> ());
> +- int start_column = location_compute_display_column (exploc);
> +- int hint_width = cpp_display_width (hint->get_string (),
> +- hint->get_length ());
> ++ int start_column = location_compute_display_column (exploc,
> context->tabstop);
> ++ int hint_width = cpp_display_width (hint->get_string (), hint-
> >get_length (),
> ++ context->tabstop);
> + int final_hint_column = start_column + hint_width - 1;
> + if (hint->insertion_p ())
> + {
> +@@ -2018,7 +2047,8 @@ get_printed_columns (const fixit_hint *h
> + {
> + exploc = expand_location (hint->get_next_loc ());
> + --exploc.column;
> +- int finish_column = location_compute_display_column (exploc);
> ++ int finish_column
> ++ = location_compute_display_column (exploc, context->tabstop);
> + return column_range (start_column,
> + MAX (finish_column, final_hint_column));
> + }
> +@@ -2035,12 +2065,14 @@ public:
> + correction (column_range affected_bytes,
> + column_range affected_columns,
> + column_range printed_columns,
> +- const char *new_text, size_t new_text_len)
> ++ const char *new_text, size_t new_text_len,
> ++ int tabstop)
> + : m_affected_bytes (affected_bytes),
> + m_affected_columns (affected_columns),
> + m_printed_columns (printed_columns),
> + m_text (xstrdup (new_text)),
> + m_byte_length (new_text_len),
> ++ m_tabstop (tabstop),
> + m_alloc_sz (new_text_len + 1)
> + {
> + compute_display_cols ();
> +@@ -2058,7 +2090,7 @@ public:
> +
> + void compute_display_cols ()
> + {
> +- m_display_cols = cpp_display_width (m_text, m_byte_length);
> ++ m_display_cols = cpp_display_width (m_text, m_byte_length,
> m_tabstop);
> + }
> +
> + void overwrite (int dst_offset, const char_span &src_span)
> +@@ -2086,6 +2118,7 @@ public:
> + char *m_text;
> + size_t m_byte_length; /* Not including null-terminator. */
> + int m_display_cols;
> ++ int m_tabstop;
> + size_t m_alloc_sz;
> + };
> +
> +@@ -2121,13 +2154,15 @@ correction::ensure_terminated ()
> + class line_corrections
> + {
> + public:
> +- line_corrections (const char *filename, linenum_type row)
> +- : m_filename (filename), m_row (row)
> ++ line_corrections (diagnostic_context *context, const char
> *filename,
> ++ linenum_type row)
> ++ : m_context (context), m_filename (filename), m_row (row)
> + {}
> + ~line_corrections ();
> +
> + void add_hint (const fixit_hint *hint);
> +
> ++ diagnostic_context *m_context;
> + const char *m_filename;
> + linenum_type m_row;
> + auto_vec <correction *> m_corrections;
> +@@ -2173,9 +2208,10 @@ source_line::source_line (const char *fi
> + void
> + line_corrections::add_hint (const fixit_hint *hint)
> + {
> +- column_range affected_bytes = get_affected_range (hint,
> CU_BYTES);
> +- column_range affected_columns = get_affected_range (hint,
> CU_DISPLAY_COLS);
> +- column_range printed_columns = get_printed_columns (hint);
> ++ column_range affected_bytes = get_affected_range (m_context,
> hint, CU_BYTES);
> ++ column_range affected_columns = get_affected_range (m_context,
> hint,
> ++
> CU_DISPLAY_COLS);
> ++ column_range printed_columns = get_printed_columns (m_context,
> hint);
> +
> + /* Potentially consolidate. */
> + if (!m_corrections.is_empty ())
> +@@ -2243,7 +2279,8 @@ line_corrections::add_hint (const fixit_
> + affected_columns,
> + printed_columns,
> + hint->get_string (),
> +- hint->get_length ()));
> ++ hint->get_length (),
> ++ m_context->tabstop));
> + }
> +
> + /* If there are any fixit hints on source line ROW, print them.
> +@@ -2257,7 +2294,7 @@ layout::print_trailing_fixits (linenum_t
> + {
> + /* Build a list of correction instances for the line,
> + potentially consolidating hints (for the sake of
> readability). */
> +- line_corrections corrections (m_exploc.file, row);
> ++ line_corrections corrections (m_context, m_exploc.file, row);
> + for (unsigned int i = 0; i < m_fixit_hints.length (); i++)
> + {
> + const fixit_hint *hint = m_fixit_hints[i];
> +@@ -2499,15 +2536,11 @@ layout::print_line (linenum_type row)
> + if (!line)
> + return;
> +
> +- line_bounds lbounds;
> + print_leading_fixits (row);
> +- print_source_line (row, line.get_buffer (), line.length (),
> &lbounds);
> ++ const line_bounds lbounds
> ++ = print_source_line (row, line.get_buffer (), line.length ());
> + if (should_print_annotation_line_p (row))
> +- {
> +- if (lbounds.m_first_non_ws != INT_MAX)
> +- lbounds.convert_to_display_cols (line);
> +- print_annotation_line (row, lbounds);
> +- }
> ++ print_annotation_line (row, lbounds);
> + if (m_show_labels_p)
> + print_any_labels (row);
> + print_trailing_fixits (row);
> +@@ -2670,9 +2703,11 @@ test_layout_x_offset_display_utf8 (const
> +
> + char_span lspan = location_get_source_line (tmp.get_filename (),
> 1);
> + ASSERT_EQ (line_display_cols,
> +- cpp_display_width (lspan.get_buffer (), lspan.length
> ()));
> ++ cpp_display_width (lspan.get_buffer (), lspan.length (),
> ++ def_tabstop));
> + ASSERT_EQ (line_display_cols,
> +- location_compute_display_column (expand_location
> (line_end)));
> ++ location_compute_display_column (expand_location
> (line_end),
> ++ def_tabstop));
> + ASSERT_EQ (0, memcmp (lspan.get_buffer () + (emoji_col - 1),
> + "\xf0\x9f\x98\x82\xf0\x9f\x98\x82", 8));
> +
> +@@ -2774,6 +2809,111 @@ test_layout_x_offset_display_utf8 (const
> +
> + }
> +
> ++static void
> ++test_layout_x_offset_display_tab (const line_table_case &case_)
> ++{
> ++ const char *content
> ++ = "This line is very long, so that we can use it to test the
> logic for "
> ++ "clipping long lines. Also this: `\t' is a tab that occupies
> 1 byte and "
> ++ "a variable number of display columns, starting at column
> #103.\n";
> ++
> ++ /* Number of bytes in the line, subtracting one to remove the
> newline. */
> ++ const int line_bytes = strlen (content) - 1;
> ++
> ++ /* The column where the tab begins. Byte or display is the same
> as there are
> ++ no multibyte characters earlier on the line. */
> ++ const int tab_col = 103;
> ++
> ++ /* Effective extra size of the tab beyond what a single space
> would have taken
> ++ up, indexed by tabstop. */
> ++ static const int num_tabstops = 11;
> ++ int extra_width[num_tabstops];
> ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> ++ {
> ++ const int this_tab_size = tabstop - (tab_col - 1) % tabstop;
> ++ extra_width[tabstop] = this_tab_size - 1;
> ++ }
> ++ /* Example of this calculation: if tabstop is 10, the tab
> starting at column
> ++ #103 has to expand into 8 spaces, covering columns 103-110, so
> that the
> ++ next character is at column #111. So it takes up 7 more
> columns than
> ++ a space would have taken up. */
> ++ ASSERT_EQ (7, extra_width[10]);
> ++
> ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
> ++ line_table_test ltt (case_);
> ++
> ++ linemap_add (line_table, LC_ENTER, false, tmp.get_filename (),
> 1);
> ++
> ++ location_t line_end = linemap_position_for_column (line_table,
> line_bytes);
> ++
> ++ /* Don't attempt to run the tests if column data might be
> unavailable. */
> ++ if (line_end > LINE_MAP_MAX_LOCATION_WITH_COLS)
> ++ return;
> ++
> ++ /* Check that cpp_display_width handles the tabs as expected. */
> ++ char_span lspan = location_get_source_line (tmp.get_filename (),
> 1);
> ++ ASSERT_EQ ('\t', *(lspan.get_buffer () + (tab_col - 1)));
> ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> ++ {
> ++ ASSERT_EQ (line_bytes + extra_width[tabstop],
> ++ cpp_display_width (lspan.get_buffer (), lspan.length
> (),
> ++ tabstop));
> ++ ASSERT_EQ (line_bytes + extra_width[tabstop],
> ++ location_compute_display_column (expand_location
> (line_end),
> ++ tabstop));
> ++ }
> ++
> ++ /* Check that the tab is expanded to the expected number of
> spaces. */
> ++ rich_location richloc (line_table,
> ++ linemap_position_for_column (line_table,
> ++ tab_col + 1));
> ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> ++ {
> ++ test_diagnostic_context dc;
> ++ dc.tabstop = tabstop;
> ++ layout test_layout (&dc, &richloc, DK_ERROR);
> ++ test_layout.print_line (1);
> ++ const char *out = pp_formatted_text (dc.printer);
> ++ ASSERT_EQ (NULL, strchr (out, '\t'));
> ++ const char *left_quote = strchr (out, '`');
> ++ const char *right_quote = strchr (out, '\'');
> ++ ASSERT_NE (NULL, left_quote);
> ++ ASSERT_NE (NULL, right_quote);
> ++ ASSERT_EQ (right_quote - left_quote, extra_width[tabstop] +
> 2);
> ++ }
> ++
> ++ /* Check that the line is offset properly and that the tab is
> broken up
> ++ into the expected number of spaces when it is the last
> character skipped
> ++ over. */
> ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> ++ {
> ++ test_diagnostic_context dc;
> ++ dc.tabstop = tabstop;
> ++ static const int small_width = 24;
> ++ dc.caret_max_width = small_width - 4;
> ++ dc.min_margin_width = test_left_margin - test_linenum_sep +
> 1;
> ++ dc.show_line_numbers_p = true;
> ++ layout test_layout (&dc, &richloc, DK_ERROR);
> ++ test_layout.print_line (1);
> ++
> ++ /* We have arranged things so that two columns will be
> printed before
> ++ the caret. If the tab results in more than one space, this
> should
> ++ produce two spaces in the output; otherwise, it will be a
> single space
> ++ preceded by the opening quote before the tab character. */
> ++ const char *output1
> ++ = " 1 | ' is a tab that occupies 1 byte and a variable
> number of "
> ++ "display columns, starting at column #103.\n"
> ++ " | ^\n\n";
> ++ const char *output2
> ++ = " 1 | ` ' is a tab that occupies 1 byte and a variable
> number of "
> ++ "display columns, starting at column #103.\n"
> ++ " | ^\n\n";
> ++ const char *expected_output = (extra_width[tabstop] ? output1
> : output2);
> ++ ASSERT_STREQ (expected_output, pp_formatted_text
> (dc.printer));
> ++ }
> ++}
> ++
> ++
> + /* Verify that diagnostic_show_locus works sanely on
> UNKNOWN_LOCATION. */
> +
> + static void
> +@@ -3854,6 +3994,27 @@ test_one_liner_labels_utf8 ()
> + }
> + }
> +
> ++/* Make sure that colorization codes don't interrupt a multibyte
> ++ sequence, which would corrupt it. */
> ++static void
> ++test_one_liner_colorized_utf8 ()
> ++{
> ++ test_diagnostic_context dc;
> ++ dc.colorize_source_p = true;
> ++ diagnostic_color_init (&dc, DIAGNOSTICS_COLOR_YES);
> ++ const location_t pi = linemap_position_for_column (line_table,
> 12);
> ++ rich_location richloc (line_table, pi);
> ++ diagnostic_show_locus (&dc, &richloc, DK_ERROR);
> ++
> ++ /* In order to avoid having the test depend on exactly how the
> colorization
> ++ was effected, just confirm there are two pi characters in the
> output. */
> ++ const char *result = pp_formatted_text (dc.printer);
> ++ const char *null_term = result + strlen (result);
> ++ const char *first_pi = strstr (result, "\xcf\x80");
> ++ ASSERT_TRUE (first_pi && first_pi <= null_term - 2);
> ++ ASSERT_STR_CONTAINS (first_pi + 2, "\xcf\x80");
> ++}
> ++
> + /* Run the various one-liner tests. */
> +
> + static void
> +@@ -3884,8 +4045,10 @@ test_diagnostic_show_locus_one_liner_utf
> + ASSERT_EQ (31, LOCATION_COLUMN (line_end));
> +
> + char_span lspan = location_get_source_line (tmp.get_filename (),
> 1);
> +- ASSERT_EQ (25, cpp_display_width (lspan.get_buffer (),
> lspan.length ()));
> +- ASSERT_EQ (25, location_compute_display_column (expand_location
> (line_end)));
> ++ ASSERT_EQ (25, cpp_display_width (lspan.get_buffer (),
> lspan.length (),
> ++ def_tabstop));
> ++ ASSERT_EQ (25, location_compute_display_column (expand_location
> (line_end),
> ++ def_tabstop));
> +
> + test_one_liner_simple_caret_utf8 ();
> + test_one_liner_caret_and_range_utf8 ();
> +@@ -3900,6 +4063,7 @@ test_diagnostic_show_locus_one_liner_utf
> + test_one_liner_many_fixits_1_utf8 ();
> + test_one_liner_many_fixits_2_utf8 ();
> + test_one_liner_labels_utf8 ();
> ++ test_one_liner_colorized_utf8 ();
> + }
> +
> + /* Verify that gcc_rich_location::add_location_if_nearby works. */
> +@@ -4272,25 +4436,28 @@ test_overlapped_fixit_printing (const li
> + /* Unit-test the line_corrections machinery. */
> + ASSERT_EQ (3, richloc.get_num_fixit_hints ());
> + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
> +- ASSERT_EQ (column_range (12, 12), get_affected_range (hint_0,
> CU_BYTES));
> + ASSERT_EQ (column_range (12, 12),
> +- get_affected_range (hint_0,
> CU_DISPLAY_COLS));
> +- ASSERT_EQ (column_range (12, 22), get_printed_columns
> (hint_0));
> ++ get_affected_range (&dc, hint_0, CU_BYTES));
> ++ ASSERT_EQ (column_range (12, 12),
> ++ get_affected_range (&dc, hint_0, CU_DISPLAY_COLS));
> ++ ASSERT_EQ (column_range (12, 22), get_printed_columns (&dc,
> hint_0));
> + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
> +- ASSERT_EQ (column_range (18, 18), get_affected_range (hint_1,
> CU_BYTES));
> + ASSERT_EQ (column_range (18, 18),
> +- get_affected_range (hint_1,
> CU_DISPLAY_COLS));
> +- ASSERT_EQ (column_range (18, 20), get_printed_columns
> (hint_1));
> ++ get_affected_range (&dc, hint_1, CU_BYTES));
> ++ ASSERT_EQ (column_range (18, 18),
> ++ get_affected_range (&dc, hint_1, CU_DISPLAY_COLS));
> ++ ASSERT_EQ (column_range (18, 20), get_printed_columns (&dc,
> hint_1));
> + const fixit_hint *hint_2 = richloc.get_fixit_hint (2);
> +- ASSERT_EQ (column_range (29, 28), get_affected_range (hint_2,
> CU_BYTES));
> + ASSERT_EQ (column_range (29, 28),
> +- get_affected_range (hint_2,
> CU_DISPLAY_COLS));
> +- ASSERT_EQ (column_range (29, 29), get_printed_columns
> (hint_2));
> ++ get_affected_range (&dc, hint_2, CU_BYTES));
> ++ ASSERT_EQ (column_range (29, 28),
> ++ get_affected_range (&dc, hint_2, CU_DISPLAY_COLS));
> ++ ASSERT_EQ (column_range (29, 29), get_printed_columns (&dc,
> hint_2));
> +
> + /* Add each hint in turn to a line_corrections instance,
> + and verify that they are consolidated into one correction
> instance
> + as expected. */
> +- line_corrections lc (tmp.get_filename (), 1);
> ++ line_corrections lc (&dc, tmp.get_filename (), 1);
> +
> + /* The first replace hint by itself. */
> + lc.add_hint (hint_0);
> +@@ -4484,25 +4651,28 @@ test_overlapped_fixit_printing_utf8 (con
> + /* Unit-test the line_corrections machinery. */
> + ASSERT_EQ (3, richloc.get_num_fixit_hints ());
> + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
> +- ASSERT_EQ (column_range (14, 14), get_affected_range (hint_0,
> CU_BYTES));
> ++ ASSERT_EQ (column_range (14, 14),
> ++ get_affected_range (&dc, hint_0, CU_BYTES));
> + ASSERT_EQ (column_range (12, 12),
> +- get_affected_range (hint_0,
> CU_DISPLAY_COLS));
> +- ASSERT_EQ (column_range (12, 22), get_printed_columns
> (hint_0));
> ++ get_affected_range (&dc, hint_0, CU_DISPLAY_COLS));
> ++ ASSERT_EQ (column_range (12, 22), get_printed_columns (&dc,
> hint_0));
> + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
> +- ASSERT_EQ (column_range (22, 22), get_affected_range (hint_1,
> CU_BYTES));
> ++ ASSERT_EQ (column_range (22, 22),
> ++ get_affected_range (&dc, hint_1, CU_BYTES));
> + ASSERT_EQ (column_range (18, 18),
> +- get_affected_range (hint_1,
> CU_DISPLAY_COLS));
> +- ASSERT_EQ (column_range (18, 20), get_printed_columns
> (hint_1));
> ++ get_affected_range (&dc, hint_1, CU_DISPLAY_COLS));
> ++ ASSERT_EQ (column_range (18, 20), get_printed_columns (&dc,
> hint_1));
> + const fixit_hint *hint_2 = richloc.get_fixit_hint (2);
> +- ASSERT_EQ (column_range (35, 34), get_affected_range (hint_2,
> CU_BYTES));
> ++ ASSERT_EQ (column_range (35, 34),
> ++ get_affected_range (&dc, hint_2, CU_BYTES));
> + ASSERT_EQ (column_range (30, 29),
> +- get_affected_range (hint_2,
> CU_DISPLAY_COLS));
> +- ASSERT_EQ (column_range (30, 30), get_printed_columns
> (hint_2));
> ++ get_affected_range (&dc, hint_2, CU_DISPLAY_COLS));
> ++ ASSERT_EQ (column_range (30, 30), get_printed_columns (&dc,
> hint_2));
> +
> + /* Add each hint in turn to a line_corrections instance,
> + and verify that they are consolidated into one correction
> instance
> + as expected. */
> +- line_corrections lc (tmp.get_filename (), 1);
> ++ line_corrections lc (&dc, tmp.get_filename (), 1);
> +
> + /* The first replace hint by itself. */
> + lc.add_hint (hint_0);
> +@@ -4689,6 +4859,8 @@ test_overlapped_fixit_printing_2 (const
> +
> + /* Two insertions, in the wrong order. */
> + {
> ++ test_diagnostic_context dc;
> ++
> + rich_location richloc (line_table, col_20);
> + richloc.add_fixit_insert_before (col_23, "{");
> + richloc.add_fixit_insert_before (col_21, "}");
> +@@ -4696,14 +4868,15 @@ test_overlapped_fixit_printing_2 (const
> + /* These fixits should be accepted; they can't be
> consolidated. */
> + ASSERT_EQ (2, richloc.get_num_fixit_hints ());
> + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
> +- ASSERT_EQ (column_range (23, 22), get_affected_range (hint_0,
> CU_BYTES));
> +- ASSERT_EQ (column_range (23, 23), get_printed_columns
> (hint_0));
> ++ ASSERT_EQ (column_range (23, 22),
> ++ get_affected_range (&dc, hint_0, CU_BYTES));
> ++ ASSERT_EQ (column_range (23, 23), get_printed_columns (&dc,
> hint_0));
> + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
> +- ASSERT_EQ (column_range (21, 20), get_affected_range (hint_1,
> CU_BYTES));
> +- ASSERT_EQ (column_range (21, 21), get_printed_columns
> (hint_1));
> ++ ASSERT_EQ (column_range (21, 20),
> ++ get_affected_range (&dc, hint_1, CU_BYTES));
> ++ ASSERT_EQ (column_range (21, 21), get_printed_columns (&dc,
> hint_1));
> +
> + /* Verify that they're printed correctly. */
> +- test_diagnostic_context dc;
> + diagnostic_show_locus (&dc, &richloc, DK_ERROR);
> + ASSERT_STREQ (" int a5[][0][0] = { 1, 2 };\n"
> + " ^\n"
> +@@ -4955,6 +5128,65 @@ test_fixit_deletion_affecting_newline (c
> + pp_formatted_text (dc.printer));
> + }
> +
> ++static void
> ++test_tab_expansion (const line_table_case &case_)
> ++{
> ++ /* Create a tempfile and write some text to it. This example
> uses a tabstop
> ++ of 8, as the column numbers attempt to indicate:
> ++
> ++ .....................000.01111111111.22222333333 display
> ++ .....................123.90123456789.56789012345 columns */
> ++ const char *content = " \t This: `\t' is a tab.\n";
> ++ /* ....................000 00000011111 11111222222 byte
> ++ ....................123 45678901234 56789012345 columns */
> ++
> ++ const int tabstop = 8;
> ++ const int first_non_ws_byte_col = 7;
> ++ const int right_quote_byte_col = 15;
> ++ const int last_byte_col = 25;
> ++ ASSERT_EQ (35, cpp_display_width (content, last_byte_col,
> tabstop));
> ++
> ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
> ++ line_table_test ltt (case_);
> ++ linemap_add (line_table, LC_ENTER, false, tmp.get_filename (),
> 1);
> ++
> ++ /* Don't attempt to run the tests if column data might be
> unavailable. */
> ++ location_t line_end = linemap_position_for_column (line_table,
> last_byte_col);
> ++ if (line_end > LINE_MAP_MAX_LOCATION_WITH_COLS)
> ++ return;
> ++
> ++ /* Check that the leading whitespace with mixed tabs and spaces
> is expanded
> ++ into 11 spaces. Recall that print_line() also puts one space
> before
> ++ everything too. */
> ++ {
> ++ test_diagnostic_context dc;
> ++ dc.tabstop = tabstop;
> ++ rich_location richloc (line_table,
> ++ linemap_position_for_column (line_table,
> ++ first_non_ws_
> byte_col));
> ++ layout test_layout (&dc, &richloc, DK_ERROR);
> ++ test_layout.print_line (1);
> ++ ASSERT_STREQ (" This: ` ' is a tab.\n"
> ++ " ^\n",
> ++ pp_formatted_text (dc.printer));
> ++ }
> ++
> ++ /* Confirm the display width was tracked correctly across the
> internal tab
> ++ as well. */
> ++ {
> ++ test_diagnostic_context dc;
> ++ dc.tabstop = tabstop;
> ++ rich_location richloc (line_table,
> ++ linemap_position_for_column (line_table,
> ++ right_quote_b
> yte_col));
> ++ layout test_layout (&dc, &richloc, DK_ERROR);
> ++ test_layout.print_line (1);
> ++ ASSERT_STREQ (" This: ` ' is a tab.\n"
> ++ " ^\n",
> ++ pp_formatted_text (dc.printer));
> ++ }
> ++}
> ++
> + /* Verify that line numbers are correctly printed for the case of
> + a multiline range in which the width of the line numbers changes
> + (e.g. from "9" to "10"). */
> +@@ -5012,6 +5244,7 @@ diagnostic_show_locus_c_tests ()
> + test_layout_range_for_multiple_lines ();
> +
> + for_each_line_table_case (test_layout_x_offset_display_utf8);
> ++ for_each_line_table_case (test_layout_x_offset_display_tab);
> +
> + test_get_line_bytes_without_trailing_whitespace ();
> +
> +@@ -5029,6 +5262,7 @@ diagnostic_show_locus_c_tests ()
> + for_each_line_table_case
> (test_fixit_insert_containing_newline_2);
> + for_each_line_table_case (test_fixit_replace_containing_newline);
> + for_each_line_table_case (test_fixit_deletion_affecting_newline);
> ++ for_each_line_table_case (test_tab_expansion);
> +
> + test_line_numbers_multiline_range ();
> + }
> +diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> +--- a/gcc/doc/invoke.texi 2021-12-24 20:23:46.876739587 -0800
> ++++ b/gcc/doc/invoke.texi 2021-12-25 01:20:53.487636494 -0800
> +@@ -293,7 +293,9 @@ Objective-C and Objective-C++ Dialects}.
> + -fdiagnostics-show-template-tree -fno-elide-type @gol
> +
> -fdiagnostics-path-format=@r{[}none@r{|}separate-events@r{|}inline-events@r{
> ]} @gol
> + -fdiagnostics-show-path-depths @gol
> +--fno-show-column}
> ++-fno-show-column @gol
> ++-fdiagnostics-column-unit=@r{[}display@r{|}byte@r{]} @gol
> ++-fdiagnostics-column-origin=@var{origin}}
> +
> + @item Warning Options
> + @xref{Warning Options,,Options to Request or Suppress Warnings}.
> +@@ -4424,6 +4426,31 @@ Do not print column numbers in diagnosti
> + diagnostics are being scanned by a program that does not understand
> the
> + column numbers, such as @command{dejagnu}.
> +
> ++@item -fdiagnostics-column-unit=@var{UNIT}
> ++@opindex fdiagnostics-column-unit
> ++Select the units for the column number. This affects traditional
> diagnostics
> ++(in the absence of @option{-fno-show-column}), as well as JSON
> format
> ++diagnostics if requested.
> ++
> ++The default @var{UNIT}, @samp{display}, considers the number of
> display
> ++columns occupied by each character. This may be larger than the
> number
> ++of bytes required to encode the character, in the case of tab
> ++characters, or it may be smaller, in the case of multibyte
> characters.
> ++For example, the character ``GREEK SMALL LETTER PI (U+03C0)''
> occupies one
> ++display column, and its UTF-8 encoding requires two bytes; the
> character
> ++``SLIGHTLY SMILING FACE (U+1F642)'' occupies two display columns,
> and
> ++its UTF-8 encoding requires four bytes.
> ++
> ++Setting @var{UNIT} to @samp{byte} changes the column number to the
> raw byte
> ++count in all cases, as was traditionally output by GCC prior to
> version 11.1.0.
> ++
> ++@item -fdiagnostics-column-origin=@var{ORIGIN}
> ++@opindex fdiagnostics-column-origin
> ++Select the origin for column numbers, i.e. the column number
> assigned to the
> ++first column. The default value of 1 corresponds to traditional
> GCC
> ++behavior and to the GNU style guide. Some utilities may perform
> better with an
> ++origin of 0; any non-negative value may be specified.
> ++
> + @item -fdiagnostics-format=@var{FORMAT}
> + @opindex fdiagnostics-format
> + Select a different format for printing diagnostics.
> +@@ -4459,11 +4486,15 @@ might be printed in JSON form (after for
> + "locations": [
> + @{
> + "caret": @{
> ++ "display-column": 3,
> ++ "byte-column": 3,
> + "column": 3,
> + "file": "misleading-indentation.c",
> + "line": 15
> + @},
> + "finish": @{
> ++ "display-column": 4,
> ++ "byte-column": 4,
> + "column": 4,
> + "file": "misleading-indentation.c",
> + "line": 15
> +@@ -4479,6 +4510,8 @@ might be printed in JSON form (after for
> + "locations": [
> + @{
> + "caret": @{
> ++ "display-column": 5,
> ++ "byte-column": 5,
> + "column": 5,
> + "file": "misleading-indentation.c",
> + "line": 17
> +@@ -4488,6 +4521,7 @@ might be printed in JSON form (after for
> + "message": "...this statement, but the latter is
> @dots{}"
> + @}
> + ]
> ++ "column-origin": 1,
> + @},
> + @dots{}
> + ]
> +@@ -4500,10 +4534,34 @@ A diagnostic has a @code{kind}. If this
> + an @code{option} key describing the command-line option controlling
> the
> + warning.
> +
> +-A diagnostic can contain zero or more locations. Each location has
> up
> +-to three positions within it: a @code{caret} position and optional
> +-@code{start} and @code{finish} positions. A location can also have
> +-an optional @code{label} string. For example, this error:
> ++A diagnostic can contain zero or more locations. Each location has
>