Files
gcc/libcpp/generated_cpp_wcwidth.h

177 lines
12 KiB
C
Raw Normal View History

Byte vs column awareness for diagnostic-show-locus.c (PR 49973) contrib/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * unicode/from_glibc/unicode_utils.py: Support script from glibc (commit 464cd3) to extract character widths from Unicode data files. * unicode/from_glibc/utf8_gen.py: Likewise. * unicode/UnicodeData.txt: Unicode v. 12.1.0 data file. * unicode/EastAsianWidth.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/gen_wcwidth.py: New utility to generate libcpp/generated_cpp_wcwidth.h with help from the glibc support scripts and the Unicode data files. * unicode/unicode-license.txt: Added. * unicode/README: New explanatory file. libcpp/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * generated_cpp_wcwidth.h: New file generated by ../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function. * charset.c (compute_next_display_width): New function to help implement display columns. (cpp_byte_column_to_display_column): Likewise. (cpp_display_column_to_byte_column): Likewise. (cpp_wcwidth): Likewise. * include/cpplib.h (cpp_byte_column_to_display_column): Declare. (cpp_display_column_to_byte_column): Declare. (cpp_wcwidth): Declare. (cpp_display_width): New function. gcc/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * input.c (location_compute_display_column): New function to help with multibyte awareness in diagnostics. (test_cpp_utf8): New self-test. (input_c_tests): Call the new test. * input.h (location_compute_display_column): Declare. * diagnostic-show-locus.c: Pervasive changes to add multibyte awareness to all classes and functions. (enum column_unit): New enum. (class exploc_with_display_col): New class. (class layout_point): Convert m_column member to array m_columns[2]. (layout_range::contains_point): Add col_unit argument. (test_layout_range_for_single_point): Pass new argument. (test_layout_range_for_single_line): Likewise. (test_layout_range_for_multiple_lines): Likewise. (line_bounds::convert_to_display_cols): New function. (layout::get_state_at_point): Add col_unit argument. (make_range): Use empty filename rather than dummy filename. (get_line_width_without_trailing_whitespace): Rename to... (get_line_bytes_without_trailing_whitespace): ...this. (test_get_line_width_without_trailing_whitespace): Rename to... (test_get_line_bytes_without_trailing_whitespace): ...this. (class layout): m_exploc changed to exploc_with_display_col from plain expanded_location. (layout::get_linenum_width): New accessor member function. (layout::get_x_offset_display): Likewise. (layout::calculate_linenum_width): New subroutine for the constuctor. (layout::calculate_x_offset_display): Likewise. (layout::layout): Use the new subroutines. Add multibyte awareness. (layout::print_source_line): Add multibyte awareness. (layout::print_line): Likewise. (layout::print_annotation_line): Likewise. (line_label::line_label): Likewise. (layout::print_any_labels): Likewise. (layout::annotation_line_showed_range_p): Likewise. (get_printed_columns): Likewise. (class line_label): Rename m_length to m_display_width. (get_affected_columns): Rename to... (get_affected_range): ...this; add col_unit argument and multibyte awareness. (class correction): Add m_affected_bytes and m_display_cols members. Rename m_len to m_byte_length for clarity. Add multibyte awareness throughout. (correction::insertion_p): Add multibyte awareness. (correction::compute_display_cols): New function. (correction::ensure_terminated): Use new member name m_byte_length. (line_corrections::add_hint): Add multibyte awareness. (layout::print_trailing_fixits): Likewise. (layout::get_x_bound_for_row): Likewise. (test_one_liner_simple_caret_utf8): New self-test analogous to the one with _utf8 suffix removed, testing multibyte awareness. (test_one_liner_caret_and_range_utf8): Likewise. (test_one_liner_multiple_carets_and_ranges_utf8): Likewise. (test_one_liner_fixit_insert_before_utf8): Likewise. (test_one_liner_fixit_insert_after_utf8): Likewise. (test_one_liner_fixit_remove_utf8): Likewise. (test_one_liner_fixit_replace_utf8): Likewise. (test_one_liner_fixit_replace_non_equal_range_utf8): Likewise. (test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise. (test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise. (test_one_liner_many_fixits_1_utf8): Likewise. (test_one_liner_many_fixits_2_utf8): Likewise. (test_one_liner_labels_utf8): Likewise. (test_diagnostic_show_locus_one_liner_utf8): Likewise. (test_overlapped_fixit_printing_utf8): Likewise. (test_overlapped_fixit_printing): Adapt for changes to get_affected_columns, get_printed_columns and class corrections. (test_overlapped_fixit_printing_2): Likewise. (test_linenum_sep): New constant. (test_left_margin): Likewise. (test_offset_impl): Helper function for new test. (test_layout_x_offset_display_utf8): New test. (diagnostic_show_locus_c_tests): Call new tests. gcc/testsuite/ChangeLog: 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (test_show_locus): Tweak so that expected output is the same as before the diagnostic-show-locus.c changes. * gcc.dg/cpp/pr66415-1.c: Likewise. From-SVN: r279137
2019-12-09 20:03:47 +00:00
/* Generated by contrib/unicode/gen_wcwidth.py, with the help of glibc's
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
utf8_gen.py, using version 17.0 of the Unicode standard. */
Byte vs column awareness for diagnostic-show-locus.c (PR 49973) contrib/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * unicode/from_glibc/unicode_utils.py: Support script from glibc (commit 464cd3) to extract character widths from Unicode data files. * unicode/from_glibc/utf8_gen.py: Likewise. * unicode/UnicodeData.txt: Unicode v. 12.1.0 data file. * unicode/EastAsianWidth.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/gen_wcwidth.py: New utility to generate libcpp/generated_cpp_wcwidth.h with help from the glibc support scripts and the Unicode data files. * unicode/unicode-license.txt: Added. * unicode/README: New explanatory file. libcpp/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * generated_cpp_wcwidth.h: New file generated by ../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function. * charset.c (compute_next_display_width): New function to help implement display columns. (cpp_byte_column_to_display_column): Likewise. (cpp_display_column_to_byte_column): Likewise. (cpp_wcwidth): Likewise. * include/cpplib.h (cpp_byte_column_to_display_column): Declare. (cpp_display_column_to_byte_column): Declare. (cpp_wcwidth): Declare. (cpp_display_width): New function. gcc/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * input.c (location_compute_display_column): New function to help with multibyte awareness in diagnostics. (test_cpp_utf8): New self-test. (input_c_tests): Call the new test. * input.h (location_compute_display_column): Declare. * diagnostic-show-locus.c: Pervasive changes to add multibyte awareness to all classes and functions. (enum column_unit): New enum. (class exploc_with_display_col): New class. (class layout_point): Convert m_column member to array m_columns[2]. (layout_range::contains_point): Add col_unit argument. (test_layout_range_for_single_point): Pass new argument. (test_layout_range_for_single_line): Likewise. (test_layout_range_for_multiple_lines): Likewise. (line_bounds::convert_to_display_cols): New function. (layout::get_state_at_point): Add col_unit argument. (make_range): Use empty filename rather than dummy filename. (get_line_width_without_trailing_whitespace): Rename to... (get_line_bytes_without_trailing_whitespace): ...this. (test_get_line_width_without_trailing_whitespace): Rename to... (test_get_line_bytes_without_trailing_whitespace): ...this. (class layout): m_exploc changed to exploc_with_display_col from plain expanded_location. (layout::get_linenum_width): New accessor member function. (layout::get_x_offset_display): Likewise. (layout::calculate_linenum_width): New subroutine for the constuctor. (layout::calculate_x_offset_display): Likewise. (layout::layout): Use the new subroutines. Add multibyte awareness. (layout::print_source_line): Add multibyte awareness. (layout::print_line): Likewise. (layout::print_annotation_line): Likewise. (line_label::line_label): Likewise. (layout::print_any_labels): Likewise. (layout::annotation_line_showed_range_p): Likewise. (get_printed_columns): Likewise. (class line_label): Rename m_length to m_display_width. (get_affected_columns): Rename to... (get_affected_range): ...this; add col_unit argument and multibyte awareness. (class correction): Add m_affected_bytes and m_display_cols members. Rename m_len to m_byte_length for clarity. Add multibyte awareness throughout. (correction::insertion_p): Add multibyte awareness. (correction::compute_display_cols): New function. (correction::ensure_terminated): Use new member name m_byte_length. (line_corrections::add_hint): Add multibyte awareness. (layout::print_trailing_fixits): Likewise. (layout::get_x_bound_for_row): Likewise. (test_one_liner_simple_caret_utf8): New self-test analogous to the one with _utf8 suffix removed, testing multibyte awareness. (test_one_liner_caret_and_range_utf8): Likewise. (test_one_liner_multiple_carets_and_ranges_utf8): Likewise. (test_one_liner_fixit_insert_before_utf8): Likewise. (test_one_liner_fixit_insert_after_utf8): Likewise. (test_one_liner_fixit_remove_utf8): Likewise. (test_one_liner_fixit_replace_utf8): Likewise. (test_one_liner_fixit_replace_non_equal_range_utf8): Likewise. (test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise. (test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise. (test_one_liner_many_fixits_1_utf8): Likewise. (test_one_liner_many_fixits_2_utf8): Likewise. (test_one_liner_labels_utf8): Likewise. (test_diagnostic_show_locus_one_liner_utf8): Likewise. (test_overlapped_fixit_printing_utf8): Likewise. (test_overlapped_fixit_printing): Adapt for changes to get_affected_columns, get_printed_columns and class corrections. (test_overlapped_fixit_printing_2): Likewise. (test_linenum_sep): New constant. (test_left_margin): Likewise. (test_offset_impl): Helper function for new test. (test_layout_x_offset_display_utf8): New test. (diagnostic_show_locus_c_tests): Call new tests. gcc/testsuite/ChangeLog: 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (test_show_locus): Tweak so that expected output is the same as before the diagnostic-show-locus.c changes. * gcc.dg/cpp/pr66415-1.c: Likewise. From-SVN: r279137
2019-12-09 20:03:47 +00:00
static const cppchar_t wcwidth_range_ends[] = {
0x2ff, 0x36f, 0x482, 0x489, 0x590, 0x5bd, 0x5be, 0x5bf,
0x5c0, 0x5c2, 0x5c3, 0x5c5, 0x5c6, 0x5c7, 0x60f, 0x61a,
0x61b, 0x61c, 0x64a, 0x65f, 0x66f, 0x670, 0x6d5, 0x6dc,
0x6de, 0x6e4, 0x6e6, 0x6e8, 0x6e9, 0x6ed, 0x710, 0x711,
0x72f, 0x74a, 0x7a5, 0x7b0, 0x7ea, 0x7f3, 0x7fc, 0x7fd,
0x815, 0x819, 0x81a, 0x823, 0x824, 0x827, 0x828, 0x82d,
contrib, libcpp, libstdc++: Update to Unicode 16.0 It is autumn again and there is a new Unicode version 16.0. The following patch updates our Unicode stuff in contrib, libcpp and libstdc++ from that Unicode version. 2024-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Update glibc git commit hash, replace Unicode 15 or 15.1 versions with 16. * unicode/gen_libstdcxx_unicode_data.py: Use 160000 instead of 150100 in _GLIBCXX_GET_UNICODE_DATA test. * unicode/from_glibc/utf8_gen.py: Updated from glibc 064c708c78cc2a6b5802dce73108fc0c1c6bfc80 commit. * unicode/DerivedCoreProperties.txt: Updated from Unicode 16.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add tests for some Unicode 16.0 characters, both normal and generated. libcpp/ * makeucnid.cc (write_copyright): Update Unicode Copyright years. * makeuname2c.cc (generated_ranges): Adjust Unicode version from 15.1 to 16.0. Add EGYPTIAN HIEROGLYPH- generated range, adjust indexes in following entries. (write_copyright): Update Unicode Copyright years. * generated_cpp_wcwidth.h: Regenerated. * ucnid.h: Regenerated. * uname2c.h: Regenerated. libstdc++-v3/ * include/bits/unicode.h (std::__unicode::__v15_1_0): Rename inline namespace to ... (std::__unicode::__v16_0_0): ... this. (_GLIBCXX_GET_UNICODE_DATA): Change from 150100 to 160000. * include/bits/unicode-data.h: Regenerated. * testsuite/ext/unicode/properties.cc: Check for _Gcb_SpacingMark on U+11F03 rather than U+1D16D as the latter lost SpacingMark property in Unicode 16.0.
2024-10-08 10:01:47 +02:00
0x858, 0x85b, 0x896, 0x89f, 0x8c9, 0x8e1, 0x8e2, 0x902,
0x939, 0x93a, 0x93b, 0x93c, 0x940, 0x948, 0x94c, 0x94d,
0x950, 0x957, 0x961, 0x963, 0x980, 0x981, 0x9bb, 0x9bc,
0x9c0, 0x9c4, 0x9cc, 0x9cd, 0x9e1, 0x9e3, 0x9fd, 0x9fe,
0xa00, 0xa02, 0xa3b, 0xa3c, 0xa40, 0xa42, 0xa46, 0xa48,
0xa4a, 0xa4d, 0xa50, 0xa51, 0xa6f, 0xa71, 0xa74, 0xa75,
0xa80, 0xa82, 0xabb, 0xabc, 0xac0, 0xac5, 0xac6, 0xac8,
0xacc, 0xacd, 0xae1, 0xae3, 0xaf9, 0xaff, 0xb00, 0xb01,
0xb3b, 0xb3c, 0xb3e, 0xb3f, 0xb40, 0xb44, 0xb4c, 0xb4d,
0xb54, 0xb56, 0xb61, 0xb63, 0xb81, 0xb82, 0xbbf, 0xbc0,
0xbcc, 0xbcd, 0xbff, 0xc00, 0xc03, 0xc04, 0xc3b, 0xc3c,
0xc3d, 0xc40, 0xc45, 0xc48, 0xc49, 0xc4d, 0xc54, 0xc56,
0xc61, 0xc63, 0xc80, 0xc81, 0xcbb, 0xcbc, 0xcbe, 0xcbf,
0xcc5, 0xcc6, 0xccb, 0xccd, 0xce1, 0xce3, 0xcff, 0xd01,
0xd3a, 0xd3c, 0xd40, 0xd44, 0xd4c, 0xd4d, 0xd61, 0xd63,
0xd80, 0xd81, 0xdc9, 0xdca, 0xdd1, 0xdd4, 0xdd5, 0xdd6,
0xe30, 0xe31, 0xe33, 0xe3a, 0xe46, 0xe4e, 0xeb0, 0xeb1,
0xeb3, 0xebc, 0xec7, 0xece, 0xf17, 0xf19, 0xf34, 0xf35,
0xf36, 0xf37, 0xf38, 0xf39, 0xf70, 0xf7e, 0xf7f, 0xf84,
0xf85, 0xf87, 0xf8c, 0xf97, 0xf98, 0xfbc, 0xfc5, 0xfc6,
0x102c, 0x1030, 0x1031, 0x1037, 0x1038, 0x103a, 0x103c, 0x103e,
0x1057, 0x1059, 0x105d, 0x1060, 0x1070, 0x1074, 0x1081, 0x1082,
0x1084, 0x1086, 0x108c, 0x108d, 0x109c, 0x109d, 0x10ff, 0x115f,
0x11ff, 0x135c, 0x135f, 0x1711, 0x1714, 0x1731, 0x1733, 0x1751,
0x1753, 0x1771, 0x1773, 0x17b3, 0x17b5, 0x17b6, 0x17bd, 0x17c5,
0x17c6, 0x17c8, 0x17d3, 0x17dc, 0x17dd, 0x180a, 0x180f, 0x1884,
0x1886, 0x18a8, 0x18a9, 0x191f, 0x1922, 0x1926, 0x1928, 0x1931,
0x1932, 0x1938, 0x193b, 0x1a16, 0x1a18, 0x1a1a, 0x1a1b, 0x1a55,
0x1a56, 0x1a57, 0x1a5e, 0x1a5f, 0x1a60, 0x1a61, 0x1a62, 0x1a64,
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
0x1a6c, 0x1a72, 0x1a7c, 0x1a7e, 0x1a7f, 0x1aaf, 0x1add, 0x1adf,
0x1aeb, 0x1aff, 0x1b03, 0x1b33, 0x1b34, 0x1b35, 0x1b3a, 0x1b3b,
0x1b3c, 0x1b41, 0x1b42, 0x1b6a, 0x1b73, 0x1b7f, 0x1b81, 0x1ba1,
0x1ba5, 0x1ba7, 0x1ba9, 0x1baa, 0x1bad, 0x1be5, 0x1be6, 0x1be7,
0x1be9, 0x1bec, 0x1bed, 0x1bee, 0x1bf1, 0x1c2b, 0x1c33, 0x1c35,
0x1c37, 0x1ccf, 0x1cd2, 0x1cd3, 0x1ce0, 0x1ce1, 0x1ce8, 0x1cec,
0x1ced, 0x1cf3, 0x1cf4, 0x1cf7, 0x1cf9, 0x1dbf, 0x1dff, 0x200a,
0x200f, 0x2029, 0x202e, 0x205f, 0x2064, 0x2065, 0x206f, 0x20cf,
0x20f0, 0x2319, 0x231b, 0x2328, 0x232a, 0x23e8, 0x23ec, 0x23ef,
0x23f0, 0x23f2, 0x23f3, 0x25fc, 0x25fe, 0x2613, 0x2615, 0x262f,
0x2637, 0x2647, 0x2653, 0x267e, 0x267f, 0x2689, 0x268f, 0x2692,
0x2693, 0x26a0, 0x26a1, 0x26a9, 0x26ab, 0x26bc, 0x26be, 0x26c3,
0x26c5, 0x26cd, 0x26ce, 0x26d3, 0x26d4, 0x26e9, 0x26ea, 0x26f1,
0x26f3, 0x26f4, 0x26f5, 0x26f9, 0x26fa, 0x26fc, 0x26fd, 0x2704,
0x2705, 0x2709, 0x270b, 0x2727, 0x2728, 0x274b, 0x274c, 0x274d,
0x274e, 0x2752, 0x2755, 0x2756, 0x2757, 0x2794, 0x2797, 0x27af,
0x27b0, 0x27be, 0x27bf, 0x2b1a, 0x2b1c, 0x2b4f, 0x2b50, 0x2b54,
0x2b55, 0x2cee, 0x2cf1, 0x2d7e, 0x2d7f, 0x2ddf, 0x2dff, 0x2e7f,
0x2e99, 0x2e9a, 0x2ef3, 0x2eff, 0x2fd5, 0x2fef, 0x3029, 0x302d,
0x303e, 0x3040, 0x3096, 0x3098, 0x309a, 0x30ff, 0x3104, 0x312f,
0x3130, 0x3163, 0x3164, 0x318e, 0x318f, 0x31e5, 0x31ee, 0x321e,
0x321f, 0xa48c, 0xa48f, 0xa4c6, 0xa66e, 0xa672, 0xa673, 0xa67d,
0xa69d, 0xa69f, 0xa6ef, 0xa6f1, 0xa801, 0xa802, 0xa805, 0xa806,
0xa80a, 0xa80b, 0xa824, 0xa826, 0xa82b, 0xa82c, 0xa8c3, 0xa8c5,
0xa8df, 0xa8f1, 0xa8fe, 0xa8ff, 0xa925, 0xa92d, 0xa946, 0xa951,
0xa95f, 0xa97c, 0xa97f, 0xa982, 0xa9b2, 0xa9b3, 0xa9b5, 0xa9b9,
0xa9bb, 0xa9bd, 0xa9e4, 0xa9e5, 0xaa28, 0xaa2e, 0xaa30, 0xaa32,
0xaa34, 0xaa36, 0xaa42, 0xaa43, 0xaa4b, 0xaa4c, 0xaa7b, 0xaa7c,
0xaaaf, 0xaab0, 0xaab1, 0xaab4, 0xaab6, 0xaab8, 0xaabd, 0xaabf,
0xaac0, 0xaac1, 0xaaeb, 0xaaed, 0xaaf5, 0xaaf6, 0xabe4, 0xabe5,
0xabe7, 0xabe8, 0xabec, 0xabed, 0xabff, 0xd7a3, 0xd7af, 0xd7c6,
0xd7ca, 0xd7fb, 0xf8ff, 0xfa6d, 0xfa6f, 0xfad9, 0xfb1d, 0xfb1e,
0xfdff, 0xfe0f, 0xfe19, 0xfe1f, 0xfe2f, 0xfe52, 0xfe53, 0xfe66,
0xfe67, 0xfe6b, 0xfefe, 0xfeff, 0xff00, 0xff60, 0xff9f, 0xffa0,
0xffdf, 0xffe6, 0x101fc, 0x101fd, 0x102df, 0x102e0, 0x10375, 0x1037a,
0x10a00, 0x10a03, 0x10a04, 0x10a06, 0x10a0b, 0x10a0f, 0x10a37, 0x10a3a,
0x10a3e, 0x10a3f, 0x10ae4, 0x10ae6, 0x10d23, 0x10d27, 0x10d68, 0x10d6d,
0x10eaa, 0x10eac, 0x10ef9, 0x10eff, 0x10f45, 0x10f50, 0x10f81, 0x10f85,
0x11000, 0x11001, 0x11037, 0x11046, 0x1106f, 0x11070, 0x11072, 0x11074,
0x1107e, 0x11081, 0x110b2, 0x110b6, 0x110b8, 0x110ba, 0x110c1, 0x110c2,
0x110ff, 0x11102, 0x11126, 0x1112b, 0x1112c, 0x11134, 0x11172, 0x11173,
0x1117f, 0x11181, 0x111b5, 0x111be, 0x111c8, 0x111cc, 0x111ce, 0x111cf,
0x1122e, 0x11231, 0x11233, 0x11234, 0x11235, 0x11237, 0x1123d, 0x1123e,
0x11240, 0x11241, 0x112de, 0x112df, 0x112e2, 0x112ea, 0x112ff, 0x11301,
0x1133a, 0x1133c, 0x1133f, 0x11340, 0x11365, 0x1136c, 0x1136f, 0x11374,
0x113ba, 0x113c0, 0x113cd, 0x113ce, 0x113cf, 0x113d0, 0x113d1, 0x113d2,
0x113e0, 0x113e2, 0x11437, 0x1143f, 0x11441, 0x11444, 0x11445, 0x11446,
0x1145d, 0x1145e, 0x114b2, 0x114b8, 0x114b9, 0x114ba, 0x114be, 0x114c0,
0x114c1, 0x114c3, 0x115b1, 0x115b5, 0x115bb, 0x115bd, 0x115be, 0x115c0,
0x115db, 0x115dd, 0x11632, 0x1163a, 0x1163c, 0x1163d, 0x1163e, 0x11640,
0x116aa, 0x116ab, 0x116ac, 0x116ad, 0x116af, 0x116b5, 0x116b6, 0x116b7,
0x1171c, 0x1171d, 0x1171e, 0x1171f, 0x11721, 0x11725, 0x11726, 0x1172b,
0x1182e, 0x11837, 0x11838, 0x1183a, 0x1193a, 0x1193c, 0x1193d, 0x1193e,
0x11942, 0x11943, 0x119d3, 0x119d7, 0x119d9, 0x119db, 0x119df, 0x119e0,
0x11a00, 0x11a0a, 0x11a32, 0x11a38, 0x11a3a, 0x11a3e, 0x11a46, 0x11a47,
0x11a50, 0x11a56, 0x11a58, 0x11a5b, 0x11a89, 0x11a96, 0x11a97, 0x11a99,
0x11b5f, 0x11b60, 0x11b61, 0x11b64, 0x11b65, 0x11b66, 0x11c2f, 0x11c36,
0x11c37, 0x11c3d, 0x11c3e, 0x11c3f, 0x11c91, 0x11ca7, 0x11ca9, 0x11cb0,
0x11cb1, 0x11cb3, 0x11cb4, 0x11cb6, 0x11d30, 0x11d36, 0x11d39, 0x11d3a,
0x11d3b, 0x11d3d, 0x11d3e, 0x11d45, 0x11d46, 0x11d47, 0x11d8f, 0x11d91,
0x11d94, 0x11d95, 0x11d96, 0x11d97, 0x11ef2, 0x11ef4, 0x11eff, 0x11f01,
0x11f35, 0x11f3a, 0x11f3f, 0x11f40, 0x11f41, 0x11f42, 0x11f59, 0x11f5a,
0x1343f, 0x13440, 0x13446, 0x13455, 0x1611d, 0x16129, 0x1612c, 0x1612f,
0x16aef, 0x16af4, 0x16b2f, 0x16b36, 0x16f4e, 0x16f4f, 0x16f8e, 0x16f92,
0x16fdf, 0x16fe3, 0x16fe4, 0x16fef, 0x16ff6, 0x16fff, 0x18cd5, 0x18cfe,
0x18d1e, 0x18d7f, 0x18df2, 0x1afef, 0x1aff3, 0x1aff4, 0x1affb, 0x1affc,
0x1affe, 0x1afff, 0x1b122, 0x1b131, 0x1b132, 0x1b14f, 0x1b152, 0x1b154,
0x1b155, 0x1b163, 0x1b167, 0x1b16f, 0x1b2fb, 0x1bc9c, 0x1bc9e, 0x1bc9f,
0x1bca3, 0x1ceff, 0x1cf2d, 0x1cf2f, 0x1cf46, 0x1d166, 0x1d169, 0x1d172,
0x1d182, 0x1d184, 0x1d18b, 0x1d1a9, 0x1d1ad, 0x1d241, 0x1d244, 0x1d2ff,
0x1d356, 0x1d35f, 0x1d376, 0x1d9ff, 0x1da36, 0x1da3a, 0x1da6c, 0x1da74,
0x1da75, 0x1da83, 0x1da84, 0x1da9a, 0x1da9f, 0x1daa0, 0x1daaf, 0x1dfff,
0x1e006, 0x1e007, 0x1e018, 0x1e01a, 0x1e021, 0x1e022, 0x1e024, 0x1e025,
0x1e02a, 0x1e08e, 0x1e08f, 0x1e12f, 0x1e136, 0x1e2ad, 0x1e2ae, 0x1e2eb,
0x1e2ef, 0x1e4eb, 0x1e4ef, 0x1e5ed, 0x1e5ef, 0x1e6e2, 0x1e6e3, 0x1e6e5,
0x1e6e6, 0x1e6ed, 0x1e6ef, 0x1e6f4, 0x1e6f5, 0x1e8cf, 0x1e8d6, 0x1e943,
0x1e94a, 0x1f003, 0x1f004, 0x1f0ce, 0x1f0cf, 0x1f18d, 0x1f18e, 0x1f190,
0x1f19a, 0x1f1ff, 0x1f202, 0x1f20f, 0x1f23b, 0x1f23f, 0x1f248, 0x1f24f,
0x1f251, 0x1f25f, 0x1f265, 0x1f2ff, 0x1f320, 0x1f32c, 0x1f335, 0x1f336,
0x1f37c, 0x1f37d, 0x1f393, 0x1f39f, 0x1f3ca, 0x1f3ce, 0x1f3d3, 0x1f3df,
0x1f3f0, 0x1f3f3, 0x1f3f4, 0x1f3f7, 0x1f43e, 0x1f43f, 0x1f440, 0x1f441,
0x1f4fc, 0x1f4fe, 0x1f53d, 0x1f54a, 0x1f54e, 0x1f54f, 0x1f567, 0x1f579,
0x1f57a, 0x1f594, 0x1f596, 0x1f5a3, 0x1f5a4, 0x1f5fa, 0x1f64f, 0x1f67f,
0x1f6c5, 0x1f6cb, 0x1f6cc, 0x1f6cf, 0x1f6d2, 0x1f6d4, 0x1f6d8, 0x1f6db,
0x1f6df, 0x1f6ea, 0x1f6ec, 0x1f6f3, 0x1f6fc, 0x1f7df, 0x1f7eb, 0x1f7ef,
0x1f7f0, 0x1f90b, 0x1f93a, 0x1f93b, 0x1f945, 0x1f946, 0x1f9ff, 0x1fa6f,
0x1fa7c, 0x1fa7f, 0x1fa8a, 0x1fa8d, 0x1fac6, 0x1fac7, 0x1fac8, 0x1facc,
0x1fadc, 0x1fade, 0x1faea, 0x1faee, 0x1faf8, 0x1ffff, 0x2a6df, 0x2a6ff,
0x2b81d, 0x2b81f, 0x2cead, 0x2ceaf, 0x2ebe0, 0x2ebef, 0x2ee5d, 0x2f7ff,
0x2fa1d, 0x2ffff, 0x3134a, 0x3134f, 0x33479, 0xe0000, 0xe0001, 0xe001f,
0xe007f, 0xe00ff, 0xe01ef,
Byte vs column awareness for diagnostic-show-locus.c (PR 49973) contrib/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * unicode/from_glibc/unicode_utils.py: Support script from glibc (commit 464cd3) to extract character widths from Unicode data files. * unicode/from_glibc/utf8_gen.py: Likewise. * unicode/UnicodeData.txt: Unicode v. 12.1.0 data file. * unicode/EastAsianWidth.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/gen_wcwidth.py: New utility to generate libcpp/generated_cpp_wcwidth.h with help from the glibc support scripts and the Unicode data files. * unicode/unicode-license.txt: Added. * unicode/README: New explanatory file. libcpp/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * generated_cpp_wcwidth.h: New file generated by ../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function. * charset.c (compute_next_display_width): New function to help implement display columns. (cpp_byte_column_to_display_column): Likewise. (cpp_display_column_to_byte_column): Likewise. (cpp_wcwidth): Likewise. * include/cpplib.h (cpp_byte_column_to_display_column): Declare. (cpp_display_column_to_byte_column): Declare. (cpp_wcwidth): Declare. (cpp_display_width): New function. gcc/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * input.c (location_compute_display_column): New function to help with multibyte awareness in diagnostics. (test_cpp_utf8): New self-test. (input_c_tests): Call the new test. * input.h (location_compute_display_column): Declare. * diagnostic-show-locus.c: Pervasive changes to add multibyte awareness to all classes and functions. (enum column_unit): New enum. (class exploc_with_display_col): New class. (class layout_point): Convert m_column member to array m_columns[2]. (layout_range::contains_point): Add col_unit argument. (test_layout_range_for_single_point): Pass new argument. (test_layout_range_for_single_line): Likewise. (test_layout_range_for_multiple_lines): Likewise. (line_bounds::convert_to_display_cols): New function. (layout::get_state_at_point): Add col_unit argument. (make_range): Use empty filename rather than dummy filename. (get_line_width_without_trailing_whitespace): Rename to... (get_line_bytes_without_trailing_whitespace): ...this. (test_get_line_width_without_trailing_whitespace): Rename to... (test_get_line_bytes_without_trailing_whitespace): ...this. (class layout): m_exploc changed to exploc_with_display_col from plain expanded_location. (layout::get_linenum_width): New accessor member function. (layout::get_x_offset_display): Likewise. (layout::calculate_linenum_width): New subroutine for the constuctor. (layout::calculate_x_offset_display): Likewise. (layout::layout): Use the new subroutines. Add multibyte awareness. (layout::print_source_line): Add multibyte awareness. (layout::print_line): Likewise. (layout::print_annotation_line): Likewise. (line_label::line_label): Likewise. (layout::print_any_labels): Likewise. (layout::annotation_line_showed_range_p): Likewise. (get_printed_columns): Likewise. (class line_label): Rename m_length to m_display_width. (get_affected_columns): Rename to... (get_affected_range): ...this; add col_unit argument and multibyte awareness. (class correction): Add m_affected_bytes and m_display_cols members. Rename m_len to m_byte_length for clarity. Add multibyte awareness throughout. (correction::insertion_p): Add multibyte awareness. (correction::compute_display_cols): New function. (correction::ensure_terminated): Use new member name m_byte_length. (line_corrections::add_hint): Add multibyte awareness. (layout::print_trailing_fixits): Likewise. (layout::get_x_bound_for_row): Likewise. (test_one_liner_simple_caret_utf8): New self-test analogous to the one with _utf8 suffix removed, testing multibyte awareness. (test_one_liner_caret_and_range_utf8): Likewise. (test_one_liner_multiple_carets_and_ranges_utf8): Likewise. (test_one_liner_fixit_insert_before_utf8): Likewise. (test_one_liner_fixit_insert_after_utf8): Likewise. (test_one_liner_fixit_remove_utf8): Likewise. (test_one_liner_fixit_replace_utf8): Likewise. (test_one_liner_fixit_replace_non_equal_range_utf8): Likewise. (test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise. (test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise. (test_one_liner_many_fixits_1_utf8): Likewise. (test_one_liner_many_fixits_2_utf8): Likewise. (test_one_liner_labels_utf8): Likewise. (test_diagnostic_show_locus_one_liner_utf8): Likewise. (test_overlapped_fixit_printing_utf8): Likewise. (test_overlapped_fixit_printing): Adapt for changes to get_affected_columns, get_printed_columns and class corrections. (test_overlapped_fixit_printing_2): Likewise. (test_linenum_sep): New constant. (test_left_margin): Likewise. (test_offset_impl): Helper function for new test. (test_layout_x_offset_display_utf8): New test. (diagnostic_show_locus_c_tests): Call new tests. gcc/testsuite/ChangeLog: 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (test_show_locus): Tweak so that expected output is the same as before the diagnostic-show-locus.c changes. * gcc.dg/cpp/pr66415-1.c: Likewise. From-SVN: r279137
2019-12-09 20:03:47 +00:00
};
static const unsigned char wcwidth_widths[] = {
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 2, 0, 1, 0, 1, 0, 1, 0, 1,
Byte vs column awareness for diagnostic-show-locus.c (PR 49973) contrib/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * unicode/from_glibc/unicode_utils.py: Support script from glibc (commit 464cd3) to extract character widths from Unicode data files. * unicode/from_glibc/utf8_gen.py: Likewise. * unicode/UnicodeData.txt: Unicode v. 12.1.0 data file. * unicode/EastAsianWidth.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/gen_wcwidth.py: New utility to generate libcpp/generated_cpp_wcwidth.h with help from the glibc support scripts and the Unicode data files. * unicode/unicode-license.txt: Added. * unicode/README: New explanatory file. libcpp/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * generated_cpp_wcwidth.h: New file generated by ../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function. * charset.c (compute_next_display_width): New function to help implement display columns. (cpp_byte_column_to_display_column): Likewise. (cpp_display_column_to_byte_column): Likewise. (cpp_wcwidth): Likewise. * include/cpplib.h (cpp_byte_column_to_display_column): Declare. (cpp_display_column_to_byte_column): Declare. (cpp_wcwidth): Declare. (cpp_display_width): New function. gcc/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * input.c (location_compute_display_column): New function to help with multibyte awareness in diagnostics. (test_cpp_utf8): New self-test. (input_c_tests): Call the new test. * input.h (location_compute_display_column): Declare. * diagnostic-show-locus.c: Pervasive changes to add multibyte awareness to all classes and functions. (enum column_unit): New enum. (class exploc_with_display_col): New class. (class layout_point): Convert m_column member to array m_columns[2]. (layout_range::contains_point): Add col_unit argument. (test_layout_range_for_single_point): Pass new argument. (test_layout_range_for_single_line): Likewise. (test_layout_range_for_multiple_lines): Likewise. (line_bounds::convert_to_display_cols): New function. (layout::get_state_at_point): Add col_unit argument. (make_range): Use empty filename rather than dummy filename. (get_line_width_without_trailing_whitespace): Rename to... (get_line_bytes_without_trailing_whitespace): ...this. (test_get_line_width_without_trailing_whitespace): Rename to... (test_get_line_bytes_without_trailing_whitespace): ...this. (class layout): m_exploc changed to exploc_with_display_col from plain expanded_location. (layout::get_linenum_width): New accessor member function. (layout::get_x_offset_display): Likewise. (layout::calculate_linenum_width): New subroutine for the constuctor. (layout::calculate_x_offset_display): Likewise. (layout::layout): Use the new subroutines. Add multibyte awareness. (layout::print_source_line): Add multibyte awareness. (layout::print_line): Likewise. (layout::print_annotation_line): Likewise. (line_label::line_label): Likewise. (layout::print_any_labels): Likewise. (layout::annotation_line_showed_range_p): Likewise. (get_printed_columns): Likewise. (class line_label): Rename m_length to m_display_width. (get_affected_columns): Rename to... (get_affected_range): ...this; add col_unit argument and multibyte awareness. (class correction): Add m_affected_bytes and m_display_cols members. Rename m_len to m_byte_length for clarity. Add multibyte awareness throughout. (correction::insertion_p): Add multibyte awareness. (correction::compute_display_cols): New function. (correction::ensure_terminated): Use new member name m_byte_length. (line_corrections::add_hint): Add multibyte awareness. (layout::print_trailing_fixits): Likewise. (layout::get_x_bound_for_row): Likewise. (test_one_liner_simple_caret_utf8): New self-test analogous to the one with _utf8 suffix removed, testing multibyte awareness. (test_one_liner_caret_and_range_utf8): Likewise. (test_one_liner_multiple_carets_and_ranges_utf8): Likewise. (test_one_liner_fixit_insert_before_utf8): Likewise. (test_one_liner_fixit_insert_after_utf8): Likewise. (test_one_liner_fixit_remove_utf8): Likewise. (test_one_liner_fixit_replace_utf8): Likewise. (test_one_liner_fixit_replace_non_equal_range_utf8): Likewise. (test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise. (test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise. (test_one_liner_many_fixits_1_utf8): Likewise. (test_one_liner_many_fixits_2_utf8): Likewise. (test_one_liner_labels_utf8): Likewise. (test_diagnostic_show_locus_one_liner_utf8): Likewise. (test_overlapped_fixit_printing_utf8): Likewise. (test_overlapped_fixit_printing): Adapt for changes to get_affected_columns, get_printed_columns and class corrections. (test_overlapped_fixit_printing_2): Likewise. (test_linenum_sep): New constant. (test_left_margin): Likewise. (test_offset_impl): Helper function for new test. (test_layout_x_offset_display_utf8): New test. (diagnostic_show_locus_c_tests): Call new tests. gcc/testsuite/ChangeLog: 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (test_show_locus): Tweak so that expected output is the same as before the diagnostic-show-locus.c changes. * gcc.dg/cpp/pr66415-1.c: Likewise. From-SVN: r279137
2019-12-09 20:03:47 +00:00
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
Byte vs column awareness for diagnostic-show-locus.c (PR 49973) contrib/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * unicode/from_glibc/unicode_utils.py: Support script from glibc (commit 464cd3) to extract character widths from Unicode data files. * unicode/from_glibc/utf8_gen.py: Likewise. * unicode/UnicodeData.txt: Unicode v. 12.1.0 data file. * unicode/EastAsianWidth.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/gen_wcwidth.py: New utility to generate libcpp/generated_cpp_wcwidth.h with help from the glibc support scripts and the Unicode data files. * unicode/unicode-license.txt: Added. * unicode/README: New explanatory file. libcpp/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * generated_cpp_wcwidth.h: New file generated by ../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function. * charset.c (compute_next_display_width): New function to help implement display columns. (cpp_byte_column_to_display_column): Likewise. (cpp_display_column_to_byte_column): Likewise. (cpp_wcwidth): Likewise. * include/cpplib.h (cpp_byte_column_to_display_column): Declare. (cpp_display_column_to_byte_column): Declare. (cpp_wcwidth): Declare. (cpp_display_width): New function. gcc/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * input.c (location_compute_display_column): New function to help with multibyte awareness in diagnostics. (test_cpp_utf8): New self-test. (input_c_tests): Call the new test. * input.h (location_compute_display_column): Declare. * diagnostic-show-locus.c: Pervasive changes to add multibyte awareness to all classes and functions. (enum column_unit): New enum. (class exploc_with_display_col): New class. (class layout_point): Convert m_column member to array m_columns[2]. (layout_range::contains_point): Add col_unit argument. (test_layout_range_for_single_point): Pass new argument. (test_layout_range_for_single_line): Likewise. (test_layout_range_for_multiple_lines): Likewise. (line_bounds::convert_to_display_cols): New function. (layout::get_state_at_point): Add col_unit argument. (make_range): Use empty filename rather than dummy filename. (get_line_width_without_trailing_whitespace): Rename to... (get_line_bytes_without_trailing_whitespace): ...this. (test_get_line_width_without_trailing_whitespace): Rename to... (test_get_line_bytes_without_trailing_whitespace): ...this. (class layout): m_exploc changed to exploc_with_display_col from plain expanded_location. (layout::get_linenum_width): New accessor member function. (layout::get_x_offset_display): Likewise. (layout::calculate_linenum_width): New subroutine for the constuctor. (layout::calculate_x_offset_display): Likewise. (layout::layout): Use the new subroutines. Add multibyte awareness. (layout::print_source_line): Add multibyte awareness. (layout::print_line): Likewise. (layout::print_annotation_line): Likewise. (line_label::line_label): Likewise. (layout::print_any_labels): Likewise. (layout::annotation_line_showed_range_p): Likewise. (get_printed_columns): Likewise. (class line_label): Rename m_length to m_display_width. (get_affected_columns): Rename to... (get_affected_range): ...this; add col_unit argument and multibyte awareness. (class correction): Add m_affected_bytes and m_display_cols members. Rename m_len to m_byte_length for clarity. Add multibyte awareness throughout. (correction::insertion_p): Add multibyte awareness. (correction::compute_display_cols): New function. (correction::ensure_terminated): Use new member name m_byte_length. (line_corrections::add_hint): Add multibyte awareness. (layout::print_trailing_fixits): Likewise. (layout::get_x_bound_for_row): Likewise. (test_one_liner_simple_caret_utf8): New self-test analogous to the one with _utf8 suffix removed, testing multibyte awareness. (test_one_liner_caret_and_range_utf8): Likewise. (test_one_liner_multiple_carets_and_ranges_utf8): Likewise. (test_one_liner_fixit_insert_before_utf8): Likewise. (test_one_liner_fixit_insert_after_utf8): Likewise. (test_one_liner_fixit_remove_utf8): Likewise. (test_one_liner_fixit_replace_utf8): Likewise. (test_one_liner_fixit_replace_non_equal_range_utf8): Likewise. (test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise. (test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise. (test_one_liner_many_fixits_1_utf8): Likewise. (test_one_liner_many_fixits_2_utf8): Likewise. (test_one_liner_labels_utf8): Likewise. (test_diagnostic_show_locus_one_liner_utf8): Likewise. (test_overlapped_fixit_printing_utf8): Likewise. (test_overlapped_fixit_printing): Adapt for changes to get_affected_columns, get_printed_columns and class corrections. (test_overlapped_fixit_printing_2): Likewise. (test_linenum_sep): New constant. (test_left_margin): Likewise. (test_offset_impl): Helper function for new test. (test_layout_x_offset_display_utf8): New test. (diagnostic_show_locus_c_tests): Call new tests. gcc/testsuite/ChangeLog: 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (test_show_locus): Tweak so that expected output is the same as before the diagnostic-show-locus.c changes. * gcc.dg/cpp/pr66415-1.c: Likewise. From-SVN: r279137
2019-12-09 20:03:47 +00:00
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 0, 1, 0, 1, 0, 1, 2, 1, 2, 1, 2, 1, 2, 0,
2, 1, 2, 1, 0, 2, 1, 2, 1, 2, 0, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 0, 1, 0,
contrib, libcpp, libstdc++: Update to Unicode 16.0 It is autumn again and there is a new Unicode version 16.0. The following patch updates our Unicode stuff in contrib, libcpp and libstdc++ from that Unicode version. 2024-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Update glibc git commit hash, replace Unicode 15 or 15.1 versions with 16. * unicode/gen_libstdcxx_unicode_data.py: Use 160000 instead of 150100 in _GLIBCXX_GET_UNICODE_DATA test. * unicode/from_glibc/utf8_gen.py: Updated from glibc 064c708c78cc2a6b5802dce73108fc0c1c6bfc80 commit. * unicode/DerivedCoreProperties.txt: Updated from Unicode 16.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add tests for some Unicode 16.0 characters, both normal and generated. libcpp/ * makeucnid.cc (write_copyright): Update Unicode Copyright years. * makeuname2c.cc (generated_ranges): Adjust Unicode version from 15.1 to 16.0. Add EGYPTIAN HIEROGLYPH- generated range, adjust indexes in following entries. (write_copyright): Update Unicode Copyright years. * generated_cpp_wcwidth.h: Regenerated. * ucnid.h: Regenerated. * uname2c.h: Regenerated. libstdc++-v3/ * include/bits/unicode.h (std::__unicode::__v15_1_0): Rename inline namespace to ... (std::__unicode::__v16_0_0): ... this. (_GLIBCXX_GET_UNICODE_DATA): Change from 150100 to 160000. * include/bits/unicode-data.h: Regenerated. * testsuite/ext/unicode/properties.cc: Check for _Gcb_SpacingMark on U+11F03 rather than U+1D16D as the latter lost SpacingMark property in Unicode 16.0.
2024-10-08 10:01:47 +02:00
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
1, 2, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 2, 1, 0,
1, 0, 1, 2, 1, 2, 1, 0, 1, 0, 2, 1, 0, 2, 1, 2, 1, 2, 1, 0, 1, 2, 1, 0,
1, 2, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
Byte vs column awareness for diagnostic-show-locus.c (PR 49973) contrib/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * unicode/from_glibc/unicode_utils.py: Support script from glibc (commit 464cd3) to extract character widths from Unicode data files. * unicode/from_glibc/utf8_gen.py: Likewise. * unicode/UnicodeData.txt: Unicode v. 12.1.0 data file. * unicode/EastAsianWidth.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/gen_wcwidth.py: New utility to generate libcpp/generated_cpp_wcwidth.h with help from the glibc support scripts and the Unicode data files. * unicode/unicode-license.txt: Added. * unicode/README: New explanatory file. libcpp/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * generated_cpp_wcwidth.h: New file generated by ../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function. * charset.c (compute_next_display_width): New function to help implement display columns. (cpp_byte_column_to_display_column): Likewise. (cpp_display_column_to_byte_column): Likewise. (cpp_wcwidth): Likewise. * include/cpplib.h (cpp_byte_column_to_display_column): Declare. (cpp_display_column_to_byte_column): Declare. (cpp_wcwidth): Declare. (cpp_display_width): New function. gcc/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * input.c (location_compute_display_column): New function to help with multibyte awareness in diagnostics. (test_cpp_utf8): New self-test. (input_c_tests): Call the new test. * input.h (location_compute_display_column): Declare. * diagnostic-show-locus.c: Pervasive changes to add multibyte awareness to all classes and functions. (enum column_unit): New enum. (class exploc_with_display_col): New class. (class layout_point): Convert m_column member to array m_columns[2]. (layout_range::contains_point): Add col_unit argument. (test_layout_range_for_single_point): Pass new argument. (test_layout_range_for_single_line): Likewise. (test_layout_range_for_multiple_lines): Likewise. (line_bounds::convert_to_display_cols): New function. (layout::get_state_at_point): Add col_unit argument. (make_range): Use empty filename rather than dummy filename. (get_line_width_without_trailing_whitespace): Rename to... (get_line_bytes_without_trailing_whitespace): ...this. (test_get_line_width_without_trailing_whitespace): Rename to... (test_get_line_bytes_without_trailing_whitespace): ...this. (class layout): m_exploc changed to exploc_with_display_col from plain expanded_location. (layout::get_linenum_width): New accessor member function. (layout::get_x_offset_display): Likewise. (layout::calculate_linenum_width): New subroutine for the constuctor. (layout::calculate_x_offset_display): Likewise. (layout::layout): Use the new subroutines. Add multibyte awareness. (layout::print_source_line): Add multibyte awareness. (layout::print_line): Likewise. (layout::print_annotation_line): Likewise. (line_label::line_label): Likewise. (layout::print_any_labels): Likewise. (layout::annotation_line_showed_range_p): Likewise. (get_printed_columns): Likewise. (class line_label): Rename m_length to m_display_width. (get_affected_columns): Rename to... (get_affected_range): ...this; add col_unit argument and multibyte awareness. (class correction): Add m_affected_bytes and m_display_cols members. Rename m_len to m_byte_length for clarity. Add multibyte awareness throughout. (correction::insertion_p): Add multibyte awareness. (correction::compute_display_cols): New function. (correction::ensure_terminated): Use new member name m_byte_length. (line_corrections::add_hint): Add multibyte awareness. (layout::print_trailing_fixits): Likewise. (layout::get_x_bound_for_row): Likewise. (test_one_liner_simple_caret_utf8): New self-test analogous to the one with _utf8 suffix removed, testing multibyte awareness. (test_one_liner_caret_and_range_utf8): Likewise. (test_one_liner_multiple_carets_and_ranges_utf8): Likewise. (test_one_liner_fixit_insert_before_utf8): Likewise. (test_one_liner_fixit_insert_after_utf8): Likewise. (test_one_liner_fixit_remove_utf8): Likewise. (test_one_liner_fixit_replace_utf8): Likewise. (test_one_liner_fixit_replace_non_equal_range_utf8): Likewise. (test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise. (test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise. (test_one_liner_many_fixits_1_utf8): Likewise. (test_one_liner_many_fixits_2_utf8): Likewise. (test_one_liner_labels_utf8): Likewise. (test_diagnostic_show_locus_one_liner_utf8): Likewise. (test_overlapped_fixit_printing_utf8): Likewise. (test_overlapped_fixit_printing): Adapt for changes to get_affected_columns, get_printed_columns and class corrections. (test_overlapped_fixit_printing_2): Likewise. (test_linenum_sep): New constant. (test_left_margin): Likewise. (test_offset_impl): Helper function for new test. (test_layout_x_offset_display_utf8): New test. (diagnostic_show_locus_c_tests): Call new tests. gcc/testsuite/ChangeLog: 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (test_show_locus): Tweak so that expected output is the same as before the diagnostic-show-locus.c changes. * gcc.dg/cpp/pr66415-1.c: Likewise. From-SVN: r279137
2019-12-09 20:03:47 +00:00
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0,
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
1, 2, 0, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
2, 1, 2, 1, 2, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
2, 1, 2, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
0, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1,
Update to Unicode 17.0.0 The following patch updates GCC from Unicode 16.0.0 to 17.0.0. I've followed what the README says and updated also one script from glibc, but that needed another Unicode file - HangulSyllableType.txt - around as well, so I'm adding it. I've added one new test to named-universal-char-escape-1.c for randomly chosen character from new CJK block. Note, Unicode 17.0.0 authors forgot to adjust the 4-8 table, I've filed bugreports about that but the UnicodeData.txt changes for the range ends and the new range seems to match e.g. what is in the glyph tables, so the patch follows UnicodeData.txt and not 4-8 table here. Another thing was that makeuname2c.cc didn't handle correctly when the size of the generated string table modulo 77 was 76 or 77, in which case it forgot to emit a semicolon after the string literal and so failed to compile. And as can be seen in the emoji-data.txt diff, some properties like Extended_Pictographic have been removed from certain characters, e.g. from the Mahjong cards characters except U+1F004, and one libstdc++ test was testing that property exactly on U+1F000. Dunno why that was changed, but U+1F004 is the only colored one among tons of black and white ones. 2025-10-08 Jakub Jelinek <jakub@redhat.com> contrib/ * unicode/README: Add HangulSyllableType.txt file to the list as newest utf8_gen.py from glibc now needs it. Adjust git commit hash and change unicode 16 version to 17. * unicode/from_glibc/utf8_gen.py: Updated from glibc. * unicode/DerivedCoreProperties.txt: Updated from Unicode 17.0.0. * unicode/emoji-data.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/GraphemeBreakProperty.txt: Likewise. * unicode/DerivedNormalizationProps.txt: Likewise. * unicode/NameAliases.txt: Likewise. * unicode/UnicodeData.txt: Likewise. * unicode/EastAsianWidth.txt: Likewise. * unicode/DerivedGeneralCategory.txt: Likewise. * unicode/HangulSyllableType.txt: New file. gcc/testsuite/ * c-c++-common/cpp/named-universal-char-escape-1.c: Add test for \N{CJK UNIFIED IDEOGRAPH-3340E}. libcpp/ * makeucnid.cc (write_copyright): Adjust copyright year. * makeuname2c.cc (generated_ranges): Adjust end points for a couple of ranges based on UnicodeData.txt Last changes and add a whole new CJK UNIFIED IDEOGRAPH- entry. None of these changes are in the 4-8 table, but clearly it has just been forgotten. (write_copyright): Adjust copyright year. (write_dict): Fix up condition when to print semicolon. * generated_cpp_wcwidth.h: Regenerate. * ucnid.h: Regenerate. * uname2c.h: Regenerate. libstdc++-v3/ * include/bits/unicode-data.h: Regenerate. * testsuite/ext/unicode/properties.cc: Test __is_extended_pictographic on U+1F004 rather than U+1F000.
2025-10-08 17:54:11 +02:00
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 0, 1, 0, 1, 0,
Byte vs column awareness for diagnostic-show-locus.c (PR 49973) contrib/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * unicode/from_glibc/unicode_utils.py: Support script from glibc (commit 464cd3) to extract character widths from Unicode data files. * unicode/from_glibc/utf8_gen.py: Likewise. * unicode/UnicodeData.txt: Unicode v. 12.1.0 data file. * unicode/EastAsianWidth.txt: Likewise. * unicode/PropList.txt: Likewise. * unicode/gen_wcwidth.py: New utility to generate libcpp/generated_cpp_wcwidth.h with help from the glibc support scripts and the Unicode data files. * unicode/unicode-license.txt: Added. * unicode/README: New explanatory file. libcpp/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * generated_cpp_wcwidth.h: New file generated by ../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function. * charset.c (compute_next_display_width): New function to help implement display columns. (cpp_byte_column_to_display_column): Likewise. (cpp_display_column_to_byte_column): Likewise. (cpp_wcwidth): Likewise. * include/cpplib.h (cpp_byte_column_to_display_column): Declare. (cpp_display_column_to_byte_column): Declare. (cpp_wcwidth): Declare. (cpp_display_width): New function. gcc/ChangeLog 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * input.c (location_compute_display_column): New function to help with multibyte awareness in diagnostics. (test_cpp_utf8): New self-test. (input_c_tests): Call the new test. * input.h (location_compute_display_column): Declare. * diagnostic-show-locus.c: Pervasive changes to add multibyte awareness to all classes and functions. (enum column_unit): New enum. (class exploc_with_display_col): New class. (class layout_point): Convert m_column member to array m_columns[2]. (layout_range::contains_point): Add col_unit argument. (test_layout_range_for_single_point): Pass new argument. (test_layout_range_for_single_line): Likewise. (test_layout_range_for_multiple_lines): Likewise. (line_bounds::convert_to_display_cols): New function. (layout::get_state_at_point): Add col_unit argument. (make_range): Use empty filename rather than dummy filename. (get_line_width_without_trailing_whitespace): Rename to... (get_line_bytes_without_trailing_whitespace): ...this. (test_get_line_width_without_trailing_whitespace): Rename to... (test_get_line_bytes_without_trailing_whitespace): ...this. (class layout): m_exploc changed to exploc_with_display_col from plain expanded_location. (layout::get_linenum_width): New accessor member function. (layout::get_x_offset_display): Likewise. (layout::calculate_linenum_width): New subroutine for the constuctor. (layout::calculate_x_offset_display): Likewise. (layout::layout): Use the new subroutines. Add multibyte awareness. (layout::print_source_line): Add multibyte awareness. (layout::print_line): Likewise. (layout::print_annotation_line): Likewise. (line_label::line_label): Likewise. (layout::print_any_labels): Likewise. (layout::annotation_line_showed_range_p): Likewise. (get_printed_columns): Likewise. (class line_label): Rename m_length to m_display_width. (get_affected_columns): Rename to... (get_affected_range): ...this; add col_unit argument and multibyte awareness. (class correction): Add m_affected_bytes and m_display_cols members. Rename m_len to m_byte_length for clarity. Add multibyte awareness throughout. (correction::insertion_p): Add multibyte awareness. (correction::compute_display_cols): New function. (correction::ensure_terminated): Use new member name m_byte_length. (line_corrections::add_hint): Add multibyte awareness. (layout::print_trailing_fixits): Likewise. (layout::get_x_bound_for_row): Likewise. (test_one_liner_simple_caret_utf8): New self-test analogous to the one with _utf8 suffix removed, testing multibyte awareness. (test_one_liner_caret_and_range_utf8): Likewise. (test_one_liner_multiple_carets_and_ranges_utf8): Likewise. (test_one_liner_fixit_insert_before_utf8): Likewise. (test_one_liner_fixit_insert_after_utf8): Likewise. (test_one_liner_fixit_remove_utf8): Likewise. (test_one_liner_fixit_replace_utf8): Likewise. (test_one_liner_fixit_replace_non_equal_range_utf8): Likewise. (test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise. (test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise. (test_one_liner_many_fixits_1_utf8): Likewise. (test_one_liner_many_fixits_2_utf8): Likewise. (test_one_liner_labels_utf8): Likewise. (test_diagnostic_show_locus_one_liner_utf8): Likewise. (test_overlapped_fixit_printing_utf8): Likewise. (test_overlapped_fixit_printing): Adapt for changes to get_affected_columns, get_printed_columns and class corrections. (test_overlapped_fixit_printing_2): Likewise. (test_linenum_sep): New constant. (test_left_margin): Likewise. (test_offset_impl): Helper function for new test. (test_layout_x_offset_display_utf8): New test. (diagnostic_show_locus_c_tests): Call new tests. gcc/testsuite/ChangeLog: 2019-12-09 Lewis Hyatt <lhyatt@gmail.com> PR preprocessor/49973 * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (test_show_locus): Tweak so that expected output is the same as before the diagnostic-show-locus.c changes. * gcc.dg/cpp/pr66415-1.c: Likewise. From-SVN: r279137
2019-12-09 20:03:47 +00:00
};