[pkgsrc/trunk]: pkgsrc/textproc/icu Update to 64.1

[ryoon <ryoon%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/613978086cd5
branches:  trunk
changeset: 394071:613978086cd5
user:      ryoon <ryoon%pkgsrc.org@localhost>
date:      Wed Apr 03 00:12:59 2019 +0000

description:
Update to 64.1

Changelog:
Common Changes

    Unicode 12: 554 new characters, including 4 new scripts and 61 new emoji characters.
    CLDR 35
        Somali and Javanese data now up to modern level
        Cebuano, Hausa, Igbo, and Yoruba data now up to basic level
        23 additional measurement units
        Many data additions and corrections in many other languages
        The following language has been added to ICU: Cebuano
        This version of ICU does not yet implement the Indic Grapheme Cluster improvements from CLDR 35.
    New Japanese calendar era from 2019: CLDR and ICU include data for testing that can be enabled. (ICU #12973, CLDR #10750)
        To enable CLDR new Japanese era placeholder name, set environment variable (and Java system property for ICU4J) ICU_ENABLE_TENTATIVE_ERA=true (This was added in ICU 63).
        Support added for Gannen year numbering (using ? for the first year of an era) in the Japanese locale Japanese-calendar full, long, and medium formats. Gannen year support is also 
automatically added for other non-numeric formats (those containing other kanji characters such as ?) derived from pattern skeletons unless specifically overridden. (ICU #20441, CLDR #11843, CLDR 
#11819)
        We are planning for an ICU 64.2 update in 2019-April which will add the new Japanese era with its real name.
    ICU 64 now uses "rearguard" TZ data. (Recent versions have used "vanguard" data with certain overrides.) (ICU-20398)
    ICU data filtering: The ICU4C build accepts an optional filter script that specifies a subset of the data to be built, with whitelists and blacklists for locales and for resource bundle paths. 
(ICU-10923, design doc)
        See this new documentation page: userguide/icu_data/buildtool.md
    MessageFormat has new pattern syntax for specifying the style of a date/time argument via a locale-independent skeleton rather than a locale-specific pattern. (ICU-9622)
        Date/time skeletons use the same "::" prefix as number skeletons.
        Example MessageFormat pattern string: "We close on {closing,date,::MMMMd} at {closing,time,::jm}."
    Many formatting APIs can now output a new type of result object which is-a FormattedValue (Java & C++), or convertible to a UFormattedValue (C).
        These combine the result strings with easy iteration over FieldPosition metadata.

ICU4C Specific Changes

    New C++ class LocaleBuilder for building a Locale from subtags, keywords, and extensions. (ICU-20328)
        Parallel to the existing ICU4J ULocale.Builder class.
    For C++ MeasureUnit instances, there are now additional factory methods that return units by value, not by pointer-with-ownership. (ICU-20337)
    Various Out-Of-Memory (OOM) issues have been fixed. (ticket query)

diffstat:

 textproc/icu/Makefile                                     |   6 +-
 textproc/icu/PLIST                                        |  17 +++--
 textproc/icu/buildlink3.mk                                |   4 +-
 textproc/icu/distinfo                                     |  20 ++---
 textproc/icu/patches/patch-CVE-2018-18928                 |  49 ---------------
 textproc/icu/patches/patch-common_umutex.h                |  30 +++++---
 textproc/icu/patches/patch-common_unicode_urename.h       |  15 ----
 textproc/icu/patches/patch-configure                      |  30 ++-------
 textproc/icu/patches/patch-configure.ac                   |  26 +------
 textproc/icu/patches/patch-i18n_double-conversion-utils.h |  16 ----
 textproc/icu/patches/patch-i18n_uspoof.cpp                |  16 ----
 11 files changed, 53 insertions(+), 176 deletions(-)

diffs (truncated from 441 to 300 lines):

diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/Makefile
--- a/textproc/icu/Makefile     Tue Apr 02 14:40:44 2019 +0000
+++ b/textproc/icu/Makefile     Wed Apr 03 00:12:59 2019 +0000
@@ -1,8 +1,7 @@
-# $NetBSD: Makefile,v 1.121 2019/02/13 20:51:57 spz Exp $
+# $NetBSD: Makefile,v 1.122 2019/04/03 00:12:59 ryoon Exp $
 
-DISTNAME=      icu4c-63_1-src
+DISTNAME=      icu4c-64_1-src
 PKGNAME=       ${DISTNAME:S/4c//:S/-src//:S/_/./g}
-PKGREVISION=   2
 CATEGORIES=    textproc
 MASTER_SITES=  http://download.icu-project.org/files/icu4c/${PKGVERSION_NOREV}/
 EXTRACT_SUFX=  .tgz
@@ -130,4 +129,5 @@
 SUBST_SED.icu-config+= -e '/l$${ICUPREFIX}/ s/$${ICULIBSUFFIX_VERSION}//g'
 .endif
 
+.include "../../lang/python/tool.mk"
 .include "../../mk/bsd.pkg.mk"
diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/PLIST
--- a/textproc/icu/PLIST        Tue Apr 02 14:40:44 2019 +0000
+++ b/textproc/icu/PLIST        Wed Apr 03 00:12:59 2019 +0000
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.44 2018/12/09 18:28:16 adam Exp $
+@comment $NetBSD: PLIST,v 1.45 2019/04/03 00:12:59 ryoon Exp $
 bin/derb
 bin/genbrk
 bin/gencfu
@@ -47,6 +47,7 @@
 include/unicode/filteredbrk.h
 include/unicode/fmtable.h
 include/unicode/format.h
+include/unicode/formattedvalue.h
 include/unicode/fpositer.h
 include/unicode/gender.h
 include/unicode/gregocal.h
@@ -54,6 +55,7 @@
 include/unicode/icuplug.h
 include/unicode/idna.h
 include/unicode/listformatter.h
+include/unicode/localebuilder.h
 include/unicode/localpointer.h
 include/unicode/locdspnm.h
 include/unicode/locid.h
@@ -139,6 +141,7 @@
 include/unicode/uenum.h
 include/unicode/ufieldpositer.h
 include/unicode/uformattable.h
+include/unicode/uformattedvalue.h
 include/unicode/ugender.h
 include/unicode/uidna.h
 include/unicode/uiter.h
@@ -201,27 +204,27 @@
 lib/libicudata.a
 lib/libicudata.so
 lib/libicudata.so.${PKGVERSION}
-lib/libicudata.so.63
+lib/libicudata.so.64
 lib/libicui18n.a
 lib/libicui18n.so
 lib/libicui18n.so.${PKGVERSION}
-lib/libicui18n.so.63
+lib/libicui18n.so.64
 lib/libicuio.a
 lib/libicuio.so
 lib/libicuio.so.${PKGVERSION}
-lib/libicuio.so.63
+lib/libicuio.so.64
 lib/libicutest.a
 lib/libicutest.so
 lib/libicutest.so.${PKGVERSION}
-lib/libicutest.so.63
+lib/libicutest.so.64
 lib/libicutu.a
 lib/libicutu.so
 lib/libicutu.so.${PKGVERSION}
-lib/libicutu.so.63
+lib/libicutu.so.64
 lib/libicuuc.a
 lib/libicuuc.so
 lib/libicuuc.so.${PKGVERSION}
-lib/libicuuc.so.63
+lib/libicuuc.so.64
 lib/pkgconfig/icu-i18n.pc
 lib/pkgconfig/icu-io.pc
 lib/pkgconfig/icu-uc.pc
diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/buildlink3.mk
--- a/textproc/icu/buildlink3.mk        Tue Apr 02 14:40:44 2019 +0000
+++ b/textproc/icu/buildlink3.mk        Wed Apr 03 00:12:59 2019 +0000
@@ -1,4 +1,4 @@
-# $NetBSD: buildlink3.mk,v 1.37 2018/07/20 03:32:09 ryoon Exp $
+# $NetBSD: buildlink3.mk,v 1.38 2019/04/03 00:12:59 ryoon Exp $
 
 BUILDLINK_TREE+=       icu
 
@@ -6,7 +6,7 @@
 ICU_BUILDLINK3_MK:=
 
 BUILDLINK_API_DEPENDS.icu+=    icu>=3.4
-BUILDLINK_ABI_DEPENDS.icu+=    icu>=62.1
+BUILDLINK_ABI_DEPENDS.icu+=    icu>=64.1
 BUILDLINK_PKGSRCDIR.icu?=      ../../textproc/icu
 .endif # ICU_BUILDLINK3_MK
 
diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/distinfo
--- a/textproc/icu/distinfo     Tue Apr 02 14:40:44 2019 +0000
+++ b/textproc/icu/distinfo     Wed Apr 03 00:12:59 2019 +0000
@@ -1,27 +1,23 @@
-$NetBSD: distinfo,v 1.81 2019/02/13 20:51:57 spz Exp $
+$NetBSD: distinfo,v 1.82 2019/04/03 00:12:59 ryoon Exp $
 
-SHA1 (icu4c-63_1-src.tgz) = ad523232f19af1c698c6489f8e15f7e9824f1662
-RMD160 (icu4c-63_1-src.tgz) = 5c895a6e2b135978df59e135ed772747aec0065f
-SHA512 (icu4c-63_1-src.tgz) = 9ab407ed840a00cdda7470dcc4c40299a125ad246ae4d019c4b1ede54781157fd63af015a8228cd95dbc47e4d15a0932b2c657489046a19788e5e8266eac079c
-Size (icu4c-63_1-src.tgz) = 23746939 bytes
-SHA1 (patch-CVE-2018-18928) = 74e8248c215bcb5ca98a63d161dc5516531a83b3
+SHA1 (icu4c-64_1-src.tgz) = 50f900fe983dcdb46c68a078b8b09afde1339ad7
+RMD160 (icu4c-64_1-src.tgz) = 9c7c07493b2700397075c50112642f849938f05c
+SHA512 (icu4c-64_1-src.tgz) = 5eca8342d8bdf902689243506643e04512744b33962687e118f6810af6f7fd073678f67b991d2ae9139d257713b63abb4222b96687234df01ad5ff62df16ede0
+Size (icu4c-64_1-src.tgz) = 24013136 bytes
 SHA1 (patch-Makefile.in) = 67440d3af9b62b8c0be258c490255ba17f778ab4
 SHA1 (patch-acinclude.m4) = f7de1a16aad0ca77c4bbc457ba76b6171199ce09
 SHA1 (patch-common_putil.cpp) = 6aa70b8698d663d3c798bafd9010a824c9609c20
 SHA1 (patch-common_putilimp.h) = a68faa97c2bffeecaca1586e26f5bbe48e71b262
-SHA1 (patch-common_umutex.h) = 096d3e15ef7b84533456af4570ed70747a4ef70c
+SHA1 (patch-common_umutex.h) = b22b7c02b4645a87f71d54885d858d0cf9005988
 SHA1 (patch-common_unicode_platform.h) = 8b7b8bcf6f5185225a1ca516ac212a495f7b47e8
-SHA1 (patch-common_unicode_urename.h) = bc27b21c9a041a16ed1dd077e5c5af7e4cee4cf3
 SHA1 (patch-common_uposixdefs.h) = 4739ca7d4662335adcb322cd0943770651d548ce
 SHA1 (patch-config_Makefile.inc.in) = e7cee161315321d2580074054d87714b55319886
 SHA1 (patch-config_icu-config-bottom) = 168b89ee9180d4ae545125866ee91eb004010501
 SHA1 (patch-config_mh-darwin) = 32f0e4c241535e37e4cad9b871ed3d36b4184199
 SHA1 (patch-config_mh-scoosr5) = 47703dcc184f58c0382da3225f849424ab74d472
 SHA1 (patch-config_mh-solaris-gcc) = e1097930f577917d3583e425d9d2affa1c3dcaf2
-SHA1 (patch-configure) = fdeae51f4390779dfd8df8471196e322cbb9ea3d
-SHA1 (patch-configure.ac) = b0291cf02351cbad9b0c7340baea9eb81cabb158
-SHA1 (patch-i18n_double-conversion-utils.h) = 773e93ed9baffa241d5af61bd631d2d7c4a867f8
-SHA1 (patch-i18n_uspoof.cpp) = b7d1757912a6da39c080b1453f4730cbb9b5fd65
+SHA1 (patch-configure) = 1f96ffb2c2c023b38a45326c0273be0ae9cc02ff
+SHA1 (patch-configure.ac) = 2cb4601e20ee6b93e08e8790df3f8212510aeb1b
 SHA1 (patch-icudefs.mk.in) = c2a9469bf896b5f0702d5795c3b1c2b394893663
 SHA1 (patch-runConfigureICU) = 3cd60613104fa4f1b81529795e75d5cf7f477b64
 SHA1 (patch-tools-toolutil-pkg_genc.cpp) = 3cbcf3387d6a39315b26a3e2a2ffb715507e9110
diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/patches/patch-CVE-2018-18928
--- a/textproc/icu/patches/patch-CVE-2018-18928 Tue Apr 02 14:40:44 2019 +0000
+++ /dev/null   Thu Jan 01 00:00:00 1970 +0000
@@ -1,49 +0,0 @@
-$NetBSD: patch-CVE-2018-18928,v 1.1 2019/02/13 20:51:57 spz Exp $
-
-fix for CVE-2018-18928 from
-https://github.com/unicode-org/icu/commit/53d8c8f3d181d87a6aa925b449b51c4a2c922a51
-
---- i18n/fmtable.cpp.orig      2018-09-29 00:34:42.000000000 +0000
-+++ i18n/fmtable.cpp
-@@ -734,7 +734,7 @@ CharString *Formattable::internalGetChar
-       // not print scientific notation for magnitudes greater than -5 and smaller than some amount (+5?).
-       if (fDecimalQuantity->isZero()) {
-         fDecimalStr->append("0", -1, status);
--      } else if (std::abs(fDecimalQuantity->getMagnitude()) < 5) {
-+      } else if (fDecimalQuantity->getMagnitude() != INT32_MIN && std::abs(fDecimalQuantity->getMagnitude()) < 5) {
-         fDecimalStr->appendInvariantChars(fDecimalQuantity->toPlainString(), status);
-       } else {
-         fDecimalStr->appendInvariantChars(fDecimalQuantity->toScientificString(), status);
-
---- i18n/number_decimalquantity.cpp.orig       2018-10-01 22:39:56.000000000 +0000
-+++ i18n/number_decimalquantity.cpp
-@@ -820,7 +820,10 @@ UnicodeString DecimalQuantity::toScienti
-     }
-     result.append(u'E');
-     int32_t _scale = upperPos + scale;
--    if (_scale < 0) {
-+    if (_scale == INT32_MIN) {
-+        result.append({u"-2147483648", -1});
-+        return result;
-+    } else if (_scale < 0) {
-         _scale *= -1;
-         result.append(u'-');
-     } else {
-
---- test/intltest/numfmtst.cpp.orig    2018-10-01 22:39:56.000000000 +0000
-+++ test/intltest/numfmtst.cpp
-@@ -9226,6 +9226,14 @@ void NumberFormatTest::Test20037_Scienti
-     assertEquals(u"Should not overflow and should parse only the first exponent",
-                  u"1E-2147483647",
-                  {sp.data(), sp.length(), US_INV});
-+
-+    // Test edge case overflow of exponent
-+    result = Formattable();
-+    nf->parse(u".0003e-2147483644", result, status);
-+    sp = result.getDecimalNumber(status);
-+    assertEquals(u"Should not overflow",
-+                 u"3E-2147483648",
-+                 {sp.data(), sp.length(), US_INV});
- }
- 
- void NumberFormatTest::Test13840_ParseLongStringCrash() {
diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/patches/patch-common_umutex.h
--- a/textproc/icu/patches/patch-common_umutex.h        Tue Apr 02 14:40:44 2019 +0000
+++ b/textproc/icu/patches/patch-common_umutex.h        Wed Apr 03 00:12:59 2019 +0000
@@ -1,19 +1,17 @@
-$NetBSD: patch-common_umutex.h,v 1.1 2014/03/04 12:07:16 obache Exp $
+$NetBSD: patch-common_umutex.h,v 1.2 2019/04/03 00:13:00 ryoon Exp $
 
 * Add NetBSD MI atomic_ops(3) support
 
---- common/umutex.h.orig       2013-10-04 20:49:16.000000000 +0000
+--- common/umutex.h.orig       2019-03-27 18:47:14.000000000 +0000
 +++ common/umutex.h
-@@ -117,6 +117,34 @@ inline int32_t umtx_atomic_dec(u_atomic_
- U_NAMESPACE_END
+@@ -60,6 +60,31 @@ template struct std::atomic<int32_t>;
  
+ U_NAMESPACE_BEGIN
  
-+#elif U_HAVE_NETBSD_ATOMIC_OPS
++#if U_HAVE_NETBSD_ATOMIC_OPS
 +/*
 + * NetBSD MI atomic_ops(3)
 + */
-+
-+U_NAMESPACE_BEGIN
 +#include <sys/atomic.h>
 +typedef volatile uint32_t u_atomic_int32_t;
 +#define ATOMIC_INT32_T_INITIALIZER(val) val
@@ -33,9 +31,17 @@
 +inline int32_t umtx_atomic_dec(u_atomic_int32_t *p) {
 +    return atomic_dec_32_nv(p);
 +}
-+U_NAMESPACE_END
-+
++#else
 +
- #elif U_HAVE_GCC_ATOMICS
- /*
-  * gcc atomic ops. These are available on several other compilers as well.
+ /****************************************************************************
+  *
+  *   Low Level Atomic Operations, ICU wrappers for.
+@@ -84,7 +109,7 @@ inline int32_t umtx_atomic_inc(u_atomic_
+ inline int32_t umtx_atomic_dec(u_atomic_int32_t *var) {
+     return var->fetch_sub(1) - 1;
+ }
+-
++#endif
+ 
+ /*************************************************************************************************
+  *
diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/patches/patch-common_unicode_urename.h
--- a/textproc/icu/patches/patch-common_unicode_urename.h       Tue Apr 02 14:40:44 2019 +0000
+++ /dev/null   Thu Jan 01 00:00:00 1970 +0000
@@ -1,15 +0,0 @@
-$NetBSD: patch-common_unicode_urename.h,v 1.1 2018/12/11 10:15:56 abs Exp $
-
-ICU-20208 uspoof.cpp function checkImpl should be static
-https://github.com/unicode-org/icu/commit/8baff8f03e07d8e02304d0c888d0bb21ad2eeb01
-
---- common/unicode/urename.h.orig      2018-10-15 18:02:37.000000000 +0000
-+++ common/unicode/urename.h
-@@ -110,7 +110,6 @@
- #define _UTF7Data U_ICU_ENTRY_POINT_RENAME(_UTF7Data)
- #define _UTF8Data U_ICU_ENTRY_POINT_RENAME(_UTF8Data)
- #define allowedHourFormatsCleanup U_ICU_ENTRY_POINT_RENAME(allowedHourFormatsCleanup)
--#define checkImpl U_ICU_ENTRY_POINT_RENAME(checkImpl)
- #define cmemory_cleanup U_ICU_ENTRY_POINT_RENAME(cmemory_cleanup)
- #define dayPeriodRulesCleanup U_ICU_ENTRY_POINT_RENAME(dayPeriodRulesCleanup)
- #define deleteAllowedHourFormats U_ICU_ENTRY_POINT_RENAME(deleteAllowedHourFormats)
diff -r 984c5c70a00c -r 613978086cd5 textproc/icu/patches/patch-configure
--- a/textproc/icu/patches/patch-configure      Tue Apr 02 14:40:44 2019 +0000
+++ b/textproc/icu/patches/patch-configure      Wed Apr 03 00:12:59 2019 +0000
@@ -1,4 +1,4 @@
-$NetBSD: patch-configure,v 1.9 2017/04/25 08:58:44 maya Exp $
+$NetBSD: patch-configure,v 1.10 2019/04/03 00:13:00 ryoon Exp $
 
 Portability fix.
 Solaris seems to need -std=c99 with recent pkgsrc so keep it.
@@ -6,9 +6,9 @@
 required.
 Add support for Bitrig.
 
---- configure.orig     2017-04-07 07:40:30.000000000 +0000
+--- configure.orig     2019-03-27 18:47:14.000000000 +0000
 +++ configure
-@@ -5175,7 +5175,7 @@ rm -f core conftest.err conftest.$ac_obj
+@@ -5242,7 +5242,7 @@ rm -f core conftest.err conftest.$ac_obj
                *-*-cygwin) icu_cv_host_frag=mh-cygwin-msvc ;;
                esac
        fi ;;
@@ -17,7 +17,7 @@
  *-*-aix*)
        if test "$GCC" = yes; then
                icu_cv_host_frag=mh-aix-gcc
-@@ -5200,6 +5200,7 @@ rm -f core conftest.err conftest.$ac_obj
+@@ -5267,6 +5267,7 @@ rm -f core conftest.err conftest.$ac_obj
  *-dec-osf*) icu_cv_host_frag=mh-alpha-osf ;;
  *-*-nto*)     icu_cv_host_frag=mh-qnx ;;
  *-ncr-*)      icu_cv_host_frag=mh-mpras ;;
@@ -25,7 +25,7 @@
  *)            icu_cv_host_frag=mh-unknown ;;
  esac