CVS commit: pkgsrc/textproc

["Ryo ONODERA" <ryoon%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   ryoon
Date:           Sun Oct 26 12:15:39 UTC 2025

Modified Files:
        pkgsrc/textproc/py-sentencepiece: PLIST distinfo
        pkgsrc/textproc/sentencepiece: Makefile Makefile.common buildlink3.mk
            distinfo
Removed Files:
        pkgsrc/textproc/py-sentencepiece/patches: patch-build__bundled.sh

Log Message:
textproc/{py-,}sentencepiece: Update to 0.2.1

Changelog:
0.2.1:
Major changes

  * [Python] Supported wheels and builds for Python 3.13 and 3.14(rc1) #1134,
    #1127, #1121, #1111, #1110, #1104, #1103, #1099, #1091
  * [Python] Added an experimental support for free-threading. #1134, #1127,
    #1110 https://github.com/google/sentencepiece/tree/master/python#free-threading-support
  * [Python] Updated the supported Python version to 3.9 or later.

New features

  * [ALL]: Added new build mode to prevent the precompiled normalization
    rules being embedded in *.so and *.a. (-DSPM_DISABLE_EMBEDDED_DATA=ON).
  * This reduces the runtime size by approximately 1-2 MB. This mode is
    enabled to build python wheels. The rules are loaded as the data package.

Bug fixes & minor changes

  * [ALL]: Security fix to address a heap overflow issue that could occur
    when using a model containing an invalid precompiled normalization model.
  * [Python]: Deprecates the wheel package for Linux i686.
  * [Python]: Supported wheel for Windows Arm64. #1114
  * [Python]: Fixed the crash issue on batch decoding #1051
  * [ALL]: Updated the Unicode normalization rule with the latest ICU/Unicode
    rules.
  * [ALL]: Unused code and build mode cleanup.


To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.3 pkgsrc/textproc/py-sentencepiece/PLIST
cvs rdiff -u -r1.3 -r1.4 pkgsrc/textproc/py-sentencepiece/distinfo
cvs rdiff -u -r1.1 -r0 \
    pkgsrc/textproc/py-sentencepiece/patches/patch-build__bundled.sh
cvs rdiff -u -r1.2 -r1.3 pkgsrc/textproc/sentencepiece/Makefile \
    pkgsrc/textproc/sentencepiece/buildlink3.mk
cvs rdiff -u -r1.8 -r1.9 pkgsrc/textproc/sentencepiece/Makefile.common
cvs rdiff -u -r1.4 -r1.5 pkgsrc/textproc/sentencepiece/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/textproc/py-sentencepiece/PLIST
diff -u pkgsrc/textproc/py-sentencepiece/PLIST:1.2 pkgsrc/textproc/py-sentencepiece/PLIST:1.3
--- pkgsrc/textproc/py-sentencepiece/PLIST:1.2  Fri Jun  6 07:51:53 2025
+++ pkgsrc/textproc/py-sentencepiece/PLIST      Sun Oct 26 12:15:39 2025
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.2 2025/06/06 07:51:53 wiz Exp $
+@comment $NetBSD: PLIST,v 1.3 2025/10/26 12:15:39 ryoon Exp $
 ${PYSITELIB}/${WHEEL_INFODIR}/METADATA
 ${PYSITELIB}/${WHEEL_INFODIR}/RECORD
 ${PYSITELIB}/${WHEEL_INFODIR}/WHEEL
@@ -10,9 +10,15 @@ ${PYSITELIB}/sentencepiece/_sentencepiec
 ${PYSITELIB}/sentencepiece/_version.py
 ${PYSITELIB}/sentencepiece/_version.pyc
 ${PYSITELIB}/sentencepiece/_version.pyo
+${PYSITELIB}/sentencepiece/package_data/nfkc.bin
+${PYSITELIB}/sentencepiece/package_data/nfkc_cf.bin
+${PYSITELIB}/sentencepiece/package_data/nmt_nfkc.bin
+${PYSITELIB}/sentencepiece/package_data/nmt_nfkc_cf.bin
+${PYSITELIB}/sentencepiece/sentencepiece.i
 ${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.py
 ${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.pyc
 ${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.pyo
 ${PYSITELIB}/sentencepiece/sentencepiece_pb2.py
 ${PYSITELIB}/sentencepiece/sentencepiece_pb2.pyc
 ${PYSITELIB}/sentencepiece/sentencepiece_pb2.pyo
+${PYSITELIB}/sentencepiece/sentencepiece_wrap.cxx

Index: pkgsrc/textproc/py-sentencepiece/distinfo
diff -u pkgsrc/textproc/py-sentencepiece/distinfo:1.3 pkgsrc/textproc/py-sentencepiece/distinfo:1.4
--- pkgsrc/textproc/py-sentencepiece/distinfo:1.3       Fri Aug 22 17:28:59 2025
+++ pkgsrc/textproc/py-sentencepiece/distinfo   Sun Oct 26 12:15:39 2025
@@ -1,6 +1,5 @@
-$NetBSD: distinfo,v 1.3 2025/08/22 17:28:59 wiz Exp $
+$NetBSD: distinfo,v 1.4 2025/10/26 12:15:39 ryoon Exp $
 
-BLAKE2s (sentencepiece-0.2.0.tar.gz) = 68609347224e87397b98a2602bb88fb15df907d2bfe7e9cbce82020fc0c67169
-SHA512 (sentencepiece-0.2.0.tar.gz) = b4214f5bfbe2a0757794c792e87e7c53fda7e65b2511b37fc757f280bf9287ba59b5d630801e17de6058f8292a3c6433211917324cb3446a212a51735402e614
-Size (sentencepiece-0.2.0.tar.gz) = 11980811 bytes
-SHA1 (patch-build__bundled.sh) = cad43cfcdb7b11acc563c1a3696975805275539c
+BLAKE2s (sentencepiece-0.2.1.tar.gz) = 631cf3f28780914a877abd233e750858d5ac5974a237c3775c88dfffafafe142
+SHA512 (sentencepiece-0.2.1.tar.gz) = 012850b63b2323e16acc5dacc0a494ad3f6375425ee86274f0946032e47c088a3b307758b99d752fcf54acf76c82d7d13d0c14bbf07aa9b612c4f1fbd30cf1cf
+Size (sentencepiece-0.2.1.tar.gz) = 13485527 bytes

Index: pkgsrc/textproc/sentencepiece/Makefile
diff -u pkgsrc/textproc/sentencepiece/Makefile:1.2 pkgsrc/textproc/sentencepiece/Makefile:1.3
--- pkgsrc/textproc/sentencepiece/Makefile:1.2  Wed Aug 20 11:35:13 2025
+++ pkgsrc/textproc/sentencepiece/Makefile      Sun Oct 26 12:15:39 2025
@@ -1,7 +1,8 @@
-# $NetBSD: Makefile,v 1.2 2025/08/20 11:35:13 wiz Exp $
+# $NetBSD: Makefile,v 1.3 2025/10/26 12:15:39 ryoon Exp $
 
 PKGCONFIG_OVERRIDE+=   sentencepiece.pc.in
 
+CMAKE_CONFIGURE_ARGS+= -DSPM_USE_BUILTIN_PROTOBUF=TRUE
 CMAKE_CONFIGURE_ARGS+= -DCMAKE_POLICY_VERSION_MINIMUM=3.5
 
 .include "Makefile.common"
Index: pkgsrc/textproc/sentencepiece/buildlink3.mk
diff -u pkgsrc/textproc/sentencepiece/buildlink3.mk:1.2 pkgsrc/textproc/sentencepiece/buildlink3.mk:1.3
--- pkgsrc/textproc/sentencepiece/buildlink3.mk:1.2     Tue Jul 11 06:09:29 2023
+++ pkgsrc/textproc/sentencepiece/buildlink3.mk Sun Oct 26 12:15:39 2025
@@ -1,4 +1,4 @@
-# $NetBSD: buildlink3.mk,v 1.2 2023/07/11 06:09:29 nia Exp $
+# $NetBSD: buildlink3.mk,v 1.3 2025/10/26 12:15:39 ryoon Exp $
 
 BUILDLINK_TREE+=       sentencepiece
 
@@ -8,7 +8,7 @@ SENTENCEPIECE_BUILDLINK3_MK:=
 # C++17
 GCC_REQD+=     7
 
-BUILDLINK_API_DEPENDS.sentencepiece+=  sentencepiece>=0.1.97
+BUILDLINK_API_DEPENDS.sentencepiece+=  sentencepiece>=0.2.0
 BUILDLINK_PKGSRCDIR.sentencepiece?=    ../../textproc/sentencepiece
 .endif # SENTENCEPIECE_BUILDLINK3_MK
 

Index: pkgsrc/textproc/sentencepiece/Makefile.common
diff -u pkgsrc/textproc/sentencepiece/Makefile.common:1.8 pkgsrc/textproc/sentencepiece/Makefile.common:1.9
--- pkgsrc/textproc/sentencepiece/Makefile.common:1.8   Fri Aug 22 17:28:59 2025
+++ pkgsrc/textproc/sentencepiece/Makefile.common       Sun Oct 26 12:15:39 2025
@@ -1,11 +1,12 @@
-# $NetBSD: Makefile.common,v 1.8 2025/08/22 17:28:59 wiz Exp $
+# $NetBSD: Makefile.common,v 1.9 2025/10/26 12:15:39 ryoon Exp $
 #
 # used by textproc/sentencepiece/Makefile
 # used by textproc/py-sentencepiece/Makefile
 
-DISTNAME=      sentencepiece-0.2.0
+DISTNAME=      sentencepiece-0.2.1
 CATEGORIES=    textproc
 MASTER_SITES=  ${MASTER_SITE_GITHUB:=google/}
+GITHUB_PROJECT=        sentencepiece
 GITHUB_TAG=    v${PKGVERSION_NOREV}
 
 MAINTAINER=    pkgsrc-users%NetBSD.org@localhost

Index: pkgsrc/textproc/sentencepiece/distinfo
diff -u pkgsrc/textproc/sentencepiece/distinfo:1.4 pkgsrc/textproc/sentencepiece/distinfo:1.5
--- pkgsrc/textproc/sentencepiece/distinfo:1.4  Fri Aug 22 17:28:59 2025
+++ pkgsrc/textproc/sentencepiece/distinfo      Sun Oct 26 12:15:39 2025
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.4 2025/08/22 17:28:59 wiz Exp $
+$NetBSD: distinfo,v 1.5 2025/10/26 12:15:39 ryoon Exp $
 
-BLAKE2s (sentencepiece-0.2.0.tar.gz) = 68609347224e87397b98a2602bb88fb15df907d2bfe7e9cbce82020fc0c67169
-SHA512 (sentencepiece-0.2.0.tar.gz) = b4214f5bfbe2a0757794c792e87e7c53fda7e65b2511b37fc757f280bf9287ba59b5d630801e17de6058f8292a3c6433211917324cb3446a212a51735402e614
-Size (sentencepiece-0.2.0.tar.gz) = 11980811 bytes
+BLAKE2s (sentencepiece-0.2.1.tar.gz) = 631cf3f28780914a877abd233e750858d5ac5974a237c3775c88dfffafafe142
+SHA512 (sentencepiece-0.2.1.tar.gz) = 012850b63b2323e16acc5dacc0a494ad3f6375425ee86274f0946032e47c088a3b307758b99d752fcf54acf76c82d7d13d0c14bbf07aa9b612c4f1fbd30cf1cf
+Size (sentencepiece-0.2.1.tar.gz) = 13485527 bytes