CVS commit: pkgsrc/textproc/py-sentencepiece

["Thomas Klausner" <wiz%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   wiz
Date:           Mon Mar 13 14:18:27 UTC 2023

Added Files:
        pkgsrc/textproc/py-sentencepiece: DESCR Makefile PLIST distinfo

Log Message:
textproc/py-sentencepiece: import py-sentencepiece-0.1.97

SentencePiece is an unsupervised text tokenizer and detokenizer
mainly for Neural Network-based text generation systems where the
vocabulary size is predetermined prior to the neural model training.
SentencePiece implements subword units (e.g., byte-pair-encoding
(BPE)) and unigram language model with the extension of direct
training from raw sentences. SentencePiece allows us to make a
purely end-to-end system that does not depend on language-specific
pre/postprocessing.

This package contains the Python module.


To generate a diff of this commit:
cvs rdiff -u -r0 -r1.1 pkgsrc/textproc/py-sentencepiece/DESCR \
    pkgsrc/textproc/py-sentencepiece/Makefile \
    pkgsrc/textproc/py-sentencepiece/PLIST \
    pkgsrc/textproc/py-sentencepiece/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Added files:

Index: pkgsrc/textproc/py-sentencepiece/DESCR
diff -u /dev/null pkgsrc/textproc/py-sentencepiece/DESCR:1.1
--- /dev/null   Mon Mar 13 14:18:27 2023
+++ pkgsrc/textproc/py-sentencepiece/DESCR      Mon Mar 13 14:18:27 2023
@@ -0,0 +1,10 @@
+SentencePiece is an unsupervised text tokenizer and detokenizer
+mainly for Neural Network-based text generation systems where the
+vocabulary size is predetermined prior to the neural model training.
+SentencePiece implements subword units (e.g., byte-pair-encoding
+(BPE)) and unigram language model with the extension of direct
+training from raw sentences. SentencePiece allows us to make a
+purely end-to-end system that does not depend on language-specific
+pre/postprocessing.
+
+This package contains the Python module.
Index: pkgsrc/textproc/py-sentencepiece/Makefile
diff -u /dev/null pkgsrc/textproc/py-sentencepiece/Makefile:1.1
--- /dev/null   Mon Mar 13 14:18:27 2023
+++ pkgsrc/textproc/py-sentencepiece/Makefile   Mon Mar 13 14:18:27 2023
@@ -0,0 +1,12 @@
+# $NetBSD: Makefile,v 1.1 2023/03/13 14:18:27 wiz Exp $
+
+PKGNAME=       ${PYPKGPREFIX}-${DISTNAME}
+
+WRKSRC=                ${WRKDIR}/${DISTNAME}/python
+
+USE_TOOLS+=    pkg-config
+
+.include "../../textproc/sentencepiece/Makefile.common"
+.include "../../textproc/sentencepiece/buildlink3.mk"
+.include "../../lang/python/egg.mk"
+.include "../../mk/bsd.pkg.mk"
Index: pkgsrc/textproc/py-sentencepiece/PLIST
diff -u /dev/null pkgsrc/textproc/py-sentencepiece/PLIST:1.1
--- /dev/null   Mon Mar 13 14:18:27 2023
+++ pkgsrc/textproc/py-sentencepiece/PLIST      Mon Mar 13 14:18:27 2023
@@ -0,0 +1,18 @@
+@comment $NetBSD: PLIST,v 1.1 2023/03/13 14:18:27 wiz Exp $
+${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
+${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
+${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt
+${PYSITELIB}/${EGG_INFODIR}/top_level.txt
+${PYSITELIB}/sentencepiece/__init__.py
+${PYSITELIB}/sentencepiece/__init__.pyc
+${PYSITELIB}/sentencepiece/__init__.pyo
+${PYSITELIB}/sentencepiece/_sentencepiece.so
+${PYSITELIB}/sentencepiece/_version.py
+${PYSITELIB}/sentencepiece/_version.pyc
+${PYSITELIB}/sentencepiece/_version.pyo
+${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.py
+${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.pyc
+${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.pyo
+${PYSITELIB}/sentencepiece/sentencepiece_pb2.py
+${PYSITELIB}/sentencepiece/sentencepiece_pb2.pyc
+${PYSITELIB}/sentencepiece/sentencepiece_pb2.pyo
Index: pkgsrc/textproc/py-sentencepiece/distinfo
diff -u /dev/null pkgsrc/textproc/py-sentencepiece/distinfo:1.1
--- /dev/null   Mon Mar 13 14:18:27 2023
+++ pkgsrc/textproc/py-sentencepiece/distinfo   Mon Mar 13 14:18:27 2023
@@ -0,0 +1,5 @@
+$NetBSD: distinfo,v 1.1 2023/03/13 14:18:27 wiz Exp $
+
+BLAKE2s (sentencepiece-0.1.97.tar.gz) = 969788b6d87e8c992f6df4349f984fb2d6e80f978d4007127174222ec7fcb3ab
+SHA512 (sentencepiece-0.1.97.tar.gz) = 4c35488e3661e45be677b04299c0d0b1f0d46421098f0b1625a1bb5e7725d175dfd55328a5a7bbf88badeb03c2ba087aef942b0d7520a29f6bf34eae211a99eb
+Size (sentencepiece-0.1.97.tar.gz) = 11945436 bytes