CVS commit: pkgsrc/converters/py-chardet

To: pkgsrc-changes%NetBSD.org@localhost
Subject: CVS commit: pkgsrc/converters/py-chardet
From: "Adam Ciarcinski" <adam%netbsd.org@localhost>
Date: Wed, 11 Mar 2026 14:03:30 +0000

Module Name:    pkgsrc
Committed By:   adam
Date:           Wed Mar 11 14:03:30 UTC 2026

Modified Files:
        pkgsrc/converters/py-chardet: Makefile PLIST distinfo

Log Message:
py-chardet: updated to 7.0.1

7.0.0 (2026-03-02)

Ground-up, MIT-licensed rewrite of chardet. Same package name, same
public API — drop-in replacement for chardet 5.x/6.x.

**Highlights:**

- **MIT license** (previous versions were LGPL)
- **96.8% accuracy** on 2,179 test files (+2.3pp vs chardet 6.0.0,
  +7.7pp vs charset-normalizer)
- **41x faster** than chardet 6.0.0 with mypyc (**28x** pure Python),
  **7.5x faster** than charset-normalizer
- **Language detection** for every result (90.5% accuracy across 49
  languages)
- **99 encodings** across six eras (MODERN_WEB, LEGACY_ISO, LEGACY_MAC,
  LEGACY_REGIONAL, DOS, MAINFRAME)
- **12-stage detection pipeline** — BOM, UTF-16/32 patterns, escape
  sequences, binary detection, markup charset, ASCII, UTF-8 validation,
  byte validity, CJK gating, structural probing, statistical scoring,
  post-processing
- **Bigram frequency models** trained on CulturaX multilingual corpus
  data for all supported language/encoding pairs
- **Optional mypyc compilation** — 1.49x additional speedup on CPython
- **Thread-safe** ``detect()`` and ``detect_all()`` with no measurable
  overhead; scales on free-threaded Python 3.13t+
- **Negligible import memory** (96 B)
- **Zero runtime dependencies**

**Breaking changes vs 6.0.0:**

- ``detect()`` and ``detect_all()`` now default to
  ``encoding_era=EncodingEra.ALL`` (6.0.0 defaulted to ``MODERN_WEB``)
- Internal architecture is completely different (probers replaced by
  pipeline stages). Only the public API is preserved.
- ``LanguageFilter`` is accepted but ignored (deprecation warning
  emitted)
- ``chunk_size`` is accepted but ignored (deprecation warning emitted)


To generate a diff of this commit:
cvs rdiff -u -r1.38 -r1.39 pkgsrc/converters/py-chardet/Makefile
cvs rdiff -u -r1.15 -r1.16 pkgsrc/converters/py-chardet/PLIST \
    pkgsrc/converters/py-chardet/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/converters/py-chardet/Makefile
diff -u pkgsrc/converters/py-chardet/Makefile:1.38 pkgsrc/converters/py-chardet/Makefile:1.39
--- pkgsrc/converters/py-chardet/Makefile:1.38  Wed Mar 11 06:57:35 2026
+++ pkgsrc/converters/py-chardet/Makefile       Wed Mar 11 14:03:30 2026
@@ -1,22 +1,20 @@
-# $NetBSD: Makefile,v 1.38 2026/03/11 06:57:35 wiz Exp $
+# $NetBSD: Makefile,v 1.39 2026/03/11 14:03:30 adam Exp $
 #
 # There is controversy about the license change in version 7.0.0.
 # Please don't update this package to 7+ for now.
 
-DISTNAME=      chardet-5.2.0
+DISTNAME=      chardet-7.0.1
 PKGNAME=       ${PYPKGPREFIX}-${DISTNAME}
-PKGREVISION=   2
 CATEGORIES=    converters python
 MASTER_SITES=  ${MASTER_SITE_PYPI:=c/chardet/}
 
 MAINTAINER=    bartosz.kuzma%gmail.com@localhost
 HOMEPAGE=      https://github.com/chardet/chardet
 COMMENT=       Universal encoding detector for Python 3
-LICENSE=       gnu-lgpl-v2.1
+LICENSE=       mit
 
-TOOL_DEPENDS+= ${PYPKGPREFIX}-setuptools>=78:../../devel/py-setuptools
-TEST_DEPENDS+= ${PYPKGPREFIX}-exceptiongroup-[0-9]*:../../devel/py-exceptiongroup
-TEST_DEPENDS+= ${PYPKGPREFIX}-hypothesis-[0-9]*:../../devel/py-hypothesis
+TOOL_DEPENDS+= ${PYPKGPREFIX}-hatch-vcs-[0-9]*:../../devel/py-hatch-vcs
+TOOL_DEPENDS+= ${PYPKGPREFIX}-hatchling-[0-9]*:../../devel/py-hatchling
 
 post-install:
        cd ${DESTDIR}${PREFIX}/bin && \

Index: pkgsrc/converters/py-chardet/PLIST
diff -u pkgsrc/converters/py-chardet/PLIST:1.15 pkgsrc/converters/py-chardet/PLIST:1.16
--- pkgsrc/converters/py-chardet/PLIST:1.15     Sun Apr 13 07:22:37 2025
+++ pkgsrc/converters/py-chardet/PLIST  Wed Mar 11 14:03:30 2026
@@ -1,156 +1,80 @@
-@comment $NetBSD: PLIST,v 1.15 2025/04/13 07:22:37 adam Exp $
+@comment $NetBSD: PLIST,v 1.16 2026/03/11 14:03:30 adam Exp $
 bin/chardetect-${PYVERSSUFFIX}
 ${PYSITELIB}/${WHEEL_INFODIR}/METADATA
 ${PYSITELIB}/${WHEEL_INFODIR}/RECORD
 ${PYSITELIB}/${WHEEL_INFODIR}/WHEEL
 ${PYSITELIB}/${WHEEL_INFODIR}/entry_points.txt
 ${PYSITELIB}/${WHEEL_INFODIR}/licenses/LICENSE
-${PYSITELIB}/${WHEEL_INFODIR}/top_level.txt
 ${PYSITELIB}/chardet/__init__.py
 ${PYSITELIB}/chardet/__init__.pyc
 ${PYSITELIB}/chardet/__init__.pyo
 ${PYSITELIB}/chardet/__main__.py
 ${PYSITELIB}/chardet/__main__.pyc
 ${PYSITELIB}/chardet/__main__.pyo
-${PYSITELIB}/chardet/big5freq.py
-${PYSITELIB}/chardet/big5freq.pyc
-${PYSITELIB}/chardet/big5freq.pyo
-${PYSITELIB}/chardet/big5prober.py
-${PYSITELIB}/chardet/big5prober.pyc
-${PYSITELIB}/chardet/big5prober.pyo
-${PYSITELIB}/chardet/chardistribution.py
-${PYSITELIB}/chardet/chardistribution.pyc
-${PYSITELIB}/chardet/chardistribution.pyo
-${PYSITELIB}/chardet/charsetgroupprober.py
-${PYSITELIB}/chardet/charsetgroupprober.pyc
-${PYSITELIB}/chardet/charsetgroupprober.pyo
-${PYSITELIB}/chardet/charsetprober.py
-${PYSITELIB}/chardet/charsetprober.pyc
-${PYSITELIB}/chardet/charsetprober.pyo
-${PYSITELIB}/chardet/cli/__init__.py
-${PYSITELIB}/chardet/cli/__init__.pyc
-${PYSITELIB}/chardet/cli/__init__.pyo
-${PYSITELIB}/chardet/cli/chardetect.py
-${PYSITELIB}/chardet/cli/chardetect.pyc
-${PYSITELIB}/chardet/cli/chardetect.pyo
-${PYSITELIB}/chardet/codingstatemachine.py
-${PYSITELIB}/chardet/codingstatemachine.pyc
-${PYSITELIB}/chardet/codingstatemachine.pyo
-${PYSITELIB}/chardet/codingstatemachinedict.py
-${PYSITELIB}/chardet/codingstatemachinedict.pyc
-${PYSITELIB}/chardet/codingstatemachinedict.pyo
-${PYSITELIB}/chardet/cp949prober.py
-${PYSITELIB}/chardet/cp949prober.pyc
-${PYSITELIB}/chardet/cp949prober.pyo
+${PYSITELIB}/chardet/_utils.py
+${PYSITELIB}/chardet/_utils.pyc
+${PYSITELIB}/chardet/_utils.pyo
+${PYSITELIB}/chardet/_version.py
+${PYSITELIB}/chardet/_version.pyc
+${PYSITELIB}/chardet/_version.pyo
+${PYSITELIB}/chardet/cli.py
+${PYSITELIB}/chardet/cli.pyc
+${PYSITELIB}/chardet/cli.pyo
+${PYSITELIB}/chardet/detector.py
+${PYSITELIB}/chardet/detector.pyc
+${PYSITELIB}/chardet/detector.pyo
 ${PYSITELIB}/chardet/enums.py
 ${PYSITELIB}/chardet/enums.pyc
 ${PYSITELIB}/chardet/enums.pyo
-${PYSITELIB}/chardet/escprober.py
-${PYSITELIB}/chardet/escprober.pyc
-${PYSITELIB}/chardet/escprober.pyo
-${PYSITELIB}/chardet/escsm.py
-${PYSITELIB}/chardet/escsm.pyc
-${PYSITELIB}/chardet/escsm.pyo
-${PYSITELIB}/chardet/eucjpprober.py
-${PYSITELIB}/chardet/eucjpprober.pyc
-${PYSITELIB}/chardet/eucjpprober.pyo
-${PYSITELIB}/chardet/euckrfreq.py
-${PYSITELIB}/chardet/euckrfreq.pyc
-${PYSITELIB}/chardet/euckrfreq.pyo
-${PYSITELIB}/chardet/euckrprober.py
-${PYSITELIB}/chardet/euckrprober.pyc
-${PYSITELIB}/chardet/euckrprober.pyo
-${PYSITELIB}/chardet/euctwfreq.py
-${PYSITELIB}/chardet/euctwfreq.pyc
-${PYSITELIB}/chardet/euctwfreq.pyo
-${PYSITELIB}/chardet/euctwprober.py
-${PYSITELIB}/chardet/euctwprober.pyc
-${PYSITELIB}/chardet/euctwprober.pyo
-${PYSITELIB}/chardet/gb2312freq.py
-${PYSITELIB}/chardet/gb2312freq.pyc
-${PYSITELIB}/chardet/gb2312freq.pyo
-${PYSITELIB}/chardet/gb2312prober.py
-${PYSITELIB}/chardet/gb2312prober.pyc
-${PYSITELIB}/chardet/gb2312prober.pyo
-${PYSITELIB}/chardet/hebrewprober.py
-${PYSITELIB}/chardet/hebrewprober.pyc
-${PYSITELIB}/chardet/hebrewprober.pyo
-${PYSITELIB}/chardet/jisfreq.py
-${PYSITELIB}/chardet/jisfreq.pyc
-${PYSITELIB}/chardet/jisfreq.pyo
-${PYSITELIB}/chardet/johabfreq.py
-${PYSITELIB}/chardet/johabfreq.pyc
-${PYSITELIB}/chardet/johabfreq.pyo
-${PYSITELIB}/chardet/johabprober.py
-${PYSITELIB}/chardet/johabprober.pyc
-${PYSITELIB}/chardet/johabprober.pyo
-${PYSITELIB}/chardet/jpcntx.py
-${PYSITELIB}/chardet/jpcntx.pyc
-${PYSITELIB}/chardet/jpcntx.pyo
-${PYSITELIB}/chardet/langbulgarianmodel.py
-${PYSITELIB}/chardet/langbulgarianmodel.pyc
-${PYSITELIB}/chardet/langbulgarianmodel.pyo
-${PYSITELIB}/chardet/langgreekmodel.py
-${PYSITELIB}/chardet/langgreekmodel.pyc
-${PYSITELIB}/chardet/langgreekmodel.pyo
-${PYSITELIB}/chardet/langhebrewmodel.py
-${PYSITELIB}/chardet/langhebrewmodel.pyc
-${PYSITELIB}/chardet/langhebrewmodel.pyo
-${PYSITELIB}/chardet/langhungarianmodel.py
-${PYSITELIB}/chardet/langhungarianmodel.pyc
-${PYSITELIB}/chardet/langhungarianmodel.pyo
-${PYSITELIB}/chardet/langrussianmodel.py
-${PYSITELIB}/chardet/langrussianmodel.pyc
-${PYSITELIB}/chardet/langrussianmodel.pyo
-${PYSITELIB}/chardet/langthaimodel.py
-${PYSITELIB}/chardet/langthaimodel.pyc
-${PYSITELIB}/chardet/langthaimodel.pyo
-${PYSITELIB}/chardet/langturkishmodel.py
-${PYSITELIB}/chardet/langturkishmodel.pyc
-${PYSITELIB}/chardet/langturkishmodel.pyo
-${PYSITELIB}/chardet/latin1prober.py
-${PYSITELIB}/chardet/latin1prober.pyc
-${PYSITELIB}/chardet/latin1prober.pyo
-${PYSITELIB}/chardet/macromanprober.py
-${PYSITELIB}/chardet/macromanprober.pyc
-${PYSITELIB}/chardet/macromanprober.pyo
-${PYSITELIB}/chardet/mbcharsetprober.py
-${PYSITELIB}/chardet/mbcharsetprober.pyc
-${PYSITELIB}/chardet/mbcharsetprober.pyo
-${PYSITELIB}/chardet/mbcsgroupprober.py
-${PYSITELIB}/chardet/mbcsgroupprober.pyc
-${PYSITELIB}/chardet/mbcsgroupprober.pyo
-${PYSITELIB}/chardet/mbcssm.py
-${PYSITELIB}/chardet/mbcssm.pyc
-${PYSITELIB}/chardet/mbcssm.pyo
-${PYSITELIB}/chardet/metadata/__init__.py
-${PYSITELIB}/chardet/metadata/__init__.pyc
-${PYSITELIB}/chardet/metadata/__init__.pyo
-${PYSITELIB}/chardet/metadata/languages.py
-${PYSITELIB}/chardet/metadata/languages.pyc
-${PYSITELIB}/chardet/metadata/languages.pyo
+${PYSITELIB}/chardet/equivalences.py
+${PYSITELIB}/chardet/equivalences.pyc
+${PYSITELIB}/chardet/equivalences.pyo
+${PYSITELIB}/chardet/models/__init__.py
+${PYSITELIB}/chardet/models/__init__.pyc
+${PYSITELIB}/chardet/models/__init__.pyo
+${PYSITELIB}/chardet/models/confusion.bin
+${PYSITELIB}/chardet/models/models.bin
+${PYSITELIB}/chardet/models/training_metadata.yaml
+${PYSITELIB}/chardet/pipeline/__init__.py
+${PYSITELIB}/chardet/pipeline/__init__.pyc
+${PYSITELIB}/chardet/pipeline/__init__.pyo
+${PYSITELIB}/chardet/pipeline/ascii.py
+${PYSITELIB}/chardet/pipeline/ascii.pyc
+${PYSITELIB}/chardet/pipeline/ascii.pyo
+${PYSITELIB}/chardet/pipeline/binary.py
+${PYSITELIB}/chardet/pipeline/binary.pyc
+${PYSITELIB}/chardet/pipeline/binary.pyo
+${PYSITELIB}/chardet/pipeline/bom.py
+${PYSITELIB}/chardet/pipeline/bom.pyc
+${PYSITELIB}/chardet/pipeline/bom.pyo
+${PYSITELIB}/chardet/pipeline/confusion.py
+${PYSITELIB}/chardet/pipeline/confusion.pyc
+${PYSITELIB}/chardet/pipeline/confusion.pyo
+${PYSITELIB}/chardet/pipeline/escape.py
+${PYSITELIB}/chardet/pipeline/escape.pyc
+${PYSITELIB}/chardet/pipeline/escape.pyo
+${PYSITELIB}/chardet/pipeline/markup.py
+${PYSITELIB}/chardet/pipeline/markup.pyc
+${PYSITELIB}/chardet/pipeline/markup.pyo
+${PYSITELIB}/chardet/pipeline/orchestrator.py
+${PYSITELIB}/chardet/pipeline/orchestrator.pyc
+${PYSITELIB}/chardet/pipeline/orchestrator.pyo
+${PYSITELIB}/chardet/pipeline/statistical.py
+${PYSITELIB}/chardet/pipeline/statistical.pyc
+${PYSITELIB}/chardet/pipeline/statistical.pyo
+${PYSITELIB}/chardet/pipeline/structural.py
+${PYSITELIB}/chardet/pipeline/structural.pyc
+${PYSITELIB}/chardet/pipeline/structural.pyo
+${PYSITELIB}/chardet/pipeline/utf1632.py
+${PYSITELIB}/chardet/pipeline/utf1632.pyc
+${PYSITELIB}/chardet/pipeline/utf1632.pyo
+${PYSITELIB}/chardet/pipeline/utf8.py
+${PYSITELIB}/chardet/pipeline/utf8.pyc
+${PYSITELIB}/chardet/pipeline/utf8.pyo
+${PYSITELIB}/chardet/pipeline/validity.py
+${PYSITELIB}/chardet/pipeline/validity.pyc
+${PYSITELIB}/chardet/pipeline/validity.pyo
 ${PYSITELIB}/chardet/py.typed
-${PYSITELIB}/chardet/resultdict.py
-${PYSITELIB}/chardet/resultdict.pyc
-${PYSITELIB}/chardet/resultdict.pyo
-${PYSITELIB}/chardet/sbcharsetprober.py
-${PYSITELIB}/chardet/sbcharsetprober.pyc
-${PYSITELIB}/chardet/sbcharsetprober.pyo
-${PYSITELIB}/chardet/sbcsgroupprober.py
-${PYSITELIB}/chardet/sbcsgroupprober.pyc
-${PYSITELIB}/chardet/sbcsgroupprober.pyo
-${PYSITELIB}/chardet/sjisprober.py
-${PYSITELIB}/chardet/sjisprober.pyc
-${PYSITELIB}/chardet/sjisprober.pyo
-${PYSITELIB}/chardet/universaldetector.py
-${PYSITELIB}/chardet/universaldetector.pyc
-${PYSITELIB}/chardet/universaldetector.pyo
-${PYSITELIB}/chardet/utf1632prober.py
-${PYSITELIB}/chardet/utf1632prober.pyc
-${PYSITELIB}/chardet/utf1632prober.pyo
-${PYSITELIB}/chardet/utf8prober.py
-${PYSITELIB}/chardet/utf8prober.pyc
-${PYSITELIB}/chardet/utf8prober.pyo
-${PYSITELIB}/chardet/version.py
-${PYSITELIB}/chardet/version.pyc
-${PYSITELIB}/chardet/version.pyo
+${PYSITELIB}/chardet/registry.py
+${PYSITELIB}/chardet/registry.pyc
+${PYSITELIB}/chardet/registry.pyo
Index: pkgsrc/converters/py-chardet/distinfo
diff -u pkgsrc/converters/py-chardet/distinfo:1.15 pkgsrc/converters/py-chardet/distinfo:1.16
--- pkgsrc/converters/py-chardet/distinfo:1.15  Wed Aug  2 06:45:12 2023
+++ pkgsrc/converters/py-chardet/distinfo       Wed Mar 11 14:03:30 2026
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.15 2023/08/02 06:45:12 adam Exp $
+$NetBSD: distinfo,v 1.16 2026/03/11 14:03:30 adam Exp $
 
-BLAKE2s (chardet-5.2.0.tar.gz) = 63c8f944cdb9ee5fee203a2616914f4439250cf5d13798fd2adea09e085476a2
-SHA512 (chardet-5.2.0.tar.gz) = c9c2d28c8783aa4513b15692c30955360b4bdb97922a8ba1ff4ec47d49be4d23d83a3e5e77fe2c9aa6bf8c966f9c021269d8e45ac7f9642b886475724ae3ae54
-Size (chardet-5.2.0.tar.gz) = 2069618 bytes
+BLAKE2s (chardet-7.0.1.tar.gz) = f9bd3b61d4d39c7bebedac55ec15af7334f52840c4a7cba35a30158d520601a6
+SHA512 (chardet-7.0.1.tar.gz) = 3b383e1da1122260c27e7f495dc5729a1549ed346fb008626baf25624d56ba3874acb5db261b19cda30e838e55c77fac19e0fe4a1896510e8fc553d548182426
+Size (chardet-7.0.1.tar.gz) = 490240 bytes

Prev by Date: CVS commit: pkgsrc/math/py-tables
Next by Date: CVS commit: pkgsrc/doc
Previous by Thread: CVS commit: pkgsrc/math/py-tables
Next by Thread: CVS commit: pkgsrc/doc
Indexes:

Home | Main Index | Thread Index | Old Index