CVS commit: pkgsrc/textproc/py-nltk

["Thomas Klausner" <wiz%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   wiz
Date:           Wed Nov 24 16:00:18 UTC 2021

Modified Files:
        pkgsrc/textproc/py-nltk: Makefile PLIST distinfo

Log Message:
py-nltk: update to 3.6.5.

Version 3.6.5 2021-10-11

* modernised nltk.org website
* addressed LGTM.com issues
* support ZWJ sequences emoji and skin tone modifer emoji in TweetTokenizer
* METEOR evaluation now requires pre-tokenized input
* Code linting and type hinting
* implement get_refs function for DrtLambdaExpression
* Enable automated CoreNLP, Senna, Prover9/Mace4, Megam, MaltParser CI tests
* specify minimum regex version that supports regex.Pattern
* avoid re.Pattern and regex.Pattern which fail for Python 3.6, 3.7

Version 3.6.4 2021-10-01

* deprecate `nltk.usage(obj)` in favor of `help(obj)`
* resolve ReDoS vulnerability in Corpus Reader
* solidify performance tests
* improve phone number recognition in tweet tokenizer
* refactored CISTEM stemmer for German
* identify NLTK Team as the author
* replace travis badge with github actions badge
* add SECURITY.md

Version 3.6.3 2021-09-19
* Dropped support for Python 3.5
* Run CI tests on Windows, too
* Moved from Travis CI to GitHub Actions
* Code and comment cleanups
* Visualize WordNet relation graphs using Graphviz
* Fixed large error in METEOR score
* Apply isort, pyupgrade, black, added as pre-commit hooks
* Prevent debug_decisions in Punkt from throwing IndexError
* Resolved ZeroDivisionError in RIBES with dissimilar sentences
* Initialize WordNet IC total counts with smoothing value
* Fixed AttributeError for Arabic ARLSTem2 stemmer
* Many fixes and improvements to lm language model package
* Fix bug in nltk.metrics.aline, C_skip = -10
* Improvements to TweetTokenizer
* Optional show arg for FreqDist.plot, ConditionalFreqDist.plot
* edit_distance now computes Damerau-Levenshtein edit-distance

Version 3.6.2 2021-04-20
* move test code to nltk/test
* clean up some doctests
* fix bug in NgramAssocMeasures (order preserving fix)
* fixes for compatibility with Pypy 7.3.4

Version 3.6 2021-04-07
* add support for Python 3.9
* add Tree.fromlist
* compute Minimum Spanning Tree of unweighted graph using BFS
* fix bug with infinite loop in Wordnet closure and tree
* fix bug in calculating BLEU using smoothing method 4
* Wordnet synset similarities work for all pos
* new Arabic light stemmer (ARLSTem2)
* new syllable tokenizer (LegalitySyllableTokenizer)
* remove nose in favor of pytest
* misc bug fixes, code cleanups, test cleanups, efficiency improvements


To generate a diff of this commit:
cvs rdiff -u -r1.3 -r1.4 pkgsrc/textproc/py-nltk/Makefile
cvs rdiff -u -r1.2 -r1.3 pkgsrc/textproc/py-nltk/PLIST
cvs rdiff -u -r1.4 -r1.5 pkgsrc/textproc/py-nltk/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/textproc/py-nltk/Makefile
diff -u pkgsrc/textproc/py-nltk/Makefile:1.3 pkgsrc/textproc/py-nltk/Makefile:1.4
--- pkgsrc/textproc/py-nltk/Makefile:1.3        Mon Aug 10 14:43:10 2020
+++ pkgsrc/textproc/py-nltk/Makefile    Wed Nov 24 16:00:18 2021
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.3 2020/08/10 14:43:10 adam Exp $
+# $NetBSD: Makefile,v 1.4 2021/11/24 16:00:18 wiz Exp $
 
-DISTNAME=      nltk-3.5
+DISTNAME=      nltk-3.6.5
 PKGNAME=       ${PYPKGPREFIX}-${DISTNAME}
 CATEGORIES=    textproc python
 MASTER_SITES=  ${MASTER_SITE_PYPI:=n/nltk/}
@@ -12,17 +12,46 @@ COMMENT=    Natural Language Toolkit (NLTK)
 LICENSE=       apache-2.0
 
 DEPENDS+=      ${PYPKGPREFIX}-click-[0-9]*:../../devel/py-click
-DEPENDS+=      ${PYPKGPREFIX}-expat-[0-9]*:../../textproc/py-expat
 DEPENDS+=      ${PYPKGPREFIX}-joblib-[0-9]*:../../devel/py-joblib
-DEPENDS+=      ${PYPKGPREFIX}-nose>=1.1.2:../../devel/py-nose
-DEPENDS+=      ${PYPKGPREFIX}-regex-[0-9]*:../../textproc/py-regex
+DEPENDS+=      ${PYPKGPREFIX}-pyparsing-[0-9]*:../../devel/py-pyparsing
+DEPENDS+=      ${PYPKGPREFIX}-regex>=2021.8.3:../../textproc/py-regex
 DEPENDS+=      ${PYPKGPREFIX}-sqlite3-[0-9]*:../../databases/py-sqlite3
 DEPENDS+=      ${PYPKGPREFIX}-tqdm-[0-9]*:../../misc/py-tqdm
-DEPENDS+=      ${PYPKGPREFIX}-yaml-[0-9]*:../../textproc/py-yaml
+TEST_DEPENDS+= ${PYPKGPREFIX}-numpy-[0-9]*:../../math/py-numpy
+TEST_DEPENDS+= ${PYPKGPREFIX}-matplotlib-[0-9]*:../../graphics/py-matplotlib
+TEST_DEPENDS+= ${PYPKGPREFIX}-test-[0-9]*:../../devel/py-test
+TEST_DEPENDS+= ${PYPKGPREFIX}-test-mock-[0-9]*:../../devel/py-test-mock
+TEST_DEPENDS+= nltk_data-averaged_perceptron_tagger-[0-9]*:../../textproc/nltk_data-averaged_perceptron_tagger
+TEST_DEPENDS+= nltk_data-averaged_perceptron_tagger_ru-[0-9]*:../../textproc/nltk_data-averaged_perceptron_tagger_ru
+TEST_DEPENDS+= nltk_data-brown-[0-9]*:../../textproc/nltk_data-brown
+TEST_DEPENDS+= nltk_data-cess_cat-[0-9]*:../../textproc/nltk_data-cess_cat
+TEST_DEPENDS+= nltk_data-cess_esp-[0-9]*:../../textproc/nltk_data-cess_esp
+TEST_DEPENDS+= nltk_data-conll2007-[0-9]*:../../textproc/nltk_data-conll2007
+TEST_DEPENDS+= nltk_data-floresta-[0-9]*:../../textproc/nltk_data-floresta
+TEST_DEPENDS+= nltk_data-gutenberg-[0-9]*:../../textproc/nltk_data-gutenberg
+TEST_DEPENDS+= nltk_data-inaugural-[0-9]*:../../textproc/nltk_data-inaugural
+TEST_DEPENDS+= nltk_data-indian-[0-9]*:../../textproc/nltk_data-indian
+TEST_DEPENDS+= nltk_data-large_grammars-[0-9]*:../../textproc/nltk_data-large_grammars
+TEST_DEPENDS+= nltk_data-nombank.1.0-[0-9]*:../../textproc/nltk_data-nombank.1.0
+TEST_DEPENDS+= nltk_data-porter_test-[0-9]*:../../textproc/nltk_data-porter_test
+TEST_DEPENDS+= nltk_data-omw-[0-9]*:../../textproc/nltk_data-omw
+TEST_DEPENDS+= nltk_data-ptb-[0-9]*:../../textproc/nltk_data-ptb
+TEST_DEPENDS+= nltk_data-pl196x-[0-9]*:../../textproc/nltk_data-pl196x
+TEST_DEPENDS+= nltk_data-punkt-[0-9]*:../../textproc/nltk_data-punkt
+TEST_DEPENDS+= nltk_data-rte-[0-9]*:../../textproc/nltk_data-rte
+TEST_DEPENDS+= nltk_data-sinica_treebank-[0-9]*:../../textproc/nltk_data-sinica_treebank
+TEST_DEPENDS+= nltk_data-stopwords-[0-9]*:../../textproc/nltk_data-stopwords
+TEST_DEPENDS+= nltk_data-treebank-[0-9]*:../../textproc/nltk_data-treebank
+TEST_DEPENDS+= nltk_data-twitter_samples-[0-9]*:../../textproc/nltk_data-twitter_samples
+TEST_DEPENDS+= nltk_data-udhr-[0-9]*:../../textproc/nltk_data-udhr
+TEST_DEPENDS+= nltk_data-universal_tagset-[0-9]*:../../textproc/nltk_data-universal_tagset
+TEST_DEPENDS+= nltk_data-wmt15_eval-[0-9]*:../../textproc/nltk_data-wmt15_eval
+TEST_DEPENDS+= nltk_data-wordnet-[0-9]*:../../textproc/nltk_data-wordnet
+TEST_DEPENDS+= nltk_data-wordnet31-[0-9]*:../../textproc/nltk_data-wordnet31
+TEST_DEPENDS+= nltk_data-wordnet_ic-[0-9]*:../../textproc/nltk_data-wordnet_ic
+TEST_DEPENDS+= nltk_data-words-[0-9]*:../../textproc/nltk_data-words
 
 REPLACE_PYTHON=                nltk/corpus/reader/knbc.py
-REPLACE_PYTHON+=       nltk/stem/rslp.py
-REPLACE_PYTHON+=       nltk/test/runtests.py
 
 PYTHON_VERSIONS_INCOMPATIBLE=  27
 
@@ -30,6 +59,9 @@ post-install:
        cd ${DESTDIR}${PREFIX}/bin && \
        ${MV} nltk nltk-${PYVERSSUFFIX} || ${TRUE}
 
+do-test:
+       cd ${WRKSRC} && ${SETENV} ${TEST_ENV} pytest-${PYVERSSUFFIX}
+
 .include "../../lang/python/application.mk"
 .include "../../lang/python/egg.mk"
 .include "../../mk/bsd.pkg.mk"

Index: pkgsrc/textproc/py-nltk/PLIST
diff -u pkgsrc/textproc/py-nltk/PLIST:1.2 pkgsrc/textproc/py-nltk/PLIST:1.3
--- pkgsrc/textproc/py-nltk/PLIST:1.2   Mon Aug 10 14:43:10 2020
+++ pkgsrc/textproc/py-nltk/PLIST       Wed Nov 24 16:00:18 2021
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.2 2020/08/10 14:43:10 adam Exp $
+@comment $NetBSD: PLIST,v 1.3 2021/11/24 16:00:18 wiz Exp $
 bin/nltk-${PYVERSSUFFIX}
 ${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
 ${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
@@ -605,6 +605,9 @@ ${PYSITELIB}/nltk/stem/api.pyo
 ${PYSITELIB}/nltk/stem/arlstem.py
 ${PYSITELIB}/nltk/stem/arlstem.pyc
 ${PYSITELIB}/nltk/stem/arlstem.pyo
+${PYSITELIB}/nltk/stem/arlstem2.py
+${PYSITELIB}/nltk/stem/arlstem2.pyc
+${PYSITELIB}/nltk/stem/arlstem2.pyo
 ${PYSITELIB}/nltk/stem/cistem.py
 ${PYSITELIB}/nltk/stem/cistem.pyc
 ${PYSITELIB}/nltk/stem/cistem.pyo
@@ -718,10 +721,10 @@ ${PYSITELIB}/nltk/test/classify_fixt.pyo
 ${PYSITELIB}/nltk/test/collections.doctest
 ${PYSITELIB}/nltk/test/collocations.doctest
 ${PYSITELIB}/nltk/test/concordance.doctest
+${PYSITELIB}/nltk/test/conftest.py
+${PYSITELIB}/nltk/test/conftest.pyc
+${PYSITELIB}/nltk/test/conftest.pyo
 ${PYSITELIB}/nltk/test/corpus.doctest
-${PYSITELIB}/nltk/test/corpus_fixt.py
-${PYSITELIB}/nltk/test/corpus_fixt.pyc
-${PYSITELIB}/nltk/test/corpus_fixt.pyo
 ${PYSITELIB}/nltk/test/crubadan.doctest
 ${PYSITELIB}/nltk/test/data.doctest
 ${PYSITELIB}/nltk/test/dependency.doctest
@@ -774,16 +777,7 @@ ${PYSITELIB}/nltk/test/probability_fixt.
 ${PYSITELIB}/nltk/test/propbank.doctest
 ${PYSITELIB}/nltk/test/relextract.doctest
 ${PYSITELIB}/nltk/test/resolution.doctest
-${PYSITELIB}/nltk/test/runtests.py
-${PYSITELIB}/nltk/test/runtests.pyc
-${PYSITELIB}/nltk/test/runtests.pyo
-${PYSITELIB}/nltk/test/segmentation_fixt.py
-${PYSITELIB}/nltk/test/segmentation_fixt.pyc
-${PYSITELIB}/nltk/test/segmentation_fixt.pyo
 ${PYSITELIB}/nltk/test/semantics.doctest
-${PYSITELIB}/nltk/test/semantics_fixt.py
-${PYSITELIB}/nltk/test/semantics_fixt.pyc
-${PYSITELIB}/nltk/test/semantics_fixt.pyo
 ${PYSITELIB}/nltk/test/sentiment.doctest
 ${PYSITELIB}/nltk/test/sentiwordnet.doctest
 ${PYSITELIB}/nltk/test/simple.doctest
@@ -792,9 +786,6 @@ ${PYSITELIB}/nltk/test/tag.doctest
 ${PYSITELIB}/nltk/test/tokenize.doctest
 ${PYSITELIB}/nltk/test/toolbox.doctest
 ${PYSITELIB}/nltk/test/translate.doctest
-${PYSITELIB}/nltk/test/translate_fixt.py
-${PYSITELIB}/nltk/test/translate_fixt.pyc
-${PYSITELIB}/nltk/test/translate_fixt.pyo
 ${PYSITELIB}/nltk/test/tree.doctest
 ${PYSITELIB}/nltk/test/treeprettyprinter.doctest
 ${PYSITELIB}/nltk/test/treetransforms.doctest
@@ -855,6 +846,9 @@ ${PYSITELIB}/nltk/test/unit/test_data.py
 ${PYSITELIB}/nltk/test/unit/test_disagreement.py
 ${PYSITELIB}/nltk/test/unit/test_disagreement.pyc
 ${PYSITELIB}/nltk/test/unit/test_disagreement.pyo
+${PYSITELIB}/nltk/test/unit/test_distance.py
+${PYSITELIB}/nltk/test/unit/test_distance.pyc
+${PYSITELIB}/nltk/test/unit/test_distance.pyo
 ${PYSITELIB}/nltk/test/unit/test_freqdist.py
 ${PYSITELIB}/nltk/test/unit/test_freqdist.pyc
 ${PYSITELIB}/nltk/test/unit/test_freqdist.pyo
@@ -867,6 +861,9 @@ ${PYSITELIB}/nltk/test/unit/test_json2cs
 ${PYSITELIB}/nltk/test/unit/test_json_serialization.py
 ${PYSITELIB}/nltk/test/unit/test_json_serialization.pyc
 ${PYSITELIB}/nltk/test/unit/test_json_serialization.pyo
+${PYSITELIB}/nltk/test/unit/test_metrics.py
+${PYSITELIB}/nltk/test/unit/test_metrics.pyc
+${PYSITELIB}/nltk/test/unit/test_metrics.pyo
 ${PYSITELIB}/nltk/test/unit/test_naivebayes.py
 ${PYSITELIB}/nltk/test/unit/test_naivebayes.pyc
 ${PYSITELIB}/nltk/test/unit/test_naivebayes.pyo
@@ -879,6 +876,9 @@ ${PYSITELIB}/nltk/test/unit/test_pl196x.
 ${PYSITELIB}/nltk/test/unit/test_pos_tag.py
 ${PYSITELIB}/nltk/test/unit/test_pos_tag.pyc
 ${PYSITELIB}/nltk/test/unit/test_pos_tag.pyo
+${PYSITELIB}/nltk/test/unit/test_ribes.py
+${PYSITELIB}/nltk/test/unit/test_ribes.pyc
+${PYSITELIB}/nltk/test/unit/test_ribes.pyo
 ${PYSITELIB}/nltk/test/unit/test_rte_classify.py
 ${PYSITELIB}/nltk/test/unit/test_rte_classify.pyc
 ${PYSITELIB}/nltk/test/unit/test_rte_classify.pyo
@@ -903,6 +903,9 @@ ${PYSITELIB}/nltk/test/unit/test_tokeniz
 ${PYSITELIB}/nltk/test/unit/test_twitter_auth.py
 ${PYSITELIB}/nltk/test/unit/test_twitter_auth.pyc
 ${PYSITELIB}/nltk/test/unit/test_twitter_auth.pyo
+${PYSITELIB}/nltk/test/unit/test_util.py
+${PYSITELIB}/nltk/test/unit/test_util.pyc
+${PYSITELIB}/nltk/test/unit/test_util.pyo
 ${PYSITELIB}/nltk/test/unit/test_wordnet.py
 ${PYSITELIB}/nltk/test/unit/test_wordnet.pyc
 ${PYSITELIB}/nltk/test/unit/test_wordnet.pyo
@@ -933,20 +936,17 @@ ${PYSITELIB}/nltk/test/unit/translate/te
 ${PYSITELIB}/nltk/test/unit/translate/test_ibm_model.py
 ${PYSITELIB}/nltk/test/unit/translate/test_ibm_model.pyc
 ${PYSITELIB}/nltk/test/unit/translate/test_ibm_model.pyo
+${PYSITELIB}/nltk/test/unit/translate/test_meteor.py
+${PYSITELIB}/nltk/test/unit/translate/test_meteor.pyc
+${PYSITELIB}/nltk/test/unit/translate/test_meteor.pyo
 ${PYSITELIB}/nltk/test/unit/translate/test_nist.py
 ${PYSITELIB}/nltk/test/unit/translate/test_nist.pyc
 ${PYSITELIB}/nltk/test/unit/translate/test_nist.pyo
 ${PYSITELIB}/nltk/test/unit/translate/test_stack_decoder.py
 ${PYSITELIB}/nltk/test/unit/translate/test_stack_decoder.pyc
 ${PYSITELIB}/nltk/test/unit/translate/test_stack_decoder.pyo
-${PYSITELIB}/nltk/test/unit/utils.py
-${PYSITELIB}/nltk/test/unit/utils.pyc
-${PYSITELIB}/nltk/test/unit/utils.pyo
 ${PYSITELIB}/nltk/test/util.doctest
 ${PYSITELIB}/nltk/test/wordnet.doctest
-${PYSITELIB}/nltk/test/wordnet_fixt.py
-${PYSITELIB}/nltk/test/wordnet_fixt.pyc
-${PYSITELIB}/nltk/test/wordnet_fixt.pyo
 ${PYSITELIB}/nltk/test/wordnet_lch.doctest
 ${PYSITELIB}/nltk/test/wsd.doctest
 ${PYSITELIB}/nltk/text.py
@@ -967,6 +967,9 @@ ${PYSITELIB}/nltk/tokenize/casual.pyo
 ${PYSITELIB}/nltk/tokenize/destructive.py
 ${PYSITELIB}/nltk/tokenize/destructive.pyc
 ${PYSITELIB}/nltk/tokenize/destructive.pyo
+${PYSITELIB}/nltk/tokenize/legality_principle.py
+${PYSITELIB}/nltk/tokenize/legality_principle.pyc
+${PYSITELIB}/nltk/tokenize/legality_principle.pyo
 ${PYSITELIB}/nltk/tokenize/mwe.py
 ${PYSITELIB}/nltk/tokenize/mwe.pyc
 ${PYSITELIB}/nltk/tokenize/mwe.pyo

Index: pkgsrc/textproc/py-nltk/distinfo
diff -u pkgsrc/textproc/py-nltk/distinfo:1.4 pkgsrc/textproc/py-nltk/distinfo:1.5
--- pkgsrc/textproc/py-nltk/distinfo:1.4        Tue Oct 26 11:23:08 2021
+++ pkgsrc/textproc/py-nltk/distinfo    Wed Nov 24 16:00:18 2021
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.4 2021/10/26 11:23:08 nia Exp $
+$NetBSD: distinfo,v 1.5 2021/11/24 16:00:18 wiz Exp $
 
-BLAKE2s (nltk-3.5.zip) = 9ae37aa65a02351d454cb98eb2a3c1375b83bba1024e89b5981b8fd8a958013c
-SHA512 (nltk-3.5.zip) = 93cb2c031bbb6d2f20a3bc739f6a2760b9073483834506b5b828f4562fd41435f30339b998d158ac90792b7908d60ebff31e9ba49b4ed4e8c38bdb0b753fc70a
-Size (nltk-3.5.zip) = 1433531 bytes
+BLAKE2s (nltk-3.6.5.zip) = e339313c2e05f3fdd0c8706346202f1e7dcd010587dbdc3552c4a9a4f1c4facb
+SHA512 (nltk-3.6.5.zip) = afddea56f092bd6e6ee119c769958afbce3f76159d94e8da28da1946e8ab63fa568762ead39e8a6fb6cbe55a548a1056e8b457c50a54e8f8ef6da610ea3bcf4b
+Size (nltk-3.6.5.zip) = 6392760 bytes