Wheelodex » Projects » polyglot-tokenizer

polyglot-tokenizer

View on PyPI — Reverse Dependencies (0)

2.0.2 polyglot_tokenizer-2.0.2-py2.py3-none-any.whl

Wheel Details

Project: polyglot-tokenizer
Version: 2.0.2
Filename: polyglot_tokenizer-2.0.2-py2.py3-none-any.whl
Download: [link]
Size: 323283
MD5: 944fefbadb4c7afdd67e45ad8684451b
SHA256: 0ddb1101fa212c35795ca916adc02436e4ea7723182449fab5185f8f94caf873
Uploaded: 2021-06-09 05:31:30 +0000

dist-info

METADATA · WHEEL · RECORD · entry_points.txt

METADATA

Metadata-Version: 2.1
Name: polyglot-tokenizer
Version: 2.0.2
Summary: Tokenizer for world's most spoken languages and social media texts like Facebook, Twitter etc.
Author: irshadbhat
Author-Email: bhatirshad127[at]gmail.com
Home-Page: https://github.com/irshadbhat/polyglot-tokenizer
Project-Url: Repository, https://github.com/irshadbhat/polyglot-tokenizer
License: MIT
Keywords: nlp,polyglot,tokenizer
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Requires-Dist: pbr (<3.0,>=2.0)
Requires-Dist: six (<2.0,>=1.12)
Description-Content-Type: text/x-rst
[Description omitted; length: 4521 characters]

WHEEL

Wheel-Version: 1.0
Generator: poetry 0.12.7
Root-Is-Purelib: true
Tag: py2.py3-none-any

RECORD

Path Digest Size
polyglot_tokenizer/__init__.py sha256=AVycOZE8fRI6yGQ1LsNgeNpy8LwPIi5p03U4QcTpFTo 3921
polyglot_tokenizer/__main__.py sha256=riWIFXEF8tvdPN1h1_g5T2f84mawFiYiI8h6ClTX0fg 335
polyglot_tokenizer/armenian_tokenizer.py sha256=--j4FyITyajiz_e2Anrdp-i73c_u7yDL9GW4fisBRqI 1266
polyglot_tokenizer/base.py sha256=6AFqOPT_wBhzmt7FJZDCb3FnnHdaon2My3KsSdXTbUQ 11886
polyglot_tokenizer/cyrillic_tokenizer.py sha256=eTRgY9GVVqUmQmDmjRMvcVCR42cnP727c0ICoMG5TEI 766
polyglot_tokenizer/data/DOMAINS sha256=iGkoah7e_Wm-pa8ijR49gjVMnYC8ive912tXVZvPT6s 293
polyglot_tokenizer/data/EMOTICONS sha256=xDWadPjU0GNG0TY7fwTd--MeD_Q0WgEpeJgfZTkruE0 3847
polyglot_tokenizer/data/nonbreaking_prefixes.ca sha256=3UpNuTCsd9fohsAaA4pSWG5TEJQ38Y7BMybQUMlWL8Q 249
polyglot_tokenizer/data/nonbreaking_prefixes.cs sha256=jGrRQWDw072t3fi6AWXZErY43lU4tUtudybjWglElek 1893
polyglot_tokenizer/data/nonbreaking_prefixes.de sha256=SJMrjU_mPM5927bWJhVdaABylPJSrARgGQHrGbSMK5A 1030
polyglot_tokenizer/data/nonbreaking_prefixes.el sha256=AWT4Yd-g8aV4SxGLoE35tEOyl6vOzp2-QKbYnVIJY-k 16832
polyglot_tokenizer/data/nonbreaking_prefixes.en sha256=NPGLV7qNlLRHakMFJ23Szc250fcUx5VMKYEVa3mkCRc 812
polyglot_tokenizer/data/nonbreaking_prefixes.es sha256=gsPBS2R7I-aHxFVLtzShlLWAEzZwtJngk_ZDPExL4h4 424
polyglot_tokenizer/data/nonbreaking_prefixes.fi sha256=K3xF_4RWJZ9fG-9LPt1PmTIk8quuei8c0pZ9hvH7Ut8 680
polyglot_tokenizer/data/nonbreaking_prefixes.fr sha256=qT15o7ow8SSmnEoGtrYf5ymBpF8qvgMckbftJUDUOxk 562
polyglot_tokenizer/data/nonbreaking_prefixes.ga sha256=6L3FNg_qda3nK92ESWgncqsmQ_bjEy9BemlknD-SckE 126
polyglot_tokenizer/data/nonbreaking_prefixes.hu sha256=Ezky9YNbHtxqaSGUCaVvuTaWN2KhCRN_6kq_fhvkFRI 297
polyglot_tokenizer/data/nonbreaking_prefixes.is sha256=cPMXu5LUrK7btsS_oG0F2yxVPQhz9436AS1Oz8hLprg 969
polyglot_tokenizer/data/nonbreaking_prefixes.it sha256=EAFxtzU2B5YiG6fSFDJlKhLz3itN2RQd0DmbhdnVk3c 640
polyglot_tokenizer/data/nonbreaking_prefixes.lt sha256=BNQFT8KI5UAcOdGdT62fwqmvZNJ89MRUwl-rK4KQxV8 1683
polyglot_tokenizer/data/nonbreaking_prefixes.lv sha256=wm11tyW-UkD7aSkEvW0UY5S10AYxJ21xQ_uU2hBrsbk 338
polyglot_tokenizer/data/nonbreaking_prefixes.nl sha256=pcrEtOKjk1wu5k9HoE_Ot7OK9QBuboe0LJpgRFqS-_A 378
polyglot_tokenizer/data/nonbreaking_prefixes.pl sha256=dJ7knTgm760wNXTphDnuwcrRtLU9fDuPNq9NNLC6F_0 1146
polyglot_tokenizer/data/nonbreaking_prefixes.pt sha256=_A6mtVJSX4AR3hTWVtKrJRur7kYy47TSz-W63ppAMXA 658
polyglot_tokenizer/data/nonbreaking_prefixes.ro sha256=Q77t-YwJLaYtTSk4kTDVYBmW7RixoRK4CGH8n3cwsEc 104
polyglot_tokenizer/data/nonbreaking_prefixes.ru sha256=fl7iHi_9UdP9LweYU0NoZzF5dfPhF8fxG6rnsI4Qm5c 1418
polyglot_tokenizer/data/nonbreaking_prefixes.sk sha256=ApqiCOjbyZNKP2DFlK2g7F0Ad7mSRtnx9ydfMtCFwh0 2460
polyglot_tokenizer/data/nonbreaking_prefixes.sl sha256=UlI0-wnD7Av8udah4PHUg5kTeOJOt5UiBvziM8nyCH0 248
polyglot_tokenizer/data/nonbreaking_prefixes.sv sha256=Osl1S4yMaFspT7cJtL8Ul9MOXFLaB7lOMENlB8doU9o 117
polyglot_tokenizer/data/nonbreaking_prefixes.ta sha256=wSX8ojerzqsv829UqJD72wNvfvPjSK08V1GjTIxKvjs 1733
polyglot_tokenizer/data/nonbreaking_prefixes.yue sha256=uMhscg8yjloHIPhlqKiOeZNaZrUvJ46lKKx9h1IzZTI 92
polyglot_tokenizer/data/nonbreaking_prefixes.zh sha256=uMhscg8yjloHIPhlqKiOeZNaZrUvJ46lKKx9h1IzZTI 92
polyglot_tokenizer/georgian_tokenizer.py sha256=Uysi04WvbVNLd5iwy0v1GPT7drG-mTWfn1m-SF8sdcM 1344
polyglot_tokenizer/greek_tokenizer.py sha256=xaJazMyTUYxr-r9cGmHLxz_zF-jOOyqEuSN2Og_2bx4 1482
polyglot_tokenizer/hebrew_tokenizer.py sha256=xGDw61hDNsf6hrmGYwztZOS--sI0Ii5bOIsp-ihPztw 1213
polyglot_tokenizer/indic_tokenizer.py sha256=fPv5B6IuDjbkn0Lqu5RYF5Bs30SFsO5MaZAWzNfTCtk 11691
polyglot_tokenizer/roman_tokenizer.py sha256=d7z1ypik_IAUmrFhSFu1AxBae8mRa3ZD3T-jSDkynn4 5322
polyglot_tokenizer/tests/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
polyglot_tokenizer/tests/ben.txt sha256=1fYzrWST1RUkFF5UaGRth4LVn_U0nW_7bpo5-khA0jU 22745
polyglot_tokenizer/tests/eng.txt sha256=hUuR-vHiWs6K213uRTD7SU5Fy5OZkDW5iUu9GufSymY 11933
polyglot_tokenizer/tests/guj.txt sha256=kwoeQ0hK9l-YSYXpo2h1eJ3-GawkD3-6iuBz3gOPrSI 24170
polyglot_tokenizer/tests/hin.txt sha256=Chc_zVZNqP-WtH_M5GtXzBqs1m3hH-yfbhrkN1nhUD8 22797
polyglot_tokenizer/tests/kan.txt sha256=ihUDAfPHXx1VdsdRjIM0podt37hC1iRdQ00GCLLncQg 31932
polyglot_tokenizer/tests/mal.txt sha256=7Q4LWXwnjJVuhplMRDTfwWm1lY1-sOfizsxuR7Ep4K8 25476
polyglot_tokenizer/tests/ori.txt sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
polyglot_tokenizer/tests/pan.txt sha256=TZvdYPs62uzQvRS54gB9zWCXHXgD3sKAazD8sGYJB6w 17290
polyglot_tokenizer/tests/tam.txt sha256=y4XIFIuj6UFTAUEifnmJzd16N4Kx8DX0ZJLBz5vWz1M 26633
polyglot_tokenizer/tests/tel.txt sha256=V0Pn2wJlNsvLDGIYnjL3HzlqP-8KIOP9zgw02ur0-7o 23544
polyglot_tokenizer/tests/test_tokenizer.py sha256=XAcIFGO0fKV6g5GQZzplc6bRWYVF7IHXa_4IiM-2xkI 1574
polyglot_tokenizer/tests/urd.txt sha256=lPETL2EIwsQ_DdsiXfGoQYKGirKD0FbikigYgxvdoZg 19817
polyglot_tokenizer/tokenizer.py sha256=xVflfCNKuFRYlEo-w2Ck2RNg3DRphL3IwhivuUHyWgo 2374
polyglot_tokenizer-2.0.2.dist-info/entry_points.txt sha256=qfXpOqnlXQs-xFnZKGchLfdfxjk3hVz1Ikfr1LIw4ls 62
polyglot_tokenizer-2.0.2.dist-info/LICENSE sha256=Q2p7ek-A9ywaJPMM-1t7Fv2-DIdl9mOiRQoWNcSK2hE 1078
polyglot_tokenizer-2.0.2.dist-info/WHEEL sha256=iLx-ZLqeEOousSGYYVht1uRLc9SimUvivVngwGv2iPE 88
polyglot_tokenizer-2.0.2.dist-info/METADATA sha256=koiP8mxY_WmTscaVxDovb7eirQ9yewqC6rV2iNyp7B4 7760
polyglot_tokenizer-2.0.2.dist-info/RECORD — —

entry_points.txt

[console_scripts]
polyglot-tokenizer = polyglot_tokenizer:main