Wheelodex » Projects » tokenizers-gt » tokenizers_gt-0.15.2.post0-cp310-none-win32.whl

tokenizers-gt

View on PyPI — Reverse Dependencies (0)

0.15.2.post0 tokenizers_gt-0.15.2.post0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-cp39-none-win_amd64.whl
tokenizers_gt-0.15.2.post0-cp39-none-win32.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-cp39-cp39-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-cp38-none-win_amd64.whl
tokenizers_gt-0.15.2.post0-cp38-none-win32.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-cp38-cp38-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-cp37-none-win_amd64.whl
tokenizers_gt-0.15.2.post0-cp37-none-win32.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-cp37-cp37m-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-cp312-cp312-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-cp311-none-win_amd64.whl
tokenizers_gt-0.15.2.post0-cp311-none-win32.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-cp311-cp311-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-cp310-none-win_amd64.whl
tokenizers_gt-0.15.2.post0-cp310-none-win32.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-cp310-cp310-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-pp39-pypy39_pp73-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-pp38-pypy38_pp73-macosx_11_0_arm64.whl
tokenizers_gt-0.15.2.post0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
tokenizers_gt-0.15.2.post0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
tokenizers_gt-0.15.2.post0-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
tokenizers_gt-0.15.2.post0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
tokenizers_gt-0.15.2.post0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
tokenizers_gt-0.15.2.post0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
tokenizers_gt-0.15.2.post0-pp310-pypy310_pp73-macosx_11_0_arm64.whl

Wheel Details

Project: tokenizers-gt
Version: 0.15.2.post0
Filename: tokenizers_gt-0.15.2.post0-cp310-none-win32.whl
Download: [link]
Size: 2048909
MD5: b0beec7f47a89171c66c83e6a60c6290
SHA256: 77fb3f947be4acf1c6d0b94853bb6d4f680bffebb4622b12901b7b258c434c14
Uploaded: 2024-02-19 01:37:31 +0000

dist-info

METADATA · WHEEL · RECORD

METADATA

Metadata-Version: 2.1
Name: tokenizers-gt
Version: 0.15.2.post0
Author: Anthony MOI <m.anthony.moi@gmail.com>
Author-Email: Nicolas Patry <patry.nicolas[at]protonmail.com>, Anthony Moi <anthony[at]huggingface.co>
Project-Url: Homepage, https://github.com/huggingface/tokenizers
Project-Url: Source, https://github.com/huggingface/tokenizers
Keywords: NLP,tokenizer,BPE,transformer,deep learning
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.7
Requires-Dist: huggingface_hub (<1.0,>=0.16.4)
Requires-Dist: pytest; extra == "testing"
Requires-Dist: requests; extra == "testing"
Requires-Dist: numpy; extra == "testing"
Requires-Dist: datasets; extra == "testing"
Requires-Dist: black; extra == "testing"
Requires-Dist: sphinx; extra == "docs"
Requires-Dist: sphinx_rtd_theme; extra == "docs"
Requires-Dist: setuptools_rust; extra == "docs"
Requires-Dist: tokenizers[testing]; extra == "dev"
Provides-Extra: testing
Provides-Extra: docs
Provides-Extra: dev
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
[Description omitted; length: 5006 characters]

WHEEL

Wheel-Version: 1.0
Generator: maturin (1.4.0)
Root-Is-Purelib: false
Tag: cp310-none-win32

RECORD

Path Digest Size
tokenizers_gt-0.15.2.post0.dist-info/METADATA sha256=-1TjEBdoq--Ge2FWcAutbzAvL1aKZITAVGHbjceodIE 6850
tokenizers_gt-0.15.2.post0.dist-info/WHEEL sha256=zWYjyrndoXjOoElok3d6rrB-N0Fv2I_2mGtMRzMv-Lk 91
tokenizers/decoders/__init__.py sha256=xffrQt9YxtuKLOBEkJnZZPu03ozf3sm-x1KgxF-f0DQ 349
tokenizers/decoders/__init__.pyi sha256=Bmoj3V9FRg7U3nzzJBoaDWXjDDYU54e_7qHTO4J2v64 7311
tokenizers/implementations/base_tokenizer.py sha256=nOUjtXTgDJSdIjDDnBzoWELUnzpRy6yLyVhUOL5t5xY 14624
tokenizers/implementations/bert_wordpiece.py sha256=jRUMSYU1C7nVEDZMKdyG3Vp7-CUongE1kGJhvOP_EHM 5671
tokenizers/implementations/byte_level_bpe.py sha256=iCCZSWW9nwQgfeXeOiwVEX7sLNfPsQxO7CCuEmTaHXw 4411
tokenizers/implementations/char_level_bpe.py sha256=AQaZFT_RK7SW1ubGsFtXOnftit_vWp0fj4-fJnzZn7c 5616
tokenizers/implementations/sentencepiece_bpe.py sha256=WzbvXIt9DNjj5uANx249wxRf6x9uM9bB_a9hg6ybPuw 3781
tokenizers/implementations/sentencepiece_unigram.py sha256=nEsscERNWT-ZdEEH9okt17jsUoh_awBHZhETz3htmHw 7656
tokenizers/implementations/__init__.py sha256=9ZI2cJHPCUCZXP37GNmRHtBzIYiKWzyqky1rtcYdrPw 316
tokenizers/models/__init__.py sha256=opRk2C1KRXY_3CfkVuYeZ8pau3IeKo6vrPUa1BC5lgI 226
tokenizers/models/__init__.pyi sha256=RWy08aNOCI9tTFDbybs-MABXfku67H4w0sfuSUB6YMg 19469
tokenizers/normalizers/__init__.py sha256=HdGKGl2qjqQN520YXGgBn6Ti3qMQ-grvNH9QgzeA-wY 837
tokenizers/normalizers/__init__.pyi sha256=QosdCP3B-MN6bhDBeVcuX07Ztturcvz9x67X33lXDGg 20168
tokenizers/pre_tokenizers/__init__.py sha256=ZWLPWG2nhoTojLyHIDUPRsa1V8drqWufJVjzbRmR7UQ 572
tokenizers/pre_tokenizers/__init__.pyi sha256=uWlLCivyHDGnUp8T6dLMk3qoSlq3kmRwYnS73y3hp5Y 23635
tokenizers/processors/__init__.py sha256=X1OKKtr3IUeede85mI1cLF0MqmDRlr7LNFMYdEy5KMU 316
tokenizers/processors/__init__.pyi sha256=cBI1jIP2DPfF9xDyw-a2OHLLQcLAANFplCjxotcxmZ8 11689
tokenizers/tools/visualizer-styles.css sha256=g98RJucUCjYnPLwXycfcV-dAbHMzqJpnZ7NoP_cCUBk 5020
tokenizers/tools/visualizer.py sha256=Er3CRAqTshUzZttZGsC4b36AGNh7ryNDuTWKHD39tfA 15024
tokenizers/tools/__init__.py sha256=mJLVrHN1FP23Cf4_EJWoYUdh8yTb5BbJFNfa8Ec5vHM 56
tokenizers/trainers/__init__.py sha256=v9CdtoVauwD6b1wAA6R5goSLZslBbPE5vZHM2d8sgKM 256
tokenizers/trainers/__init__.pyi sha256=ZPBCBM4u9PLnJyXklRKTiRRjp2qs_pmI-GR1qr5MTBM 5542
tokenizers/__init__.py sha256=AJ5NdKOV_dZ_SOw9oFF1M4ln0fh1hod-voqzeYHS71U 2715
tokenizers/__init__.pyi sha256=O7p6xv31eVIrojoB5NBFuY9GVG-o7bdx8wbwtIBoskU 39582
tokenizers/tokenizers.cp310-win32.pyd sha256=PlKa9k8Zl88Fwb5hPVjafVNJfHQQnhKbQoXWBjFFg20 4996096
tokenizers_gt-0.15.2.post0.dist-info/RECORD — —