Wheelodex » Projects » curated-tokenizers

curated-tokenizers

View on PyPI — Reverse Dependencies (2)

2.0.0 curated_tokenizers-2.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
curated_tokenizers-2.0.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
curated_tokenizers-2.0.0-cp39-cp39-win_amd64.whl
curated_tokenizers-2.0.0-cp39-cp39-macosx_10_9_x86_64.whl
curated_tokenizers-2.0.0-cp39-cp39-macosx_11_0_arm64.whl
curated_tokenizers-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
curated_tokenizers-2.0.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
curated_tokenizers-2.0.0-cp312-cp312-win_amd64.whl
curated_tokenizers-2.0.0-cp312-cp312-macosx_10_9_x86_64.whl
curated_tokenizers-2.0.0-cp312-cp312-macosx_11_0_arm64.whl
curated_tokenizers-2.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
curated_tokenizers-2.0.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
curated_tokenizers-2.0.0-cp311-cp311-win_amd64.whl
curated_tokenizers-2.0.0-cp311-cp311-macosx_10_9_x86_64.whl
curated_tokenizers-2.0.0-cp311-cp311-macosx_11_0_arm64.whl
curated_tokenizers-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
curated_tokenizers-2.0.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
curated_tokenizers-2.0.0-cp310-cp310-win_amd64.whl
curated_tokenizers-2.0.0-cp310-cp310-macosx_10_9_x86_64.whl
curated_tokenizers-2.0.0-cp310-cp310-macosx_11_0_arm64.whl

Wheel Details

Project: curated-tokenizers
Version: 2.0.0
Filename: curated_tokenizers-2.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Download: [link]
Size: 774766
MD5: 3ddccaddb09d131c66204ee5e952eab5
SHA256: 6acd7931ff5ff620a6f84ea8279312ad1d1e87a1e2014d4367c9e58b8ef23e0d
Uploaded: 2024-04-15 17:19:27 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · zip-safe

METADATA

Metadata-Version: 2.1
Name: curated-tokenizers
Version: 2.0.0
Summary: Lightweight piece tokenization library
Author: Explosion
Author-Email: contact[at]explosion.ai
Home-Page: https://github.com/explosion/curated-tokenizers
License: MIT
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Programming Language :: Cython
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.9
Requires-Dist: regex (>=2022)
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 927 characters]

WHEEL

Wheel-Version: 1.0
Generator: bdist_wheel (0.43.0)
Root-Is-Purelib: false
Tag: cp39-cp39-manylinux_2_17_x86_64
Tag: cp39-cp39-manylinux2014_x86_64

RECORD

Path Digest Size
curated_tokenizers-2.0.0.dist-info/zip-safe sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs 1
curated_tokenizers-2.0.0.dist-info/RECORD — —
curated_tokenizers-2.0.0.dist-info/LICENSE sha256=Sdj--WzZJxnLUkLvTE-VTtwAikb898qr7K4U28xES1A 2642
curated_tokenizers-2.0.0.dist-info/top_level.txt sha256=EKBDsRk94HtBI7QQVDtt8uMqpwI--el9tTFO5T99Yjw 19
curated_tokenizers-2.0.0.dist-info/METADATA sha256=L0lrPnkB_7rlNYLXz7ifs-_JhuGmttdNl0Yv2TfBV18 1911
curated_tokenizers-2.0.0.dist-info/WHEEL sha256=rY0Y6THYM7EImsHfF-zs67o8pQciAsMw9_YuSvftjrQ 148
curated_tokenizers/_bbpe.cpython-39-x86_64-linux-gnu.so sha256=Fkxc7XykFPWvecSy0511lO_boCTeat-Ukmygh827mu0 170160
curated_tokenizers/config.h sha256=bSpQjNpV_vzGjOb2J5zcjg1H7z1_wOmWBFjIY1JBc7k 156
curated_tokenizers/util.hh sha256=VT-IgsTPBVqCxPvmh2rDT6qCzbxyWLgx22J9a2OmTuc 333
curated_tokenizers/_spp.cpython-39-x86_64-linux-gnu.so sha256=qe_JYY0R_fk6so0o74oZGWPY8veuOHJfmejAsHVjYpo 1263456
curated_tokenizers/_spp.pyx sha256=yLFPj1k5m093V_KRKa2jZhxzJ8LhyrtL-rpAKRwbQJY 8321
curated_tokenizers/_wordpiece.cpython-39-x86_64-linux-gnu.so sha256=IMLIcroE3MMgBSmbjYUEYFN66QxjZ_5w0h74frEuHXs 120752
curated_tokenizers/_wordpiece.pxd sha256=j72fwJnyhdk1vD-uWiDZev288a-e_KVtfCqm6TYfLeY 655
curated_tokenizers/types.py sha256=rtD_qNf50uhUL_xSWNfmRQq8b311mf25nqnsD-2gsno 157
curated_tokenizers/merges.hh sha256=uzHG-HQ6NmgXfNzJfxv9DLRBMq9GmI4F3zYCgH8mosE 1287
curated_tokenizers/merges.cc sha256=svdmhrwsFGoBoifWB-IRBZHmfgcSKk4ruyWzGQG-eHc 2145
curated_tokenizers/_bbpe.pxd sha256=OVhMk1A6-NStdb7CIVZKhDHrgCvMyeaHbtz25_rz_80 347
curated_tokenizers/_bbpe.pyx sha256=hzziFcnttv3NIhsbh-hkaQx9S18-L-gdbF5l0ISClEg 7537
curated_tokenizers/wordpiece.cc sha256=m6XOiqoslH6ieBnavtI2TAWSgjMNuft0bTm9U6Q2vmM 1083
curated_tokenizers/_wordpiece.pyx sha256=u5XNm3D8C0w_C-5riVmfpD0ywVNcXnTJGxJ40wkdJJk 5250
curated_tokenizers/py.typed sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
curated_tokenizers/_util.py sha256=K8gvB9KTl0QFbDK34ZpmnD2b-_pQ4uFYpAlFrKfRFt8 1025
curated_tokenizers/_spp.pxd sha256=4CtoM_uyL3IVohGG3W4QAFjQquCVfyBBjr04DY8aaUc 2150
curated_tokenizers/__init__.py sha256=Dtn4OlRTn_YX-6pOTp47sG9PhJBTjtvk3Cb6CUM283E 120
curated_tokenizers/wordpiece.hh sha256=YShSvQUOxmye0W6hzlES44reBstFRP7oCZGH94UB-S0 1011
curated_tokenizers/tests/test_word_piece_processor.py sha256=mJXw-1tpkwzYj-k-FpqHVvUuW1-LMvjmCOHxTFQeY4Q 2479
curated_tokenizers/tests/toy-word-pieces.txt sha256=pX4mNd-dx7EsFcMJy90uHFU297daQ3kfRBH30UV9QQw 31
curated_tokenizers/tests/toy.model sha256=aOgs9MnD7wvh8lp8riyDb8r6fTyDgh-5hkEeVnea0eI 253270
curated_tokenizers/tests/test_bbpe_processor.py sha256=19_Em4vMbykSBL8N8T6eZ4A3DGTnWOIN1Vf_KYMR9RU 3858
curated_tokenizers/tests/robbert-merges-1000.txt sha256=ouOja6engvjR8K136noUyvaRihzu_QyDh8BsVnhsr5c 6291
curated_tokenizers/tests/conftest.py sha256=l7gBiNGtU1QLfgm2mdAojCMoxvGOD-BIXOGjiO7i6Gc 1114
curated_tokenizers/tests/incorrect-merges.txt sha256=J3bMlGl31fOTeHCw6Hkf0jViEyh86ca-BiGM37JkFZY 19
curated_tokenizers/tests/robbert-vocab-1000.json sha256=OA7hBeYFTAIiNV01E0qG8amdTkA-bzOc_6MGkVQPjOA 16815
curated_tokenizers/tests/troonrede.txt sha256=RHz6WLyDJzVbQPuNFX4_I0sE5Gaf3W4PFYiaQwlGK6E 69708
curated_tokenizers/tests/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
curated_tokenizers/tests/test_sp.py sha256=wu7LDKTfj-np8WJobWngUS5vIvk7IpF942qs3CnOJxw 4448
curated_tokenizers/tests/compat.py sha256=VQ-PRA08sq26jBiYDhJKXhngaddyrwzwmFooeyEfyiY 159

top_level.txt

curated_tokenizers

zip-safe