Wheelodex » Projects » epochraft

epochraft

View on PyPI — Reverse Dependencies (0)

0.1.0.dev20231107 epochraft-0.1.0.dev20231107-py3-none-any.whl

Wheel Details

Project: epochraft
Version: 0.1.0.dev20231107
Filename: epochraft-0.1.0.dev20231107-py3-none-any.whl
Download: [link]
Size: 40109
MD5: ffeebe545e246f25f96e7ffa884769ef
SHA256: c007e78581e0b1931b6a7354a6befd533cd3f701e8ffa8ba9f5a1c104316f1d6
Uploaded: 2023-11-07 06:40:34 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt

METADATA

Metadata-Version: 2.1
Name: epochraft
Version: 0.1.0.dev20231107
Summary: Supercharge Your LLM Training with Checkpointable Data Loading
Author: Takuya Akiba
Author-Email: takuya.akiba[at]stability.ai
Project-Url: repository, https://github.com/iwiwi/epochraft
License: MIT License Copyright (c) 2023 Takuya Akiba Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Requires-Dist: torch
Requires-Dist: numpy
Requires-Dist: braceexpand
Requires-Dist: smart-open
Requires-Dist: black; extra == "development"
Requires-Dist: blackdoc; extra == "development"
Requires-Dist: flake8; extra == "development"
Requires-Dist: isort; extra == "development"
Requires-Dist: mypy; extra == "development"
Requires-Dist: pytest; extra == "development"
Requires-Dist: mosaicml-streaming; extra == "development"
Requires-Dist: transformers; extra == "development"
Requires-Dist: sentencepiece; extra == "development"
Requires-Dist: cbor2; extra == "development"
Provides-Extra: development
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 5956 characters]

WHEEL

Wheel-Version: 1.0
Generator: bdist_wheel (0.41.3)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
epochraft/__init__.py sha256=swBRKJfnx_pqtiEqmc0injki7KcFmKyXm0BJsUKOIHA 553
epochraft/base.py sha256=05RmrrHcGwr-kgb20Gn88agZYtuEJxLKdS9TbWDzhac 10753
epochraft/caching.py sha256=xY_uJyW79IAJfmZPX65hbqOSSv5TxOlvEIJdMjKnoaY 4360
epochraft/testing.py sha256=jUuibs7HVMqTUTm5luE71C75ubYUI3tb-3587d7Ay6A 3485
epochraft/version.py sha256=xMUsI6SPP6qBE2aco2_l6pGEAh77XCUeWDLdXcjh1Pw 33
epochraft/combinations/__init__.py sha256=DLmJxtKrticE7nP0eARNTXCvkLCR4RdmMG5C1l0Ju_o 135
epochraft/combinations/concat.py sha256=Qi9ovbLHwKgLFD9sWVeHSr9AxzfRavI4Psy9gi3I58k 2265
epochraft/combinations/interleave.py sha256=YaxwXWfTWiyQ6cCTQ2p7ugt8JefnCoL3rsnuIl9ii-c 3394
epochraft/sources/__init__.py sha256=EWKtQsBTw_gFJpvuqvxyTGfvlW9RjOwVsTWuBQpW13Q 251
epochraft/sources/iterable.py sha256=EUARhFzBRzQIXVea2IpL4ixGZHynFHWlDftqOGnOGkE 1947
epochraft/sources/mosaicml.py sha256=w7xZD_xKydM7J7pl1--0EpNLHbHkm9OTttxk7Q-pS8k 3089
epochraft/sources/sequence.py sha256=G6sR0t35kQXHa-ylIlAMfUtecnKRYIW5OwoJ7mYepNE 2014
epochraft/sources/files/__init__.py sha256=aBXTdYLIOa5jfxPn23qSz2F2rsEvdgzVTZo__2iemB4 63
epochraft/sources/files/dataset.py sha256=1e2UWJmZEHvRhSpbdGhRnz6HMTdJolYey9fEfHXhFYs 2131
epochraft/sources/files/delay_handler.py sha256=8168eSngujQp6O_Miba6GMRMsbJvejFEcTeNVZMio2c 2843
epochraft/sources/files/generators.py sha256=Wjtx6yYP4i3wjJslbtHYjxvQGsQ_9nIoXGZNWC-rZm8 6172
epochraft/sources/files/shard_reader.py sha256=YsAQP96-Uv6I4EmqLBksxOg-KNSftAAlbgWv9ps9gWc 3582
epochraft/sources/files/shards_mux.py sha256=00V3svQbfwxn2QQKL_sunKrrJm3MvjFBOgkNJatUUx0 5925
epochraft/transforms/__init__.py sha256=4mV3YxhpN2EB33X_IrOpT67I-SlgKrPZbadKWZBQ96A 621
epochraft/transforms/basic/__init__.py sha256=XRvyF8ZUjnLV9u12mQDjIa0h8oBEeTCdA4NVjNpSesM 388
epochraft/transforms/basic/batch.py sha256=aPe77jXG6txhKHbgyupAR-lT6IuZtLWE1RK4Augev_o 1325
epochraft/transforms/basic/count.py sha256=bDK5lF2fIvQiUniHcxnhORez-ETQ6CiALYiXhIvGZqc 1783
epochraft/transforms/basic/filter_map.py sha256=BS5K-U4bwm96RgcX647V62Rk36ZqhWoa-wEpDtiUeRY 1652
epochraft/transforms/basic/shuffle.py sha256=Oh9KQvsUmUSSr-3uTgBSzZSwAQgPNtAJo9axLpB47WU 2290
epochraft/transforms/basic/stride.py sha256=wni7g5Msyw1SWsRUmaEIdmaBsWtc6AG_rPdFcRSvK34 1766
epochraft/transforms/basic/parallel_filter_map/__init__.py sha256=uDHJSuUbTUIVWZGHNHv6V3XQP12g-eMiqM0gSa2Q2mA 154
epochraft/transforms/basic/parallel_filter_map/dataset.py sha256=BBt8_aEAGFTdI95aWn4c9FE4umWLCLsL8rmm6zBuoY4 4477
epochraft/transforms/basic/parallel_filter_map/imap.py sha256=gpZkMrxWPAM6dNVZz-Z7ty3VmVEQTobzSOnperfkocA 9390
epochraft/transforms/language_modeling/__init__.py sha256=-JdgXiVbF-nHNGT7dIrlpFJgwiMfAOfy5ElEINg9CQ0 593
epochraft/transforms/language_modeling/bos_eos.py sha256=_nnC_SUrFuOYROQ_IxdGaP2YejPmS_IkIFRao0wMaLk 4155
epochraft/transforms/language_modeling/chunk.py sha256=fcZOrwPuhXkWwvrzb8PiP7E3gId9xVpfA5Kbddqr8Og 2290
epochraft/transforms/language_modeling/concat_chunk.py sha256=Q1CSeJOiX0FxihJY86Rl9Dm3_NcB2SVqHIXh1CE3Ml0 1955
epochraft/transforms/language_modeling/pack_chunk.py sha256=PfcGizcXB1dj0LVgxveJCt1OicHNxdB6JE5cnrwEFpo 3499
epochraft/transforms/language_modeling/padding.py sha256=PkOx_KYReZVJYbIdV4Ev5EYX5oak5nWYSwXygMRsaVI 1246
epochraft/transforms/language_modeling/tokenization.py sha256=pcjAbaAV8UfYLVStaZiTqYnx68eBbLJ6S1tJVZk_55w 1335
epochraft/transforms/language_modeling/tokenizer_utils.py sha256=qUD1jiFxYGZH7Er1tf6qoaYuXkgEZk0G6lFv3_BZh7o 4029
epochraft-0.1.0.dev20231107.dist-info/LICENSE sha256=LTedoJIxoK9ShJ1r9_p0u34Q_JDVOIyGSg8-wgzLctw 1069
epochraft-0.1.0.dev20231107.dist-info/METADATA sha256=P9HTEc7o0LjoUyZfAGb2gyF8ZozBJu58FJubSFgs0j4 9007
epochraft-0.1.0.dev20231107.dist-info/WHEEL sha256=Xo9-1PvkuimrydujYJAjF7pCkriuXBpUPEjma1nZyJ0 92
epochraft-0.1.0.dev20231107.dist-info/top_level.txt sha256=Fzri4Qf6Xs7OO2DmG5_Ya41xkUwhw-gw06JnTk1D7KA 10
epochraft-0.1.0.dev20231107.dist-info/RECORD — —

top_level.txt

epochraft