Wheelodex » Projects » lighteval

lighteval

View on PyPI — Reverse Dependencies (2)

0.8.1 lighteval-0.8.1-py3-none-any.whl

Wheel Details

Project: lighteval
Version: 0.8.1
Filename: lighteval-0.8.1-py3-none-any.whl
Download: [link]
Size: 411130
MD5: 0ca050453463e940f4b2e51721e871c8
SHA256: 420d142c079bc9256bd15e612a9744e2e465ef3823be1683ad889ad24101068b
Uploaded: 2025-03-24 10:14:25 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · entry_points.txt

METADATA

Metadata-Version: 2.4
Name: lighteval
Version: 0.8.1
Summary: A lightweight and configurable evaluation package
Author-Email: Nathan Habib <nathan.habib[at]huggingface.com>, Clémentine Fourrier <clementine[at]huggingface.com>, Thomas Wolf <thom[at]huggingface.com>
Maintainer-Email: Nathan Habib <nathan.habib[at]huggingface.com>, Clémentine Fourrier <clementine[at]huggingface.com>
Project-Url: Homepage, https://github.com/huggingface/lighteval
Project-Url: Issues, https://github.com/huggingface/lighteval/issues
License: MIT License
Keywords: evaluation,nlp,llm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.10
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Requires-Dist: transformers (>=4.38.0)
Requires-Dist: accelerate
Requires-Dist: huggingface_hub (>=0.23.0)
Requires-Dist: torch (<3.0,>=2.0)
Requires-Dist: GitPython (>=3.1.41)
Requires-Dist: datasets (>=2.14.0)
Requires-Dist: numpy (<2)
Requires-Dist: typer
Requires-Dist: termcolor (==2.3.0)
Requires-Dist: pytablewriter
Requires-Dist: rich
Requires-Dist: colorlog
Requires-Dist: aenum (==3.1.15)
Requires-Dist: nltk (==3.9.1)
Requires-Dist: scikit-learn
Requires-Dist: sacrebleu
Requires-Dist: rouge_score (==0.1.2)
Requires-Dist: sentencepiece (>=0.1.99)
Requires-Dist: protobuf (==3.20.*)
Requires-Dist: pycountry
Requires-Dist: fsspec (>=2023.12.2)
Requires-Dist: litellm; extra == "litellm"
Requires-Dist: diskcache; extra == "litellm"
Requires-Dist: text-generation (>=0.6.0); extra == "tgi"
Requires-Dist: optimum (==1.12.0); extra == "optimum"
Requires-Dist: bitsandbytes (>=0.41.0); extra == "quantization"
Requires-Dist: auto-gptq (>=0.4.2); extra == "quantization"
Requires-Dist: peft (==0.3.0); extra == "adapters"
Requires-Dist: nanotron; extra == "nanotron"
Requires-Dist: tensorboardX; extra == "nanotron"
Requires-Dist: tensorboardX; extra == "tensorboardx"
Requires-Dist: vllm (>=0.7.0); extra == "vllm"
Requires-Dist: ray; extra == "vllm"
Requires-Dist: more_itertools; extra == "vllm"
Requires-Dist: ruff (==v0.2.2); extra == "quality"
Requires-Dist: pre-commit; extra == "quality"
Requires-Dist: pytest (==7.4.0); extra == "tests"
Requires-Dist: lighteval[accelerate,extended_tasks,math,multilingual,quality,tests]; extra == "dev"
Requires-Dist: hf-doc-builder; extra == "docs"
Requires-Dist: watchdog; extra == "docs"
Requires-Dist: langdetect; extra == "extended-tasks"
Requires-Dist: openai; extra == "extended-tasks"
Requires-Dist: tiktoken; extra == "extended-tasks"
Requires-Dist: s3fs; extra == "s3"
Requires-Dist: stanza; extra == "multilingual"
Requires-Dist: spacy[ja,ko,th]; extra == "multilingual"
Requires-Dist: jieba; extra == "multilingual"
Requires-Dist: pyvi; extra == "multilingual"
Requires-Dist: latex2sympy2_extended (==1.0.6); extra == "math"
Provides-Extra: litellm
Provides-Extra: tgi
Provides-Extra: optimum
Provides-Extra: quantization
Provides-Extra: adapters
Provides-Extra: nanotron
Provides-Extra: tensorboardx
Provides-Extra: vllm
Provides-Extra: quality
Provides-Extra: tests
Provides-Extra: dev
Provides-Extra: docs
Provides-Extra: extended-tasks
Provides-Extra: s3
Provides-Extra: multilingual
Provides-Extra: math
Description-Content-Type: text/markdown
Dynamic: license-file
License-File: LICENSE
[Description omitted; length: 5691 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (77.0.3)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
lighteval/__init__.py sha256=5NEYneDuYWDp83SJx108iOr1b-T08TkewQevIIeJK4E 1206
lighteval/__main__.py sha256=HiEs0QpUS95d3evosxf7nL5hA_fw4FmYOkfGXgmypI0 3024
lighteval/data.py sha256=q-tGrfYeIu7M41_XAVhXmaPZbwlpTuQ_JuUU-ofE8dk 13795
lighteval/main_accelerate.py sha256=auBI8Wdn5r4WFf_fqEpmT1eO_CHMYWW0vxIjNM7tOJs 9223
lighteval/main_baseline.py sha256=FGvv4249Ra3_YxAANWyOCxpGlB-Jp4QcLuwMNCyamik 4916
lighteval/main_endpoint.py sha256=bczWm6bzJ7rojpGu2Z2Ql9HnMil05s_438DIX1myKrc 24094
lighteval/main_nanotron.py sha256=DkIWlyBILSichvob9dFZg_k7JtUmyu6i9BAPIA5dsaQ 4781
lighteval/main_sglang.py sha256=AdVPJecbvlGgEwFYsr1QQV7pF2NtTwKyApECRPaCTd0 6311
lighteval/main_tasks.py sha256=8vtcxXHpFac88CBIyHn6fKj_iSBSq07K6IZ8BtAWKhY 3554
lighteval/main_vllm.py sha256=OHOZQD2o8_ZIze1i9bV4J6_kpAPKXXUsjSzgpa3vgkI 6709
lighteval/pipeline.py sha256=xhWS9Ev710_3bDveixSolK8rva360q3RKPhjv4ZDsuQ 24993
lighteval/config/lighteval_config.py sha256=oxa1lmQEv-4AIjpeM3Ljt5tZWLwxG3Cix5B1HVZPK48 3357
lighteval/logging/evaluation_tracker.py sha256=p_86xx6esHLvBOFk72lz-DCNcTs3nEPKGrg4stnoZxQ 33314
lighteval/logging/info_loggers.py sha256=moMto7XqEdddgh0kN0Oksdej18GhY-6UuZO2wnaRv3Q 29294
lighteval/metrics/__init__.py sha256=XcqtKnBGQeYp4V_pPjPVIUsZekDLkn2IxOhHliv7o8k 9423
lighteval/metrics/dynamic_metrics.py sha256=FlXtthsbLCVvnvtHZFZX-xPXz66TSswal0XCJiZIeWo 12326
lighteval/metrics/judge_prompts.jsonl sha256=PfMpr2gGvHRFp66RepgZ0YuiXnPBR22P7yjeLA__uv4 10451
lighteval/metrics/llm_as_judge.py sha256=oB56NM-xTxywSXRi-LDVGhQVpoDdxyVLla0E4z55NlM 13633
lighteval/metrics/metrics.py sha256=pvFqTtux3bzkeBuvpcXL0cElBTJKxbkZE-mAiOwEXpM 24919
lighteval/metrics/metrics_corpus.py sha256=TcifHObv1HZZRsLhB5Ngt3uUG8D2edBoP_CqA6fvpn4 6705
lighteval/metrics/metrics_sample.py sha256=J8wrK-DMmTf3rZNC9bCPE2xnohpCobkI18u0U8PdWpM 49652
lighteval/metrics/normalizations.py sha256=zRqbQ1U5j9TlfynvYe6HyTQyvNnAsoJBPD6ObPa8lwg 16691
lighteval/metrics/sample_preparator.py sha256=cHzpxy8SZXjetiXuNDSX42vwoPsvQd9qGUp3Bjg1Eb8 5752
lighteval/metrics/stderr.py sha256=2Y51BYaaIR4zLwGeQLH5wTjKCi4M0c66bXAwWP3r320 4411
lighteval/metrics/harness_compatibility/drop.py sha256=JVQJewQFHjqsFVRPVYxdeCUzWpYHUGJmTicN4YI9YIM 7172
lighteval/metrics/harness_compatibility/truthful_qa.py sha256=7PfI5-LMEVdXOjvJqkgnV3FeHloMmlECYjR60m-BAOU 2832
lighteval/metrics/imports/__init__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/metrics/imports/bert_scorer.py sha256=XzV2iPTR3gi6Lb6FDondF4eVfbaL-DcUwAJQpocQUPw 18493
lighteval/metrics/imports/data_stats_metric.py sha256=Vyf8EwMeMSS7zXWRV2AterDEt4M6uHxlDHKoNbph21I 5887
lighteval/metrics/imports/data_stats_utils.py sha256=eca-eXrrcyjZwcJbW0bmSYAY3jt-n827gKbIsTJnW58 6116
lighteval/metrics/imports/summac.py sha256=cW3YpHUt-g6qHxJz7ouU2WH9UFblJygXnzYp0jnkkb8 11683
lighteval/metrics/utils/__init__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/metrics/utils/extractive_match_utils.py sha256=BWXe4Pb1qdGgP0txyYkRU-1GO2mDD_WRQpigeOiqSKA 25581
lighteval/metrics/utils/linguistic_tokenizers.py sha256=8rqgyKEZuGxct85rY90AeXD6d5AeKOZ0no9ywcCimKI 11283
lighteval/metrics/utils/math_comparison.py sha256=UPUYV7ep1Mur_A0tCcUhsWDhNA4LJu8qU58Wf4hHgFo 21862
lighteval/metrics/utils/metric_utils.py sha256=jmWjzVsySC7mEvAeHbN_z6e2SnsPaEZD98FN0IpMWWE 3471
lighteval/models/abstract_model.py sha256=El4vYOh2qQ-zb7GqXwKJmwJBizVMgM9stklaf9L_mw4 8435
lighteval/models/litellm_model.py sha256=fYaxwgg9RjJ_Xg3fPGereQ8gKCAIzAlDKIqNzi8YoR8 13126
lighteval/models/model_input.py sha256=LiNx0iwHwJ_PIwu0690NvR7o3rYJYiFjT6L5fMNLp8E 10193
lighteval/models/model_loader.py sha256=60_ooPM4frL2myyUD7PPFT7MpdMTUrPmUwLp5PcUtNk 7210
lighteval/models/model_output.py sha256=7MwaBLWk169eJyVeIk0Kc_qWrv-jFMK5jKj_F5AgUME 2860
lighteval/models/utils.py sha256=rmQ-KTXTDJuNrFsihvZifsjDdAkooc0v4JtKdDoB79A 3824
lighteval/models/dummy/dummy_model.py sha256=Khl5fhrpmz9j4m7tTMCiOLgfMMmbg0ItcMqpT6ou1qY 3602
lighteval/models/endpoints/endpoint_model.py sha256=Ofz5fW2_l1jBcV1Wtu-imdtR5erTCkPCQobvp1N9Cuw 28770
lighteval/models/endpoints/inference_providers_model.py sha256=A7Kby2oOObfw5-r8ENrIrsBuJOq7GwDS60DZeh-3Iec 10194
lighteval/models/endpoints/openai_model.py sha256=_IXRFWvNoo1WErY9-BXZjsnn8hgXWHAYN9gRw_3eXuU 11661
lighteval/models/endpoints/tgi_model.py sha256=MOlkMKtnd2gcUryMm9CQACw01MTUDNJiq89mFBHvMrE 5878
lighteval/models/nanotron/nanotron_model.py sha256=SS8s0rObns90EgniKpfg1mjOqlpNpd1aOIEUzUVq1dQ 65089
lighteval/models/sglang/sglang_model.py sha256=XKkU9sdJvE7u5SYfVgeKZlAyCylFI4tL_oERZGPUPi8 13804
lighteval/models/transformers/adapter_model.py sha256=gCHzKMK0b2WAyv1OKpJg51l-uthakRKh2HuYFBLP13U 5240
lighteval/models/transformers/delta_model.py sha256=fTunYtW-y_VXx5E4iJ2p__m-vBOay0Az53s3E6UASwU 4172
lighteval/models/transformers/transformers_model.py sha256=pT9LGm2HMCkyplsrS7KfGH8q4s9HpMEtg57T_nSZcG8 65612
lighteval/models/vllm/vllm_model.py sha256=B0_zhVWhfjIEjECBpU51fV4SyHMeZQNnZODY4yOaWKk 18793
lighteval/tasks/__initi__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/tasks/default_prompts.py sha256=9ajwc3lrfwPL38vwHkBrjOPSZoLilEp_7GLkqMJ-kn8 106423
lighteval/tasks/default_tasks.py sha256=_oLIl3SflMY1mivMkV8fln0FlTlMhNLOt9APUhDWa5Q 669561
lighteval/tasks/lighteval_task.py sha256=o2ApuLaD56lHjcRvNPHHgZxqE9TOB9wIyexFh68rjHw 28639
lighteval/tasks/prompt_manager.py sha256=pJVNfHACj-1V0Zebt0TaTtQtCVvBNt4iM1ZoD7kezws 18867
lighteval/tasks/registry.py sha256=LIDV1aYEqIkvykOgJBypkE1M6DlbGJ2wQZuo8fiHmUQ 15227
lighteval/tasks/requests.py sha256=fdeO-vAuawrgLK0Gnos17YEjQhpwo5kXhwSEl2N0cz8 7096
lighteval/tasks/extended/__init__.py sha256=G-lOZwxko34bXK_qsxdqWW-VhWJ6OclrRugQUTeDMug 1802
lighteval/tasks/extended/hle/main.py sha256=uUK-TH922b4wX3uIDNLYvdYXyjmia7tEDHzaOcNxtYk 8658
lighteval/tasks/extended/ifeval/instructions.py sha256=jZgpr8Jg6gx27IS5GF6aovVLQ-Wi1zFhinuN-tF8gUA 56053
lighteval/tasks/extended/ifeval/instructions_registry.py sha256=iRVMuVdoCBc3HyTHDxPkxi9hKOyZhWkEuxWSpa1Z4eM 7258
lighteval/tasks/extended/ifeval/instructions_utils.py sha256=08GaBGzAFUuwipF1fO70nORGh5799vtH4TdDHJdqvu8 25894
lighteval/tasks/extended/ifeval/main.py sha256=xNspHITl1GNxiPpl2oHYVzkzhFbuyirim0KQzSIMWds 5692
lighteval/tasks/extended/lcb/codegen_metrics.py sha256=SSeXplyDLkaeMqIZBcWgXt4BGMEXmWPqtQTEiwDIoV0 21752
lighteval/tasks/extended/lcb/main.py sha256=QZiN9V4rEOpeblHFhrjjwdz_xb6RQuSoF6pRlwf9whw 6187
lighteval/tasks/extended/mix_eval/judge_prompts.py sha256=n5EciOYpKIJDz-mvb0LlT_qCF8O_ukXZiXeoo6IwEVA 13347
lighteval/tasks/extended/mix_eval/main.py sha256=9eyhU3gB4yKPh2vKme9Xor1XiEl8R2Z98TmJpY1Nyxw 8470
lighteval/tasks/extended/mix_eval/prompts.py sha256=UNw1V2VNeUEFumnXdrh9SKn8W8u8zdoiZKjg_4wC6Ms 9704
lighteval/tasks/extended/mt_bench/judge_prompt_templates.py sha256=ytWhGfSma_ukwwBhEqDToljUWvPOHQXYPZAsuy4DLS8 6308
lighteval/tasks/extended/mt_bench/main.py sha256=tKdvtKP5mPZ_kaORiihgOvMFv5MpdPz1migaFOfqeJ0 3553
lighteval/tasks/extended/olympiade_bench/main.py sha256=wJvD8d5N7xtTU3GKqUoefpk_XP5Vbc6CuIQbZN8ZZbk 9968
lighteval/tasks/extended/tiny_benchmarks/main.py sha256=6WJ3-ICFiDe1hv3tFYpMw_N4nCJPmG2xr0LTdGol8Bc 11176
lighteval/tasks/multilingual/__init__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/tasks/multilingual/adapters.py sha256=SUSouX4v1iw6pev1X_KIXDZoLX05YT8wx1HlY47GML4 12244
lighteval/tasks/multilingual/tasks.py sha256=x9A0argJPUtd2LoCNb8_-_3UOQfF63m2dSNgm4KoN_g 127582
lighteval/tasks/multilingual/utils/__init__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/tasks/multilingual/utils/adapters_utils.py sha256=U9uiDViC9fhEHhGkTFlaRRGEV5KMDE8VHXmchiLC2GY 5476
lighteval/tasks/multilingual/utils/task_utils.py sha256=1y42LnO37VtEViYCWE6xLZA5OGW_I2mivsKjAJ1SCPI 1863
lighteval/tasks/templates/__init__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/tasks/templates/boolq.py sha256=f0_GJRdLqB6KxaRvBD3jv3FHZtP3pt1sLLe_AinQneg 4380
lighteval/tasks/templates/continuation.py sha256=VJiEbQmChskCd8xHgedDXftnWMWZ91YvBWbpSKCMDn8 8029
lighteval/tasks/templates/copa.py sha256=OHNF2XTxsJC82ofTPE3bgyZBjzSpyrGt0d7YfaOXheE 5989
lighteval/tasks/templates/hellaswag.py sha256=q9tN_nf3--N3bMMeSjsUueaB-82lBkyGNtD_A9RiVT8 6038
lighteval/tasks/templates/multichoice.py sha256=ohbv0-rXOFhiyFpPYSH_RVCr79BqubYP4RO_tB5J4Ug 6270
lighteval/tasks/templates/nli.py sha256=gcdRm-fUzL97xcRYYa0m3lWEduGE_itukRH-RZBbE0w 11104
lighteval/tasks/templates/qa.py sha256=JcACl8lAATazRE-PRt1HH6gICRVxeyT89eTkc5iyamI 2801
lighteval/tasks/templates/translation.py sha256=l8M94xtEVZTMrIHlVmgU4MV1YlXfKZOn2a6StbqZ7B8 6034
lighteval/tasks/templates/utils/__init__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/tasks/templates/utils/adapter_utils.py sha256=zoiTe5Z7KzJUQfU17702deVJjig32au49IWmb15_if8 1760
lighteval/tasks/templates/utils/formatting_utils.py sha256=5A8O9bD7Rt7eHZSLzS1nD8F8a2obh8S3D-Y5AwRx3Yw 4150
lighteval/tasks/templates/utils/formulation.py sha256=BA3ji-xhfhUUGcOAb5Ndw9flRJqrvrucLlhCxqfbw_E 5715
lighteval/tasks/templates/utils/translation_literals.py sha256=-kBI_--Io-mNLElCsm_xcV-UX8L24JAopb3a_BLdSV0 33570
lighteval/utils/__init__.py sha256=8mjC9aG83BbFsrdndqAJEEAijCoOA1KNOY7cXq8NlUk 1111
lighteval/utils/imports.py sha256=I_qs5x_wkkxWG_79dLSr8gmpwYd08QXijUS2K6NmWS4 6031
lighteval/utils/language.py sha256=OVIiXNfSp_Xb6N32ZjFnhNSy3MYN04hIHDURCTZci0g 6881
lighteval/utils/parallelism.py sha256=BK3xUbXhtOGjPe0RRvZiraU3DnKMqQwTWKdPz48lHwg 6273
lighteval/utils/timeout.py sha256=vPvpfRsVYFVH6Bj94T7_z2KbfkDIBXc2SC7EhvZVLJI 3590
lighteval/utils/utils.py sha256=vhYCLOBbgy6tYn9QHambueSV7OlyMH0ipgEX3ZzE0xw 7807
lighteval-0.8.1.dist-info/licenses/LICENSE sha256=tPk5wOXOUTsipCsjMpS1bCJqJ5Nudwj1HtE3xgLo6w8 1069
lighteval-0.8.1.dist-info/METADATA sha256=B4B_m2FYB8gcnVr_42G0gzpfn4wbjF90bNB5c-7FQCQ 9062
lighteval-0.8.1.dist-info/WHEEL sha256=1tXe9gY0PYatrMPMDd6jXqjfpz_B-Wqm32CPfRC58XU 91
lighteval-0.8.1.dist-info/entry_points.txt sha256=eW_fUc4_bJVN6LJSjI1wB39LbT6pllEOUEUbY3qPXlc 53
lighteval-0.8.1.dist-info/top_level.txt sha256=dvkN9L0K1BpVFmb-YCakjCmC13DtEVvsyMi5V12jdvg 10
lighteval-0.8.1.dist-info/RECORD — —

top_level.txt

lighteval

entry_points.txt

[console_scripts]
lighteval = lighteval.__main__:app