Wheelodex » Projects » crfm-helm

crfm-helm

View on PyPI — Reverse Dependencies (2)

0.5.0 crfm_helm-0.5.0-py3-none-any.whl
0.4.0 crfm_helm-0.4.0-py3-none-any.whl

Wheel Details

Project: crfm-helm
Version: 0.4.0
Filename: crfm_helm-0.4.0-py3-none-any.whl
Download: [link]
Size: 1597329
MD5: ba7ff8a10701578a6f97e0cacd421f5c
SHA256: 3fc9c3721f78f48632cad6dfb04851de7055f83d900125fd8d247b9503a99a27
Uploaded: 2023-12-20 22:00:00 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · entry_points.txt

METADATA

Metadata-Version: 2.1
Name: crfm-helm
Version: 0.4.0
Summary: Benchmark for language models
Author: Stanford CRFM
Author-Email: contact-crfm[at]stanford.edu
Home-Page: https://github.com/stanford-crfm/helm
License: Apache License 2.0
Keywords: language models benchmarking
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.8
Classifier: License :: OSI Approved :: Apache Software License
Requires-Python: <3.11,>=3.8
Requires-Dist: cattrs (~=22.2.0)
Requires-Dist: dacite (~=1.6.0)
Requires-Dist: importlib-resources (~=5.10.0)
Requires-Dist: Mako (~=1.2.3)
Requires-Dist: numpy (~=1.23.3)
Requires-Dist: pyhocon (~=0.3.59)
Requires-Dist: retrying (~=1.3.4)
Requires-Dist: spacy (~=3.5.3)
Requires-Dist: tqdm (~=4.64.1)
Requires-Dist: zstandard (~=0.18.0)
Requires-Dist: sqlitedict (~=1.7.0)
Requires-Dist: bottle (~=0.12.23)
Requires-Dist: datasets (~=2.5.2)
Requires-Dist: pyarrow (>=11.0.0)
Requires-Dist: nltk (~=3.7)
Requires-Dist: pyext (~=0.7)
Requires-Dist: rouge-score (~=0.1.2)
Requires-Dist: scipy (~=1.10.0)
Requires-Dist: uncertainty-calibration (~=0.1.4)
Requires-Dist: scikit-learn (~=1.1.2)
Requires-Dist: transformers (~=4.33.1)
Requires-Dist: torch (<3.0.0,>=1.12.1)
Requires-Dist: torchvision (<3.0.0,>=0.13.1)
Requires-Dist: google-api-python-client (~=2.64.0)
Requires-Dist: aleph-alpha-client (~=2.14.0); extra == "aleph-alpha"
Requires-Dist: tokenizers (~=0.13.3); extra == "aleph-alpha"
Requires-Dist: crfm-helm[proxy-server]; extra == "all"
Requires-Dist: crfm-helm[human-evaluation]; extra == "all"
Requires-Dist: crfm-helm[scenarios]; extra == "all"
Requires-Dist: crfm-helm[metrics]; extra == "all"
Requires-Dist: crfm-helm[plots]; extra == "all"
Requires-Dist: crfm-helm[slurm]; extra == "all"
Requires-Dist: crfm-helm[cleva]; extra == "all"
Requires-Dist: crfm-helm[images]; extra == "all"
Requires-Dist: crfm-helm[models]; extra == "all"
Requires-Dist: crfm-helm[mongo]; extra == "all"
Requires-Dist: anthropic (~=0.2.5); extra == "anthropic"
Requires-Dist: websocket-client (~=1.3.2); extra == "anthropic"
Requires-Dist: unidecode (==1.3.6); extra == "cleva"
Requires-Dist: pypinyin (==0.49.0); extra == "cleva"
Requires-Dist: jieba (==0.42.1); extra == "cleva"
Requires-Dist: opencc (==1.1.6); extra == "cleva"
Requires-Dist: langdetect (==1.0.9); extra == "cleva"
Requires-Dist: pytest (~=7.2.0); extra == "dev"
Requires-Dist: black (~=22.10.0); extra == "dev"
Requires-Dist: mypy (~=1.5.1); extra == "dev"
Requires-Dist: pre-commit (~=2.20.0); extra == "dev"
Requires-Dist: flake8 (~=5.0.4); extra == "dev"
Requires-Dist: google-cloud-aiplatform (~=1.36.4); extra == "google"
Requires-Dist: scaleapi (~=2.13.0); extra == "human-evaluation"
Requires-Dist: surge-api (~=1.1.0); extra == "human-evaluation"
Requires-Dist: accelerate (~=0.23.0); extra == "images"
Requires-Dist: pillow (~=9.4.0); extra == "images"
Requires-Dist: numba (~=0.56.4); extra == "metrics"
Requires-Dist: pytrec-eval (==0.5); extra == "metrics"
Requires-Dist: sacrebleu (~=2.2.1); extra == "metrics"
Requires-Dist: summ-eval (~=0.892); extra == "metrics"
Requires-Dist: crfm-helm[aleph-alpha]; extra == "models"
Requires-Dist: crfm-helm[anthropic]; extra == "models"
Requires-Dist: crfm-helm[google]; extra == "models"
Requires-Dist: crfm-helm[openai]; extra == "models"
Requires-Dist: crfm-helm[tsinghua]; extra == "models"
Requires-Dist: crfm-helm[yandex]; extra == "models"
Requires-Dist: pymongo (~=4.2.0); extra == "mongo"
Requires-Dist: openai (~=0.27.8); extra == "openai"
Requires-Dist: tiktoken (~=0.3.3); extra == "openai"
Requires-Dist: colorcet (~=3.0.1); extra == "plots"
Requires-Dist: matplotlib (~=3.6.0); extra == "plots"
Requires-Dist: seaborn (~=0.11.0); extra == "plots"
Requires-Dist: gunicorn (~=20.1.0); extra == "proxy-server"
Requires-Dist: gdown (~=4.4.0); extra == "scenarios"
Requires-Dist: sympy (~=1.11.1); extra == "scenarios"
Requires-Dist: xlrd (~=2.0.1); extra == "scenarios"
Requires-Dist: simple-slurm (~=0.2.6); extra == "slurm"
Requires-Dist: icetk (~=0.0.4); extra == "tsinghua"
Requires-Dist: sentencepiece (~=0.1.97); extra == "yandex"
Provides-Extra: aleph-alpha
Provides-Extra: all
Provides-Extra: anthropic
Provides-Extra: cleva
Provides-Extra: dev
Provides-Extra: google
Provides-Extra: human-evaluation
Provides-Extra: images
Provides-Extra: metrics
Provides-Extra: models
Provides-Extra: mongo
Provides-Extra: openai
Provides-Extra: plots
Provides-Extra: proxy-server
Provides-Extra: scenarios
Provides-Extra: slurm
Provides-Extra: tsinghua
Provides-Extra: yandex
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 8756 characters]

WHEEL

Wheel-Version: 1.0
Generator: bdist_wheel (0.42.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
helm/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/py.typed sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/config_registry.py sha256=jEUik24cpHer0KrKTQQGcRf6p4NW0peRwaXToHjqnZE 1295
helm/benchmark/data_preprocessor.py sha256=aNdM-o2t4qkLIQHiQeWUFg03DjjJ8HTBIphYCK8pXVo 2173
helm/benchmark/executor.py sha256=zi3YzO1rFHegwWYZcu7HC6ClqgqjhzSAdVgIOMk20G8 3586
helm/benchmark/huggingface_registration.py sha256=kDnnXTKAE6tPEGZRaZ-Dkly32ooGNB03Wbtm1A1uX8o 3350
helm/benchmark/model_deployment_registry.py sha256=Hftahy-eHm6pI815JEsdNVlv-ln4z2ISl4ojo3fAIYw 5460
helm/benchmark/model_metadata_registry.py sha256=1NaHyk1ojwSYPf8UH-mAqEZZSoiqlKUKn8MtQK1robI 6458
helm/benchmark/run.py sha256=qGdBjTTlYN996CAs5WSqf2yFFLNN85TP53FR0tjhnSw 11409
helm/benchmark/run_expander.py sha256=6QhHcPYDzOnV4gBF1QPw7vHlSnZjv6yZjT_0SzEqoMk 41365
helm/benchmark/run_specs.py sha256=RC3BN-fPGHKgIu03Ew07p7EYhzxWsr4beU5Xz-p82Xg 104887
helm/benchmark/runner.py sha256=S38RHCOObuoUyfF7OAB0H_mgSaTz-Ptzwlz2oj4EmxE 14262
helm/benchmark/server.py sha256=iBODLZC_-GVFNUvkPBHKFEhtSdZCrgDwvs5ka2NQcmg 5127
helm/benchmark/slurm_jobs.py sha256=eNCAoaWDfT0Wk32ZJRIGo-x8kgjhDPnPB4Xrvw_eLB0 3225
helm/benchmark/slurm_runner.py sha256=_5tqyap8EbqOXKtT_8ZuB3ChVSlsNTNSSaO-axSNkYM 15795
helm/benchmark/test_data_preprocessor.py sha256=xIwFuhb7WpdGLfFq9gfBEzPlsmyIEZMuHW4iPtq2HEY 2180
helm/benchmark/test_model_deployment_definition.py sha256=EZ9WqI9y6u-XfK7XRsk0Ew_o5tnoyvOL0UMNkKDt8NM 4537
helm/benchmark/test_model_properties.py sha256=ioi7CotXyXtlhP6XOEVQYQqcMvnh5gX2QlcdjbSuY_g 68560
helm/benchmark/test_run_expander.py sha256=nVxlsu5mFC-omMTTSDR0CHAaDDnI1sXbLxjCj4Xuu6I 1161
helm/benchmark/tokenizer_config_registry.py sha256=ZOImg38ta0FXZYAWna6q7A5xrG2mU7Ofr-8j4EqGlUY 1585
helm/benchmark/vlm_run_specs.py sha256=aC5p74RzpJenyVb8f9JcIyUnYa2rl5yQuFDbrLggBog 3100
helm/benchmark/adaptation/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/adaptation/adapter_spec.py sha256=9nyHrSrC0EoS6c1q8ocY4cnvdiCtU4Ndl-GONr1fe4g 3295
helm/benchmark/adaptation/prompt.py sha256=n0Ka3RGSWMr3CBnJrPNPy626x9TJE3k677wKbG8hO9A 2133
helm/benchmark/adaptation/request_state.py sha256=o3OpZbB0TJFiZ2Nmhvg3vWmByaUSYTffT_WnoNb7w68 2712
helm/benchmark/adaptation/scenario_state.py sha256=ZflBuNgvN0JqUhshFcy0kTweO1WJs6j5UCaTxWTMe0o 1747
helm/benchmark/adaptation/adapters/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/adaptation/adapters/adapter.py sha256=rFqfjTsha3r6CgRat2FKwIJb3pl-IyyoGUznwQtcHag 1236
helm/benchmark/adaptation/adapters/adapter_factory.py sha256=VUBOfOWKC8k0FimJ5ZW8CZZVK0W33eg801wK1ZhYIjg 2627
helm/benchmark/adaptation/adapters/binary_ranking_adapter.py sha256=zF5KcGip5CAMfg5mzisYQ2jlIuChbraujN-UaxCV8E8 5782
helm/benchmark/adaptation/adapters/generation_adapter.py sha256=IqCFpU3r4YuIeqbTgK0MPJtvLNXomkIpgld5Jj7ciZw 2059
helm/benchmark/adaptation/adapters/in_context_learning_adapter.py sha256=wJO8-vOMk07tH3X6PCo4LXC6eSZpSzvMxdKObq6-4Cg 14244
helm/benchmark/adaptation/adapters/language_modeling_adapter.py sha256=Vs0AU-laV89aPF9CZaOYG8nf8Ukpmc5ozXNZ-Amu5b0 14969
helm/benchmark/adaptation/adapters/multiple_choice_calibrated_adapter.py sha256=VJ66MfIGQWJg0VXCV0MJEMwF9Jx1DeJ7RxsgYlOTx_4 1889
helm/benchmark/adaptation/adapters/multiple_choice_joint_adapter.py sha256=k8wSxv9pK8wtbQNBzWYPkGEUKJb8tcVi41Y1M3fPQT4 3985
helm/benchmark/adaptation/adapters/multiple_choice_separate_adapter.py sha256=aMe-y4iiyEnM7_bqIoddeZBsVLoDxMmjKY2eZKB6Y2Q 2156
helm/benchmark/adaptation/adapters/test_adapter.py sha256=u-c7K6VH6yvymRtPjoWRBIjk0vs41g7IWOfv1w9Sxn0 620
helm/benchmark/adaptation/adapters/test_generation_adapter.py sha256=rbZlC_xHEKCXo5GnDt7C8856fVDjX18QGpMvvn0RRaE 11765
helm/benchmark/adaptation/adapters/test_language_modeling_adapter.py sha256=VCD-9EKDi9_xElUKU6cjAgUoPTqx5UGF2tiXAxsV45E 8490
helm/benchmark/adaptation/adapters/test_multiple_choice_joint_adapter.py sha256=CiIpclOvpIDigOVS8NgjCQOqt1olcxs5xPFVMlmuckM 7379
helm/benchmark/adaptation/adapters/multimodal/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/adaptation/adapters/multimodal/generation_multimodal_adapter.py sha256=o7CGClyVWYOuJ4G56-whq5fTvCr7QIn51Mo6DTdvwg0 1881
helm/benchmark/adaptation/adapters/multimodal/in_context_learning_multimodal_adapter.py sha256=9_9PgNIiJDoTm1NIhCKWytCYe3hHzHJUhaAdd8hM0PQ 6300
helm/benchmark/adaptation/adapters/multimodal/multimodal_prompt.py sha256=5jdVVfWf0lBErwzDamIHozZSbf1mEkpmACX8mPlHJc0 1907
helm/benchmark/adaptation/adapters/multimodal/test_in_context_learning_multimodal_adapter.py sha256=GANySWgsdt5SyajawgMPPqCF82bsd27VBHDfGngupc0 10047
helm/benchmark/adaptation/adapters/multimodal/test_multimodal_prompt.py sha256=2OZhhOtEZ7RvItHZD4AGsefXvVUwEQZL2T-ESHPsvdI 3482
helm/benchmark/augmentations/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/augmentations/cleva_perturbation.py sha256=arUkY_luc274YEMZocOos9rpAZVbEFZphbMlobAxTy0 29208
helm/benchmark/augmentations/contraction_expansion_perturbation.py sha256=yni1UR2fviN0Wig8MpOp0zzLn4H-gYocTjKTpxBwywg 4850
helm/benchmark/augmentations/contrast_sets_perturbation.py sha256=OvuntCLHYZoWHXO1JTixWmnZD5iQDNy_yYrLwgAO15M 3338
helm/benchmark/augmentations/correct_to_misspelling.json sha256=L44RiJXlJCa6zQzTLf0MFHCOhFyRDRKfLQNXH-n3XIs 213429
helm/benchmark/augmentations/data_augmenter.py sha256=57LA6h7z1tVMy_xGcW46F1KRi3D4wnv0fi8XeJjsi2c 3849
helm/benchmark/augmentations/dialect_perturbation.py sha256=4DJB2gDj-JzhYtu-uxh9zgebChRDHpwMgKhju8OqP44 6317
helm/benchmark/augmentations/extra_space_perturbation.py sha256=D7y7oA4LwKJnVgTuLZOF3nQjmYRtkezuxY75qVQkD8I 843
helm/benchmark/augmentations/filler_words_perturbation.py sha256=NihLbpyxlXnncxM635pIyQ5LJvu4uPb4oKiLb8UU5dQ 2922
helm/benchmark/augmentations/gender_perturbation.py sha256=XGkEM-PnlH7aRWdz4zPkJ37j0b2XKhTFa_UlRl6bmKA 9389
helm/benchmark/augmentations/lowercase_perturbation.py sha256=e-lhkuKOV5QR_GnOp71vr2lHI97tiWSZ_jz57jcFbvg 519
helm/benchmark/augmentations/mild_mix_perturbation.py sha256=q27-c8_di1jaLhUuo5LzqOStnJcWIjSJ0a8Sr3S5TOs 1787
helm/benchmark/augmentations/misspelling_perturbation.py sha256=IE45qEE0HuIln62tAGvCP9B_RjNMJ5Xu06goL6b0fWQ 2144
helm/benchmark/augmentations/person_name_perturbation.py sha256=AIF5VVjNJ1OlUyCC1bt2dcP1KDn4gdgqd7Y2KGvHPqs 14422
helm/benchmark/augmentations/perturbation.py sha256=6JTCpKbV_PLcvLHkZT9OUFn4WnTMfIWZZazbKGLHdMA 2905
helm/benchmark/augmentations/perturbation_description.py sha256=VU3lo-7QJsHZNHKzQjYRRqxF73xUQ8AhQAaJUJk7x4Y 1185
helm/benchmark/augmentations/space_perturbation.py sha256=g4rbyoureBaOVf_lrRXIWYlL6YZpqbCOJk554iOCuIs 935
helm/benchmark/augmentations/synonym_perturbation.py sha256=komOV5M342_8unopnwN6gkPWpJIZXidywiu6PO9_riU 4151
helm/benchmark/augmentations/test_perturbation.py sha256=SNvAdUs2ZHP6OQAeOPMnvJgHldzdB2lVAS25SOU3rsc 11537
helm/benchmark/augmentations/typos_perturbation.py sha256=_F9zwvrLie8hX7mzUtQmYq6oq6yqaFiKGsvc9LAuBr4 2798
helm/benchmark/data_overlap/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/data_overlap/data_overlap_spec.py sha256=aj_l1l0qxUbUMrSWr70-Sb1j_JN-7WYop5BXPG_xj44 1998
helm/benchmark/data_overlap/export_scenario_text.py sha256=OiFsU_qME2_I87yDSNPfuAVI5Y9MbPEcEliYK6TaAEE 4527
helm/benchmark/data_overlap/light_scenario.py sha256=UFUr8plZD32e2TJTjFZLxTwD-ZRb9eYnHk2U3ZD8P40 1074
helm/benchmark/efficiency_data/inference_denoised_runtimes.json sha256=ios_dt-_8wtXvkVAx0iI2zwCxqHvk3XKTx31qHPalsI 4203
helm/benchmark/efficiency_data/inference_idealized_runtimes.json sha256=5w7reeZc0yc4cjH8kJGxQQSoe8yaRVX2SSlSrx0QWFQ 12348
helm/benchmark/efficiency_data/training_efficiency.json sha256=aH2moiBLStOLVi8Ci2KTK5ZkWlTBLK-B3fRfNZwhoSg 9763
helm/benchmark/metrics/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/metrics/basic_metrics.py sha256=523j_t0UuuEGpIEyuk0u1TyQ2kY-vwwiwjg6TP5OkYM 43762
helm/benchmark/metrics/bbq_metrics.py sha256=H44mwKXLJ0PXo-sVKgRHCEKjZGnCahDD6GOQMWpnbOQ 6061
helm/benchmark/metrics/bias_metrics.py sha256=uVJFQvDSzvPR1ELu0FNYyExjRy2ThaJRCw8beEMDqJs 11309
helm/benchmark/metrics/bias_word_lists.py sha256=mx5JjW3mHffXIqo4GcQN-zENUEttBqQnEjPTz3J3J_4 13909
helm/benchmark/metrics/classification_metrics.py sha256=pgY8BToHw61lfPTswEW2Da8L4rkWJhthMBYVWnngzuw 5442
helm/benchmark/metrics/cleva_accuracy_metrics.py sha256=nSrtxGtIXPFcLrz9mNMYRfJqgQMzoCuf4i3TcQx6Rzw 2580
helm/benchmark/metrics/cleva_harms_metrics.py sha256=MrsSAvpOuGGFtVL8RfrG5bNbVTr3LmfhuNfnV4PrAuE 11085
helm/benchmark/metrics/cleva_metrics_helper.py sha256=8UwiGhekUmp7DxYWU4rxqX2v3ewkg-O5-jOh49iOGmc 304
helm/benchmark/metrics/code_metrics.py sha256=e0aqLcxBAdCc0qAqebzK40Ilv2Py6xZbosud5v169x8 5121
helm/benchmark/metrics/code_metrics_helper.py sha256=byyuI1lJgbIDPVJzywaBsam9zFMPPyn28g1grsK9xyA 22336
helm/benchmark/metrics/copyright_metrics.py sha256=X9j3YsfzWEoGpgPpIvCzm18-JggLAW5QFooifE1KqaM 7729
helm/benchmark/metrics/disinformation_metrics.py sha256=UVipWjmqiK3VONVTspCvAY9gTH8wE84eAJ1JRUT2iKA 10777
helm/benchmark/metrics/dry_run_metrics.py sha256=I6NfrWW9WsmFCoqCRIW_nSiFQW0gnyRl2_9JkIEqpNM 3656
helm/benchmark/metrics/instruction_following_critique_metrics.py sha256=-cHgmSKy-Z7ZT4ZFguI4EY6OiEp4MltXBq8zTEXBtaY 9757
helm/benchmark/metrics/metric.py sha256=GEa8xt7SSIaoZMs76bImII1HUxs9mhchIiNFaoG8IIk 18767
helm/benchmark/metrics/metric_name.py sha256=hk5WQ6uj_9EjgKKFawPenL2-XOMf-aKvNRkOxlu4nCo 1355
helm/benchmark/metrics/metric_service.py sha256=W3BqPoWikPQ8eGTUmcDIqMQK1IM1tymrjL741G2BSjY 989
helm/benchmark/metrics/numeracy_metrics.py sha256=panMWD3a1NPerg3Ix7l6NhR7jGOIQOQV9i_KysBeDA8 2818
helm/benchmark/metrics/paraphrase_generation_metrics.py sha256=EbdVAJkK9ueA6k4rEdB34jbE_xeh-o164sPiWieLAYo 1885
helm/benchmark/metrics/ranking_metrics.py sha256=oZgTHlCjjQ6imzrddPMZteTmQjG9QqLUxpiJzyqGREw 17341
helm/benchmark/metrics/statistic.py sha256=FuxNxMtAfiCkOxBS9KHlhEyxe61e0YXt2emvsufgPZQ 3424
helm/benchmark/metrics/summarization_critique_metrics.py sha256=Lf7PDuce62HDzyofsyxaOvH0QvzcaS-vJvDWtIs8xKk 4694
helm/benchmark/metrics/summarization_metrics.py sha256=gZ2MpzX0kQstRUnqzJjqTJ6z9h6kp4v_qhW1ahSIMVQ 16420
helm/benchmark/metrics/test_basic_metrics.py sha256=zbTTPxTKGLjGXQHyydar4YAd0JDZsPzju8J6ELGndOc 1076
helm/benchmark/metrics/test_bias_metrics.py sha256=N22a5UKqSd7TfYBrZS6ciFSOIc1s85pYMwk7m9ufpG0 6274
helm/benchmark/metrics/test_classification_metrics.py sha256=0snnCDOYoJqOb7xyQBgSd9bip19DB-WXRXR-Dcbb6QA 5550
helm/benchmark/metrics/test_metric.py sha256=S7LGHNCHuhMk582eHylw1tOasUBEf_7F0T4u3tey7b4 757
helm/benchmark/metrics/test_numeracy_metrics.py sha256=ls1ZIHDePKpHMoqAbf4HmJ1SIBjLFuLIzGbfg6OiZvM 4162
helm/benchmark/metrics/test_statistic.py sha256=AejuYLSeUwEOqpEMRKZFjnxu4HKUraeExU8TPmZEqW4 1229
helm/benchmark/metrics/toxicity_metrics.py sha256=6OsuOYX57m6Lvnpe6iYmJAjTGofQ73LBbyCtFCQ2Cbs 4033
helm/benchmark/metrics/summac/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/metrics/summac/model_summac.py sha256=HNWB5-C0Xnqml4TnXyPZdVegNu4hHJsY2IkzX3rm5Rg 17414
helm/benchmark/metrics/summac/utils_misc.py sha256=7_Q1c72cKt8PWtxn8u4R8nB53HK6_JF2nP8bBXYNk-A 1485
helm/benchmark/metrics/tokens/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/metrics/tokens/ai21_token_cost_estimator.py sha256=WeNP4yiM4TVrD9Kid-uVRmWIVDqETnBsMycZmIBiTZ0 665
helm/benchmark/metrics/tokens/auto_token_cost_estimator.py sha256=MAVCme5cK_ryPjoA6cQPvfpKwqdSfnliTaxwT0vi4R4 1974
helm/benchmark/metrics/tokens/cohere_token_cost_estimator.py sha256=5igmDhWu7H8-qiJnAQqCTQd1WXRewMrpdJdo4qhvSWc 523
helm/benchmark/metrics/tokens/free_token_cost_estimator.py sha256=G_6UK6Js_NZ_eqY0ZQnrC9QJVMERGhV1f6v7xq2lM-Y 461
helm/benchmark/metrics/tokens/gooseai_token_cost_estimator.py sha256=vcIy78rZfjrgYksIWojDckf5Op24yIKkmeTXfhFf6mQ 1186
helm/benchmark/metrics/tokens/openai_token_cost_estimator.py sha256=7jgjcgmbcVfLA_nTOrWXKIF8TEXng_KnE6cSgsSXWmE 1398
helm/benchmark/metrics/tokens/test_ai21_token_cost_estimator.py sha256=l9UQZ0aAIhCYuFbIjU3j3A3XXoMvwUvz1kvRtlDbtOo 1079
helm/benchmark/metrics/tokens/test_openai_token_cost_estimator.py sha256=_wJ3E3LbJB9XPLixTH82BYQbp32o3oij6Sz3lsZL30E 2648
helm/benchmark/metrics/tokens/token_cost_estimator.py sha256=fTGUfhHV6yMwpTkCEMTGMxKO8jskqJz4sAtwXT6M_C8 425
helm/benchmark/presentation/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/presentation/contamination.py sha256=PiIdcaD3-xfExjOmyL5q4Ao2ASa-OlScJAB9u1Zxe7o 2811
helm/benchmark/presentation/create_plots.py sha256=lKD42IWdIFnYvlionF50erc94-wJQRZxFqqAfT4jxtk 28715
helm/benchmark/presentation/run_display.py sha256=9d_zOFK7-MCSqjRPTVnOHU8GATznyeXarS7YZRF1r4s 11232
helm/benchmark/presentation/run_entry.py sha256=J1QgLOP99N7N4bs7nzXWxyU3pOd-a1j8xwL9ag1nP_Y 1158
helm/benchmark/presentation/schema.py sha256=dzYMP9FFiY_3BkeAvz-i3iSeeIXoaSrnqh6l0ggc9cw 9634
helm/benchmark/presentation/summarize.py sha256=tzE--9WQ9DJi9tn3oQ_aIR4_wPT1giRP-1IPucyhlIA 65041
helm/benchmark/presentation/table.py sha256=VzVMwsgP3kItAM6FPRUaTphzJ-ZjriiuFbWlO1rJUMU 2879
helm/benchmark/presentation/test_contamination.py sha256=vEzr_CnNAWM5BUBe1yuc57uiRQ_bDNKPFJ059O1Wklc 517
helm/benchmark/presentation/test_create_plots.py sha256=5PPPegMTdBZurxyyUxI4rN13AVsjV3eQrwFqlobJ8UA 1286
helm/benchmark/presentation/test_run_entry.py sha256=r7jOIvNmGLyiZFFfkXGgDE6_SzhQ3Y7hLgmbzRPONWo 793
helm/benchmark/presentation/test_summarize.py sha256=pm9XbFsXfPEjWdx8cggC7CSDt4XDQoB5kGYyp5mMlNs 1629
helm/benchmark/scenarios/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/scenarios/anthropic_hh_rlhf_scenario.py sha256=Wyt7J5BAvAqC5JTqCW4fh7ex9-itX11P_9rLTocqvtk 4973
helm/benchmark/scenarios/babi_qa_scenario.py sha256=S1tPQY2x1I3hQL1JQ6wvUwvKyiSe7SqpRSW6N3_T0mo 5043
helm/benchmark/scenarios/bbq_scenario.py sha256=AMUeFLaO2jiB-FnBbe_a5sZ8OXBuG7_3lgld3dn_jtE 9581
helm/benchmark/scenarios/big_bench_scenario.py sha256=bSk8Ia4u_6OqMjiyadpYQAWN-8GFWqvd3Ft3JiVGpi8 8081
helm/benchmark/scenarios/blimp_scenario.py sha256=o1MDcHT14KFDET4K9otx8pDiIgXrhsD19pvO0mR2ADU 6260
helm/benchmark/scenarios/bold_scenario.py sha256=-Gg9C7lY8VFXU-sv5RmNOUZ3NXOJqvZtWsZC2iBEAlo 4110
helm/benchmark/scenarios/boolq_scenario.py sha256=rvSp5SwXMCVzBo5BFxfhj1Xv06_ksqKrtTQR7nPiS-o 8013
helm/benchmark/scenarios/civil_comments_scenario.py sha256=VO5G-cQ9qctmBN0O76uSewnO_mFslMo5mbR2ZTrjuds 4851
helm/benchmark/scenarios/cleva_scenario.py sha256=xhwZ616iz0CN3fYIfrXHcV1XlcRQjyPSzML8fq8D3l4 57939
helm/benchmark/scenarios/code_scenario.py sha256=BDjKQbFbV7btd_pRjUD0YA06zCnqqXCi8ekxjE0xws4 12170
helm/benchmark/scenarios/code_scenario_apps_pinned_file_order.py sha256=KC-5MQ-d8Nn46aDN4FaPxmd6yk1DtVUmVR-CIZsNCp4 1738
helm/benchmark/scenarios/code_scenario_helper.py sha256=TnXAlY-wdAFwIDylFItf0z7HOu93WD6dNThwzZYe330 5904
helm/benchmark/scenarios/commonsense_scenario.py sha256=Wgry4P4W0hHmeBrN61u5h6VTjET5TZI0oozC2ri3nbQ 9487
helm/benchmark/scenarios/copyright_scenario.py sha256=qw-ijottZ28DjMqMWSZzL9_FuRDAHf0YmGV27YstmVI 3673
helm/benchmark/scenarios/covid_dialog_scenario.py sha256=LqNtfb9XQestluN0Bz7yi-Zo0VRkLjl4F9IOYzhPw3I 3971
helm/benchmark/scenarios/custom_mcqa_scenario.py sha256=aZkt1Ums-kIVpLfd4Pfhe7Rf_fZLxjO2ANLL2aN2qVM 1915
helm/benchmark/scenarios/dialogue_scenarios.py sha256=MyF8CWkd_L42_gGLqG19YjCvpSpChCHeTj4jMUn_IkI 5630
helm/benchmark/scenarios/disinformation_scenario.py sha256=kQi0MVVoSDhx2vOTnUaCIttPXMf8zz7Eld2FD_77tnA 8504
helm/benchmark/scenarios/dyck_language_scenario.py sha256=vMxND9wPJenrGlCLhSw5UxOw3TV2Jq8cTmIXGpzEWaA 9318
helm/benchmark/scenarios/entity_data_imputation_scenario.py sha256=n2mnkmSeTznEy7S-GVumqpD9bt27yctbuEmtgQrG-Y0 6399
helm/benchmark/scenarios/entity_matching_scenario.py sha256=YjBX61TlL3CDQ3X6D-JyR-qlOYGLdoRXJxl9AEeqxYs 7022
helm/benchmark/scenarios/entity_matching_scenario_fixed_random_state.py sha256=TklbX7Kx4y-estV-YHUbI5O08q2qCZRrOmX9D3gZS9c 2193
helm/benchmark/scenarios/grammar.py sha256=Pb9vEP_0Ki87UdQCj1ym7QWJ24M4DRP6TXB5d3GnhLs 5597
helm/benchmark/scenarios/grammar_scenario.py sha256=bl-Cm9caDs077zSu38mzaS9maZ2gM-QazgjOEMFvxYg 1454
helm/benchmark/scenarios/gsm_scenario.py sha256=9fV2SEw3ocKNAD-TrDZZTpq4l7mbttQQWbO0YNz4e6k 2613
helm/benchmark/scenarios/ice_scenario.py sha256=vvk11cFPGUhg_CcGh3wEfVsGzrvMFgkByN-xcF-OOjI 16473
helm/benchmark/scenarios/ice_scenario_pinned_file_order.py sha256=fuirubIdi-rkJMfSd7YoDdBX2q0f5K7GGTN4XVapAUY 1613
helm/benchmark/scenarios/imdb_scenario.py sha256=-bfjm_43j0lsfGskz2THNeKABCld90rFTZ5yWt8FTHE 6175
helm/benchmark/scenarios/imdb_scenario_pinned_file_order.py sha256=fjW0Gkzg2Y3IAbtYJ3KC7MueWd9U8h0tlcBCqxYmRrM 1621
helm/benchmark/scenarios/interactive_qa_mmlu_scenario.py sha256=PuwcuHnx3nCuRYEEzEQ6aRI9i7QAHtfg4DmLAG4M6AM 2205
helm/benchmark/scenarios/koala_scenario.py sha256=ihx39W1C9KyOh5MkdMqw1twUwzA8S5QPsBKeQRZcZGM 1365
helm/benchmark/scenarios/legal_summarization_scenario.py sha256=DTYYsf-74NFjHvrNa5ZWSgMywq9ss16r5eFZI2-xZJk 8644
helm/benchmark/scenarios/legal_support_scenario.py sha256=hu8SlDGrU6oUTQjnrLEagjFNniW96wMahKjnrtuyjzo 3925
helm/benchmark/scenarios/legalbench_scenario.py sha256=55D09TaELT_B__ERyyWUp92gGwXifNgX1eIOzw2pgBE 4398
helm/benchmark/scenarios/lex_glue_scenario.py sha256=vsy8Y03hoCwTJFgBdQf0koxlHDRFm2V9wT04n2K1tlk 10268
helm/benchmark/scenarios/lextreme_scenario.py sha256=HEpXLdLLywo6yfXaGF3zRgx1PiGPE2ILnpKYs9Dmsow 20396
helm/benchmark/scenarios/lsat_qa_scenario.py sha256=Ui68mi05N3WxGMZfCznJBscKkqaJ0ZEUO3PhIiD1xNE 6129
helm/benchmark/scenarios/math_scenario.py sha256=6HZ6y1kjE64tywZIee0ocVj-tqxXyvR6VfBldn7vftY 13849
helm/benchmark/scenarios/me_q_sum_scenario.py sha256=WLfED6DiIOUPd4PowydPmrQ4v26OsjG4iHumq7ka9do 4001
helm/benchmark/scenarios/med_dialog_scenario.py sha256=f7hTcEpxGfoxJ-WXNrSRrgVfbyuYpB624HbyRuMP-KI 7295
helm/benchmark/scenarios/med_mcqa_scenario.py sha256=JdVQuzKM4TRyWT6c1zpsMvNk1RYHnKdy_fRM6W7PCLI 5035
helm/benchmark/scenarios/med_paragraph_simplification_scenario.py sha256=e9WRx41XKJI06l3ZGkZe_LKNx-qd4hoWopfDncrr3KE 7591
helm/benchmark/scenarios/med_qa_scenario.py sha256=R0iDMzu3U0TnYEOiyjnlOWHGFwhvOJBVwJ5-Hos2eP8 4417
helm/benchmark/scenarios/mmlu_scenario.py sha256=wZjoGjaZNERyqSDHgIsv8fIANyUUQZ1ExQAa4cPzmM8 3790
helm/benchmark/scenarios/msmarco_scenario.py sha256=5DTWSDuinXVomlUTYfwOijXLifCSeG4be9Tvb7SANQM 34026
helm/benchmark/scenarios/narrativeqa_scenario.py sha256=FIzd4CTS6ZTOI_TJ8aPlkXYNfqbTIs13ZUqAt7ie8U4 5608
helm/benchmark/scenarios/natural_qa_scenario.py sha256=khKCR8xVhRqJtZoAfGQ9_CV6REkfrYn5ahLpci6rUzk 12540
helm/benchmark/scenarios/newsqa_scenario.py sha256=g7bFXWCTvRjofmmUJhx3jmCOvcVwi_P9LSVrNofvMLg 7260
helm/benchmark/scenarios/numeracy_scenario.py sha256=FxJJztIs16oCvMak6QpcycQKFlSu6oG3HMY9QicsSVw 30823
helm/benchmark/scenarios/open_assistant_scenario.py sha256=PH8F8zqYXXakr1xttBtFmENGMGciIpyijK4RCkJol_0 5495
helm/benchmark/scenarios/opinions_qa_scenario.py sha256=QmcqzokfZxskHa2HoEcaJAxdxZ1iQh-Qp-u4BWqzj2A 7375
helm/benchmark/scenarios/pubmed_qa_scenario.py sha256=zVL1gb3eVz-LbK2hfdnRR9ItaMSPlAGJorByWLt-4wk 7506
helm/benchmark/scenarios/quac_scenario.py sha256=SRAhMp6TAsmTRq6VRONLl3SEayFIe23He_mBhzkZ7qM 6628
helm/benchmark/scenarios/raft_scenario.py sha256=_5QhHS3opxxML7Rek6F-q5NVOf0M2UgbC6OTnQZ4C1U 4452
helm/benchmark/scenarios/real_toxicity_prompts_scenario.py sha256=GkgJo_13MWQQQTZbhlknvTR6ZrYr7NEn1WdMZrPs4y4 2400
helm/benchmark/scenarios/scenario.py sha256=do_ZO-vRnFREHosrloKDt-vFpGK0Qnml8wZc1eLKkVk 8091
helm/benchmark/scenarios/self_instruct_scenario.py sha256=jZ2MksT4N_4g_sp5egw7ycrsM-Ya786_RFmiYYdMvG8 2285
helm/benchmark/scenarios/simple_scenarios.py sha256=TbdQoeIxQAfcUOrFEGgnFrnSswzIuLPcek5RRV54GTs 1965
helm/benchmark/scenarios/summarization_scenario.py sha256=MlNMgsY369DC04nhMUdG2o9Ydi6yze1fGOjC0bK-UwQ 6847
helm/benchmark/scenarios/synthetic_efficiency_scenario.py sha256=pzifpsJJbucmTjujNqQnwQa4Y7wpQjkS6QjNXOrgTAQ 3096
helm/benchmark/scenarios/synthetic_reasoning_natural_scenario.py sha256=1b3e3WpFMNBV3li17-0Ug6QCSKO4qRFaWDF23bYNsvQ 16326
helm/benchmark/scenarios/synthetic_reasoning_scenario.py sha256=k8IGK6VABOr6wuha4HynP47peoAkmIViAVhScOtCANo 8345
helm/benchmark/scenarios/test_grammar.py sha256=sPlA36sHpThbXgnGlXyOuqHfDPe2epIafmzIeL0nkoU 1364
helm/benchmark/scenarios/test_math_scenario.py sha256=89rAt6GOAE5gBCWD5HXtCUEMcInKze-F-Is5r5LXFCo 725
helm/benchmark/scenarios/test_scenario.py sha256=OBcsrF7yCAbd3LNYo2pwFExZmixtdAyWaKRax5ngxyU 1870
helm/benchmark/scenarios/the_pile_scenario.py sha256=RqU8yXQJ4FkmEc6rO9J3QMXenyUZrsEZlLAQUx4-Wnk 4995
helm/benchmark/scenarios/truthful_qa_scenario.py sha256=iqL-tuqUQZjF9-DKAnI6wV-oLGC_I3aFuofdtJYHt8E 6035
helm/benchmark/scenarios/twitter_aae_scenario.py sha256=CRlPxVfkg3HPZV-lUMyCUSFOiAqg5IIPt-dq3qR9LU4 2096
helm/benchmark/scenarios/verifiability_judgment_scenario.py sha256=0buyJWLhy6edykqEiLrz0pZz-lA5u1UpqYny5aPuDO4 6011
helm/benchmark/scenarios/vicuna_scenario.py sha256=5z82_vz3I5pbWctDzC4kKHmtattkalSWQFURB1APYwg 1659
helm/benchmark/scenarios/wikifact_scenario.py sha256=cOVKgDisBdjPcmVMCLhTekdgX3hpDJUT-aKbvRSaMoM 5791
helm/benchmark/scenarios/wikitext_103_scenario.py sha256=PAPkmZdC4aIBQ1k29dDvTFBEFaPV1ZR1Ifif4FHoZqs 3087
helm/benchmark/scenarios/wmt_14_scenario.py sha256=Rsx3RpY7NYTS3Oz1PuGSaJLnDQoXTeSxWg2ddt8Q4ak 4474
helm/benchmark/scenarios/vision_language/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/scenarios/vision_language/viz_wiz_scenario.py sha256=zrG5D-tJvPIxJRrMY3GDbko8UQAAr8CGV7660KoAl0w 4202
helm/benchmark/scenarios/vision_language/vqa_scenario.py sha256=w2acgyq0kFhh3iGkyCcHKAqWcEZvNEukCd0Nh5ay8TE 5111
helm/benchmark/static/benchmarking.css sha256=7PsUCff8YcoVxyWuALItfB4TZL55T7MbVQpoc9nZznc 2104
helm/benchmark/static/benchmarking.js sha256=hqRbqumir7DfnhWhT_bdGwo0bftd10YTONaGetsklpY 54529
helm/benchmark/static/config.js sha256=kIfkgr6gaMdFOAdqB35EvuBohq0DWYSQZbe_pTK09VM 103
helm/benchmark/static/contamination.yaml sha256=rAfh1DqwyUcDtyzHPQ2QiUK5eY7QfuuRtBXpZMn4TeA 3171
helm/benchmark/static/general.js sha256=qcsntanG5UMWK2vznSVAVFy9zd3BMc8DFfNa7KKezew 3053
helm/benchmark/static/index.html sha256=xIJGjMg0qn9eemfdBiNbTI0jzPfBD5x0v8HJF-dMqBc 3561
helm/benchmark/static/info-icon.png sha256=P-PW3Ek3NGiRAW5BXOjJRPBfMVqprjAqtQheGWu7zNI 3428
helm/benchmark/static/json-urls.js sha256=AaULgfHw8OLfrQLJpBHfcC013uavQnlNNFS9vzb0qOg 1981
helm/benchmark/static/plot-captions.js sha256=bTR8gYx-QqF_RJyKX-L-eQP7hSEtawfJSoADCvgjKag 3011
helm/benchmark/static/schema_classic.yaml sha256=S5Z3vMdWM9SGABDCWhQfQuXcuPS-f0BSjJjeTQbFDeU 151999
helm/benchmark/static/schema_lite.yaml sha256=r5XkUMX001FyguskqudMPjyCLjw01EJVT3DdelV3G8g 51318
helm/benchmark/static/utils.js sha256=bgN0PT53Dregc-nLmEmAEmg2psufWpS8jTf74WoypHw 7681
helm/benchmark/static/images/crfm-logo.png sha256=dDkauL_wJR_Luu7L7pltphS3a9HSLjDkpVLa6C9vcA4 62712
helm/benchmark/static/images/helm-logo-simple.png sha256=LtVAC4OgcWgMAob53rTrf7cRDu-O0z85ZOGGj9wR9hw 86133
helm/benchmark/static/images/helm-logo.png sha256=GTqbrxJr0oQXbBRq-8v6afY5zB5x0M6PhEbKRIX9qIE 280667
helm/benchmark/static/images/language-model-helm.png sha256=mG0-bkdziXeiF0wOGd67y2jnYmVKJYqhD2N5Q8VIF8Q 26563
helm/benchmark/static/images/scenarios-by-metrics.png sha256=F7g9mvIYopm-n7sDGg-7I0XCyZvloKsi2wIq1i6da_Q 51331
helm/benchmark/static/images/taxonomy-scenarios.png sha256=2MiuCLaxnuHvwsWWJHnZFc-rvoQIi_tNIjDatY7I-Dg 100766
helm/benchmark/static/images/organizations/ai21.png sha256=Drkew6Vlwi2_4_S8hjagK2x8smOwLKTNiXIT3rDiurs 10208
helm/benchmark/static/images/organizations/anthropic.png sha256=cNi8OdIshIIb8PdodcX8mAj-khaUD0O6nhah-_6nYfs 8017
helm/benchmark/static/images/organizations/bigscience.png sha256=fwQAwN1x2Fr_ztD_HZdcOkdFcyxuDjtS3B5-VuRNkuc 19036
helm/benchmark/static/images/organizations/cohere.png sha256=7cr4LI8WK9yPryQboyWK_T5baSND-d-tVrlPNflLQMg 8757
helm/benchmark/static/images/organizations/eleutherai.png sha256=uUURFF8YWY85mwGoKVEjArO5DUBCy4es5naCXsBzn6c 4526
helm/benchmark/static/images/organizations/google.png sha256=BtmXrVQZHr3WH5c8c23ent2FO8aPWeNwO8czl22lDCo 4914
helm/benchmark/static/images/organizations/meta.png sha256=VYDp8arkAe2eYRJhAOcIAsZY1qY0hqyOEQDgVMbX9M8 4646
helm/benchmark/static/images/organizations/microsoft.png sha256=9e5QFl23yTbnAk8u7lZKaQOf4oPHbr_aiQda5n4MZqE 50850
helm/benchmark/static/images/organizations/nvidia.png sha256=hvp1wZMwYxkfrVMvJs73PX71JwY5L8ZvxIH_fL4n6Po 27945
helm/benchmark/static/images/organizations/openai.png sha256=P4ZT5ISIlt6Dl0mOp7juSM4Y7dfyRNPqdc0PJuwNoqg 16877
helm/benchmark/static/images/organizations/together.png sha256=pmWjW4r7GnlKqFhKLPTiBeILiOighL3XzcSCsxWtB7U 48053
helm/benchmark/static/images/organizations/tsinghua-keg.png sha256=l9SzlZCsLF18BY876wYJcVgiQbgvwte7uoILPDcVwHk 7776
helm/benchmark/static/images/organizations/yandex.png sha256=OOCdcKubAP4x7h4VW7z5a-AHPWBiSDTjsIJea6ZiovA 27964
helm/benchmark/window_services/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/benchmark/window_services/ai21_window_service.py sha256=aV8uHeAc5_DmbBnh4tgLEgcC_QxdkPotl8Asyr2K1KA 12826
helm/benchmark/window_services/cohere_window_service.py sha256=dp4KTmbwxdHwxy7Eogfr_p0_F5gayFR86PNhUHaqb1M 7091
helm/benchmark/window_services/default_window_service.py sha256=TTVs6MXmyHQaZESd_J0DoYpDgCeTm9HIJQfBF0UlctI 1595
helm/benchmark/window_services/encoder_decoder_window_service.py sha256=Dg_rD0RtKIALtpvT4Wi4Am3zgFcIvgxfzdglw5fbbTU 2478
helm/benchmark/window_services/flan_t5_window_service.py sha256=39IZX89_tay3bpSGVoWDoekmhW-RUNATQuT-bNyFRTs 915
helm/benchmark/window_services/gpt2_window_service.py sha256=d-ys9FHoVI1u1GZOH734JPAOc8W6QZb1N3ZjooKmwz8 990
helm/benchmark/window_services/huggingface_window_service.py sha256=n6x2yWSUSO7bTlQqne4MbPhmZDLuN5KF8TkdTVVfxvM 2357
helm/benchmark/window_services/ice_window_service.py sha256=5z52rP-xAF_jckIPoogyoNFW6FQXqaq3SHybCaBuRn0 2005
helm/benchmark/window_services/local_window_service.py sha256=9nGiWMvmmqmuky3fe7rAAae9vGQYiD1HlSCtSUXk37c 4547
helm/benchmark/window_services/no_decoding_window_service.py sha256=s_i_cqIuU9p0GDRIBApaOHzjH7gHrBPTJ2X5NEcN33Y 1375
helm/benchmark/window_services/t0pp_window_service.py sha256=oa1vJRiyFPbkTb8eYnfjZNyKFJSvceipOq0U3Ys5e04 1196
helm/benchmark/window_services/t511b_window_service.py sha256=8CvkSfuG_Bg17gAEre_bmM4tUwoi9fAWpPxx6qCJwAE 1005
helm/benchmark/window_services/test_ai21_window_service.py sha256=HkpNSaJAClZfaa-bQZ2BrRm1UB_u4sLAGSBlGQqRUD4 8221
helm/benchmark/window_services/test_anthropic_window_service.py sha256=Zo3IHsKtE2yf3E29_FgBd_dNbwtgTO6LPArzXYEwI-8 4092
helm/benchmark/window_services/test_bloom_window_service.py sha256=fG1oJdfErc0m-r-x7WalQ4mW49R76Wt2F_bUuYDSEb0 4184
helm/benchmark/window_services/test_cohere_window_service.py sha256=j36eIUywlVqDyDnDlgKA4HgT8YE-ItmvOqCgaVk3csM 3240
helm/benchmark/window_services/test_cohere_window_service_utils.py sha256=sf25f9MeXzoqsbDzZ7d7le13hm8RkDe54nhLtKF2pqo 158150
helm/benchmark/window_services/test_flan_t5_window_service.py sha256=xv_EXbiRklveJPQtThYCSYF7qBYwjL7K4wH3Xu5z2Fg 591
helm/benchmark/window_services/test_gpt2_window_service.py sha256=3k25pLa_z__g4yoQL40DEXj-T4dGtrgif13N2NXs59U 2568
helm/benchmark/window_services/test_gpt4_window_service.py sha256=IR8JGLViwiFDmHOLgBfnPGeAJlV6IyRKt1BHOQ4P4X0 948
helm/benchmark/window_services/test_gptj_window_service.py sha256=sxsTpozKv9N-wZXtGl1prkQr9Md_q-tnCjO9zt226Co 2267
helm/benchmark/window_services/test_gptneox_window_service.py sha256=VwawHQnvrUKUr9Tl0eKjNLtZLEmpJPGxAUsAYMY1hlg 4103
helm/benchmark/window_services/test_ice_window_service.py sha256=hSJ52MQ2Edh6gTvgzCHFMvlZuidRhZF_t2E-RR4H_6A 23475
helm/benchmark/window_services/test_mt_nlg_window_service.py sha256=2PtD1orlVYYNa_JOV0_3bQPct3zvUx0YWzTnwwKZ74I 2212
helm/benchmark/window_services/test_openai_window_service.py sha256=q4ou-AK7-NvUkwAQP4ZuldrAgoGeDnfqM8Syj-7KZfA 2208
helm/benchmark/window_services/test_opt_window_service.py sha256=Fn_-q-yfh8wHhIdJ4ysTDmRkbz5S8I806NWLwRlTcW4 4108
helm/benchmark/window_services/test_palmyra_window_service.py sha256=WYSpFyr8Ll0-umMnqvUbIz69Vs4urqH4regmeC4sed0 4108
helm/benchmark/window_services/test_t0pp_window_service.py sha256=O99CgeJWEqfh5t6__jr_RrcYUvm2JplDbD3MpN9Ay4I 3894
helm/benchmark/window_services/test_t511b_window_service.py sha256=YtwycL2-oPLkWkZduSkm7AWGWzXJQC9A3sLjo7dy1-w 3897
helm/benchmark/window_services/test_ul2_window_service.py sha256=otiqcOPPFDgPzB2avNOvdmJwVUZH443J2KiQQ787xcI 3957
helm/benchmark/window_services/test_utils.py sha256=TMmPTkJ4vlFKv4vzSO35B6-0gnXbQ0nvNZJR6CZN6qo 3227
helm/benchmark/window_services/test_yalm_window_service.py sha256=NDvZyDkgO-eYjt1oEieAJiNFFgt9qqXAa38CqWaWq4c 4197
helm/benchmark/window_services/tokenizer_service.py sha256=dPsEhA1I4SUvsR0UBdzJ2wsFahw8GuF6qhE8CxiPZRY 1051
helm/benchmark/window_services/ul2_window_service.py sha256=R_VEzOb59zQE9mmbTLunQeIvLAtK3-97h-B2_oc0Uxs 1021
helm/benchmark/window_services/wider_ai21_window_service.py sha256=o52sBY0OgHEiPrMv5rve8FzE4uXy_QuDFdjTgqQpZM8 727
helm/benchmark/window_services/window_service.py sha256=YmMyFRBkQ1sH2bGPEuUBC7x4nt6n8wKRVWyENlr6B28 3361
helm/benchmark/window_services/window_service_factory.py sha256=SVyBM3nRITDc9fOk8KUfmXZ79hE-P3UBrFtPHFxLbYQ 3230
helm/benchmark/window_services/yalm_window_service.py sha256=tPOAiUsz3Z6aAU-nBq81lLDtYttZopRGJEcLNfCVBak 1813
helm/common/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/common/authentication.py sha256=RlMx29_TSrfU7ujE7dJkxmFub5EqLj2NswV5lAVFFDk 179
helm/common/cache.py sha256=uq95XlMjFRsQ9g6yCrfX4SE-daaVi59K14CcxGZPGmg 7199
helm/common/cache_utils.py sha256=sLv6YBu6Js2oGa_c2LXrch3bInu7scthQoMoWzbgTi4 522
helm/common/codec.py sha256=gTh6AwIQ0Bbul_QSnIO7eItwMZmYtnkIrG1jkc4GOL4 7100
helm/common/concurrency.py sha256=8THtHlCtXo5c8iCuz_UcBBdzZX6aiEALLc4u0M4SYL0 856
helm/common/credentials_utils.py sha256=O-57nUgkWLbZF0k3lsSaVGPPHj2_OYeVuCMe0to3bRE 1118
helm/common/critique_request.py sha256=Exu8Ans05zCU5d5-AglEbG40mBwKYED2Z3WqY_XjXBY 2772
helm/common/general.py sha256=nlZ6jRgdSNpM66BWEmjTkun8AXm0IaLyBZnrnnsLyk0 11679
helm/common/gpu_utils.py sha256=pmLq6ipYNLEm28VxxSNeZuVt-gAw-WnYmBvxP1P1p6M 480
helm/common/hierarchical_logger.py sha256=EnKLnfbQftca08EJfjGEQb4tcnCKbx-JtwLnoCnhMQs 2908
helm/common/images_utils.py sha256=Jp6nlqYfrBvtwuWdWThQvuSHWWw1qM2WQH7KJfxQ0VI 1514
helm/common/key_value_store.py sha256=cZPYMc4Fp3oIk3O908YCyIFIx-KVIBrVKUO0qeLf9WU 2333
helm/common/media_object.py sha256=nAl4phYCAl9TL1q61TlW4zwInDcdgwzYr2X4JZfFJSU 4709
helm/common/mongo_key_value_store.py sha256=Lzi7m5NeJeHBOGEZUMPQiaFxz_U88xzyXNdU6OQkAec 3796
helm/common/object_spec.py sha256=_usgTDQULBF6_jy7C6m-9ZNVvNxbGoTE_CdGcSvBASU 4327
helm/common/optional_dependencies.py sha256=Qam3QCHff8tuXbS-fCw-MVe-pK18gSvHw-uQoXXxT7M 616
helm/common/perspective_api_request.py sha256=WAVwtajNVmi5XJNsPcorGEAVrqkpPSk-Kd3b0hJghbA 2427
helm/common/request.py sha256=mWDXulEc_o6OQlrgvykGttWE_2XAXNnlGTRl1hCQFWc 8019
helm/common/test_cache.py sha256=XqboYHQAkFWIHPsuIjuageRSLeN7QoATKF7wwxggPqE 7054
helm/common/test_codec.py sha256=igL--k-2DwAy0eoMr8D9Xs8MOjBoT0LutbMPzDlTNkM 5885
helm/common/test_general.py sha256=zOxSwWNgWnWHsXKcG4NZ50GkWicn4uZ4jPVypSwFaQE 1672
helm/common/test_media_object.py sha256=AAm9DD7MC-ZvTwiqXA-e52U6L4S1noxItW8f7ARi6DY 1650
helm/common/tokenization_request.py sha256=fjEU2yi15vbf16tuoGEMEJ5vKpxfn_nWXwIstZ3M7gU 3546
helm/config/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/config/model_deployments.yaml sha256=a35_i5UbzfnbvyNBrgA4lWXVp4Bx_ORhyW23MYXZrSQ 46211
helm/config/model_metadata.yaml sha256=49aFr-2zWlyf9sjobOxZjChPd6xO6Ui0cFNPfnHZJCI 67906
helm/config/tokenizer_configs.yaml sha256=4I0gfNlOM0rrTnVnJ-ntAdw9hhcas18Nx1DvcTzFRSk 8731
helm/proxy/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/accounts.py sha256=AZ0hCCD3ygSXtgZtnFJFJE1jOSRh5gMtQSM8TeJzqn0 13533
helm/proxy/cli.py sha256=l8F7UYqrIOoBD9ZCIxJFA4fhxlzhae0-2Nn8A7FMkzk 8244
helm/proxy/example_queries.py sha256=qTUY41MK-i62TvqOIQCsb365-RdxMb6j5lVF3LNqX2g 4263
helm/proxy/query.py sha256=eftbiUICMh8QIHVs-7cLtv_rDXKeKdRPmwjLMu0TDxQ 645
helm/proxy/retry.py sha256=iLZmKATEJQa9jsSpOIx6YDRhmrA8G1Qm21cUxCuo2Ug 3490
helm/proxy/server.py sha256=iIfZdVYmfYcanseWIiaHRGJX5Wa5Oj0BVkanomvsCIQ 8020
helm/proxy/test_retry.py sha256=8h398auzjW9VnlTJWllxR-bdpub-XFp8EN8LWDEnEHM 1049
helm/proxy/clients/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/clients/ai21_client.py sha256=-ju4YwPGtimP3Jp2q8YLFRZh3BlHGfhg0l00vDxWdwA 5865
helm/proxy/clients/ai21_utils.py sha256=mlg3h615kyckccGZv9rqsP4Y60O3XpwyE-UURRMrxII 471
helm/proxy/clients/aleph_alpha_client.py sha256=eCW0nXGFQevbm3YzJ9SM6Udo4N7D6Hra4KdB9OybG1g 3987
helm/proxy/clients/anthropic_client.py sha256=uUGIQcyH4mjmG6zuOrdyGCcZXoj937dyIJdOAIePcz8 23945
helm/proxy/clients/auto_client.py sha256=1UTOwsgpVI3gRSHVQZBGtbGGPPeb29sbzF_nB3qH8jo 9469
helm/proxy/clients/client.py sha256=rCvFnrPXggXvCv_iVoF8WEjRF_K7uqsQG1tDkiFcRKE 5753
helm/proxy/clients/cohere_client.py sha256=6JfFwWXnGB3e_iDJfAtrSz9fVy1VWz5PTl2LkDRs3gY 7565
helm/proxy/clients/cohere_utils.py sha256=aYmj60m0e9RF9BIdxp1vmA-uZv17TEALw0dbgTUSpCc 504
helm/proxy/clients/google_client.py sha256=cPcab3GBYIS4cMRLzgAAwYGR9uzN2ArAuvTeJRI3XiE 2906
helm/proxy/clients/goose_ai_client.py sha256=D3U0At_wiTDMttVbEo6b_x7QpBWM1zYhlZb2GruG-b8 3817
helm/proxy/clients/http_model_client.py sha256=efpoT2y5aDd-Fr5hTF_4j9buv6_4kv1wpbY2_mGAyiI 2826
helm/proxy/clients/huggingface_client.py sha256=mGBqVj-QfZ61cF89FFfPLaWwzqNqV1aoGmtmAaFq8QE 15061
helm/proxy/clients/lit_gpt_client.py sha256=FTcIU4oHAeAych6I68SHwYaNw4ngY7jzxLBRhjgyQZc 6214
helm/proxy/clients/lit_gpt_generate.py sha256=8DdBE9ReQ00NbV3KMFYc--PlO9X-HMOR0Rhm5CADWEA 3103
helm/proxy/clients/megatron_client.py sha256=hFJW_1PFvirYW3vUnooCUX3_xhx5nxbw58tx7_CmVrA 4122
helm/proxy/clients/microsoft_client.py sha256=1-vTybeAxz_mgaQZ4JBfVhiQVA2Zir-2o9ebM7Ze7KM 7838
helm/proxy/clients/openai_client.py sha256=w_ZRbMWknO4zyC5jfYM2hkXUvLBktDSLi4uk3lujAwo 9822
helm/proxy/clients/palmyra_client.py sha256=TNv1PXkzXj61TRgekHFJLdYyaXmjL1L2IYKUyBM_VHE 6565
helm/proxy/clients/perspective_api_client.py sha256=wwfD62GdxKsULZNXHuzx-_JUde96MFIBwZw5QGrVzc0 5880
helm/proxy/clients/simple_client.py sha256=uLa8fD56UV7EMs_F4CnpLwHXZD8vfijv_hFYzyM0dbI 2124
helm/proxy/clients/test_auto_client.py sha256=q_UWDZ4C34KKXT8rxmO-4ZR81KXqYUIcjrT6rxLG-Ns 3587
helm/proxy/clients/test_client.py sha256=Oqj53D5JC3-FdfOGHUdOvnOvklKTd_XVf7Tf7Xo8yno 1722
helm/proxy/clients/test_huggingface_client.py sha256=Eg1Qjr2uLScgUdlvpup-lTloH0YzXTlR74z_mgNnoNo 2613
helm/proxy/clients/test_together_client.py sha256=SDhvSTO4QhzQsd5l3b6Sn_XEABuUwalmw19cB6pOz14 3432
helm/proxy/clients/together_client.py sha256=JX1l2sVjVlVgwldHxhgvyxbVGo0-jP7ta-Q38XXP5kQ 15713
helm/proxy/clients/toxicity_classifier_client.py sha256=AI_FizxMurubTIyeceRdkixSnhWQbcD-oEEONj5ve7o 464
helm/proxy/clients/vertexai_client.py sha256=9jE7gVT0NfhICAe1nLbK1cQBWHncT8UJ6cFbNU0_jgQ 5195
helm/proxy/clients/vision_language/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/clients/vision_language/idefics_client.py sha256=WJAAFE13n6qlj7YqCYJesozRtGq13a99qW_7la88gHQ 7252
helm/proxy/critique/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/critique/critique_client.py sha256=ATZuXw77lejwtpgLg3Soy3VDyv8D8xetl0o4guxDM2M 1764
helm/proxy/critique/mechanical_turk_critique_client.py sha256=OcppmFOMweBSfVTiLIICIwjvPpHHTkdu9fFUTaubitQ 574
helm/proxy/critique/mechanical_turk_critique_exporter.py sha256=taULrc_cIP0O9c5UpGz3l9DmWQadTVzN_v-qzTgMoyo 8470
helm/proxy/critique/mechanical_turk_critique_importer.py sha256=fhMf4O9O712GEQD2lLv7HQDtN2BARhOCrnDmlMYbwmw 5535
helm/proxy/critique/mechanical_turk_utils.py sha256=mKpUv4zz3s5ptzDY7UrwuI7Cr5HmNgSjPC10BnN9AL4 1766
helm/proxy/critique/model_critique_client.py sha256=VrZrYQIUur8NT-jA18lh8aJItP3jRJdHBx4euXY0Kk8 11144
helm/proxy/critique/scale_critique_client.py sha256=B4povtceyfal95eE3N7em9cC_B5Vy4jMrHXcsXc_5m4 15889
helm/proxy/critique/surge_ai_critique_client.py sha256=HnzgAoF4Du9Me0GS_lbNaozZslS4a2OZx735gh-coo0 8357
helm/proxy/services/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/services/remote_service.py sha256=YudZE8RWX4cB0YqYCC8yUPLrT_nfZmaSpD1tubhkT-Q 7395
helm/proxy/services/server_service.py sha256=doMe3U2AFS-Rm9xfBTct4BUMXenpXG9lY25ns-_FdcE 7931
helm/proxy/services/service.py sha256=pZicxz9eOQn-CmQ-EHrZ0Apfp-OZXjopiLW5XduWMdE 4900
helm/proxy/services/test_remote_service.py sha256=qZrDGDpvhxn3Ip3hjWo0kJ9POtSV2W3oQy67UhrlGSo 6527
helm/proxy/services/test_service.py sha256=IdKg_Qs64w9--HhQnbl5XnHZNI4F4tZ9YNzrCT8frjE 9553
helm/proxy/token_counters/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/token_counters/ai21_token_counter.py sha256=Y0LmjyLarUIlshQIhcsvD-nUJQ1KmEAW-jJuinYAJrQ 962
helm/proxy/token_counters/auto_token_counter.py sha256=4lbYAsVwXMjbN1blac_6zgLTpSJaThQhYp_yw_TU_s8 1866
helm/proxy/token_counters/cohere_token_counter.py sha256=DHzZ0V3CnUvWvTdgCBjC-SEBbYxu_A7kfXsRdaFr0ro 488
helm/proxy/token_counters/free_token_counter.py sha256=DVdKgaQDxd42A8D9elcSunmtxAAtzUr1sEPRLJQ4ZSk 399
helm/proxy/token_counters/gooseai_token_counter.py sha256=cY5-3MJcTf0Wzpk5c3-8C8rXkEmdr4yBPv60SuBFtPI 1071
helm/proxy/token_counters/openai_token_counter.py sha256=ZIrKmogZZPFn_ay9HFZLRNK4s9YVh9Byxz64QkUQdOE 1117
helm/proxy/token_counters/test_ai21_token_counter.py sha256=rDRA_TNbH1rLpiBnsJH7sg1DFQxHg1-1ULwXvLZp3q8 5610
helm/proxy/token_counters/test_openai_token_counter.py sha256=k_HH3pP-fKxCIUDM59HLbpSWdvrDINj7j7JVN1QF9qY 5024
helm/proxy/token_counters/token_counter.py sha256=x8KyTR82EedgCQUuneQiVq9AiU1B3_CHPmKPNumClHc 429
helm/proxy/tokenizers/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/tokenizers/ai21_tokenizer.py sha256=snBAumRIGwhTFm3oeyVexpla7g6paeTIIIMr51f-S44 2238
helm/proxy/tokenizers/aleph_alpha_tokenizer.py sha256=UlWC_SjObBvexpZ3OfKZT2yjhbSsHlKjQe_oWuRrXno 3818
helm/proxy/tokenizers/anthropic_tokenizer.py sha256=15nsRyrm_sKWgrdR0sX95UDd5PGR8aAp-NjyZKbeaNU 2102
helm/proxy/tokenizers/auto_tokenizer.py sha256=1mG2TbznYuQikOALDHDf0Rq50Qf1o4s0t7-A4m0cFHY 4299
helm/proxy/tokenizers/caching_tokenizer.py sha256=kSegrCFotRevSDgJsn0g52dWiSUCNa7_EZpRNrELeUE 8163
helm/proxy/tokenizers/cohere_tokenizer.py sha256=9mRLdwJhy_JgUW-y6-CFc_iez2raky3S7UqPJ0dcCbE 3745
helm/proxy/tokenizers/http_model_tokenizer.py sha256=nfIANZt_0UCh6SM7VZI72wmBov80PkE38v5kgeVPD70 3062
helm/proxy/tokenizers/huggingface_tokenizer.py sha256=G2V4B7SwXS1Vsd5eytFV1xqgMVMzKFQyPHL4prIdbI4 8379
helm/proxy/tokenizers/ice_tokenizer.py sha256=4ZTIRpmt2cqwcxnmrDpCRhiJ0BI3ELE-GHoBuHWgrDA 1200
helm/proxy/tokenizers/lit_gpt_tokenizer.py sha256=LMrpaje64UmnDKoYjPG_RQeXVA4xQUwW5t48IJIeLaQ 1660
helm/proxy/tokenizers/simple_tokenizer.py sha256=GjQz0D1IQ3reXO5e6fkYkn7a9b_vmmIKch2wT13qljE 1134
helm/proxy/tokenizers/test_anthropic_tokenizer.py sha256=nN2YQMWtNCCFmETZA6OBGhh9NDoz6t0xnA5OLTHoTVg 3798
helm/proxy/tokenizers/test_huggingface_tokenizer.py sha256=4A7F-9eCWOlQJk8QRwzX1h37YZJf6_3b7DwXUBc7BPo 6041
helm/proxy/tokenizers/test_ice_tokenizer.py sha256=-xi_f8TBSkAYr5CcA56HDq7rZ9HAGd99J7twNfkLzFU 2619
helm/proxy/tokenizers/test_yalm_tokenizer.py sha256=qWpKnUuAlePd6t-UJB_mAiBwtAacnC8caKXLJ_GdTkk 2477
helm/proxy/tokenizers/tiktoken_tokenizer.py sha256=FU2g_FF0pVoyspYhHcz3SyCBGNbsTby-nWVrj0Cq4_c 1265
helm/proxy/tokenizers/tokenizer.py sha256=1zi9fhgrs2znINKDnoEhW-6UatsVj30hRjQbTDbOdZg 1583
helm/proxy/tokenizers/vertexai_tokenizer.py sha256=mzlc7ZJt1fRT2Ib82-Am-UALOBjjLSgmYbDG3VdhkdM 4111
helm/proxy/tokenizers/yalm_tokenizer.py sha256=uN7ZLiqG2uCmL1iWy6Y_Je6XuATyvqupWPjJDLnjdNk 1411
helm/proxy/tokenizers/yalm_tokenizer_data/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
helm/proxy/tokenizers/yalm_tokenizer_data/test_yalm_tokenizer.py sha256=W9p5QNn1GSm-y85yVEQe_82zn5CVK_vR6jvhk7JTs_k 869
helm/proxy/tokenizers/yalm_tokenizer_data/yalm_tokenizer.py sha256=wXP-LmFLUjRfCExkNHbwVUINE8qFPwvv9h7Sf_pVNbw 6395
crfm_helm-0.4.0.dist-info/LICENSE sha256=bJiay7Nn5SHQ2n_4ZIT3AE0W1RGq4O7pxOApgBsaT64 11349
crfm_helm-0.4.0.dist-info/METADATA sha256=xHtXfsOCRLde00vrfKYVHJ9018nSXred3WxwFXUWkxw 13501
crfm_helm-0.4.0.dist-info/WHEEL sha256=oiQVh_5PnQM0E3gPdiz09WCNmwiHDMaGer_elqB3coM 92
crfm_helm-0.4.0.dist-info/entry_points.txt sha256=AvH9soAH3uey9xffisWewd0yrmPWGASC036jHd1SFyg 300
crfm_helm-0.4.0.dist-info/top_level.txt sha256=s9yl-XmuTId6n_W_xRjCS99MHTwPXOlkKxmTr8xZUNY 5
crfm_helm-0.4.0.dist-info/RECORD — —

top_level.txt

helm

entry_points.txt

[console_scripts]
crfm-proxy-cli = helm.proxy.cli:main
crfm-proxy-server = helm.proxy.server:main
helm-create-plots = helm.benchmark.presentation.create_plots:main
helm-run = helm.benchmark.run:main
helm-server = helm.benchmark.server:main
helm-summarize = helm.benchmark.presentation.summarize:main