Wheelodex » Projects » llm-serve

llm-serve

View on PyPI — Reverse Dependencies (0)

0.0.1 llm_serve-0.0.1-py3-none-any.whl

Wheel Details

Project: llm-serve
Version: 0.0.1
Filename: llm_serve-0.0.1-py3-none-any.whl
Download: [link]
Size: 114920
MD5: 04819ca2c42cb33d74fc19a47505cbc0
SHA256: b2176a89759929ba4af2c0a602ac6cff2e4c211fc42a0d9c188a9f22f6a8d226
Uploaded: 2024-03-03 12:10:34 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · entry_points.txt · dependency_links.txt

METADATA

Metadata-Version: 2.1
Name: llm-serve
Version: 0.0.1
Summary: An LLM inference solution to quickly deploy productive LLM service
Author: llm-serve authors
Author-Email: jasonhe258[at]163.com
Requires-Python: >=3.10
Requires-Dist: typer (>=0.9)
Requires-Dist: rich
Requires-Dist: async-timeout; extra == "backend"
Requires-Dist: datasets; extra == "backend"
Requires-Dist: ftfy; extra == "backend"
Requires-Dist: tensorboard; extra == "backend"
Requires-Dist: sentencepiece; extra == "backend"
Requires-Dist: Jinja2; extra == "backend"
Requires-Dist: numexpr (>=2.7.3); extra == "backend"
Requires-Dist: hf-transfer; extra == "backend"
Requires-Dist: evaluate; extra == "backend"
Requires-Dist: bitsandbytes; extra == "backend"
Requires-Dist: numpy (<1.24); extra == "backend"
Requires-Dist: ninja; extra == "backend"
Requires-Dist: protobuf (<3.21.0); extra == "backend"
Requires-Dist: optimum; extra == "backend"
Requires-Dist: safetensors; extra == "backend"
Requires-Dist: pydantic (==1.10.7); extra == "backend"
Requires-Dist: einops; extra == "backend"
Requires-Dist: markdown-it-py[plugins]; extra == "backend"
Requires-Dist: scipy (==1.11.1); extra == "backend"
Requires-Dist: jieba (==0.42.1); extra == "backend"
Requires-Dist: rouge-chinese (==1.0.3); extra == "backend"
Requires-Dist: nltk (==3.8.1); extra == "backend"
Requires-Dist: sqlalchemy (==1.4.41); extra == "backend"
Requires-Dist: typing-extensions (==4.5.0); extra == "backend"
Requires-Dist: linkify-it-py (==2.0.2); extra == "backend"
Requires-Dist: markdown-it-py (==2.2.0); extra == "backend"
Requires-Dist: gradio; extra == "backend"
Requires-Dist: httpx[socks] (==0.23.3); extra == "backend"
Requires-Dist: torch (==2.1.2); extra == "backend"
Requires-Dist: torchaudio (==2.1.2); extra == "backend"
Requires-Dist: torchvision (==0.16.2); extra == "backend"
Requires-Dist: accelerate (==0.25.0); extra == "backend"
Requires-Dist: deepspeed (==0.12.6); extra == "backend"
Requires-Dist: torchmetrics (==1.2.1); extra == "backend"
Requires-Dist: llama-cpp-python (==0.2.20); extra == "backend"
Requires-Dist: transformers (==4.33.3); extra == "backend"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: ruff (==0.0.270); extra == "dev"
Requires-Dist: black (==23.3.0); extra == "dev"
Requires-Dist: mkdocs-material; extra == "docs"
Requires-Dist: gradio; extra == "frontend"
Requires-Dist: aiorwlock; extra == "frontend"
Requires-Dist: pymongo; extra == "frontend"
Requires-Dist: pandas; extra == "frontend"
Requires-Dist: boto3; extra == "frontend"
Requires-Dist: pytest; extra == "test"
Requires-Dist: vllm (==0.2.7); extra == "vllm"
Provides-Extra: backend
Provides-Extra: dev
Provides-Extra: docs
Provides-Extra: frontend
Provides-Extra: test
Provides-Extra: vllm
License-File: LICENSE
[No description]

WHEEL

Wheel-Version: 1.0
Generator: bdist_wheel (0.42.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
llmserve/__init__.py sha256=JzpP2A0yqn1EOzxHW5lbYomewWFcykXpL0hG26SfJgM 31
llmserve/api/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llmserve/api/cli.py sha256=KK9mW1aTsMZnuJTEEdrQ-s2u2RAL_tkHvp-42UZokWs 10715
llmserve/api/env.py sha256=LSNvOGhPin2elv73nungOgs9IxRD3MOQ8oGQsR6YjZM 649
llmserve/api/sdk.py sha256=HlEnDs7B8HixzL1nyUhnrPwve8EAwJfP5HTzLKLunq8 11044
llmserve/backend/__init__.py sha256=eFjYCCaT2vIFtX1ZDbmhf6o1T5PdTegX56OOq8rKyyM 187
llmserve/backend/logger.py sha256=xrDl8FdEhl4x0UiiNkvBmgtGEnOx0CE3s4FPARSd3l0 550
llmserve/backend/llm/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llmserve/backend/llm/predictor.py sha256=r1ziCAJ2bqaU2Y1Le3ojM8DLYd_79IUqXhj0_l-LuFo 15617
llmserve/backend/llm/utils.py sha256=aH8_5o2D-7cW3jQFPVyeq69dPao13Hjh3T5csuazCpk 10619
llmserve/backend/llm/ft/__init__.py sha256=ZWAeE1n6QY4dH26EnetNsj-Kaly9sxe29XZapPaLt3Y 128
llmserve/backend/llm/ft/_base.py sha256=wzaEtbh9f4JD7yfy8SMrJIST6HfpDac0ytauQ-rWAYA 1665
llmserve/backend/llm/ft/callback.py sha256=xqGYqy45LH6KmTT5wqZLxgMAj2fOY9SERm-HtPzTvN8 1024
llmserve/backend/llm/ft/const.py sha256=4UGWg9otzKEDRWvctWhSUotDHMHAXQ2wcTJmzC-eI8k 31
llmserve/backend/llm/ft/transformer.py sha256=UYlHXKTmJqmfUkmc5jYx2BOtdHu_PXhhB9UvhDk3FF4 9949
llmserve/backend/llm/ft/utils.py sha256=br_yEBOqI47rRvUJK7epPrUSUcaNLve3_0gZJgUyYOw 379
llmserve/backend/llm/ft/tasks/__init__.py sha256=joG9iX4EH1SrStePxueMMMKXzksqTsgul1m1vDHRU4Y 862
llmserve/backend/llm/ft/tasks/_base.py sha256=Hy3eP-51HMiYh47gXkiGxSCYm_Fc0IsSbbU-CyI8d1Q 3422
llmserve/backend/llm/ft/tasks/maskedlm_imdb.py sha256=9H6xfphMiKW79unBquZPb7fKA9Qwrbe9FonY1f7OtJE 4267
llmserve/backend/llm/ft/tasks/noheader_shibing624_AdvertiseGen.py sha256=mkafDubK9q8URiiIlANaxPlpebebdtX2k3mV6ZHdO90 4769
llmserve/backend/llm/ft/tasks/sequenceclassification_glue_cola.py sha256=WphXsup5uE71irt_Oiquk53DunxS6QhOHG2oU5i6ijg 2356
llmserve/backend/llm/ft/tasks/sequenceclassification_glue_mrpc.py sha256=jdYM33YxfDAed9u8YchD6Ko7evoiZAhAPnCXM9Qh3SA 2493
llmserve/backend/llm/ft/tasks/sequenceclassification_yelp_review_full.py sha256=keoat8D16vp_qusgGJ8Uz4W7wtTYMxQsfhY1fAJPnCA 2677
llmserve/backend/llm/ft/tasks/tokenclassification_conll2003.py sha256=4UgdNnFva99_VTy4GB8nJlqd-rzq72SGOLw4NzauOFg 4798
llmserve/backend/llm/initializers/__init__.py sha256=3Moxbi7TZVoTdoBbpI8UAbCu2Q-CLPBKUlcwrbQaUhU 948
llmserve/backend/llm/initializers/_base.py sha256=PtWB0Fb2jF56HyZRGsticzxoEtqUnVhNhzOgvEiNfgc 2487
llmserve/backend/llm/initializers/llamacpp.py sha256=pttHkJp9kqVQmwlhzxsrixfB4BkmV-KtYKxPXcYwR5U 3251
llmserve/backend/llm/initializers/models.py sha256=FNLpsLibxhWmJG5WGMuUm4H-DbSi0pVkYjjbdFVaYWU 1422
llmserve/backend/llm/initializers/vllm.py sha256=MszDrCr7-Vu9eF2_Pf5jnrdDHIBS4PPSwCR5qQghSrA 2394
llmserve/backend/llm/initializers/hf_transformers/__init__.py sha256=ikwnY490_oV4F05_jLG_n9vO1VUH-omtTBMFVrrCVfo 436
llmserve/backend/llm/initializers/hf_transformers/base.py sha256=XIgdipvmJZxayhVuV87-xuvx2M03XZ6eKDHztbzujGs 18149
llmserve/backend/llm/initializers/hf_transformers/deepspeed.py sha256=q4So2Z2Q4Qm3QAhNv9_mqnH2ecKcgQn-GPCv-80Np8s 9039
llmserve/backend/llm/pipelines/__init__.py sha256=B5fFn9EIjM6gNSLvJIPFOhOMo3PWnlLyRVIwwzyPrx8 674
llmserve/backend/llm/pipelines/_base.py sha256=Td_3evYt9QiHKCTVlwZC7EXVWil9o3jlaff2PB8CbuY 14050
llmserve/backend/llm/pipelines/default_pipeline.py sha256=Mnr4VjbyDq-tO9AANF4cdfTdvpQH-fln62Qff0rJX10 5633
llmserve/backend/llm/pipelines/default_transformers_pipeline.py sha256=KSSI5-qamWfDAb_fwxKseZTie3HVQKYLawc7fTHNlmM 8151
llmserve/backend/llm/pipelines/processors.py sha256=wBQ9Z_3zMomUxE4A902j6ezitSs-FGQMPR44XMoT3Z0 3204
llmserve/backend/llm/pipelines/utils.py sha256=TrI65-Z0vBNloGnl9u3YDrAPnXFVUZujqfzqC1c7EHI 4047
llmserve/backend/llm/pipelines/llamacpp/__init__.py sha256=59OXmbnsTchSQSvfXTC35pZJtlX29ROoOuCSCkEiwV0 79
llmserve/backend/llm/pipelines/llamacpp/llamacpp_pipeline.py sha256=bYHQSUNkgYqgLmkffnwr_-nDQd11R0A9P9mo7Qww-Js 6897
llmserve/backend/llm/pipelines/llamacpp/processors.py sha256=WqUFfD_HAPZq8OND2T3gOuc7jbJysKWGp8CK7foo_jc 700
llmserve/backend/llm/pipelines/vllm/__init__.py sha256=KPIjX5krhTlNsVK-4RzWOOYABNAvCpUcH-rNzTyeje0 67
llmserve/backend/llm/pipelines/vllm/error_handling.py sha256=CLckbYSQT_VGTmL0PlgeaF71aET4Yq9WOn3OHZKMH9I 1224
llmserve/backend/llm/pipelines/vllm/generation.py sha256=ExQgq194oeSFORi_n644sfHr0XSHn-XCwEoJiURDCT4 639
llmserve/backend/llm/pipelines/vllm/models.py sha256=4PEwM2VpXrUn3R63t8zQv4W8V1rEOIjojhivfuL6deA 4838
llmserve/backend/llm/pipelines/vllm/vllm_pipeline.py sha256=aBTWlzF2k_FxSt1Sp35rcESTIE7IjuA99YomWO4ZGZU 6847
llmserve/backend/server/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llmserve/backend/server/_batch.py sha256=WLBAD14IHGun1HEa5dPTaFXQHcoSRmxCjaWPFEKdx44 9416
llmserve/backend/server/app.py sha256=e47U6AUjVNwTNgg5APiiayVthL4w6c_wPo1t1dDTz4w 41020
llmserve/backend/server/config.py sha256=mMroszPRRAEqc50j5O-cGhjEYqtOSpHEUkHjuvrBjao 5505
llmserve/backend/server/exceptions.py sha256=4V0mIHq_Gb88cOXDdaNeZUOlRjz53abd024CSoXr1HE 47
llmserve/backend/server/models.py sha256=mPvV5URWxu-UVpRI5Wulrmtej3WdQXocmlYx-duAgsE 17692
llmserve/backend/server/run.py sha256=CnroQJbPfwrHeOoP9hnldugEsvXCI-5T9kW897ryo4I 9850
llmserve/backend/server/utils.py sha256=ZyUomXfGSxQ4D69uL0p6-6leonmMn1F2IsQ7QNcx70s 13285
llmserve/common/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llmserve/common/backend.py sha256=IqMRM2On0hh23WVwbwypE5VWw4CiV3gjAaDUk5hw9q4 6542
llmserve/common/constants.py sha256=9hjRCeYPQ4kB4HNotbsnsfpOSL6E-aJDhgVQNpVWf5c 28230
llmserve/common/evaluation.py sha256=6DXp8n6Tn9gxhLodkFTzv3J2D3iaTNdMdJY4FXc6fxM 3048
llmserve/common/llm_event.py sha256=XRePDED-xzKOG6J4_Os1OkVhurW-q6RRTmbkh_oT4mY 1101
llmserve/common/utils.py sha256=sl6sOvG8KiY3k-zpcFIw_XLaDuowTmexlzNXqXHWAzE 153
llmserve/frontend/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llmserve/frontend/app.py sha256=8wU2O680845R4Gpna4rYuX8nVrE3oY4q5J62CeShjK0 14156
llmserve/frontend/javascript_loader.py sha256=UiA-OG85U7FwgCFyh-kD7nXlBsD8pnX6ydF4fkUupG8 1973
llmserve/frontend/leaderboard.py sha256=dUs061jIwQn-sYikoMcELQVMaE3v9yIpxmkqdLS8-CU 4200
llmserve/frontend/mongo_logger.py sha256=ycNSPu4QgbXIR_BTtYwYKyWtGFg7a9borfDZpLPfzB8 2860
llmserve/frontend/mongo_secrets.py sha256=FitmI6txqIK8FNesvMZXqNMYUeP1s9sH7Y4amYozf3A 1093
llmserve/frontend/utils.py sha256=5jp2-Wf1iFok7mkTSs_E7hYX7DdADBdsWu3WJlKill4 2025
llmserve/frontend/javascript/llmserve.js sha256=Bwy8gzP2XCQH6tin4JugviSjb_E1QuZ4DYBEIOYdcCw 2187
llm_serve-0.0.1.dist-info/LICENSE sha256=g2WwaQi2u9vTc7ZDYWAU8-0CVUF9ju3cq2lJSh9jKsI 11338
llm_serve-0.0.1.dist-info/METADATA sha256=o2dnZUeAjqzMBKothyCeca8p89WKiUOnCiO-zjuloKw 2767
llm_serve-0.0.1.dist-info/WHEEL sha256=oiQVh_5PnQM0E3gPdiz09WCNmwiHDMaGer_elqB3coM 92
llm_serve-0.0.1.dist-info/dependency_links.txt sha256=Y7lsZ8fUqQwcwTUsPzmDtLq2_sPrMr_DOjqtXnh5j58 39
llm_serve-0.0.1.dist-info/entry_points.txt sha256=hPxNowf4xR7yCD0aQRyJt1V58mcPwtJb7GaTWy2T2xk 51
llm_serve-0.0.1.dist-info/top_level.txt sha256=MbghAu7jTk-L1zgYTxHFAczfg0vrWF7QYqu7mST9LTg 9
llm_serve-0.0.1.dist-info/RECORD — —

top_level.txt

llmserve

entry_points.txt

[console_scripts]
llm-serve = llmserve.api.cli:app
https://download.pytorch.org/whl/cu118