Wheelodex » Projects » crawler4py

crawler4py

View on PyPI — Reverse Dependencies (0)

0.5 crawler4py-0.5-py3-none-any.whl

Wheel Details

Project: crawler4py
Version: 0.5
Filename: crawler4py-0.5-py3-none-any.whl
Download: [link]
Size: 34065
MD5: fd7944cf62c5fca37958662eaabf98f1
SHA256: baae0f07631a9dbbcafeab4244e4033c6177c1549c3d480e168303f08b5c77f7
Uploaded: 2020-08-29 23:05:26 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · entry_points.txt

METADATA

Metadata-Version: 2.1
Name: crawler4py
Version: 0.5
Summary: A distributed crawler framework based on Python
Author: winner-hub
Author-Email: 1344246287[at]qq.com
Home-Page: https://github.com/winner-hue/crawler4py
License: Apache 2.0
Keywords: crawler4py crawler spider pyspider
Requires-Dist: DBUtils (>=1.3)
Requires-Dist: Faker (>=4.1.1)
Requires-Dist: PyMySQL (>=0.9.3)
Requires-Dist: PyYAML (>=5.3.1)
Requires-Dist: beautifulsoup4 (>=4.9.1)
Requires-Dist: bs4 (>=0.0.1)
Requires-Dist: certifi (>=2020.6.20)
Requires-Dist: chardet (>=3.0.4)
Requires-Dist: gne (>=0.2.1)
Requires-Dist: idna (>=2.10)
Requires-Dist: loguru (>=0.5.0)
Requires-Dist: lxml (>=4.5.2)
Requires-Dist: numpy (>=1.19.0)
Requires-Dist: pika (>=1.1.0)
Requires-Dist: psutil (>=5.7.0)
Requires-Dist: pymongo (>=3.10.1)
Requires-Dist: python-dateutil (>=2.8.1)
Requires-Dist: redis (>=3.5.3)
Requires-Dist: requests-file (>=1.5.1)
Requires-Dist: requests (>=2.24.0)
Requires-Dist: six (>=1.15.0)
Requires-Dist: soupsieve (>=2.0.1)
Requires-Dist: text-unidecode (>=1.3)
Requires-Dist: tldextract (>=2.2.2)
Requires-Dist: urllib3 (>=1.25.9)
[Description omitted; length: 50 characters]

WHEEL

Wheel-Version: 1.0
Generator: bdist_wheel (0.34.2)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
crawler4py/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
crawler4py/command.py sha256=uZw3ikhr_ONcN1vyQWRyKHeJau1BPxiwNUeOk71mKyo 5404
crawler4py/crawler.py sha256=voz5HUO4LBgKI6eYIkVZ-yrzK3XLvYg50kuB5SXcURo 358
crawler4py/log.py sha256=FDE_JoLgkpGT6FddjxF_Fxar856lySlRVrX9bPZ7uhk 1437
crawler4py/manager.py sha256=pljsPUHNagveW0ooTJo4p11N5XX7aYM8Oe5SS2Nk3Rg 4086
crawler4py/setting.py sha256=JCk-R9-l53rGP0BuYPUTyggkVsh4N0lB_9RyHLLsNRI 2230
crawler4py/setup.py sha256=Q6fUWdWJlJNCEDhF4lyY_OiNUJuL-qrwhUvgJWCFZkQ 1470
crawler4py/test.py sha256=CE0Yc5mAXuqOMkN-oAwZ5d4MJvCOifeZVLRkaTsD4AM 172
crawler4py/dispatch/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
crawler4py/dispatch/dispatch.py sha256=MNsvLgAxeoo65iTjROMFPgq89s8JbLD_sB0ugWwU1cU 7216
crawler4py/dispatch/monitor.py sha256=IpsdBlBNF4pbJPZAkXfCFvJAv7NkxTkKlnkkkjnkfEU 5755
crawler4py/dispatch/starter.py sha256=oY6ILMdFVDbQBGGkHixtOVXdldNL5V_5v3h-jDM9cJI 3672
crawler4py/download/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
crawler4py/download/base_download.py sha256=CcPKmbHoez-2-dT3g6K1kF5Q8aXdKhAVE22cIjdEmmQ 2083
crawler4py/download/downloader.py sha256=pmHTpKn7EaKcPtnD4gwQWdCpyal3Zd3aWFuwpi8IKTo 5511
crawler4py/download/process.py sha256=TzzTJ46ed4r4QqqAneJjRCJ27VqYDSqNkH84FAws480 2202
crawler4py/download/request.py sha256=JiF2FP6X-wT5KRsCefRG6QNApC3Y-kTdLZL8z7Kzb_o 709
crawler4py/extractor/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
crawler4py/extractor/analysis.py sha256=1f9T0TYV9kZ05suwY1ohgK-8-9E7H1aFIfdL39fDSiQ 2200
crawler4py/extractor/base_extract.py sha256=qBOw9L4U8R3mpoZ-LFA-O5041nBWaP91wPU4RXkKRIs 2568
crawler4py/extractor/extractor.py sha256=Z8Lf2vpchcWIy4TwWtlIVQxGeEi7B-Pd3Rl4H_ZLars 2424
crawler4py/storage_dup/__init__.py sha256=-5k0of_RHqSIBKMsUCzKykxToUnE1RYbw3hlQm3cdYE 4072
crawler4py/storage_dup/base_storage_dup.py sha256=hVMhvvi16AArtxtkp2IrUGkrDVDGltF4AQPguPk3HlQ 2359
crawler4py/storage_dup/storage_dup.py sha256=WHVCLJfOM0jhaTmzM2ryd6Y2GUImTLAU2cHOkDar0Cc 2157
crawler4py/util/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
crawler4py/util/commonutil.py sha256=VckzH0E16XDg6pGxXsporYbL4xNQejwDY_KM4H5ppk4 1137
crawler4py/util/mongoutil.py sha256=002t2Z-Teryl3bqw1B8J6ptH5cBKzyUZ558fNMkawX0 2793
crawler4py/util/rabbitmqutil.py sha256=wqePxIMRBd_SbXetUPD_apOFUYS7j1jBkL4YPTpWCGk 3009
crawler4py/util/redisutil.py sha256=UMM19gzB0FgLk14YX1hrFPWslON8E7y6tvEy2SsFqvo 6893
crawler4py/util/running_params.py sha256=tVVT0cIHm1zBAVRewM4gjlJ5zZ_hs6rfUyNiY6Zdi5w 98
crawler4py/util/sqlutil.py sha256=kRKphrnwdNBs6GpjH6mMRN7R62a030K3aAhxvVaiAnw 2927
crawler4py-0.5.dist-info/LICENSE sha256=xx0jnfkXJvxRnG63LTGOxlggYnIysveWIZ6H3PNdCrQ 11357
crawler4py-0.5.dist-info/METADATA sha256=ry3fbqcJ6JZE0tsKgZaFKx7M8nyElgAgJwhT2g0t7BM 1179
crawler4py-0.5.dist-info/WHEEL sha256=g4nMs7d-Xl9-xC9XovUrsDHGXt-FT0E17Yqo92DEfvY 92
crawler4py-0.5.dist-info/entry_points.txt sha256=yhfK7z1UvWifChFl10Xs5zLqgQKdE53XzZUw-yMFwGA 56
crawler4py-0.5.dist-info/top_level.txt sha256=0gxzL09QCG8HtirV3mUaexzSR_7L0EobiGtc_FoBvCk 11
crawler4py-0.5.dist-info/RECORD — —

top_level.txt

crawler4py

entry_points.txt

[console_scripts]
crawler4py = crawler4py.manager:main