Reverse Dependencies of Scrapy

The following projects have a declared dependency on Scrapy:

• Advertools — Productivity and analysis tools for online marketing
• AdyanUtils — Special package
• aha-scrapyd — Pre-built of Scrapyd
• allusgov — This project attempts to map the organization of the US Federal Government by gathering and consolidating information from various directories.
• angeltools — personal python small tools collection
• anikore — Anikore API for Python
• apify — Apify SDK for Python
• arachnado — Scrapy-based Web Crawler with an UI
• ArachneScrapy — API for Scrapy spiders
• aroay-cloudscraper — scrapy的一个下载中间件，绕过cloudflare检测
• aroay-pyppeteer — scrapy的一个下载中间件，无缝对接pyppeteer
• aroay-translate — scrapy的异步翻译，最准确的翻译器
• article-extract — Article extractor can extract title, time, author, article content, etc. according to article URL.
• ayugespidertools — scrapy 扩展库：用于扩展 Scrapy 功能来解放双手。
• bibscrap — Semi-automated tools for systematic literature reviews.
• board-game-scraper — Board games data scraping and processing from BoardGameGeek and more!
• bocfx — Easy API to get foreign exchange rate from Bank of China.
• braise — Lunch without #lunch
• bwt-scrapy-proxy — My short description for my project.
• c4v-py — Code for Venezuela python library.
• camcops-server — CamCOPS server
• catlyst — Extracting Data using scrapy framework
• cewler — Custom Word List generator Redefined
• chady — A package for ML libraries
• chronos_ai — no summary
• city-scrapers-core — Core functionality for City Scrapers projects
• city-scrapers-sentry — Scrapy extension that logs errors to Sentry
• ClappScrapers — Clappform Python scraper
• cloudacademy-crawler — A simple cloudacademy course crawling & downloading tool
• clutch.co-scraper — clutch.co-scraper is a command-line application written in Python that scrapes and saves information about firms according to the user-defined filters.
• cohospider — A package to scrape Companies House data
• coupang — Python wrapper for coupang open api
• crau — Easy-to-use Web archiver
• crawlab-ai — SDK for Crawlab AI
• crawlab-demo — Demo for Crawlab
• crawlab-sdk-ssl — Python SDK for Crawlab
• crawler-test — This is a web application that extracts images URLs from web pages.
• crawlib — tool set for crawler project.
• crawltools — Simple crawlers
• crawlutils — Base Scrapy project
• crwutils — Base Scrapy project
• data-engineering-job-market — A data engineering batch pipeline to analyse job postings.
• data-job-crawler — no summary
• datalad-crawler — DataLad extension package for crawling external web resources into an automated data distribution
• datasource-contributor — A CUI tool that automatically crawls website data and contributes to http://www.citybrain.org
• deepctrl-scrapy-redis — Redis-based components for Scrapy.
• dj-kaos-webclipper — Clip web pages html and store in Django
• django-covid19 — A django app of covid-19 API in countries around the world, provinces and cities in China, and states in the USA.
• django-generic-search — A Django app to conduct generic search on a django web application.
• docrawl — Do automated crawling of pages using scrapy
• docscraper — A web crawler to scrape documents from websites
• dr-zoidberg — Are you ready to operate, Doctor? - I'd love to, but first I have to perform surgery.
• DtCrawlEngine — 一个使用http api提供爬虫脚本服务的引擎
• e-models — Tools for helping build of extraction models with scrapy spiders.
• eagle-kaist — Stock Extractor library
• easydown — Downloader by scrapy
• ecoindex-cli — `ecoindex-cli` is a CLI tool that let you make ecoindex tests on given pages
• eintf — no summary
• elcrawlsdk — Python SDK for elcrawl
• energy-trading-api — API Wrappers for the Energy Markets
• ensembl-rest — An interface to the Ensembl REST APIs, biological data at your fingertips.
• esgf-scraper — Keeps a local data repository in syncronisation with ESGF
• favorites-crawler — Crawl your personal favorite images, photo albums, comics from website. Support pixiv, yande.re for now.
• finance-cn — python for wps
• finscraper — Web scraping API for Finnish websites
• frontera-seedloader-mongodb — no summary
• fxportia — Convert portia spider definitions to python scrapy spiders
• fzutils — A Python utils for spider
• gather-vision — Obtain, extract, organise, and store information.
• geo-spider — Crawling all GEO metadata.
• gerapy — Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js.
• gerapy-item-pipeline — Item Pipeline Components for Scrapy & Gerapy
• gerapy-playwright — Playwright Components for Scrapy & Gerapy
• gerapy-prometheus-exporter — Prometheus Exporter Components for Scrapy & Gerapy
• gerapy-proxy — Proxy Components for Scrapy & Gerapy
• gerapy-pyppeteer — Pyppeteer Components for Scrapy & Gerapy
• gerapy-redis — Distribution Support for Scrapy & Gerapy using Redis
• gerapy-selenium — Selenium Components for Scrapy & Gerapy
• gesp — convenient scraping of german court decisions
• GFP-TEST — no summary
• gggspider — 通用采集框架。
• gpt-web-crawler — A web crawler for GPTs to build knowledge bases
• gzSpiderTools — 魔改使用工具库
• habra-favorites — Sort your favorites posts from Habrahabr.ru
• haipproxy2 — High aviariable proxy pool client for crawlers.
• hepcrawl — Scrapy project for feeds into INSPIRE-HEP (http://inspirehep.net).
• herodotus — Package for fast integration between SQLAlchemy models and Scrapy spiders
• htmlparsingbs4based — This package extracts/parses information from source HTML.
• iipg — This is just a test
• imgqa — Test Automation Framework
• ironarms — Awesome ironarms created by oirontro
• ivystar — python tools package of ivystar
• jgdv — no summary
• jk-sgp-lib — Make Scrapy easier and more versatile.
• kafka_scrapy_connect — Integrating Scrapy with kafka using the confluent-kafka python client
• kk-scrapy-redis — Redis-based components for Scrapy.
• koya-scraper — no summary
• kstocks — Korean Stock Module
• kw618 — integrated commonly used third-party libraries to personal use
• langroid — Harness LLMs with Multi-Agent Programming