Scrapy project

scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.

scrapy/scrapy’s past year of commit activity

Python 52,198 BSD-3-Clause 10,448 449 (21 issues need help) 220 Updated Aug 26, 2024
itemadapter Public
Common interface for data container classes

scrapy/itemadapter’s past year of commit activity

Python 60 BSD-3-Clause 13 6 4 Updated Aug 15, 2024
scrapy.org Public
The scrapy.org website

scrapy/scrapy.org’s past year of commit activity

HTML 60 137 1 1 Updated Aug 7, 2024
scrapyd Public
A service daemon to run Scrapy spiders

scrapy/scrapyd’s past year of commit activity

Python 2,921 BSD-3-Clause 570 9 0 Updated Jul 27, 2024
scrapyd-client Public
Command line client for Scrapyd server

scrapy/scrapyd-client’s past year of commit activity

Python 763 BSD-3-Clause 146 10 1 Updated Jul 21, 2024
form2request Public
Python 3.8+ library to build HTTP requests out of HTML forms

scrapy/form2request’s past year of commit activity

Python 4 BSD-3-Clause 3 1 0 Updated Jul 15, 2024
parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

scrapy/parsel’s past year of commit activity

Python 1,117 BSD-3-Clause 141 28 (1 issue needs help) 12 Updated Jun 14, 2024
w3lib Public
Python library of web-related functions

scrapy/w3lib’s past year of commit activity

Python 388 BSD-3-Clause 104 11 (1 issue needs help) 4 Updated Jun 12, 2024
itemloaders Public
Library to populate items using XPath and CSS with a convenient API

scrapy/itemloaders’s past year of commit activity

Python 44 BSD-3-Clause 16 17 4 Updated Jun 4, 2024
protego Public
A pure-Python robots.txt parser with support for modern conventions.

scrapy/protego’s past year of commit activity

DIGITAL Command Language 54 BSD-3-Clause 28 7 (1 issue needs help) 1 Updated May 28, 2024

View all repositories

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scrapy project

Pinned Loading

Repositories

People

Top languages

Most used topics