GitHub - FanFicDev/python-weaver: python web crawling utilities

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 14 Commits
sql		sql
src		src
README		README
bulkDump.py		bulkDump.py
bulkImport.py		bulkImport.py
dumpId.py		dumpId.py
dumpIdRange.py		dumpIdRange.py
dumpIdRange_xz.py		dumpIdRange_xz.py
dumpUrl_raw.py		dumpUrl_raw.py
dump_minerva_fs.py		dump_minerva_fs.py
enqueue.py		enqueue.py
prescrape_queue.py		prescrape_queue.py
prescrape_remote.py		prescrape_remote.py
wcache_bench.py		wcache_bench.py

Repository files navigation

python-weaver is a small utility for web crawling in python.

This includes a python library called weaver (found in src/) along with a
number of utility scripts written against it (top level *.py files).