Name		Name	Last commit message	Last commit date
README.md		README.md
__main__.py		__main__.py
_gitignore		_gitignore
domain.py		domain.py
general.py		general.py
link_finder.py		link_finder.py
spider.py		spider.py

Repository files navigation

Python Web Crawler

Developed a web-crawler using multi-thread operation to gather all the links from any particular website.

main.py to run the code

To change the site name open main.py and change HOMEPAGE variable to the required site.

About

Developed a web-crawler using multi-thread operation to gather all the links from any particular website.

0 stars

1 watching

Report repository

Releases

Packages

Contributors

Languages