Science for SEO

I’m often asked how to make a spider to crawl the web, or a specific site, a directory, look for hubs, and so on. They’re not hard to make and there are a ton load of them out there. There is really no need to reinvent the wheel and write a load of other ones [...][...]