Web Jargon Explained: Spiders for Search Engine Optimisation
A Spider (also known as web crawlers, web bots, web scutters, ants, indexers, and worms) is a program or script that browses the web in a process called spidering or web crawling. Spiders are mainly used by search engines, and their job is to collect and store a copy (a cache) of the website for [&h...