|
R is for Robots, Spiders and Crawlers
Search engines, such as Google, Yahoo, and MSN, use special programs to access your website, evaluate it, and index it. These are variously known as robots, crawlers and spiders. The metaphorical allusions tell us that it's an automatic process and that it makes it way endlessly across the WWW. Google's programme is called Googlebot. It makes it way through the web searching out links (HREF and SCR links). Yahoo doesn't seem to provide the equivalent information. I've had to go to third parties, i.e. other websites, to find info about Yahoo's ways of doing things. MSN follows Google's example and info for website owners is easy to get to from its frontpage - MSNBot.
|
|
Or take the Brighton
Quiz by clicking here |
||
