BellaOnline Forums Computers Web Design spiders

At one time I think I understood what spiders are; what is the definition today?

I'm not sure that the definition has changed? To me they're always robots that scan through a website in order to index it for whatever nefarious reason the company chooses

Is that like spyware, Lisa?

That is what I thought it was previously - search engines indexing websites. Are the posts on this site indexed on search engines?

LOL Phyllis I'm sure the search engines would bristle at that comparison!

Spyware installs on your PRIVATE computer - without your knowledge - and steals your information without your permission.

Spiders index PUBLIC webpages which are on the internet and deliberately there for public viewing. They index that public information in a helpful way. Like a Yellow Pages. If you want your phone # not in the yellow pages, you can opt out, but otherwise it's a public resource. In the same way, if you want your website private, you can hide it.

So spiders are much different than spyware.

Yes, this website is a public website. So posts made on this site are public, are viewable by the public, and therefore spiders can see them too.

SEO spidering is a very important part of a popular web site. You WANT Google etc. to spider your site silly!

The best way to achieve this is to have excellent descriptive, well defined metatags in the head of your html or css pages.
Well chosen keywords are also essential to include in your metatags.