Back in 1972 there were a grand total of 213 computers worldwide that were part of a very exclusive club called the internet. By 1992 that number had risen to just under one million. That's still not a really large number when you think about it.
Back then there really wasn't much 'searching' going on, you either knew where something was, or you just left it alone.
The proliferation of the 'World Wide Wobble' has changed all that. Even the smallest company has a web site, many individuals have web sites, hell, peoples pets have their own web sites, the web has exploded. This vastness of knowledge and garbage has created an environment where in order to locate information you must use a search engine.
By far the most popular is Google, but there is also MSN, Yahoo, AOL, everyone and their dog wants in on this game!But where is the incentive? I hear you ask. Why would you want to spend millions of dollars cataloging the net, so that people can use you services for free?
Oh, there is big money involved here, Google makes its living primarily from the advertising revenue it charges. Have you never noticed the 'paid for' ad's when you do a search? The amount of money being generated by search engines is obscene (damn I wish I had thought of this business opportunity in the early 90's).
All of these search engines are plowing money into make theirs better than the rival, so that they attract more revenue. This is where the problems start.Let me try and explain a little.
I am sure that you are all familiar with e-mail spam, and I am sure you hate receiving all the offers to make things bigger, live longer, get rich quick, and all of the other scams out there. The World Wide Web is also over filling with spam. Spam Websites offering the same pills and potions, this in its self is not a problem, or at least it should not be.
If you type into google "Cheap Hotel in Las Vegas" you expect to get a list of hits about cheap hotels in Las Vegas, unfortunately what was happening was Google was also returing hits about Viagra, and other delightful things.This was not because Google was doing anything wrong, it was because web site owners were manipulating the search engine by methods like 'key word stuffing', this is where you include (un seen to the naked eye) keywords on your page that the Google web crawlers will index.These less than upstanding web owners also found a variety of other techniques to fool the search process.
Obviously this is not a good thing for the consumer, nor is it a good thing for the advertiser that is in effect financing the entire show.
Google has made it clear to the tech world that it is no longer tolerating this situation. Unfortunately what seems to have now happened is that it is next to impossible to actually get your new web site listed, and existing web sites that are well meaning, but maybe not professionally developed are being de-listed.Over the past few months I have created several sites, mostly non-commercial, mostly fun, or information sites and none of them have made it to the Google index. Yahoo, Msn, Altavista, have happily picked them up tho.
So the moral of the story, if you are a small e-commerce company do not put all of your eggs in one basket, and if you are a consumer looking for something, give the other search engines a try.