Information Retrieval

Sorting the Wheat from the Chaff
In the classic world of Information Retrieval, when no commercial interests existed in the databases, very simplistic algorithms could be used to return high quality results. On the world wide web, however, the opposite is true. Commercial interests in the SERPs are a constant issue for modern search engines. With every new focus on quality control and growth in relevance metrics, there are thousands of individuals (many in the field of SEO) dedicated to manipulating these metrics in order to control the SERPs, typically by aiming to list their sites/pages first.
The worst kind of results are what the industry refers to as "search spam" - pages and sites with little real value that contain primarily re-directs to other pages, lists of links, scraped (copied) content, etc. These pages are so irrelevant and useless that search engines are highly focused on removing them from the index. Naturally, the monetary incentives are similar to email spam - although few visit and fewer click on the links (which are what provide the spam publisher with revenue), the sheer quantity is the decisive factor in producing income.
Other "spam" results range from sites that are of low quality or affiliate status that search engines would prefer not to list, to high quality sites and businesses that are using the link structure of the web to manipulate the results in their favor. Search engines are focused on clearing out all types of manipulation and hope to eventually achieve fully relevant and organic algorithms to determine ranking order. So-called "search engine spammers" engage in a constant battle against these tactics, seeking new loopholes and methods for manipulation, resulting in a never-ending struggle.
This guide is NOT about how to manipulate the search engines to achieve rankings, but rather how to create a website that search engines and users will be happy to have ranking permanently in the top positions, thanks to its relevance, quality, and user friendliness.
Source: seomoz.org