Reasons that search engines repeat websites in search results

Like the pop-up ad, identical search results can be a common Internet problem. When a person is trying to find some specific information on the Internet, it can be very annoying to have to wade through multiple instances of the identical-but-not-quite-relevant search results. Many search engines even have outright clauses against duplicate information, in fact, as the duplicate results do not always have perfectly innocent causes. This article will cover a few of the more harmless causes first, and then some of the not-quite-so-innocent ones.

Legitimately placed advertisements are one reason why a person may see multiple listings of the same basic website. Sometimes larger companies will own other companies, or sometimes they will sponsor activities or forums related to important causes or charities of some kind. (For example, companies like Bing and Google sometimes like to sponsor certain types of worthwhile organizations.) The funds for this obviously need to come from somewhere, and they frequently come from advertising. Besides contributing to causes like public education or environmental protection, the income sometimes also goes to support things like the arts, development in underprivileged areas, or medical research and availability.

Another reason why a person might see duplicate search results is word of mouth, word of text, or other discussion on multiple forums. There are all kinds of different reasons why a person may inadvertently enter duplicate information into a search engine, often for no other reason than to advance the discussion about an important social, consumer, or political issue. For example critiques, blogs or review websites may cover virtually any topic from art or education to technology or medical products. Discussion on popular news websites may also result in people re-posting identical information without even realizing it, which is likewise often done for perfectly innocent reasons. Additionally, some websites will list small, technically permissible amounts of duplicate content on different pages simply to make certain commonly-investigated information easier for people to find.

Another harmless or innocent reason why a search engine may return identical website results is that people may forget to report or delete a certain webpage which is no longer being used. If a person happens to stumble upon one of these webpages and has a few extra moments of time, then a perfectly respectable white hat, ‘responsible Internet user’ approach is to simply contact the IT or Customer Support email address for the website provider’s domain name (and/or the comparable addresses for the search engine) to report the address of the broken or outdated link. As it is usually expensive for companies to try to maintain more bandwidth than necessary, and especially if you have nothing better to do at the time, then it is basically considered ‘good Internet karma’ to report the problem. This type of action falls into more or less the same kind of category as picking up a random piece of litter or making small change for someone who needs it: Little effort is involved, it’s not always expected, yet it’s still nice to have happen every once in awhile.

On a slightly darker note, however, there some non-innocent reasons why a web search may return duplicate content. Piracy or other copyright violations can, unfortunately, also result in either duplicate content or potentially dangerous malware being posted or uploaded onto multiple search engine sites. If someone suspects that some of a search engine’s returned hits may contain copyright violations or malware which may still be spread through a download of some sort, then it is perfectly justified to be concerned about it. Additionally, any duplicate or other website which shows signs of being part of a scam should likely be reported to either the search engine’s IT people and/or to the relevant country’s Internet Fraud Reporting website(s)- possibly both, though the search engine companies have more direct access to their own content and can often act more quickly to remove any potentially dangerous links or websites.

So, when a search engine returns several identical hits, it is not necessarily due to malicious, negligent, or avaricious types of reasons. While certain instances of identical results can contribute to malware spread or make the Internet appear more cluttered, some of these duplicate results actually do offer the technical advantage of making certain types of commonly researched information more readily accessible to Internet users. If someone feels that someone else may actually be using certain websites to spread viruses or to manipulate consumers (i.e. over-seeding content), then it is certainly more than helpful to contact the search engine’s Technical or Customer Support email addresses to report the suspected problem. If nothing else, that person will have done a good deed for the day and may even help to restore some poor, overworked technical support person’s confidence in the human race. After all, the Internet is about nothing if not human interaction.

On a final note, there are also certain things which can be done to help minimize unhelpful or potentially hazardous search results. The former of these things is frequently best accomplished by learning how to use the more advanced search options which are available on most large search engines. (For an explanation of advanced search engine techniques for a specific search engine, just look for that search engine’s Advanced Search tutorial.) It is also useful to examine a search engine’s security ratings for listed set of results, if available, to see which websites are the least likely to contain malicious content.