Google changes algorithm to fight search spam

0

SAN JOSE, Calif. — Facing stinging criticism about the quality of its search results, Google
late Thursday made a significant change to its search engine to try to
combat the growing number of websites that exist primarily to land high
on its rankings.

These “content farms” and other websites that produce low-quality content were damaging the usefulness of Google’s search results, many outside experts charged.

What Google
called “a major improvement” was designed to highlight sites with
high-quality content and will noticeably affect about 12 percent of all
U.S. searches. It is also a clear signal that the company was also
concerned about the problem. Google Fellow Amit Singhal, who is in
charge of Google’s search algorithm, said in an interview Thursday that Google recognized the problem more than 15 months ago, and has been working on a solution long before the current wave of criticism.

“Most of the changes that we make, and we make lots of them, are nowhere close to this level of impact,” Singhal said.

It’s too soon to know whether the change will be
enough to silence critics, some of whom said before Thursday’s change
was announced that Google has not fought hard enough against these sites because it shares in the advertising revenue they generate.

“This is probably about the worst I’ve ever seen it,” Danny Sullivan, editor-in-chief of Search Engine Land, said of the complaints about Google’s search quality. “This does feel fairly unprecedented.”

Critics include Vivek Wadhwa, a visiting scholar at the University of California-Berkeley; Michael Arrington, founder of the TechCrunch blog; and researchers at the University of London. Google is dominant in search in North America and much of Europe,
with the criticism appearing not to have a significant impact on its
market share. And the company fired back at critics who have said it
has not been doing enough to keep the interests of advertisers and
users separate.

“I’m absolutely concerned about that perception because it could not be further from the truth,” Matt Cutts, Google principal engineer, said in an e-mail to the Mercury News.

On Feb. 7, researchers at the University of London published a paper saying that Google’s
efforts to personalize search results may actually do more to help
advertisers target users than help searchers find what they want. And
Wadhwa, who posted an influential blog Jan. 1 criticizing Google’s search results, has continued to speak out.

Critics like Wadhwa accused Google
of not doing enough to stop content farms from manipulating its search
results. When some people invariably click on those results, they go to
pages that often include Google text or display ads, Web traffic with the potential to earn revenue for the website—and for Google. In that scenario, Google is not just allowing poor content, it is directly profiting from it.

Google’s search
quality “has been getting worse for the past two or three years,”
Wadhwa said. “I’ve had tremendous feedback from all around the world;
people agree with what I said. Google is now on the spot.”

A large share of the search company’s profit, said
Wadhwa, a researcher and writer who specializes in entrepreneurship,
education, software development and is also affiliated with Duke University and Harvard Law School, “is coming from this spam.”

Reached after Google announced the change late Thursday, Wadhwa said he has been very critical about Google’s efforts to stop content farms because Google has an inherent conflict. But he said he thought Google’s response showed that the company is serious about fixing the problem, and is doing many of the right things.

Harry Shum, Microsoft’s corporate vice president for search product development, said at a recent conference that Google was not only allowing its search results to be degraded by content farms, but also benefiting financially from the situation.

“There must be an economic incentive to create this
kind of shallow content, this barely mediocre content,” Shum said. “Why
do they do that? Google ads.”

Unless Google
and its competitors work cooperatively to address the problem, “as an
industry, we really have a problem,” Shum said. “We really need to take
search quality very seriously; otherwise, we are all going down the
toilet.”

Industry experts differentiate between “webspam,”
when websites try to deceive search engines to gain a higher ranking,
and content farms, which produce low-quality content authored primarily
to feed popular trending topics on Google, Yahoo or Bing.

Blekko, a Redwood City
search startup that allows users to customize their search results,
several weeks ago began blocking all results from the 20 websites most
frequently marked by its users as being generators of poor-quality
content, including Demand Media’s
eHow and Answerbag. The company’s executives said this week that they
plan to start aggressively defending the company’s content. Such sites
can be lucrative: Demand Media recently raised $151.3 million in an IPO.

“We’re not being mealy-mouthed about this: There are good sites, and there are bad sites,” said Rich Skrenta, CEO of Blekko, which is funded by prominent investors including Ron Conway and Marc Andreessen.

Google knows it
will lose search share if users start to believe that advertising is
influencing the search engine’s results, Cutts said.

Google also
recently introduced an anti-spam tool, dubbed “Personal Blocklist,”
that can be installed in the company’s Chrome browser and that allows
users to block specific sites. More than 75,000 people installed it
during its first week, and 84 percent of the sites users complained
about will be affected by the change. While Google
has allowed users some limited control of the ranking of their search
results in the past, this is the first time it has allowed users to
automatically block all results from a site.

Sullivan, who takes pains to be a neutral voice in the search wars, said the Google tool “is half a good PR move and half a good user move.” One of Google’s problems is that spammers and content farms are getting smarter, said Vanessa Fox, who once worked at Google fighting spam. She said she doesn’t believe Google is selling out in that fight to make a buck in advertising.

“From the outside, it could easily look like the
wall is not so strong. But I think it’s that spammers are getting
smarter,” said Fox, who now runs Nine By Blue, an online marketing firm
in Seattle.

———

BLOCKING SPAM AND ‘CONTENT FARMS’:

—On the Web: The search engine Blekko.com allows
users to click on any result to designate it as “spam,” blocking future
results from that site. And on Thursday, Google revised its search engine to try to combat such sites.

—In your browser: A Google Chrome software tool at https://chrome.google.com/extensions allows users to block specific sites.

—The blocked: Blekko announced it would block
results from the top 20 websites designated as spam by its users,
including ehow.com, experts-exchange.com and naymz.com.

———

(c) 2011, San Jose Mercury News (San Jose, Calif.).

Visit MercuryNews.com, the World Wide Web site of the Mercury News, at http://www.mercurynews.com.

Distributed by McClatchy-Tribune Information Services.