Regular users of Google News may soon find they’re getting less content from major online news sources than before in their searches, as Google tries to accommodate claims by publishers that it is “stealing” their news.
Media magnate Rupert Murdoch has led the charge against free access to news online. He says he is planning to charge Web users to read items on sites he controls (eg: The Times, The Australian, The Wall Street Journal). Murdoch and other publishers say Google News is allowing content from their sites that would normally be accessible only through paid subscription, to be read for free and thus robbing them of revenue.
Google’s search engines are constantly trawling through Web sites, ‘caching’ pages which then turn up when people use Google to search for items on specific topics. Google News lets them search for news items in this way.
Now Google is making it easier for publishers to stop it caching their pages. It has installed a new ‘web crawler’ for Google News which checks a specific file on news sites that tells it whether it is allowed to cache any, or all, of the site’s content. Previously, publishers had to fill out a specific request to Google to restrict caching – now it will be done automatically.
The move doesn’t block these pages from specific Google searches; it just stops them from appearing in Google News.
“Most people put their content on the Web because they want it to be found, so very few choose to exclude their material from Google,” an official Google Blog posting said. “But we respect publishers' wishes. If publishers don't want their Web sites to appear in Web search results or in Google News, we want to give them easy ways to remove it.”