Naked Capitalism is on the receiving end of a nasty bit of synchronicity. Readers may recall that we wrote last week about the role that monopolies and oligopolies play in reducing competitiveness and promoting inequality. Oligopolies and monopolies are a classic example of rent extraction, where the incumbents can distort pricing due to their outsized position. The end result is less competitive products (dominant players don’t need to reinvest since their customers lack good options) and a transfer from customers to providers. We discussed a few examples of monopolists and oligopolists, but one we neglected to mention was Google.
Google’s search business has very strong network effects. Even though some users contend that competitors like Bing produce better results, particularly since Google in recent years has come to emphasize shopping, recency, and natural language searches, its challengers remain mere ankle-biters. Google’s ascendancy thus means if ranking high in a general search is important or even critical, as in you as a business want or need people who have no prior knowledge of your existence to be able to find you, Google can make or break you. As a result, there’s a mini industry devoted to catering to the whims of Google called “search engine optimization” or SEO. Established readers may recall that we looked into getting SEO advice but the service we used, Yoast, provided recommendations that in too many cases were designed for a product site, and thus were a poor fit for us. So we haven’t gone the route of rearranging our lives to please the Google gods.
I don’t monitor traffic to the site all that often, so when I looked at a summary of the last 7 days from Sitemeter, I was shocked at how low it was, particularly since the fall in page views (people who visit www.nakedcapitalism) was inconsistent with our readership levels via RSS syndication (which records how many people actually click on our articles in their RSS readers). If the problem was content, you’d expect to see a decline in both places.
I had our code jockey/WordPress expert Kristin look into it, and this was her report:
One of the things I had vowed to do was to wipe out all of the old rule list of what bots got blocked on the site. Cloudflare had several rules set up to keep the Chinese attack bots at bay, but when I noticed in Webmaster Tools that there were suddenly a significant number of URLs that were blocking the Googlebots, I got worried that the Cloudflare rules had recently trapped one or more of the Googlebots. Once I wiped out all the trapping rules in Cloudflare, The number of URLs blocking googlebots went straight to zero, so that got fixed. So, earlier, I was thinking possibly, that’s taking a few weeks to recover, but I feel it should have by now.
My next theory that I am afraid is probably true, however, is the fact that there have been a whole truckload of Google algorithm updates in the last 30-45 days.
Google Algorithm & Ranking Shifts On Fire This Month
Notice the update at the bottom of the post:
“Update: On May 20th we got confirmation from Google that this was an update to the Google Spam algorithm. And Google has also released a major update to the Panda algorithm today, which began rolling out today, May 20th.”
And, you can see the list of the updates just for the last month right here:
Again, I am not an SEO person, so I do not for the life of me what precisely about nakedcapitalism.com these rollouts affect, but it does not seem to be coincidence when I can directly match up the dates of the most major “update” and the dates the traffic slipped to its current levels.
I do wish we had an SEO person that we trust to consult.
We’ve done a lot more investigation into our traffic over the last few months. While the data is noisy and traffic often falls off in the summer, that usually doesn’t become pronounced until July and August. But we can see a clear decline in how often we come up in searches, and we don’t see a similar decay in traffic from readers who’ve previously visited the site. For instance:
We’ve also checked with sites whose type of content overlaps with ours and would be seen by Google in some other respects (high reading level and a moderately high reliance on cross posts, both of which Google penalizes in its search rankings). They haven’t seen any adverse traffic impact since the new Google spam-Panda rollouts.
The problem is the two likeliest culprits are core to our brand. The first is our name. Google may well have downgraded us for the word “naked”. This line of thinking isn’t paranoid. Long-standing readers may recall that in 2008, when this blog was young and still on Blogger (and before it had a daily Links feature), Google shut it down as a spam blog due to its name and number of links. Fortunately, we had a C-level contact and we were able to get that reversed in 24 hours (it normally would have taken two to three weeks). Thus, the second likely cause is our number of outbound links, meaning our daily Links feature. Google takes a dim view of outbound links. For instance, Yoast told us that our modest-by-financial-or-political-blog-standard blogroll was “enormous” and we needed to trim it way down to make Google happy (we haven’t).
The problem is our high-level Google contact has now moved on to another Silicon Valley giant. We do have a mid-level resource at Google, and we’ll clearly ask him for help, but we do not know yet whether he will be able to give us guidance, much the less run interference. If any readers know senior-level people at Google and would be willing to contact them on our behalf, please ping me at yves-at-nakedcapitalism.com so I can give you more background. It does seem absurd that a site that was listed in Wired in the last year as one of Wired’s “favorite sources of news covering the world of business and finance” and singled out by the New York Times’ Gretchen Morgenson as one of her only two non-MSM daily must reads is now deemed by Google’s formulas to be trashy, but welcome to the world of big data.
Admittedly, the impact of Google’s algo changes on us has not been as extreme as for some sites. Some webhosts saw their traffic go to zero. Nevertheless, the last week’s results were troubling enough that I feel compelled to take action, since if it were sustained, the drop in traffic (which translates into lower ad revenues) means we’d have to cut back on a lot of stuff we do for the site (and for moi to keep me from burning out, like vacation coverage).
We will be implementing one change we had discussed some months ago, which is to truncate our RSS feed. I read many sites on RSS and my reaction is similar to the overwhelming majority of readers who gave us feedback: I don’t mind truncation as long as I get enough text (one to three paragraphs) so I have enough of an idea of what the article is about to decide whether or not to click through.
The reason I’ve held off is that there does not seem to be a good solution for our e-mail readers. At least with WordPress, there is no off-the-shelf way to give e-mail readers full text while truncating the RSS, and a custom solution is prohibitive for a site of our scale. E-mail readers will also get a truncated version and will need to click through to the site to read our articles in full. Some e-mail readers said they’d be willing to subscribe to keep getting a full text version via e-mail. We looked into that too and came up empty-handed. If anyone knows of an alternative, we’d be happy to entertain it, but we spent real tech dollars investigating options.
I know this will make some long-standing members of our community mighty unhappy, but the alternative is likely to be to cut Links down to ten or fewer links a day, which would represent an even greater reduction in the service we now offer. And while the RSS and e-mail readers may initially chafe at having to click through to the site, I suspect many of you don’t realize what you are missing by not reading our lively, informative, and often funny comments. Many members of the NC community regularly say they get as much value from reader comments as they do the articles proper. So I hope you’ll find that this change in habit actually puts you net ahead.
Finally, we do have some feature enhancements in the works, although dealing with this Google mess will delay the rollout. I’d rather be bearing good tidings before bad, but clearly these events were not of our choosing.
While this RSS change should put us back where we were in terms of official page views (and thus stem the revenue loss), that is far from an ideal remedy. Being greatly downgraded in searches reduces our ability to influence opinion and reach new readers. If we had a C-level contact, as we did before, we’d probably be able to get this problem remedied relatively quickly. Lacking that sort of internal advocate now, I have no idea whether we can get this situation redressed.
As Lambert often says, the Web is a hostile environment. I’m sorry that some readers may take umbrage at the upcoming changes (which we will implement this week) but I trust you understand that they appear to be the least bad of our available choices. And as always, I thank you for your interest in our work and your ongoing support.